Auto Keyboard Switch For Multi-Layer Input Methods: Feature Request

Imagine a world where your keyboard effortlessly adapts to your input, switching layers automatically as you type. This is the vision behind this feature request, aimed at enhancing the functionality of multi-layer input methods within Trime. Currently, users of input methods like 声韵分层 (Shengyun) face the cumbersome task of manually embedding keyboard switch commands in every key of the second layer. This approach not only increases maintenance efforts but also hinders true automatic transitions based on input state. Let's dive into the details of this proposal and explore how it can revolutionize the user experience.

The Problem: Manual Keyboard Switching

The core issue lies in the manual nature of keyboard switching in multi-layer input methods. The current system requires developers to explicitly define keyboard switch commands for each key on secondary layers. This approach presents several challenges:

• Increased Maintenance Burden: Manually managing keyboard switches for every key is time-consuming and prone to errors. Any changes or updates require tedious modifications across numerous configurations.
• Lack of Automatic Transitions: The absence of event-based triggers prevents seamless transitions between keyboard layers based on the user's input state. This limitation hinders the natural flow of typing and diminishes the overall user experience.
• Repetitive Configuration: The need to repeat keyboard switch commands across multiple keys leads to significant configuration duplication, making the setup process cumbersome and inefficient.

Use Case: Shengyun (声韵) Input Method

To illustrate the problem and the potential solution, let's consider the example of Shengyun, a two-layer Pinyin input method designed for kindergarten children learning Chinese Pinyin. This method exemplifies the challenges and opportunities associated with multi-layer input.

Background:

Shengyun is designed to simplify the learning process by dividing Pinyin input into two layers:

1. Initial Layer (声母层): This layer comprises 23 initial consonants, such as b, p, m, f, d, t, n, l, g, k, h, j, q, x, zh, ch, sh, r, z, c, s, y, and w.
2. Finals Layer (韵母层): This layer contains 35 finals for each initial, including a, o, e, ai, ei, ao, ou, an, en, ang, eng, ong, and er.

Desired Workflow:

The ideal user experience with Shengyun would involve the following steps:

1. The user clicks an initial consonant, such as "l," on the initial layer.
2. The system automatically switches to the finals layer, presenting the corresponding finals for the selected initial.
3. The user clicks a final, such as "iang." The system commits the complete Pinyin syllable ("liang") to RIME and automatically switches back to the initials layer.
4. RIME displays a list of candidate characters for the input syllable, such as 两 (two), 亮 (bright), and 量 (measure).

Current Workaround:

Currently, achieving this workflow requires a cumbersome workaround:

# Must add this to EVERY finals button in EVERY keyboard
- {click: liang, select: shengyun_initials, label: "iang", width: 11.11}

This approach results in several drawbacks:

• 23 Separate Finals Keyboards: The need to define a separate finals keyboard for each initial consonant leads to a proliferation of keyboard configurations.
• ~1,295 Lines of Repetitive Configuration: The sheer volume of repetitive configuration lines makes the setup process tedious and error-prone.
• No Response to RIME Engine Events: The current system lacks the ability to respond to RIME engine events, such as auto-returning to the initials layer upon committing a syllable or after backspacing when the input is empty.

Current Limitations

What Trime Currently Supports

Trime offers some features that partially address the needs of multi-layer input methods:

• Manual Keyboard Switching: Users can manually switch between keyboards using click: Keyboard_X or select: keyboard_name.
• Special Keyboard IDs: Trime provides special keyboard IDs like .default, .last, and .next for convenient keyboard switching.
• send_bindings: false: This option prevents double input during transitions.

What's Missing

However, several crucial features are missing:

• No Event Hooks: Trime lacks event hooks for on_commit, on_backspace, and on_candidate_select, which would enable automatic transitions based on input events.
• No RIME Processor/Translator Integration: The absence of integration with RIME processors and translators hinders keyboard state management based on input context.
• No State Machine Support: Trime does not offer a state machine mechanism for defining automatic layer transitions based on predefined rules.
• No Lua API: The lack of a Lua API for keyboard switching (e.g., rime.keyboard.switch()) prevents dynamic keyboard control from within Lua scripts.

Why Lua Approach Failed

Attempts to use RIME Lua processors for automatic switching have been unsuccessful due to the following error:

LuaProcessor::ProcessKeyEvent error(2): attempt to call a nil value

This error indicates that RIME Lua processors cannot access Trime's keyboard switching API, limiting the flexibility of Lua-based solutions.

Requested Features

To address the limitations outlined above, the following features are requested:

1. Event Hook: on_commit / on_backspace_empty

The ability to trigger keyboard switches based on input events would greatly simplify multi-layer input methods. The following YAML configuration demonstrates how this could be implemented:

preset_keyboards:
  shengyun_finals:
    on_commit: shengyun_initials        # Auto-return after input committed
    on_backspace_empty: shengyun_initials  # Return when backspace on empty input

2. RIME Processor Integration

Allowing RIME schema processors to send keyboard switch commands to Trime would enable dynamic keyboard control based on input context. The following Lua code illustrates how this could be achieved:

-- In rime.lua processor
function processor.func(key, env)
  if key:repr() == "Return" then
    env.keyboard.switch("shengyun_initials")  -- Switch keyboard from Lua
  end
end

3. State Machine Support

Defining keyboard transitions declaratively through a state machine mechanism would provide a flexible and maintainable approach to managing multi-layer input methods. The following YAML configuration demonstrates how this could be implemented:

keyboard_transitions:
  - from: shengyun_initials
    to: shengyun_finals_*
    trigger: on_key_press
  - from: shengyun_finals_*
    to: shengyun_initials
    trigger: on_commit | on_candidate_select | on_backspace_empty

4. Lua API Extension

Extending the Lua API to include keyboard switching functions would provide developers with greater control over keyboard behavior. The following API functions are proposed:

-- Proposed API
rime.keyboard.switch(keyboard_id)    -- Switch to specific keyboard
rime.keyboard.current()              -- Get current keyboard ID
rime.keyboard.on_commit(callback)    -- Register commit event listener
rime.keyboard.on_backspace(callback) -- Register backspace event listener

Similar Use Cases

This feature would benefit a wide range of input method scenarios:

1. 双拼 (Shuangpin) Keyboards: Seamlessly switch between initial and finals layers.
2. 三拼 (Triple Pinyin) Keyboards: Support multi-layer phonetic input with ease.
3. Shape-Based + Phonetic Dual-Layer: Enable 形音双层布局 for efficient input.
4. Educational Input Methods: Create step-by-step learning keyboards for children.
5. Symbol/Emoji Layers: Automatically return to the main keyboard after symbol selection.

Technical Benefits

The implementation of these features would yield several technical benefits:

• Reduced Configuration Duplication: Reduce the number of keyboards from 23 to 1 base + 1 finals template.
• Enabled Dynamic Keyboard Generation: Generate keyboards dynamically based on RIME state.
• Better Separation of Concerns: Allow RIME to handle logic while Trime focuses on display.
• Support for Complex Multi-Layer Workflows: Enable educational, professional, and accessibility use cases.
• Improved Maintainability: Centralize transition logic instead of relying on per-key switches.

Alternative Approaches Considered

Current Manual Approach

• Pros: Works reliably.
• Cons: Requires 1000+ manual select: statements, cannot respond to RIME events, and has a high maintenance burden.

RIME Lua Processors

• Pros: Can access RIME input state.
• Cons: Cannot control Trime keyboard state due to the lack of an API and encounters LuaProcessor::ProcessKeyEvent errors.

Custom Trime Fork

• Pros: Full control over behavior.
• Cons: Difficult to maintain (track upstream changes) and does not benefit the broader community.

Proposed Implementation (Optional)

If the Trime maintainers find this feature valuable, I am willing to contribute code. A potential implementation approach could involve the following steps:

1. Add Event Callbacks to the Keyboard Class:

data class Keyboard(
  val onCommit: String? = null,  // Target keyboard ID
  val onBackspaceEmpty: String? = null
)

2. Hook into RIME Events in RimeEngine.kt:

override fun onCommit() {
  currentKeyboard.onCommit?.let { switchKeyboard(it) }
}

3. Expose Lua API via RimeLuaApi:

fun keyboard_switch(keyboard_id: String) {
  Rime.switchKeyboard(keyboard_id)
}

Summary

Multi-layer input methods like Shengyun require automatic keyboard transitions based on input events. Current manual approaches are verbose and inflexible. Adding event hooks (on_commit, on_backspace_empty) and RIME integration would:

• Reduce configuration complexity
• Enable dynamic keyboard workflows
• Benefit educational, dual-layer, and multi-step input methods
• Improve user experience for non-traditional keyboard layouts

Would the Trime maintainers be open to this feature? I'm happy to discuss implementation details or contribute code if this aligns with Trime's roadmap.

---

Environment:

• Trime version: latest (from official release)
• RIME version: librime (via Trime)
• Android version: 14
• Device: Tested on multiple Android devices

Related:

• Shengyun project: https://github.com/gshmu/shengyun
• Current implementation: https://github.com/gshmu/shengyun/blob/main/shengyun.trime.yaml

To learn more about Rime and its capabilities, visit the official Rime Input Method Engine Documentation website.