Skip to content

ZQ-77: add composer dictation and Desktop native-input foundation - #175

Draft
zqbake wants to merge 33 commits into
QwenAudio:mainfrom
zqbake:zq-77-fresh-composer-dictation
Draft

ZQ-77: add composer dictation and Desktop native-input foundation#175
zqbake wants to merge 33 commits into
QwenAudio:mainfrom
zqbake:zq-77-fresh-composer-dictation

Conversation

@zqbake

@zqbake zqbake commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add opt-in Gateway 2.1.0 composer dictation with an independent Qwen ASR adapter, capability gating, bounded session state, exact-once commit receipts, and explicit Memory-correction handling
  • wire Web and TUI composers for partial/final text, keyboard interleaving, deterministic recent-range edits, continuous dictation, pause/cancel, and existing composer submission
  • integrate the macOS system-input feature into the same Desktop product: a packaged universal InputMethodKit bundle, Desktop-owned transient Bridge, signed local IPC, lifecycle settings, global shortcut, shared Gateway/microphone ownership, and no independent daemon or provider credential
  • install, repair, register, status-check, and uninstall Qwen Input from the Desktop settings flow; the user must explicitly enable and select it in macOS, and Qwen never silently changes the selected input source

Architecture and safety

Desktop renderer (existing microphone + Gateway WS) -> Electron main (narrow IPC) -> QwenInputBridge (signed stdio/socket broker) -> Qwen Input (InputMethodKit) -> focused macOS text client

  • composer and native-input features remain disabled by default
  • the IME and Bridge do not capture audio, connect to providers, hold provider credentials, or persist transcript/draft/target text
  • partial text uses the target's marked-text composition; final/edit operations are limited to ranges owned by the current session and guarded by session/generation/target/sequence checks
  • secure, stale, hidden, focus-lost, replayed, malformed, and uncorrelated operations fail closed
  • Bridge loss removes the owned partial, reconnects to a replacement Bridge, and never writes into a new target
  • ordinary keyboard input passes through; starting while another input source is selected fails visibly and leaves that source unchanged
  • Accessibility, Input Monitoring, Full Disk Access, and administrator access are not required for the base InputMethodKit path; optional Voice Send remains separate and disabled

Installed-app and GUI evidence

On macOS 26.5.1 arm64 with the Debug/ad-hoc build:

  • real install/register/status/repair/uninstall completed without administrator access; macOS System Settings was used to explicitly add and enable Qwen Input
  • TextEdit accepted real marked partial, final, pause/resume, deterministic replace, and cancel operations
  • Safari textarea and contenteditable accepted real partial/final/edit operations
  • Safari password input failed closed without insertion; focus loss rejected the later final and did not write into the new target
  • terminating the Bridge during an active marked partial removed the partial and allowed a replacement Bridge to reconnect to the same live IME controller
  • cleanup restored ABC, removed the installed input method and runtime socket, and left no Bridge/IME process

Automated verification

  • npm run native-input:test
  • node --test desktop/test/native-input-*.test.mjs (34/34)
  • npm test
  • npm run lint
  • npm run build
  • git diff --check
  • arm64 and universal DMGs pass hdiutil verify; packaged Desktop, Bridge, and IME pass strict/deep code-sign verification
  • the universal Desktop, Bridge, and IME each contain both x86_64 and arm64

Remaining release gates

This PR remains Draft. The installed InputMethodKit path has been exercised in TextEdit and Safari, but the physical-microphone/provider path in the packaged Desktop UI, raw Terminal/Secure Keyboard Entry matrix, Intel runtime, and Developer ID hardened-runtime/notarized/stapled release have not been completed. The local artifacts are ad-hoc signed and are not production release evidence. The remaining matrix is documented in docs/desktop/native-input-testing.md and its Chinese counterpart.

Closes ZQ-77

Related #165

zqbake and others added 3 commits August 21, 2026 13:09
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
@zqbake
zqbake marked this pull request as draft August 21, 2026 13:42
zqbake and others added 14 commits August 22, 2026 12:44
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
@zqbake zqbake changed the title ZQ-77: add private composer dictation for Web and TUI ZQ-77: add composer dictation and Desktop native-input foundation Aug 22, 2026
zqbake and others added 11 commits August 22, 2026 14:23
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
@zqbake
zqbake marked this pull request as ready for review August 24, 2026 04:07
@zqbake
zqbake marked this pull request as draft August 24, 2026 15:07
zqbake and others added 5 commits August 24, 2026 23:15
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
Co-authored-by: multica-agent <github@multica.ai>
@zqbake
zqbake marked this pull request as ready for review August 24, 2026 17:16
@zqbake
zqbake marked this pull request as draft August 25, 2026 01:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant