Skip to content

Conversation

@shinaoka
Copy link

@shinaoka shinaoka commented Dec 14, 2025

element-meta Issue #2883: i18n support for full-text search in encrypted rooms

Summary

This PR adds tokenizer mode support to element-desktop, enabling users to choose between standard (language-based) and N-gram tokenization for the local message search index.

Changes

  • Add tokenizerMode option to EventIndex initialization
  • Accept tokenizerMode from element-web via IPC
  • Support both N-gram and language-based tokenization modes
  • Default to language-based tokenization for backward compatibility

Files Changed

  • src/@types/matrix-seshat.d.ts - Add TypeScript types for tokenizer options
  • src/seshat.ts - Pass tokenizerMode to Seshat, handle IPC from element-web

Dependencies

⚠️ This PR depends on:

Seshat PR must be merged and a new npm package version released before this PR can be merged.

After Seshat is released, update package.json to use the new matrix-seshat version (e.g., ^4.1.0).

Merge Order

1. Seshat PR #150  →  npm release (e.g., [email protected])
2. element-desktop (this PR)  →  update matrix-seshat dependency
3. element-web  →  UI for tokenizer mode selection

Checklist

  • Ensure your code works with manual testing.
  • New or updated public/exported symbols have accurate TSDoc documentation.
  • Linter and other CI checks pass.
  • I have licensed the changes to Element by completing the Contributor License Agreement (CLA)

@shinaoka shinaoka requested a review from a team as a code owner December 14, 2025 23:11
@github-actions github-actions bot added the Z-Community-PR Issue is solved by a community member's PR label Dec 14, 2025
@shinaoka shinaoka requested a review from a team as a code owner December 14, 2025 23:21
- Add tokenizerMode option to EventIndex initialization
- Accept tokenizerMode from element-web via IPC
- Support both N-gram and language-based tokenization modes
- Default to language-based tokenization for backward compatibility
@dbkr
Copy link
Member

dbkr commented Dec 15, 2025

This needs the other two PRs to land first (which will need a look from product).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

T-Enhancement Z-Community-PR Issue is solved by a community member's PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants