Skip to content

feat: FIFO command queue + channel slot virtualization for multi-client access#13

Merged
rgregg merged 17 commits intomainfrom
feature/command-queue
Mar 26, 2026
Merged

feat: FIFO command queue + channel slot virtualization for multi-client access#13
rgregg merged 17 commits intomainfrom
feature/command-queue

Conversation

@rgregg
Copy link
Copy Markdown
Owner

@rgregg rgregg commented Mar 26, 2026

Summary

  • Adds an asyncio.Queue to serialize commands from multiple TCP clients to the radio
  • Adds opt-in channel slot virtualization (--virtualize-channels / VIRTUALIZE_CHANNELS=true) that gives each TCP client its own virtual channel slot space mapped to physical radio slots
  • Physical slots are deduplicated: identical channels from different clients share one slot on the radio
  • LRU eviction when all 40 physical slots are exhausted (safe because TCP clients always re-send SET_CHANNEL)
  • Eager release on reassignment, compatible with Remote-Terminal's channel cycling pattern
  • Adds environment variable support for all CLI settings (Docker-friendly)

Motivation

Support running Home Assistant integration, MeshCore client, and Remote-Terminal simultaneously against the same radio. The command queue prevents command interleaving, and channel virtualization prevents clients from overwriting each other's channel slot configurations.

New files

  • src/meshcore_proxy/channel_virtualizer.py - ChannelSlotAllocator with all virtualization logic
  • tests/test_channel_virtualizer.py - 20 unit tests

Environment variables

All CLI flags now accept env var fallbacks:
SERIAL_PORT, BLE_ADDRESS, TCP_HOST, TCP_PORT, BAUD_RATE, BLE_PIN, LOG_LEVEL, LOG_JSON, DEBUG, VIRTUALIZE_CHANNELS

Test plan

  • test_commands_serialized_through_queue - verifies FIFO ordering through queue
  • test_tcp_client_commands_go_through_queue - verifies end-to-end TCP client -> queue -> radio path
  • test_commands_dropped_when_radio_disconnected - verifies graceful handling when radio is down
  • 20 unit tests for channel slot allocator (allocation, dedup, LRU eviction, response rewriting, disconnect cleanup)
  • 2 integration tests for virtualization in the proxy
  • Pull dev Docker image ghcr.io/rgregg/meshcore-proxy:dev-13 and test with VIRTUALIZE_CHANNELS=true
  • Test with Remote-Terminal cycling through channels
  • Test with two clients sharing a channel (dedup)
  • Verify existing behavior unchanged without --virtualize-channels

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings March 26, 2026 03:03
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a FIFO asyncio.Queue–backed command pipeline so multiple TCP clients can safely share a single radio connection without interleaving outbound commands.

Changes:

  • Introduces a background queue worker to serialize outbound commands to the radio.
  • Updates TCP client handling to enqueue commands instead of sending directly.
  • Adds async tests covering FIFO ordering, TCP → queue → radio flow, and disconnected-radio behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
src/meshcore_proxy/proxy.py Adds command queue + worker, routes TCP client commands through the queue, and cancels/drains the queue on shutdown.
tests/test_proxy.py Adds tests for queue serialization and disconnected-radio handling.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +269 to +272
await self._send_to_radio(payload)
except Exception as e:
logger.error(f"Queue worker error: {e}")
finally:
Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_run_command_queue catches Exception, which includes asyncio.CancelledError. If the worker is cancelled while awaiting _send_to_radio, the cancellation will be swallowed and the task will keep running, potentially causing stop() to hang while awaiting _queue_worker_task. Handle asyncio.CancelledError explicitly (re-raise) before the broad exception handler so cancellation reliably terminates the worker.

Copilot uses AI. Check for mistakes.
Comment on lines +143 to +144
self._command_queue: asyncio.Queue = asyncio.Queue()
self._queue_worker_task: Optional[asyncio.Task] = None
Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_command_queue is unbounded. Since TCP handlers now put() quickly without waiting on radio I/O, a fast/malicious client can enqueue commands faster than the single worker can drain, leading to unbounded memory growth. Consider giving the queue a maxsize and applying backpressure (awaiting put) or dropping commands with a clear warning when the queue is full.

Copilot uses AI. Check for mistakes.
Comment on lines +252 to +255
# Command should not appear in send buffer (it was dropped)
# The radio had no sends after disconnection
send_count_before = len(mock_radio.send_buffer)
assert len(mock_radio.send_buffer) == send_count_before
Copy link

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion is a no-op: send_count_before is captured after the enqueue + wait, and then compared to the current length, so it will always pass. Capture send_count_before before enqueuing (or assert mock_radio.send_buffer is unchanged/empty) so the test actually verifies that the command was dropped while disconnected.

Copilot uses AI. Check for mistakes.
rgregg and others added 5 commits March 26, 2026 03:25
Add asyncio.Queue and worker task to MeshCoreProxy so that commands
are serialized through the queue and sent to the radio one at a time
in FIFO order. The worker is started in run() and cancelled in stop().

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Adds a publish-ghcr-dev job that builds and pushes
ghcr.io/rgregg/meshcore-proxy:dev-<PR number> on pull requests.
Requires approval via the dev-deploy environment.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Re-raise CancelledError in queue worker so shutdown isn't blocked
- Add maxsize=100 to command queue to bound memory usage
- Fix no-op assertion in disconnect test (capture count before enqueue)
- Prevent reconnection in disconnect test to avoid race condition

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@rgregg rgregg force-pushed the feature/command-queue branch from 0543ae5 to 44253bb Compare March 26, 2026 03:26
Reject frames that don't start with 0x3C (client -> server) instead
of blindly parsing them. Logs a warning with the invalid byte value.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
rgregg and others added 7 commits March 26, 2026 18:34
Cover SEND_CHAN_MSG rewriting, reassignment, LRU eviction,
response rewriting, client disconnect, GET_CHANNEL, and pass-through.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Code review caught that the V3 variant of channel message responses
wasn't being rewritten. Consolidated the response type check into a
single tuple membership test.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
All CLI flags now accept environment variable fallbacks, making
Docker container configuration easier without modifying the command.

Environment variables:
  SERIAL_PORT, BLE_ADDRESS, TCP_HOST, TCP_PORT, BAUD_RATE, BLE_PIN,
  LOG_LEVEL (off/summary/verbose), LOG_JSON, DEBUG, VIRTUALIZE_CHANNELS

Updated docker-compose.yml to use environment variables instead of
command arrays.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@rgregg rgregg changed the title Add FIFO command queue for multi-client radio access feat: FIFO command queue + channel slot virtualization for multi-client access Mar 26, 2026
@rgregg rgregg deployed to dev-deploy March 26, 2026 19:01 — with GitHub Actions Active
rgregg and others added 3 commits March 26, 2026 19:03
Skip the dev-deploy environment approval gate for PRs from branches
in this repo (trusted). PRs from forks still require manual approval.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
The Dockerfile CMD was --help, which caused containers configured
purely via environment variables to loop (print help, exit, restart).

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Replace --quiet, --log-events, --log-events-verbose, and --debug
with a single --log-level flag: off, error, warning, info, debug,
verbose. Simplifies Docker configuration to just LOG_LEVEL=debug.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@rgregg rgregg merged commit d9d9883 into main Mar 26, 2026
8 checks passed
@rgregg rgregg deleted the feature/command-queue branch March 26, 2026 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants