Skip to content

[Performance]: Reduce live Markdown re-render churn while streaming #49

Description

@shauryagangrade

Problem

RichUI re-renders the streaming Markdown every 80 new characters
(_TRUNCATE_STEP = 80) under rich.live.Live. Long model responses cause
hundreds of full-screen re-renders, which is CPU-heavy and can visibly lag on
slower terminals (and is brutal for the vhs demo recordings).

Proposed approach

  • Benchmark re-render cost; consider: larger/adaptive step, rendering the
    final block only on stream end, or throttling to N fps.
  • Keep the "current partial output visible while streaming" behavior.

Where to look

  • gcode/ui.py:20 (_TRUNCATE_STEP), :132 (the Live refresh handler),
    :164-191 (assistant_start/token/assistant_end).

Acceptance criteria

  • A long model response streams with a measurable reduction in re-renders/Live updates.
  • No visual regression in partial-token display.

Difficulty

Medium.

Metadata

Metadata

Assignees

No one assigned

    Labels

    performanceLatency, throughput, and efficiency improvements

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions