add text processor for data channel

[eliteprox]

This pull request adds support for text frame processing and publishing throughout the PyTrickle pipeline. It introduces new TextFrame and TextOutput classes, enables text processing in the core client and stream processor, and ensures text data can be handled, processed, and published similarly to audio and video. The changes also include robust fallback and caching logic for frame processing, and update protocol and logging to handle the new text frame type.

Text frame and output support:

• Added TextFrame and TextOutput classes to represent and handle text data (e.g., for transcription workflows) in frames.py. [1] [2]
• Updated imports and type handling in client.py, protocol.py, and stream_processor.py to include TextFrame and TextOutput. [1] [2] [3]

Text processing pipeline integration:

• Extended FrameProcessor and its async processing methods to support text frames, including a new process_text_with_fallback method for robust text handling. [1] [2]
• Updated StreamProcessor to accept a text_processor function, validate at least one processor is provided, and process text frames in the internal processor. [1] [2] [3] [4] [5]

Client and protocol logic updates:

• Modified the client ingress loop to process, send, and publish text frames, including fallback and logging logic for empty or failed frames. [1] [2] [3]
• Updated protocol logic to handle text frames and outputs, including debug logging and ensuring text frames do not affect FPS metrics. [1] [2]

Frame caching and fallback improvements:

• Added frame caching and fallback logic for video, audio, and text frames in FrameProcessor to improve robustness in case of processing failures.

These changes collectively enable robust, efficient, and extensible support for text data in the PyTrickle streaming and processing pipeline.

[Copilot]

Pull Request Overview

This pull request adds comprehensive text frame processing capabilities to the PyTrickle pipeline, enabling text data to be processed and published alongside audio and video streams. The changes implement text publishing functionality for processing workflows like transcription services.

• Introduces text frame processing infrastructure with TextFrame and TextOutput classes
• Adds text publishing capabilities to processing functions through a publish_text parameter
• Implements robust frame caching and fallback mechanisms for improved reliability

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
pytrickle/frames.py	Adds import for time module (minor addition)
pytrickle/frame_processor.py	Adds async text publishing method and timing reset hook
pytrickle/stream_processor.py	Adds text publishing support and validation for processor functions
pytrickle/client.py	Implements comprehensive frame caching and fallback logic with text publishing
examples/requirements.txt	Adds scipy dependency for audio processing examples
examples/process_video_with_text.py	New example demonstrating text publishing with passthrough processing
examples/process_audio_example.py	New example showing advanced audio effects processing
.vscode/launch.json	Adds debug configurations for new examples

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[pschroedl]

lets rename this example to video_to_text or something similar

[eliteprox]

This example counts audio frames and publishes some basic stats to the data publisher to demonstrate text publishing. Should we name it text_output_example.py?

[pschroedl]

planning to implement in this PR?

[eliteprox]

Removed in version in favor of a shared text_queue for publishing outputs