Add VOD browsing, Baichuan recording playback, and live streaming - #164
Open
lorek123 wants to merge 11 commits into
Open
Add VOD browsing, Baichuan recording playback, and live streaming#164lorek123 wants to merge 11 commits into
lorek123 wants to merge 11 commits into
Conversation
- Two-way audio (talk) via Baichuan protocol:
* cmd_id 201 (TalkAbility): query duplex mode and audio config from camera
* cmd_id 202 (Talk): send IMA ADPCM audio data without encryption
* cmd_id 203 (TalkConfig): configure and start a talk session
* cmd_id 204 (TalkReset): stop an active talk session
* Baichuan.build_bcmedia_adpcm() helper to wrap ADPCM blocks in BcMedia framing
* send_binary_no_reply() internal method for fire-and-forget binary payloads
* Capability detection updated to use cmd_id=201 as the authoritative check
- Public API additions on Host:
* two_way_audio_support(channel) – bool capability check
* start_talk(channel) – starts session, returns audio config dict
* send_talk_data(channel, bcmedia_data) – sends BcMedia-framed ADPCM audio
* stop_talk(channel) – stops session
- RTSP VOD replay support:
* VodRequestType.RTSP added to enums
* get_vod_source(..., request_type=VodRequestType.RTSP) returns an RTSP URL
for cameras/NVRs that support RTSP playback of recorded files
- XML templates for TalkAbility and TalkConfig added to baichuan/xmls.py
References: apocaliss92/nodelink-js (cmd_id 201-204), QuantumEntangledAndy/neolink PR#396
https://claude.ai/code/session_0116gKmpLmMXRwA5DM8VBzfc
…owser Adds two convenience wrappers over request_vod_files() that make it easy to implement Home Assistant's media browser for recorded clips: - get_recording_days(channel, year, month) -> set[int] Returns the set of day-numbers (1–31) in the given month that contain at least one recording. Intended for calendar/month navigation in the media browser. Backed by request_vod_files(status_only=True). - get_recordings_for_day(channel, day, stream=None, trigger=None) -> list[VOD_file] Returns all recordings for the given calendar date, sorted chronologically and deduplicated (a file matching multiple triggers only appears once). Supports optional trigger filter (VOD_trigger.MOTION, PERSON, etc.). Each returned VOD_file exposes: - start_time / end_time / duration - triggers (VOD_trigger flags: MOTION, PERSON, VEHICLE, …) - file_name → pass to get_vod_source() for playback URL - size Typical HA media-browser flow: 1. days = await host.get_recording_days(ch, year, month) # calendar 2. files = await host.get_recordings_for_day(ch, date(...)) # day listing 3. mime, url = await host.get_vod_source(ch, file.file_name) # playback https://claude.ai/code/session_0116gKmpLmMXRwA5DM8VBzfc
Adds raw H264 VOD streaming over the Baichuan TCP protocol for baichuan_only cameras that lack HTTP download or RTSP VOD support. New public API on Host: - stream_recording_bc(channel, file_name, start_time, stream_type) Async generator yielding (timestamp_us, h264_annex_b_bytes) tuples. Authenticates lazily via the Baichuan TCP connection — no HTTP get_host_data()/get_states() calls needed (~0.4 s startup). New Baichuan methods: - search_recording_days_bc() — days with recordings in a given month - search_recordings_for_day_bc() — file list for a given day - replay_seek_bc() — seek to a position in an open replay session - stream_replay_bc() — sends StartSearch/StartPlay commands, yields raw decrypted binary chunks from the camera - parse_bcmedia_frames() — parses BcMedia binary framing (I/P-frame video, AAC/ADPCM audio, Info packets), extracts H264 Annex-B payloads with microsecond timestamps; uses 8-byte-aligned scanning to avoid false-positive magic matches inside H264 payload data New XML templates in xmls.py: DayRecords, ReplaySeek, ReplayStart, ReplayStop. reolink-stream-vod console script (reolink_aio/scripts/stream_vod.py): Intended as a go2rtc exec: source. Rate-limits H264 delivery using BcMedia microsecond timestamps so go2rtc wall-clock RTP timestamps produce correct playback speed. Handles u32 timestamp wraparound (~71 min recordings). Usage: reolink-stream-vod HOST USER PASS CHANNEL FILE_NAME START_TIME Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix BCMEDIA_ADPCM_MAGIC byte order: was b"bw10" (big-endian), must be
struct.pack("<I", 0x62773130) = bytes [30 31 77 62] (little-endian u32).
The camera silently discarded all audio frames due to magic mismatch.
- Fix build_bcmedia_adpcm: add proper frame header (payload_size duplicated,
sub-magic 0x0100, half_block=2 constant per Ghidra/pcap analysis).
Previously the function sent raw ADPCM blocks with no framing at all.
- Fix cmd_ids (confirmed via Ghidra libBCSDKWrapper.so + pcap):
TalkAbility=10, TalkConfig=201, TalkReset=11 (was 201/203/204)
- Fix capability detection: probe baichuan_only cameras via cmd_id=10;
detect two_way_audio by presence of <duplex> element (not audioStreamMode)
- Add CHANNEL_BINARY_EXTENSION_XML with <binaryData>1</binaryData> for
audio talk data packets (cmd_id=202)
- Remove unused TalkAbilityGet XML template
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move `import calendar` to top-level in api.py and baichuan.py (C0415) - Add `from err` to raise in stream_replay_bc timeout handler (W0707) - Reformat with black (api.py, baichuan/baichuan.py) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The upstream non-ASCII fix (bfc2234) changed _aes_encrypt() to accept bytes instead of str. Update _send_streaming() to encode ext/body as UTF-8 bytes and use byte lengths for mess_len/payload_offset, consistent with the existing send() method. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ation - Extract _parse_cover_preview_frame() helper that accepts both b"1001" and b"1002" stream header magic (E1 cameras return 1002) - snapshot_past() now uses the helper, fixing the UnexpectedDataError bug - Remove public get_recording_thumbnail() API: CoverPreview on E1 returns only SPS/PPS headers (no decodable picture), so thumbnail generation is better handled by the HA integration using the streaming path Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Author
|
Update: Added one more commit fixing a bug in the existing
|
|
Nice timing! I independently implemented the same feature (PR #165) this week, unaware of your work. We arrived at the exact same approach — IMA ADPCM encoding, BcMedia framing, cmd_id 201→202→11 lifecycle — which is a good sign that the reverse engineering is solid. I've tested against a Reolink Home Hub + Argus 4 Pro with both sine wave tones and ElevenLabs TTS speech, and it works well. Your split API (start_talk/send_talk_data/stop_talk) is more flexible than my single talk() method — better for live streaming use cases. Happy to close mine if this gets merged, or contribute anything useful from my implementation (e.g. the 30 unit tests in tests/test_talk.py). Cheers, good work! |
The E1 camera sends replay packets in two flavors: packets with an explicit encryptLen in the Extension XML (partial AES region), and packets with just <binaryData>1</binaryData> but no encryptLen (payload is plaintext). The SDK treats absent encryptLen as 0xFFFFFFFF which cast to signed is -1, so `0 < (int)encryptLen` fails and decrypt is skipped. Previously we fell through to a probe/full-AES-decrypt heuristic on packets without encryptLen, producing garbled video (20 ffmpeg decode errors). Now we only decrypt when encryptLen is explicitly present and > 0, matching the SDK behavior. Verified on E1: 0 decode errors. Also stores the negotiated Baichuan crypto tier (Aes vs FullAes) from the login response status word, so Aes-tier cameras (like Argus 2) skip binary decrypt entirely per neolink PR #396. Made-with: Cursor
Author
|
Hey Fredrik, thanks! Great to see independent implementations converge on the same approach — definitely confirms the protocol is right. I tested your PR #165 against my Reolink E1 (standalone camera) and found a bug: if self.http_api.api_version("talk", channel) > 0 or self.http_api.baichuan_only:Happy to contribute this fix + the split API (start_talk/send_talk_data/stop_talk) to your branch if that works for you. I'll then strip 2-way audio out of this PR so it stays focused on VOD replay — cleaner for review. Let me know if you're open to that! |
|
Hi! Thanks! Good catch on the E1 bug — I only have hub-based cameras so wouldn't have caught that. Happy to have you push the fix + split API to my branch. I'll review when it's ready. |
Two-way audio (talk) is being contributed to the standalone PR at fredrsat#1 which focuses on that feature. This keeps PR starkillerOG#164 focused on VOD replay streaming and recording browsing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ent search
Port the following protocol-level improvements from our neolink work:
1. H.265 support in parse_bcmedia_frames() — read video_type bytes [4:8] from
BcMedia frame header ("H264" or "H265") and yield (microseconds, bytes, codec)
instead of the previous (microseconds, bytes) 2-tuple.
2. NAL-level codec detection (_detect_codec_from_nal) — override the BcMedia
header codec when firmware labels H.265 frames as "H264" (e.g. Argus PT).
Detects H.265-exclusive VPS/SPS/PPS NAL types (0x40, 0x42, 0x44).
3. MSG 0x17d (desktop binary replay) fallback — when both MSG 5 and MSG 8 return
400, try the desktop protocol that sends a 0x944-byte binary payload (channel +
file path). Adds _send_streaming_binary() for raw (unencrypted) payloads and
_build_desktop_replay_payload() for the struct layout (neolink replay.rs).
4. Size-based stream end — parse the 32-byte replay header (bytes [16:24] and
[24:32]) for expected file size and stop streaming when bytes received reach
it. Needed for cameras (e.g. E1) that never send response 300/331.
5. Multi-stream listing — search_recordings_for_day_bc() now queries both
mainStream and subStream when stream=None/default and merges results
(deduplicating by file_name). Cameras like Argus 3 store separate files
per stream; old behaviour returned only mainStream.
6. MSG 175 alarm/event search — new search_recordings_by_event_bc() sends
findAlarmVideo with server-side alarm-type filter. Adds FindAlarmVideoOpen
and FindAlarmVideoNext XML templates.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Author
|
@fredrsat fredrsat#1 let me know what you think |
Implements host.stream_live_bc(channel, stream_type) — an async generator yielding (microseconds, video_bytes, codec) tuples for live H264/H265 frames via the Baichuan Preview/VideoStart protocol (MSG 3 start, MSG 4 stop). Applies the same FullAes AES-CFB decryption logic as stream_replay_bc for cameras that encrypt the first chunk of each streaming packet (e.g. E1). Tested on Reolink E1 (baichuan_only): 2304×1296 H264, valid NALs confirmed via ffprobe. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lorek123
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
get_recording_days/get_recordings_for_day): calendar-level and day-level recording search. Falls back to Baichuan MSG 142/14/15 whenbaichuan_only=True, otherwise uses the existing HTTPrequest_vod_filespath. Designed for use in the Home Assistant media browser.stream_recording_bc): async generator that yields(microseconds, video_bytes, codec)tuples via the Baichuan replay protocol (MSG 5/8/0x17d) + BcMedia frame parser. Supports H.264 and H.265. Forbaichuan_onlycameras where HTTP download is unavailable.stream_live_bc): async generator that yields(microseconds, video_bytes, codec)tuples for live H.264/H.265 frames via the Baichuan Preview/VideoStart protocol (MSG 3 start, MSG 4 stop). Applies the same FullAes AES-CFB decryption logic as VOD replay for cameras that encrypt streaming packets.VodRequestType.RTSP): adds an RTSP playback URL variant toget_vod_source().Usage
Test plan
get_recording_days(0, 2026, 2)returns correct set of days with recordingsget_recordings_for_day(0, date(2026, 2, 13))returns recordings with correct times/triggers/filenamesstream_recording_bcstreams H.264 frames from a recording; verified with ffprobe (H.264 High, 2304×1296)stream_live_bcstreams live H.264 frames; verified with ffprobe (H.264, 2304×1296), 50 frames decoded cleanlybaichuan_only=True)Notes
_send_streamingupdated to encode XML as UTF-8 bytes, consistent with the non-ASCII fix from Fix Baichuan header lengths for non-ASCII XML #163.🤖 Generated with Claude Code