feat(example): add example about advanced streaming insert. #462
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds comprehensive examples demonstrating advanced streaming INSERT operations with backpressure handling for the ClickHouse Node.js client. This addresses issue #390 by providing practical guidance for handling high-throughput streaming scenarios.
Key changes:
- Adds advanced streaming example with event-driven data source and proper backpressure management
- Includes simplified streaming example showing essential backpressure patterns
- Updates documentation to reference the new streaming examples
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| examples/node/insert_streaming_with_backpressure.ts | Advanced streaming example with event-driven data source, burst mode simulation, and comprehensive backpressure handling |
| examples/node/insert_streaming_backpressure_simple.ts | Simplified streaming example demonstrating core backpressure concepts with interval-based data generation |
| examples/README.md | Documentation updates adding references to the new streaming examples |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| // Push all pending data, but stop if we hit backpressure again | ||
| while (this.#pendingData.length > 0 && !this.#streamPaused) { | ||
| const data = this.#pendingData.shift() | ||
| if (!data || !this.#pushData(data)) { |
There was a problem hiding this comment.
The condition !data will never be true since this.#pendingData.shift() is only called when this.#pendingData.length > 0 is verified on line 97. The !data check is redundant and could mask logic errors.
| if (!data || !this.#pushData(data)) { | |
| if (!this.#pushData(data)) { |
Copilot uses AI. Check for mistakes.
There was a problem hiding this comment.
It's not right, because the data might be undefined.
|
|
||
| await client.command({ | ||
| query: ` | ||
| CREATE TABLE ${tableName} |
There was a problem hiding this comment.
Nit: just CREATE OR REPLACE should be sufficient
|
Thanks, this is very cool. I will try to find time to review it more thoroughly soon. |
Summary
This pr is try to fix #390
Note
This pr only add example codes, no any other feature or bug fix changes.
Checklist
Delete items not relevant to your PR: