Skip to content

Why does strict allow 'highWaterMark' pending writes, instead of just 1? #22

Closed

Description

@pimterry

There may very well be a good answer to this, so this is partly just for my own understanding and to document the reasoning, since I can't see anything about this written down anywhere.

The arguments I can see for pending writes = 1 in strict mode:

  • It bounds the buffered chunks much closer to what you'd naively expect: if you set highWaterMark = N, it's surprising that a writer will hold 2N in flight. N + 1 is much closer & more justifiable bound given what you asked for.
  • If you do correct single-writer backpressure, you never need more than one pending write (multi-writer backpressure is complex and I think for strict + async at least it's not obvious how to do that safely, and an arbitrary parallelism limit is even worse, so better to fail clearly upfront anyway).
  • It tightens the constraint & strictness, so it's much more likely to fail faster if you're doing something wrong (instead of failing unpredictably later).
  • If you aren't doing correct backpressure in some subtle way you're eventually going to get into a world of trouble with strict regardless, and relaxing this doesn't help there - it'd be better to use block and accept this explicitly.

The Node docs themselves say:

If you properly await each write, you can only ever have one pending write at a time (yours), so you never hit the pending writes limit.

So why allow more than 1?

(Obviously inspired by #20, since this applies there too, and the double bound is extra surprising - why not just always allow exactly 1 pending slot?)

Activity

  1. jasnell commented on Jun 2, 2026

    @jasnell
    Collaborator

    The way I've envisioned it is like a hose filling up a bucket. Both the bucket and the hose each have a capacity. The water in the bucket (the slot queue) is what we're currently processing. The water in the hose (the pending queue) is what we're processing immediately next. What we don't want is water spilling on the ground.

        hose          bucket
     +-+-+-+-+-+    +-+-+-+-+-+
      1 2 3 4 5  =>  1 2 3 4 5
     +-+-+-+-+-+    +-+-+-+-+-+
    

    The strict policy just says that we can't overflow both the bucket or the hose. However, in order to keep data flowing smoothly we always want to try to keep water in both the bucket and the hose flowing at a steady rate. For strict, ideally the capacity of the hose matches the capacity of the bucket.

    The highWaterMark primarily specifies the capacity of the bucket, but we also use it to determine the capacity of our hose such that in strict, the hose never contains more than what the bucket can hold. This means that in strict, the actual effective total capacity is highWaterMark times 2.

  2. jasnell commented on Jun 2, 2026

    @jasnell
    Collaborator

    Here's a more structured example

    push({ highWaterMark: 3, backpressure: 'strict' })
    
    Producer awaits each write. Consumer is slow.
    
    Step 1: write('a') — bucket has space, goes directly in
        hose               bucket
     +-+-+-+             +-+-+-+
      . . .    =======>   . . a
     +-+-+-+             +-+-+-+
    
    Step 2: write('b') — bucket has space
        hose               bucket
     +-+-+-+             +-+-+-+
      . . .    =======>   . b a
     +-+-+-+             +-+-+-+
    
    Step 3: write('c') — bucket is now full
        hose               bucket
     +-+-+-+             +-+-+-+
      . . .    =======>   c b a
     +-+-+-+             +-+-+-+
    
    Step 4: write('d') — bucket full, write goes into hose (pending queue).
             Producer blocks (awaits) until space opens up.
        hose               bucket
     +-+-+-+             +-+-+-+
      . . d    =======>   c b a
     +-+-+-+             +-+-+-+
    
    Step 5: write('e') — also goes into hose. Producer blocks.
        hose               bucket
     +-+-+-+             +-+-+-+
      . e d    =======>   c b a
     +-+-+-+             +-+-+-+
    
    Step 6: Consumer reads! Bucket drains completely. Hose refills bucket.
             Pending writes resolve; producers unblock.
    
      before read:
        hose               bucket
     +-+-+-+             +-+-+-+
      . e d    =======>   c b a
     +-+-+-+             +-+-+-+
    
      consumer gets: [c, b, a]
    
      after drain + promote:
        hose               bucket
     +-+-+-+             +-+-+-+
      . . .    =======>   . e d
     +-+-+-+             +-+-+-+
    
    Step 7: write('f') — bucket has space again
        hose               bucket
     +-+-+-+             +-+-+-+
      . . .    =======>   f e d
     +-+-+-+             +-+-+-+
    
    And the fire-and-forget case that `strict` catches:
    
    What if the producer does NOT await?
    
        hose               bucket
     +-+-+-+             +-+-+-+
      f e d    =======>   c b a       — both full
     +-+-+-+             +-+-+-+
    
    write('g') — no room in bucket OR hose!
                 strict: RangeError "too many pending writes"
                 (the water would spill on the ground)
    
  3. pimterry commented on Jun 2, 2026

    @pimterry
    Author

    So that example is assuming multiple parallel producers, is that right? I think if it were just a single producer, AFAICT it blocks at step 4 and doesn't continue until the read happens, and so for that single well-behaved producer case your example never writes more than a single value into the pending queue at a time.

    So in effect, in a strict + correct-backpressure world, we're using the pending queue size purely to set the number of parallel async-writing producers supported, is that right? That does make sense, and I can see how >1 is required to support that. It's a bit confusing though - it's not obvious that in the above example this is safe with 3 producers, but could fail at any time with 4, even if they're independently handling backpressure correctly.

    I do see from your example nearly why the hose should never be bigger then the bucket though - that's required for efficient promotion, otherwise we have to go through an unbounded number of reads to empty all pending writes, whereas a pending size <= buffer size guarantees that we'll always have a pending slot after read, and need two reads at most to empty everything. Neat.

    Interesting to think about anyway. We could consider defining this better in the docs to make the concurrent-producer behaviour (and expected patterns) clearer in general imo. Right now there's clear solutions for concurrent reads, but concurrent writes is an exercise for the reader. Not sure if you already have a suggested best practice & external contract for that somewhere?

  4. jasnell commented on Jun 2, 2026

    @jasnell
    Collaborator

    So that example is assuming multiple parallel producers, is that right?

    The number and type of producer is irrelevant here and the model makes no such assumptions.

    I think if it were just a single producer, AFAICT it blocks at step 4 and doesn't continue until the read happens, and so for that single well-behaved producer case your example never writes more than a single value into the pending queue at a time.

    For strict, the writes would not start rejecting unless both the slot queue (the bucket) and the pending queue (the hose) are both full.

    The slots here represent number of write operations, not amount of data. In the current design, if we specify highWaterMark: 1, that places no constraints on the actual number of Uint8Arrays included in the write.

    writer.write(new Uint8Array(1));  // consumes 1 slot
    writer.writev([... 100 Uint8Arrays ...]);  // consumes 1 slot
    

    Whether the writes are coming from multiple producers (sync, async, whatever) is not a factor.

  5. jasnell commented on Jun 2, 2026

    @jasnell
    Collaborator

    Ah, I see my example is a bit off at step 4. In both strict and block, with an async writers, when a write goes directly to the bucket, the returned promise is returned resolved (meaning, the write was immediately accepted into the bucket). If the write goes into the pending, the returned promise is pending. In that example, for strict, the caller can write up to two more times without running afoul of the strict policy and having writes reject. With the block policy, the pending writes just keep accumulating in the pending queue.

  6. pimterry commented on Jun 3, 2026

    @pimterry
    Author

    So that example is assuming multiple parallel producers, is that right?

    The number and type of producer is irrelevant here and the model makes no such assumptions.

    Yep, I think from your second comment you see what I mean. I was confused by blocking in step 4, but then continuing to step 5 without a read.

    More generally though, I've done some tests, and I think there is a real parallel producer problem: I think if you have highWaterMark = N on a strict stream and N + 1 parallel producers, even if they always correctly use the patterns from the docs, currently any backpressure will always trigger a violation. Demo:

    import * as SI from 'stream/iter';
    
    const { writer, readable } = SI.push({ highWaterMark: 3 });
    
    async function startWriting(id) {
        for (const item of ['a', 'b', 'c', 'd', 'e']) {
            console.log(`${id} writing ${item}...`);
            await writer.write(item);
            console.log(`${id} wrote ${item}`);
        }
    }
    
    const PRODUCER_COUNT = 4;
    for (let i = 0; i < PRODUCER_COUNT; i++) {
        startWriting(i);
    }
    
    // Sleep - don't read immediately
    await new Promise(resolve => setTimeout(resolve, 1000));
    
    const i = readable[Symbol.asyncIterator]();
    console.log('Reading...');
    const { value } = await i.next();
    console.log('Read', value);

    That's the canonical correct writer pattern, and they crash immediately: 4 producers write a, 4th producers sees backpressure & waits, 3 remaining producers all write('b') => Backpressure violation.

    Parallel producers can detect that there's backpressure (desiredSize === 0) but AFAICT we don't expose any promise to wait for drain except via write(), so there's no way they can safely & independently wait for it even if they want to.

    Only solution would be a global backpressure promise that all producers carefully share and manage somehow, but that is a major constraint that's non-trivial to implement due to the TOCTOU & coordination involved. We don't have a way to handle this in our own code either - N+1 pipeTo fails under backpressure the same way.

    Is there any way to rewrite the above example (4 independent producers writing to a 3-slot writer) so it handles backpressure correctly today?

    For strict, the writes would not start rejecting unless both the slot queue (the bucket) and the pending queue (the hose) are both full.

    They won't reject, but they'll block, and that guarantees a maximum one item in the pending queue given a single producer I think. Defining that more clearly:

    import * as SI from 'stream/iter';
    
    const { writer, readable } = SI.push({ highWaterMark: 3 });
    
    async function startWriting() {
        for (const item of ['a', 'b', 'c', 'd', 'e']) {
            console.log(`Writing ${item}...`);
            await writer.write(item);
            console.log(`Wrote ${item}`);
        }
    }
    
    startWriting();
    
    // Sleep - don't read immediately
    await new Promise(resolve => setTimeout(resolve, 1000));
    
    const i = readable[Symbol.asyncIterator]();
    console.log('Reading...');
    const { value } = await i.next();
    console.log('Read', value);

    This blocks after writing 'c' => that's 3 slots used (a/b/c), one pending queue (d). AFAICT there is no reader behaviour where this producer (the correct backpressure example from the Node docs) ever uses more than one pending slot. Every producer flow I can see that does use the queue requires either parallel producers, or the single producer to ignore a backpressure signal before a write() call.

  7. pimterry commented on Jul 13, 2026

    @pimterry
    Author

    Resolved - as of #23, pending writes is now 1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions