Backpressure
The problem backpressure solves
Section titled “The problem backpressure solves”Imagine a fire hose attached to a garden sprinkler. If water arrives faster than the sprinkler can spray it, the hose eventually bursts. The same thing happens in Node streams: if a fast Readable source produces data faster than a slow Writable can consume it, buffered chunks pile up in memory until the process runs out of RAM or crashes.
Backpressure is the mechanism by which a slow consumer tells a fast producer to slow down. It is the stream equivalent of flow control in TCP.
How backpressure works in Node
Section titled “How backpressure works in Node”The Writable stream’s write() method is the key signal:
- Returns
true— the internal buffer is below the high-water mark (default: 16 KB (16384 bytes) for byte streams, or 16 objects for object-mode streams;fs.createReadStreamis the exception at 64 KB). The producer can keep sending. - Returns
false— the buffer is at or above the high-water mark. The producer must pause and wait for thedrainevent before writing more.
The high-water mark is not a hard limit; it is a soft advisory. Node buffers chunks beyond it, but doing so costs memory. Respecting the signal keeps memory usage flat.
Manual backpressure
Section titled “Manual backpressure”Before pipe and pipeline, you had to handle this yourself:
function pumpWithBackpressure(readable, writable) { readable.on('data', function(chunk) { var ok = writable.write(chunk); if (!ok) { readable.pause(); // stop the source writable.once('drain', function() { readable.resume(); // resume when ready }); } });
readable.on('end', function() { writable.end(); });}This pattern is verbose and error-prone. One forgotten readable.pause() call causes the buffer to grow without bound.
How pipe and pipeline handle it automatically
Section titled “How pipe and pipeline handle it automatically”readable.pipe(writable) and stream.pipeline() implement the above pattern internally. They:
- Forward each chunk from the readable to the writable.
- Pause the readable when
write()returnsfalse. - Resume the readable on the
drainevent.
You never need to manage pausing and resuming manually when using pipe or pipeline. This is one of the primary reasons to prefer them over raw event listeners.
Visualising backpressure with custom high-water marks
Section titled “Visualising backpressure with custom high-water marks”The demo below creates a Readable and a Writable with a very small high-water mark (1 byte) to make backpressure trigger on every chunk. Watch how the readable is automatically paused and resumed.
Needs the Node.js runtime — open in StackBlitz to run.