Worked review 2: a file-processing CLI

A small CLI-style script: read a text file, uppercase every line, write the result to another file — the kind of “streams” example an AI assistant reaches for constantly. It writes its own input file so the demo is self-contained, and cleans up both files on exit. Four bugs from the AI bug catalog are planted across about 35 lines.

The program

Node.js

// @expect-throw
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { Transform } from 'node:stream';

const inputPath = path.join(os.tmpdir(), 'wr2-input.txt');
const outputPath = path.join(os.tmpdir(), 'wr2-output.txt');

process.on('exit', function () {
  try { fs.unlinkSync(inputPath); } catch {}
  try { fs.unlinkSync(outputPath); } catch {}
});

fs.writeFileSync(inputPath, 'hello\nสวัสดี\nCORRUPT_ROW\nworld\n', 'utf8');

// BUG 1: reads the whole file into memory synchronously before streaming even starts
const wholeFile = fs.readFileSync(inputPath);
console.log('read', wholeFile.length, 'bytes synchronously before streaming even starts');

function slowHash(text) {
  // BUG 4: synchronous CPU work per chunk -- blocks the event loop while "streaming"
  let sum = 0;
  for (let i = 0; i < text.length; i++) {
    for (let j = 0; j < 5000; j++) { sum = (sum + text.charCodeAt(i) * j) % 100000007; }
  }
  return sum;
}

const upper = new Transform({
  transform(chunk, encoding, callback) {
    const text = chunk.toString('utf8'); // BUG 3: decodes each chunk independently
    slowHash(text);
    if (text.includes('CORRUPT')) {
      callback(new Error('corrupt row encountered'));
      return;
    }
    callback(null, text.toUpperCase());
  }
});

const source = fs.createReadStream(inputPath, { highWaterMark: 8 });
const dest = fs.createWriteStream(outputPath);

source.pipe(upper).pipe(dest); // BUG 2: no error handling anywhere in this pipe chain

dest.on('finish', function () {
  const written = fs.readFileSync(outputPath, 'utf8');
  console.log('output:', JSON.stringify(written));
});

Output (verified on Node 24)

read 43 bytes synchronously before streaming even starts

Only one line of stdout, then a crash — dest.on('finish', ...) never fires, so no output: line ever appears.

Real output (stderr, after the line above):

node:events:486
      throw er; // Unhandled 'error' event
      ^
Error: corrupt row encountered
    at Transform.transform [as _transform] (...)
Emitted 'error' event on Transform instance at:
    at Transform.onerror (...)

Find the bugs before reading on

Four planted bugs — one memory choice, one lifetime, one boundary, one blocking call. Try to name all four before opening the answers below.

Bug 1 — fs.readFileSync(inputPath) right before streaming the same file

Symptom: the whole point of a stream is to avoid holding an entire file in memory at once — this line defeats that immediately, loading the complete file synchronously before a single streamed chunk is even read.

Evidence: on this tiny demo file it costs nothing visible; on a multi-gigabyte log file it is the difference between constant memory usage and an out-of-memory crash. The “43 bytes” in the real output above is misleadingly small — the bug’s cost scales with file size, not with whether the bug is present.

Fix: delete the line entirely. Nothing downstream in this file uses wholeFile for anything — it exists only as a diagnostic that was never removed.

Bug 2 — source.pipe(upper).pipe(dest); with no error handling

Symptom: .pipe() forwards data between streams but never forwards 'error' events — an error on any stream in the chain has nowhere to go.

Evidence: the real crash above. The Transform’s callback(new Error(...)) on the corrupt row raises an 'error' event on upper; with nothing listening for it, Node treats it as fatal and kills the whole process before dest ever finishes.

Fix: stream.pipeline() from node:stream/promises, which propagates any stream’s error into one place and also destroys every stream in the chain so nothing is left half-open.

Bug 3 — chunk.toString('utf8') inside the transform

Symptom: each chunk is decoded to a string independently, with highWaterMark: 8 deliberately small enough to split the multi-byte Thai text mid-character.

Evidence: this is invisible in the crash above (the corrupt row crashes the process before this bug’s own output would appear), but running only the first two lines of input (no corrupt row) makes the mangled Thai text directly visible in the output file — the same � replacement-character corruption as pattern 15 in the AI bug catalog.

Fix: node:string_decoder’s StringDecoder, which buffers an incomplete multi-byte sequence until the chunk that completes it arrives.

Bug 4 — slowHash(text) called synchronously inside transform()

Symptom: a CPU-heavy nested loop runs synchronously for every single chunk, blocking the event loop for its entire duration — inside code that is nominally “streaming” specifically to avoid blocking.

Evidence: invisible in this small demo’s output (its result, sum, is not even used), which is exactly the danger: it costs real, measurable latency under load without ever producing a visibly wrong answer.

Fix: delete it — nothing in this script actually needs a checksum. If a real per-chunk computation were needed, it would belong off the main thread (a worker_thread) rather than blocking every other request or timer while it runs.

The fixed version

Node.js

import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { Transform } from 'node:stream';
import { pipeline } from 'node:stream/promises';
import { StringDecoder } from 'node:string_decoder';

const inputPath = path.join(os.tmpdir(), 'wr2-input.txt');
const outputPath = path.join(os.tmpdir(), 'wr2-output.txt');

process.on('exit', function () {
  try { fs.unlinkSync(inputPath); } catch {}
  try { fs.unlinkSync(outputPath); } catch {}
});

fs.writeFileSync(inputPath, 'hello\nสวัสดี\nworld\n', 'utf8');

// FIX 1: no upfront readFileSync -- go straight to streaming

function makeUpperTransform() {
  const decoder = new StringDecoder('utf8'); // FIX 3: buffers incomplete multi-byte sequences
  return new Transform({
    transform(chunk, encoding, callback) {
      const text = decoder.write(chunk); // FIX 4: no synchronous per-chunk busy work either
      callback(null, text.toUpperCase());
    },
    flush(callback) {
      const rest = decoder.end();
      callback(null, rest ? rest.toUpperCase() : undefined);
    }
  });
}

const source = fs.createReadStream(inputPath, { highWaterMark: 8 });
const dest = fs.createWriteStream(outputPath);

try {
  await pipeline(source, makeUpperTransform(), dest); // FIX 2: pipeline forwards errors and cleans up every stream
  const written = fs.readFileSync(outputPath, 'utf8');
  console.log('output:', JSON.stringify(written));
} catch (err) {
  console.log('pipeline failed:', err.message);
}

Output (verified on Node 24)

output: "HELLO\nสวัสดี\nWORLD\n"

The Thai line survives the round trip intact this time — StringDecoder held the split bytes from the first small chunk until the next chunk completed the character, instead of decoding each incomplete half on its own. Had this run included the same CORRUPT_ROW line as the buggy version, pipeline()’s catch block would have printed pipeline failed: corrupt row encountered and exited cleanly at code 0, instead of crashing the process the way the buggy version’s unhandled .pipe() chain does.

What the tools caught

npx eslint on the buggy version: zero problems. None of these four bugs — an unnecessary upfront readFileSync, a .pipe() chain with no error handling, per-chunk toString(), and a synchronous loop inside a stream callback — match a rule in eslint-plugin-n or eslint-plugin-promise.

node --trace-warnings on the buggy version: nothing — this crashes via an unhandled 'error' event on a stream, which is a different mechanism from the warning event that flag is built to trace.

Running the file caught bug 2 immediately (the crash) and, indirectly, bug 1 (the synchronous read that runs right before it, visible in the one line of real output). Bugs 3 and 4 produced no visible symptom in this exact run — 3 needs a run without the crash to see the mangled output, and 4 needs either a much bigger input or a profiler (--cpu-prof, from the Verification Tools lesson) to show up as measurable latency.