Streams

Code

import { createReadStream, createWriteStream } from "fs";
import { Transform, pipeline } from "stream";
import { createGzip } from "zlib";

// Transform stream that uppercases text
const upper = new Transform({
  transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

// Pipeline handles backpressure and error propagation
pipeline(
  createReadStream("input.txt"),
  upper,
  createGzip(),
  createWriteStream("output.txt.gz"),
  err => {
    if (err) console.error("pipeline failed", err);
    else console.log("done");
  }
);

// Consuming a stream with async iteration
import { stdout } from "process";
for await (const chunk of createReadStream("input.txt")) {
  stdout.write(chunk);
}

Explanation

Streams process data in chunks instead of buffering it all in memory, essential for large files or network traffic. Transform streams sit between Readable and Writable streams to modify data in flight. pipeline wires streams together with proper backpressure and error handling, replacing the older pipe() method.

More Node.js Snippets