3x Faster Stream processing

3x Faster Stream Processing in Node.js: Boosting throughput by eliminating buffering.

Hi Folks,

This newsletter comes back to a theme I love: Node.js streams. I expect more contents about the fundamentals in the future, as I'm scavenging some old research to help folks prioritize work on Node.js itself.

3x Faster Stream Proessing

8 years ago, I explored how to make Node.js Streams significantly faster. This research never landed in Node.js, but given the renewed interest in Performance... who know? Anyhow, I was able to 3x the throughput of a Node.js stream pipeline.

A primer on Node.js streams

Node.js streams are the key way we use to transfer data using constant memory. There is a Readable side (upstream), and a Writable side (downstream), and the data flow in this direction at the maximum throughput that downstream allows. In order to accomodate to the different throughput, there are buffers on either side.

Streams manage backpressure, which could be called flow control in text books, which have a readable pause the flowing of data if the Writable own buffer has reached a threshold, which we call highWaterMark. If that happens, a Readable will accumulate data in its own buffer, up until its own highWaterMark.

Data flowing from one readable to one writable is partially useful. The power from streams come from using one or more Transform between Readable and Writable.

Transform is both a Writable and Readable at the same time, and it has both their buffers.

Here is a pipeline example:

import { createReadStream } from 'node:fs'
import { pipeline } from 'node:stream/promises'
import { Transform } from 'node:stream'

await pipeline(
  createReadStream(import.meta.filename),
  new Transform({
    transform(chunk, enc, cb) {
      this.push(chunk.toString().toUpperCase())
      cb()
    }
  }),
  process.stdout)

The problem with the above code is that by adding a Transform, you are adding two layers of buffering, which add overhead and memory usage.

The need of those two extra layers exists because backpressure is not enforced: a writable will accept data even if its buffer is full.

How SyncThrough increases throughput by 3x

Making any code faster involves reducing overhead, and SynchThrough implements the same API of transform without the use of any buffers by forcing the data transformation step to be synchronous. Lucky for us, this is the most common use case for Transform streams.

Here is a pipeline example:

import { createReadStream } from 'node:fs'
import { pipeline } from 'node:stream/promises'
import { syncthrough } from 'syncthrough'

await pipeline(
  createReadStream(import.meta.filename),
  syncthrough(function (chunk) {
    // there is no callback here
    // you can return null to end the stream
    // returning undefined will let you skip this chunk
    return chunk.toString().toUpperCase()
  }),
  process.stdout)

Benchmarks:

Here is the benchmarks results:

benchThrough2*10000: 1.680s
benchThrough*10000: 356.651ms
benchPassThrough*10000: 766.079ms
benchSyncThrough*10000: 260.583ms
benchThrough2*10000: 1.625s
benchThrough*10000: 336.915ms
benchPassThrough*10000: 745.687ms
benchSyncThrough*10000: 248.093ms

Should you contribute to Open Source?

I don't think you should contribute to Open Source to land a job, but it's often a clear way up if you don't have many opportunities.

In the video, I tell my experience

Releases

• @fastify/cors v9.0.0 adds Vary header only for non-static origin option. @fastify/cors v9.0.1
• fastq v1.17.0 consistently respect the configured concurrency
• fastify-cli v6.1.0 makes require work in esm projects.
• fastify v4.26.0
• @fastify/compress v7.0.0 reduces default brotli compression to 4.
• borp v0.6.0 adds reporters; v0.7.0 adds a flag to disable typescript compilation. v0.8.0 switches to using @reporters/github and removes the markdown reporter. v0.9.0 adds support for reporters in node_modules. v0.9.1 do not crash when using the gh reporter.
• fast-json-stringify optimizes the performance of required properties. v5.11.1 reverts all changes introduced by v5.11.0 because they were breaking. v5.12.0 re-apply the patches of v5.11.0.
• @fastify/http-proxy v9.4.0 adds WebSocket host constraints.
• undici v6.6.0 include some spec updates to fetch() and some bugfixes.
• pino v8.18.0 add the ability to override custom levels comparison.
• @fastify/static v7.0.0 update to glob v10. Drop Node.js v14 and v16.
• [@fastify/auth v4.5.0] extend composite auth to support "or" relations.
• @matteo.collina/snap v0.1.2 adds missing slash dependency.
• fastq v1.17.1 fixes a regression that prevented the drain callback from being called after an empty pause/resume cycle.
• avvio v8.3.0 fixes a type.
• fastify-undici-dispatcher v0.5.1

Articles

• Open Source and a Healthy Dose of Capitalism
• OpenJS Security Audit for nvm Completed Successfully | OpenJS Foundation
• Now it's PostgreSQL's turn to have a bogus CVE
• Streams and React Server Components
• A 2024 Wishlist for Node's Test Runner
• GitHub-hosted runners: Double the power for open source
• React, where are you going?
• Node.js, TypeScript and ESM: it doesn't have to be painful
• Leaky Vessels: Docker and runc Container Breakout Vulnerabilities - January 2024 | Snyk
• Patch management needs a revolution, part 4: Sane patching is safe patching is selective patching
• htmx ~ Examples ~ Async Authentication
• Method Shorthand Syntax Considered Harmful
• Hot Module Replacement is Easy - Bjorn Lu

Events

Node.js HTTP Clients Masterclass

It's a new month & that means it's time for a new Platformatic masterclass! We'll be exploring the complexities surrounding HTTP Clients, looking at: ‣Choosing between libraries ‣Addressing unexpected breakages ‣Building an HTTP client with Platformatic For the chance to win a Platformatic swag bag, sign up to the masterclass, share it on LinkedIn or Twitter and tag us (@platformatic), and attend! We will announce the winner at the end of the masterclass.

Register here.

CityJS London

I'll be in London from the 2nd to the 6th of April, for Node.js Collaborator Summit and to Keynote CityJS London "The Alleged 'End' of Node.js is Much Ado About Nothing"

Tickets are available here.