April 6, 2021

Adventures in Nodeland by Matteo Collina - Issue #4

Hi Everyone! I’ve started a newsletter to share my thoughts on technology, the advancements of Node.js and to keep all of you up-to-date of what I’m working on. This newsletter might be weekly, biweekly or monthly… we’ll see how it evolves in the coming months.

If you have been contributing Open Source on GitHub in the last couple of months, you might have noticed that there was a significant change in GitHub actions due to the security risk that GitHub actions posed. As an example, this broke a significant part of the Fastify automation as well as something that powered Node.js. Anyway, read up on the story that lead to that security bug, as well as how we re-implemented the Fastify automation using a Github App:

Fastify

One of the biggest news in Fastify this week is the release of Tap v15 by Isaacs - this version ships a few breaking changes that I have been awaiting for a long time:

1. the removal of the esm module and full support for native ESM as it is now stable;
2. the removal of the typescript dependency - you would need to install typescript yourself;
3. JSX is enabled by default only on .jsx files;
4. 100% code coverage is enforced by default (while it can be easily disabled);
5. deprecation or removal of a lot of assertions.

All those changes combined broke the build of most of the Fastify projects 😭. I wrote a quick script that replaces all the deprecations in a single sweep: https://gist.github.com/mcollina/e799ebb3fa9260abbe8ea56a8434ef24. If you would like to help, send a PR to one of the many projects at https://github.com/fastify.

Supabase, the Open Source Firebase Alternative, has used Fastify to create their storage API. Read up on their choice in the following article:

Node.js Core

Last week I discovered a significant performance regression in the HTTP server (https://github.com/nodejs/node/issues/37937) while I was preparing for the previous edition of this newsletter - I wanted to speak how fast Node.js v16 would be… and it was significantly slower!

I performed several git bisects (https://git-scm.com/docs/git-bisect) as well as flamegraphs with Clinic Flame (https://www.clinicjs.org/). I identified the following commits to be problematic.

“requestTimeout” df08d52 dropped the number req/s from 80k to 76k; I co-wrote this feature and we benchmarked it extensively, I’m really surprised we missed it! I plan to work on fixing this in the future as it’s going to be extremely hard because it was introduced following a security report.

“SafeArrayIterator” 0b6d307, made debuglog safer to use - however it dropped our req/s from 72k to 67k. Turns out there was not a problem with this commit at all, but it was rather a problem with how debuglog was cached with Micheal fixed in https://github.com/nodejs/node/pull/37966.

“http: align with stream.Writable” e2f5bb7 brought us down from 67k to 60k, which I reverted at #37963. I have been extremely sorry for this, as I was excited it finally happened.

Fixing the problems introduced by “SafeIterator” and “align with stream.Writable” did not bring back the full throughput - so I dig deeper and I found that “perf_hooks: complete overhaul of the implementation” f3eb224 added two process.hrtime() calls to ServerResponse.

During this investigation I also found that https://github.com/nodejs/node/pull/37964 introduced a severe Denial of Service attack where a few closures were leaked in every request. Luckily, this did not ship into a release.

Read up more!

My friend Laurie told the story of her recent interviews at three tech companies. I’ve found this article really interesting as there are not many write-ups that explain how it is to be interviewed for senior tech roles. Congrats Laurie for your new job!

Last year at SnykCon 2020, I spoke about the conundrum of between Security and Open Source, focusing on sustainability and the value chain. I recently blogged about it:

...and read up on how I fixed a bug!

sonic-boom is a faster replacement for fs.createWriteStream() that we use in Pino (the logger). The goal for sonic-boom is to provide two working modes, one sync where all data written are synchronously written out and one asynchronous, where data is buffered and written as soon as it’s possible. The latter is faster if the destination can absorb all data “fast enough” - otherwise we are exposing ourselves to the risk of memory leaks. Pino ships with synchronous logging by default as a safety measure.

A long time ago I added a bug in sonic-boom in synchronous mode: in case there was a problem in accessing the destination, I would both throw an error and emit it after a nextTick (trying to avoid releasing zalgo). Unfortunately there would usually be no one listening to the ‘error’ event in synchronous mode, making it a sure path to ‘uncaughtException’ and a likely crash. The fix was trivial, I had to not emit ‘error’ if there were no listeners and only rely on throwing the error. Check for yourself:

Thanks!

Thanks for the extremely warm response to every edition of this newsletter and on twitter! I just want to let you know about one more thing… NodeConf Remote is back for 2021. Registrations and the Call for Proposals are both open - signup or let us know what topic you’d like to speak about.