Asynchronicity and the Tartarus of Node.js

By John Mercouris

If you are lucky, you will not know that JavaScript running in the Node.js runtime does not support synchronous I/O. That's right, in the year 2024, it is not possible to do ANY synchronous I/O (Why? Probably because of Node.js' Titanic concurrency model). I say, if you are lucky, because knowing this implies that you have seen the River Styx with your own eyes.

I imagine that we spent somewhere close to two months of developer time struggling with Node.js' feeble concurrency/I/O model. What follows below is a short glimpse into our epic.

Background

Nyxt catches all input events and, depending on the key combination, it determines whether it is forwarded to the renderer. Let's take a simple input event - pressing down the "X" key. When a command bound to "X" exists, Nyxt invokes it and prevents the renderer from processing it further. Otherwise, the event is passed to the renderer and the character "X" is inserted. Think of Nyxt as an input event gatekeeper that filters those that are to be processed by the renderer.

The Electron Callback

In order to know which keys are pressed we need to add what is commonly referred to as a "listener". This is a function that gets invoked every time a particular event occurs. In other words we are "listening" for an event, and then we get to do something. In Electron this looks like this:

const { app, BrowserWindow } = require('electron/main')

app.whenReady().then(() => {
  const win = new BrowserWindow({ width: 800, height: 600 })

  win.loadFile('index.html')
  win.webContents.on('before-input-event', (event, input) => {
    if (input.control && input.key.toLowerCase() === 'i') {
      console.log('Pressed Control+I')
      event.preventDefault()
    }
  })
})

This might look like a lot of gibberish, so let's break it down. The first few lines are basically loading Electron, and then creating a window when we are ready. The key here is the line that says:

win.webContents.on('before-input-event', (event, input) => { ... }

What this line does is register our listener function within the { brackets } to execute before an input event is created. This allows us to intercept any input events and perform our own operations.

Should we want to do something with that input event we can say:

event.preventDefault(), and that event will no longer be bubbled throughout our Electron window. That is, if we had typed "X", it would NOT go through to the underlying web page.

Sounds simple: before-input-event is triggered, we fire off a question to our Lisp server, ask it whether we should consume the key ("X" in this case), and then based on its answer we either reply event.preventDefault or we do nothing. Easy.

NOT SO EASY- said the immortal guardians of JavaScript. They pulled out their tridents and laughed maniacally into the sky…

You see, this function before-input-event MUST return a result synchronously. WITHOUT DELAY. We cannot pause input events to wait. However, our communication with our Lisp server IS asynchronous. Try as you might, the Guardians will not let you pass.

The Imperfect Solution

So, how do we get an asynchronous response from our Lisp server synchronously?- by abusing a part of the Node.js API. You see, basically no I/O in Node.js is synchronous, except some VERY limited I/O operations. Luckily for us, one of these operations is used to launch a new program. We can synchronously launch a program, and return its output. If you are thinking what I am thinking, you know what we are about to do is sacrilege.

Whenever the user presses a key, launch a NEW Node.js process which listens on a socket, and then returns a result.

That's right, we are launching a NEW program for every keystroke. This was, predictably, as fast as frozen molasses being poured from a decanter in deep space.

I wish I could take credit for this brilliant monstrosity, but that credit belongs to someone else. If you want to read more about this approach, please see here

https://github.com/JacobFischer/sync-socket

While this method was indeed a crime, it did work. And while it was slow, it helped us get to the next step. We were slowly building our tower of JavaScript babel to the sky above. I was certain at any moment that one of the angry Gods would smite us for our insolence.

The Perfect Solution

We all know there is no perfect solution, but we got close. All we had to do was write some C++ code.

What we did was create our own JavaScript library called synchronous-socket, you can see it here:

https://github.com/atlas-engineer/synchronous-socket

What this library does is create synchronous read and write functions for Unix domain sockets so that we can converse with our Lisp process before the end of before-input-event, such that we can provide a response to Electron before it loses its mind.

Fortunately, I am neither an expert in C++, nor JavaScript. It is for this reason I can heartily assure you that our synchronous-socket communication library is bug free!

If the above sounds like the rant of an addled computer scientist, that's because it is. JavaScript has all but eroded my psyche to a smooth tapioca pudding like consistency.

Thanks for reading :-)