Promises for the (Web) Worker

While working on the Barcode Detector Polyfill, I needed to wrap a web worker with a promise based API.

The use of a promise in this case was dictated by the standard proposal, and the use of a web worker was needed because of the heavy calculations that the detector requires.

Web workers have been introduced about 8 years ago so they are really not new, but they are mature and very well supported.

In a nutshell, web workers allow to run code in a separate processing thread, so lengthy computations won’t block the main thread. This is important because when the main thread is blocked, the web app will be unresponsive.

As browsers become more powerful, they need to handle more computational tasks, so using web workers makes sense in more cases.

Promises as a concept have also been around for several years, and have been standardized as part of ES6 (ES2015). They are supported in all modern browsers and there are good polyfills for older ones.

In two words, they allow to write asynchronous code in more synchronous way, making it more readable and maintainable.

So, how can we combine these two technologies?

The challenge

As web workers run in a separate thread, they need some way to communicate with the main thread. This is done by exchanging messages which can contain arbitrary data and are sent and received asynchronously between the main and worker threads. By default, these messages don’t have any context other than their content.

Now in our case, if a web worker computation was initiated inside a promise, we want that same promise to be resolved (or rejected) when that specific computation is complete and the result is available.

The way to achieve this, is with some book keeping. A wrapper around the web worker will remember which callback functions belong to which computation and will call the right callback function upon getting a response message.

This is simpler than it sounds, and easier to explain with an example.

Code Example

In the code below I’ll be using ES2015 syntax and features.
A complete, live example can be found here.

Wrapper.js:

const resolves = {}
const rejects = {}
let globalMsgId = 0
// Activate calculation in the worker, returning a promise
function sendMsg(payload, worker){
  const msgId = globalMsgId++
  const msg = {
    id: msgId,
    payload
  }
  return new Promise(function (resolve, reject) {
    // save callbacks for later
    resolves[msgId] = resolve
    rejects[msgId] = reject
    worker.postMessage(msg)
  }) 
}
// Handle incoming calculation result
function handleMsg(msg) {
  const {id, err, payload} = msg.data
  if (payload) {
    const resolve = resolves[id]
    if (resolve) {
      resolve(payload)
    }
  } else {
    // error condition
    const reject = rejects[id]
    if (reject) {
        if (err) {
          reject(err)
        } else {
          reject('Got nothing')
        }
    }
  }
  
  // purge used callbacks
  delete resolves[id]
  delete rejects[id]
}
// Wrapper class
class Wrapper {
  constructor() {
    this.worker = new Worker('./worker.js')
    this.worker.onmessage = handleMsg
  }
  
  oche(str) {
    return sendMsg(str, this.worker)
  }
}
export default Wrapper

The wrapper needs to do most of the bookkeeping.

When the process function oche is called, a new promise is returned, but before that we do 3 things (see sendMsg in the code):

1. Define a unique ID for this call
2. Store the resolve and reject callback functions of the promise
3. Send a message to the worker with the input data for the computation along with the unique message ID.

In the handler for incoming messages from the worker (see handleMsg in the code), we also have 3 main steps:

1. Fetch the callback functions that we stored using the incoming message unique ID
2. Call the relevant callback function, thus resolving or rejecting the original promise
3. Do some cleanup by deleting the stored callback functions for this message, as we don’t need them anymore.

worker.js:

// Simulate lengthy calculation or an async call
function doCalculation(data, cb) {
  let result = null, err = null
  
  if (typeof data === 'string') {
    result = data.split('').reverse().join('')
  } else {
    err = 'Not a string'
  }
  const delay = Math.ceil(Math.random() * 1000)
  setTimeout(function() {
    cb(err, result)
  }, delay)
}
// Handle incoming messages
self.onmessage = function(msg) {
  const {id, payload} = msg.data
  
  doCalculation(payload, function(err, result) {
    const msg = {
      id,
      err,
      payload: result
    }
    self.postMessage(msg)
  })
}

The worker can do whatever it needs with the messages it receives. It has just two requirements:

1. Reply with exactly one outgoing message for each incoming message.
2. Include the ID from the incoming message in the outgoing message, together with any result data.

index.js:

import Wrapper from './Wrapper'
const wrapper = new Wrapper()
console.log('Calling')
wrapper.oche('hello world').then(
  (res) => console.log('Got result: ' + res)
).catch(
  (err) => console.log('Got error: ' + err)
)

Using the wrapper is just standard promise use. Everything worker related is encapsulated and handled by the wrapper, so we don’t need anything special on this side.

Final notes

The above example is pretty basic and was meant to convey the main idea.
Of course it can be extended as needed, for example:

• The wrapper and worker can be made to support more processing actions using a similar pattern
• The wrapper can be made more robust by keeping a timer for each call which will prevent the callback stores from filling with zombie entries in cases the worker does not reply for some reason.

Also, while I think that this pattern is pretty straight-forward and easy to implement, if you prefer something ready-made, there’s a library available that implements it.