Better Queue in Node JS

Reading Time: 2 minutes


Good practice in software development is to delegate as much heavy work as possible to background jobs to avoid blocking the main execution of your application that can be a web app, mobile app, or desktop.

Send email notifications it’s the typical scenario where you should execute it in background.

More scenarios

  • Image processing
  • Data aggregation / migration / conversion
  • Push notifications

Some platforms offer cheaper CPU running in background so you can save money in addition to user experience.

Why is it important

Imagine several users making requests to your server that lasts more than 30 seconds or one minute. Your web app will get slow soon because HTTP connections are not infinite.

Queue several jobs it’s pretty easy but what it’s no easy is to process them one by one or in batches, set states, retry if some of them fail, etc.

This is a common problem so you shouldn’t implement a solution from scratch

Better Queue

From many solutions we have in Node JS, better-queue module is a good one.

Better Queue is designed to be simple to set up but still let you do complex things.

From better-queue:
By default, it uses an in-memory queue but to configure a persistent queue with Redis or MySQL is pretty easy because there are drivers written for better-queue.

More features

  • Persistent (and extendable) storage
  • Batched processing
  • Prioritize tasks
  • Merge/filter tasks
  • Progress events (with ETA!)
  • Fine-tuned timing controls
  • Retry on fail
  • Concurrent batch processing
  • Task statistics (average completion time, failure rate and peak queue size)
Photo by Halacious on Unsplash

About the author

Andrés Canavesi
Andrés Canavesi

Software Engineer with 15+ experience in software development, specialized in Salesforce, Java and Node.js.

Related posts

Leave a Reply

%d bloggers like this: