Fun with Queues


written by: Dev

I moved my job queue from ActiveMQ and Lambda functions to Express and Inngest, and it’s been a huge improvement in developer experience.

If you're not familiar with Inngest, it's a service that manages background jobs. This is a good place to get acquainted. Basically, you write the workflows, and Inngest handles the scheduling.

The first benefit of Inngest is that I can write functions just like the rest of the code running on my server. Lambdas are small, ephemeral functions spun up to meet demand and are thus subject to certain constraints — like execution time limits*. You’ll often need to adapt your code specifically to work in this context, and if you’re not fully aware of the ins and outs of these limitations, debugging can be a hassle.

Inngest offers an emulator that delivers all I need for local testing. With one command I can spin up my Express server and the emulator and watch for changes. This gives me a web interface where I can send events and watch every step of a workflow. The old approach was more manual. I ran a command to compile my Typescript, and to test I hacked together something with AWS SAM CLI. But it was less than ideal; function behavior varied between my local machine and the hosted environment. I’ve heard that tools like LocalStack and SST make this less painful, but I've yet to try them out. I’m interested in using SST for a project, so I’ll report back with my experience. But what's great about Inngest is that it gives me one less thing to think about. I don't need to search for a separate tool to make local development bearable. The emulator handles everything.

The improvement in observability is probably the biggest win. While it’s nice that Lambda functions have CloudWatch logging built-in, it’s not the easiest platform to navigate. With Inngest you break jobs into steps. If something goes wrong, you see exactly where — a database query, an API call, etc. You can return a value at the end of each step and at the end of the job. With a tiny bit of code you can add middleware to log each piece, and when combined with a decent logging platform like Betterstack (formerly known as Logtail), it’s easy to trace exactly what happened when.

Error handling is a breeze. Inngest has a default retry policy with incremental backoff, but if you want more control over that schedule, you can throw a RetryAfterError. If a certain error should immediately end the job, there's NonRetriableError. I'd previously written my own version of this, but Inngest makes complex error handling much easier. You can write a function to capture all errors by listening for the inngest/function.failed event. From here you can start with generic error handling behavior like logging, before funneling to a specific handler.

It’s a complete toolbox. There’s something for every kind of scheduled task — cron jobs, waiting for a specific event or some static amount of time, or canceling if some other event is received. Fan-out jobs have been very useful for data migrations. One job grabs relevant records, maps over them, making any necessary transformations, before finally sending an array of events that run in parallel. If some fail, I can quickly get to the root cause, make some updates, and try again.

The tools they started with were already fantastic, and they’re still adding new tools that give me even more flexibility when designing my workflows. They're moving fast but not breaking things. Updating from v2 to v3 involved a few small changes and went off without incident. And these frequent updates come with meaningful improvements. A recent addition allows you to invoke one function from another and handle the result, and the new webhook platform makes it simple to receive requests and transform data for later use in jobs.

Inngest has made my life easier in a number of ways – fewer resources to manage on AWS, a phenomenal local development experience, greater error visibility, and better error handling. It transformed the most difficult-to-work-with part of my stack into the part I enjoy working with most. Give it a shot. Get started with the documentation here.

  • This best practices guide goes over many of the considerations when using Lambda functions.
Powered by