I recently moved my job queue from ActiveMQ and Lambda functions to Express and Inngest, and it's been a huge improvement in developer experience.

For those unfamiliar with Inngest, it's a service that manages background jobs. You write the workflows, and Inngest takes care of the scheduling. If you want to learn more, their documentation is a good place to start.

One of the main benefits of Inngest is that I can write jobs just like the rest of the code running on my server. This is a notable difference from working with Lambda functions, which are small, ephemeral, and subject to limitations like execution time constraints. With Lambda, you often need to adapt your code to work within these limitations, and debugging can be a pain if you're not well-versed in Lambda's quirks*.

Inngest's emulator is a game-changer for local testing. With a single command, I can spin up my Express server and the emulator, and it watches for changes. The web interface allows me to send events and monitor each step of a workflow. Previously, I had to compile my TypeScript and hack together a testing setup using AWS SAM CLI, but it wasn't ideal due to behavioral differences between my local machine and the hosted environment. Tools like SST might make this less painful, but with Inngest, I don't need to hunt for a separate tool to make local development tolerable – the emulator handles it all.

The improved observability is arguably the biggest win. While Lambda functions have built-in CloudWatch logging, navigating it can be cumbersome. Inngest allows you to break jobs into steps, making it easy to pinpoint issues, whether it's a database query, an API call, or something else. You can return values at each step and at the end of the job, and with a bit of middleware, you can log each piece. When paired with a solid logging platform like Betterstack (formerly Logtail), tracing becomes a breeze.

Inngest simplifies complex error handling. It has a default retry policy with incremental backoff, but you can throw a RetryAfterError for more control over the retry schedule. For errors that should immediately end the job, there's NonRetriableError. I used to have to write my own version of this. You can also write a function to capture all errors by listening for the inngest/function.failed event, starting with generic error handling like logging before funneling to a specific handler.

Inngest is a complete toolbox for scheduled tasks, whether it's cron jobs, waiting for specific events or time intervals, or canceling based on another event. Fan-out jobs have been particularly useful for data migrations. One job grabs relevant records, maps over them, makes necessary transformations, and then sends an array of events that run in parallel. If any fail, I can quickly identify the cause, make updates, and retry.

The initial set of tools was already fantastic, and they continue to add new features that provide even more flexibility in workflow design. They're moving fast without breaking things – updating from v2 to v3 required only minor changes and went smoothly. These frequent updates bring meaningful improvements, like the recent addition of invoking one function from another and handling the result, and the new webhook platform that simplifies receiving requests and transforming data for later use in jobs.

Inngest has made my life easier in several ways: fewer AWS resources to manage, a superb local development experience, better error visibility, and improved error handling. It turned the most challenging part of my stack into the most enjoyable to work with.

*This best practices guide goes over many of the considerations when using Lambda functions.