Queues and workers

A job is a row in the maple.jobs table: a kind, arguments as JSON, a queue and a time it may run at. Workers take available jobs, run them and mark them done.

#Queues

Queues are names; there is nothing to create. Put jobs that must not wait behind others, such as password reset emails, into a queue of their own and give it its own workers:

$ maple worker --queues mail --concurrency 5
$ maple worker --queues default,reports

#How a worker takes a job

Workers select jobs with FOR UPDATE SKIP LOCKED, so any number of them can work on the same queue without taking the same job twice. New jobs send a notification on insert, and an idle worker picks them up within milliseconds instead of waiting for the next poll.

#Running your code

A worker hands each job to your application: it POSTs the job as JSON to handler.url and treats a 2xx answer as success. Anything else, or no answer within timeout, is a failure and the job is retried.