Queues

How queues work

Every job belongs to exactly one queue. Workers subscribe to one or more queues and ask Halyard for work; the queue decides which job to hand out next and whether another one may start right now. Queues are created from halyard.yaml on halyard deploy, or on first use with default settings.

Split work into queues by how it should behave under load, not by feature. A good rule: if two kinds of jobs need different limits or one must never wait behind the other, give them separate queues.

Concurrency

concurrency is the maximum number of jobs in running on the queue at once, counted across all your workers. It protects whatever the jobs talk to: a database, a third-party API, a GPU.

Workers have their own limit too (HALYARD_CONCURRENCY, default 5). The effective number of running jobs is the smaller of the queue limit and the sum of free worker slots. With concurrency: 20 and three workers at 5 each, at most 15 jobs run; add a fourth worker to reach 20.

Priority ordering

With priority: on (the default) the queue hands out the highest priority ready job first, then the oldest. With priority: off it is strict FIFO by run_at, which is cheaper and keeps ordering predictable for pipelines.

Priority only orders jobs within a queue. A critical job on reports still waits for a reports slot; that is why time-sensitive work belongs on its own queue.

Rate limits

A rate limit caps how many jobs may start per interval, independent of concurrency. Use it for APIs that limit requests per second. Halyard uses a token bucket: limit tokens refill per interval, up to burst. Jobs that hit the limit stay queued and do not use an attempt.

Keyed rate limits

Set key to a payload field to keep one bucket per value. On the webhooks queue below, each customer endpoint gets 200 deliveries per second, so one slow customer can't starve the rest.

Pausing and draining

Pause a queue to stop new jobs from starting while running ones finish. Jobs can still be created and wait in queued. Resume picks up where it left off, in order. Pausing is the right tool when a downstream service is down: nothing fails, nothing burns attempts.

# Pause: queued jobs wait, running jobs finishcurl -X POST https://api.halyard.example/v3/queues/emails/pause \  -H "Authorization: Bearer hy_test_51Hq8Zx2rTmV9cKpL3d" # Resumecurl -X POST https://api.halyard.example/v3/queues/emails/resume \  -H "Authorization: Bearer hy_test_51Hq8Zx2rTmV9cKpL3d"

await halyard.queues.pause("emails");// ...await halyard.queues.resume("emails");

halyard.queues.pause("emails")# ...halyard.queues.resume("emails")

Draining before a deploy

Drain stops workers on this deployment from claiming new jobs and waits until their running jobs finish, then exits. Other workers keep serving the queue. halyard deploy drains automatically; on your own platform, send SIGTERM and the SDK worker drains for up to the queue timeout.

{  "id": "que_7Hn2Lp4Wq9",  "object": "queue",  "name": "emails",  "state": "draining",  "concurrency": 20,  "rate_limit": { "limit": 50, "per": "second", "burst": 50 },  "counts": { "queued": 0, "delayed": 12, "running": 3 },  "oldest_queued_age_ms": 0}

Choosing queue settings

Settings from a typical production project. Start here and tune with the queue metrics in the dashboard.

Queue configuration

Declare queues in halyard.yaml and apply them with halyard deploy. Changes take effect within a few seconds without restarting workers. Queues missing from the file are left as they are; remove one with halyard queues delete.

# halyard.yamlversion: 3project: my-app queues:  emails:    concurrency: 20        # jobs running at once, across all workers    rate_limit: 50/s       # starts per second (token bucket)    priority: on    timeout: 30s           # lease length before an attempt is presumed lost    retry:      max_attempts: 8   media:    concurrency: 8    timeout: 10m    priority: off          # strict FIFO   webhooks:    concurrency: 50    rate_limit:      limit: 200/s      burst: 400      key: payload.endpoint_id   # one bucket per endpoint    timeout: 15s   reports:    concurrency: 2    timeout: 1h    retry:      max_attempts: 3