Skip to main content

Queue Management

Allegro runs background jobs on Laravel Cloud managed queues. Laravel Cloud provisions each queue, runs its workers on dedicated instances separate from web traffic, and scales the worker pool to the work waiting to be processed — including down to zero.

There is no supervisor to configure and no worker process count to tune. Queues are created and sized in the Laravel Cloud canvas, not in this repository.

Queues

Allegro routes jobs across three queues. A managed queue serves exactly one queue name, so each of these needs its own managed queue in every Cloud environment.

QueueCompute classPurpose
highFlexMagic link and OTP email — the only jobs a person is waiting on
defaultFlexGeneral background work: profile pre-warming, webhook deliveries, event backfill, domain polling, newsletter subscribes, template sync, and the audience sync dispatcher
bulkProLong-running work: the audience sync member jobs, and CSV exports

Jobs are routed by the #[Queue] attribute on the job class, or by onQueue() in a Mailable's constructor. A job with neither lands on default, which is where most of them belong.

The same list lives in allegro.queues (packages/allegro-platform/config/allegro.php), which is what the health endpoint reports pending counts for. Adding a queue means adding it there and creating the managed queue in Cloud — a job routed to a name with no managed queue behind it is accepted and then never processed.

Compute class

Flex workers scale to zero and wake in under a second, but cap a job at 90 seconds. Pro workers are always on with a minimum of one worker and have no runtime limit.

Two jobs declare a $timeout above the Flex cap, and they are why bulk is Pro: SyncUserAttributeFromExternalProfilesJob (3600s) and GenerateExportJob (3600s). Every other queued class runs at or below the framework default of 60s, so high and default are both Flex and both scale to zero between bursts.

bulk also carries the only workload that can fan out without bound — one sync tick can page through every member of every tenant. Both reasons point the same way: keep the slow, wide work off the queues that serve logins.

DispatchAudienceSyncChunkJob is the deliberate exception: it runs on default at a 60s timeout, above the queue it feeds, so the dispatcher never starves behind the member jobs it enqueues.

Priority is per-queue, not across queues

high, default and bulk each have their own managed queue and their own independent worker pool, so high does not preempt default — but it also cannot be starved by a bulk flood, which is the failure that strict priority exists to prevent. The name is a description of the latency budget, not a rank.

Configuration

Deploying a managed queue sets QUEUE_CONNECTION=cloud for the environment automatically. The cloud connection itself is injected at runtime by the framework from LARAVEL_CLOUD_MANAGED_QUEUES_CONFIG, which is why config/queue.php does not define one.

VariableWhereDescription
QUEUE_CONNECTIONSet by Laravel Cloudcloud on Cloud environments; redis locally
REDIS_HOSTLocalStill required — the event and heartbeat queues use Redis directly
REDIS_PORTLocalRedis port (default: 6379)

Local development

Managed queues exist only on Laravel Cloud. Locally, jobs run through the redis connection with plain workers:

composer run dev

That runs php artisan dev, which starts one worker per queue named in allegro.queues alongside Octane, Pail, and Vite. A worker each mirrors Cloud, where every managed queue has its own workers, and keeps a long-running bulk job from stalling the high queue that OTP and magic-link mail rides on. Adding a queue to allegro.queues adds its worker automatically.

To run one in isolation:

php artisan queue:work --queue=high --tries=1

A single worker can also drain several queues in priority order by passing a comma-separated list, which is a queue:work feature:

php artisan queue:work --queue=high,default,bulk --tries=1

That list is not valid as a managed queue name — commas are not permitted, and a managed queue serves one name.

Monitoring

Queue depth, throughput, memory, and worker counts are on the Queues dashboard under an environment's Monitoring tab in Laravel Cloud. Failed jobs are listed there too, with retry and bulk actions.

Failed jobs are pruned after 7 days

A scheduled queue:prune-failed --hours=168 command runs daily, so failed jobs older than 7 days drop off the list automatically. Retry or inspect a failed job before then.

Allegro also exposes a machine-readable summary at the platform health endpoint for external monitors.

Slow and failing jobs are tracked in Nightwatch, which is where alerting belongs.

Pausing and purging

To stop a queue from processing during an incident or downstream maintenance, pause it from the queue's dropdown menu on the Cloud canvas. Workers scale to zero, in-flight jobs finish gracefully, and dispatched jobs accumulate until you resume.

warning
queue:pause is not supported

The queue:pause Artisan command does not work against managed queues. Pause from the Laravel Cloud canvas instead.

Purging a queue from the same menu discards every waiting job. It cannot be undone and does not affect jobs already being processed.

Job design constraints

Managed queues are backed by SQS, which imposes limits the previous Redis backend did not:

  • At-least-once delivery. A job whose worker crashes or is terminated becomes visible again and runs on another worker. Jobs must tolerate running more than once.
  • Delays cap at 15 minutes. Requesting longer from ->delay() throws.
  • Releases cap at 12 hours. SendWebhookDeliveryJob clamps a receiver's Retry-After to this ceiling.
  • Payloads cap at 256 KB. Pass scalar identifiers to job constructors rather than models or large arrays.