All case studies
07 · fintech · infrastructure
Scaling queue workers for a 2.5M+ recipient messaging engine
Hundreds of thousands of SMS and email messages a day go out on queue workers that scale out automatically, isolated from the web app.
- Client
- Stepping Stone Loans, a US consumer lender
- Role
- Senior full-stack & AI engineer
- Period
- Dec 2024 – present
- Engagement
- Long-term contract through Upwork
Problem
Stepping Stone Loans reaches an audience of 2.5M+ recipients by SMS and email, with hundreds of thousands of messages going out every day.
The messaging engine, with sender warming, engagement tracking and bot filtering, had to send at that volume without slowing the web app or other background work.
How it works
- 01QueueSend jobs go onto Redis queues.
- 02Dedicated workersWorker processes drain those queues, isolated from the web app and other background work.
- 03AutoscaleHorizon adjusts worker processes as ready work changes, and Laravel Cloud scales worker instances out horizontally.
- 04SendMessages go out with sender warming, engagement tracking and bot filtering.
What I built
- Deployed the messaging engine on Laravel Cloud, with send jobs on Redis queues drained by dedicated workers.
- Set up horizontal autoscaling, so worker capacity follows the send volume.
- Tuned Horizon to adjust worker processes within configured limits as ready work changes.
- Split background work into dedicated Horizon supervisors per workload, so sends don’t compete with AI, document or export jobs.
Key decisions
- Dedicated workers, not the web servers
- Send spikes don’t slow the web app or other background work.
- One Horizon supervisor per workload
- Sends, AI, document and export jobs each get their own workers, so one backlog doesn’t hold up the others.
- Scale out automatically
- Laravel Cloud adds worker capacity horizontally as the volume grows, with no change to the web app.
Result
Capacity follows the send volume, and send spikes stay off the web app and other background work.
media note
The lender’s internal systems are private, so this case study uses words and a diagram instead of screenshots.