lucrolearning

Aug 12, 202610 min read27 views

Building a Task Queue From Scratch

Why BullMQ and Celery Can't Share One Queue

Inside the Pipeline: Clustering, Summarizing, and Mapping Themes at Scale
Aug 3, 202611 min read7 views
Inside the Pipeline: Fetching, Cleaning, and Embedding Survey Data at Scale
Aug 2, 202611 min read9 views
Making Workers Production Ready
Aug 2, 20265 min read14 views
"Inside the Worker: Building a Resilient AI Execution Platform"
Jul 18, 20266 min read14 views

The Listener Loop: Turning Jobs into Work
Putting It All Together
Jul 17, 20266 min read4 views
Building the Execution Engine: Queue Manager, Workers and Process Orchestration
Part 3 of the "Beyond FastAPI" series. In the previous article, we built a thin orchestration layer using FastAPI. The API validates incoming requests, persists them as pending jobs, and immediately
Jul 16, 20267 min read8 views
Building the Orchestration Layer: Designing a Thin FastAPI API for Long-Running AI Workloads
Accepting requests elegantly
Jul 16, 20269 min read12 views
Beyond FastAPI: Building a Production-Ready Asynchronous Job Orchestrator in Python
How I built a scalable execution platform for long-running AI workloads using FastAPI, asyncio, multiprocessing, and MongoDB.
Jul 15, 20267 min read18 views