Celery
Open SourceDistributed task queue for Python.
Published 12 October 2026
Scores
Popularity3/5
The best-known task queue in the Python ecosystem with tens of millions of monthly downloads, but developers outside Python backends rarely meet it.
Learning Curve3/5
A first task and worker take a few lines, but brokers, result backends, acknowledgement settings, and worker pools add a configuration layer that takes time to get right in production.
Flexibility4/5
Two stable brokers, many result backends, canvas workflows, scheduling, and routing cover most background-job patterns, though it stays Python-only and not async-native.
Performance4/5
Scales horizontally by adding workers, and the prefork pool handles CPU-bound jobs well; overall throughput depends on the broker underneath.
Portability5/5
BSD-licensed, self-hosted, and broker-agnostic between Redis and RabbitMQ, so it runs on any host or cloud without lock-in.
About Celery
Celery is an open-source distributed task queue for Python. You decorate a function as a task, call it from your web app with .delay(), and a separate worker process picks it up and runs it outside the request. Sending email, resizing images, calling slow third-party APIs, generating reports, and processing webhooks are the usual jobs.
Celery is a library, not a server, so it needs a broker to carry the messages between your app and the workers. Redis and RabbitMQ are the two stable choices: Redis is the simpler setup and is often already in the stack for caching, while RabbitMQ adds richer routing and stronger delivery guarantees. A separate result backend, such as Redis or a database through the Django ORM or SQLAlchemy, stores task return values when you need them.
On top of single tasks, the canvas primitives compose work into chains, groups, and chords, so a job can fan out into parallel subtasks and gather the results. Celery beat schedules periodic tasks on intervals or crontab expressions, and every task can set retries with backoff, rate limits, time limits, and routing to dedicated queues. Workers run with a prefork process pool by default, or with threads, gevent, or eventlet for I/O-heavy jobs.
Celery works with any Python web framework. Django has the deepest integration through the official setup guide and django-celery-beat and django-celery-results, and FastAPI and Flask apps use it the same way. Flower adds a web dashboard for watching workers and tasks. The project is BSD-licensed and community-maintained, with no paid edition.
Key Features
- Decorate a function as a task and run it in a separate worker process
- Redis and RabbitMQ as stable brokers, with Amazon SQS and others experimental
- Celery beat for periodic tasks on intervals or crontab schedules
- Canvas primitives: chains, groups, and chords for multi-step workflows
- Retries with backoff, rate limits, and time limits per task
- Result backends including Redis, the Django ORM, and SQLAlchemy
- Prefork, thread, gevent, and eventlet worker pools
- Task routing to dedicated queues and workers
Pros
- The default answer for background jobs in Django, FastAPI, and Flask apps
- Mature, with a large body of tutorials and answered questions for common problems
- Works with either Redis or RabbitMQ, so it fits whichever broker the team already runs
- Scheduling, retries, and workflows are built in rather than added through plugins
- Free and BSD-licensed, with no paid tier holding back features
Cons
- Configuration has many settings and defaults that surprise newcomers, such as visibility timeouts with Redis
- Requires running and monitoring a broker and worker processes alongside the web app
- No native asyncio support, so async FastAPI code needs wrappers inside tasks
- Debugging failed or stuck tasks is harder than with newer durable-execution platforms
- Community-maintained with a small core team, so some issues stay open for a long time
Celery Pricing
Open SourceTech Stacks with Celery
Airflow Self-Hosted
InfrastructureThe reference scheduler for data pipelines on your own server: Airflow's official Docker Compose deployment, with a SQL metadata database (PostgreSQL by default, MySQL also supported) and Redis as the Celery worker queue.
Superset Self-Hosted
InfrastructureApache Superset on your own server with Docker: a SQL metadata database (PostgreSQL by default, MySQL and MariaDB also supported) and Redis behind the Celery workers, keeping every dashboard, query, and user record on your infrastructure.
Python Web (FastAPI + React)
ProjectA clean separation of concerns: React on the frontend, FastAPI serving a typed REST API, and PostgreSQL for persistent storage (MySQL, MariaDB, and serverless Postgres hosts are also available). Docker keeps environments consistent.
Tools Related to Celery
Works well with Celery(1)
FastAPI's own BackgroundTasks run inside the API process; its docs point to Celery for heavy or long-running work, which separate Celery workers take over with retries and scheduling.
Integrates with Celery(3)
Celery documents Django as a first-class integration: the Celery project maintains django-celery-beat and django-celery-results, and delay_on_commit queues a task only after the Django transaction commits.
Redis is one of Celery's two stable brokers and a common result backend; it is the simpler setup, and is often already in the stack for caching.
RabbitMQ is one of Celery's two stable brokers, reached through its built-in AMQP transport; it adds richer routing, priorities, and stronger delivery guarantees than the Redis broker.
Built on (1)
Celery is a distributed task queue library for Python applications.
Alternatives to Celery(1)
Celery is the Python task queue and works with Redis or RabbitMQ; BullMQ is the Node.js and TypeScript job queue built on Redis. The choice follows the backend's language.