Distributed Task Queue

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.

The execution units, called tasks, are executed concurrently on a single or more worker servers. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

Celery is already used in production to process millions of tasks a day.

Celery is written in Python, but the protocol can be implemented in any language. It can also operate with other languages using webhooks.

The recommended message broker is RabbitMQ, but support for Redis and databases is also available.

You may also be pleased to know that full Django integrationexists, delivered by the django-celery package.

Example

This is a simple task adding two numbers:

from celery.decorators import task

@task
def add(x, y):
    return x + y

You can execute the task in the background, or wait for it to finish:

>>> result = add.delay(8, 8)
>>> result.wait() # wait for and return the result
16

Getting Started

1. Install celery by download or pip install -U celery
2. Set up RabbitMQ or one of the ghetto queue solutions.
3. Select one of the following guides:
   • First steps with Python
   • First steps with Django

Community

There is a mailing-list available for general discussion.

For those craving real, human interaction, there is also an IRC channel (#celery on irc.freenode.net).

Finally, if you find a bug or would like to request a feature, please submit an issue.