Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.
The execution units, called tasks, are executed concurrently on a single or more worker servers. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).
Celery is already used in production to process millions of tasks a day.
Celery is written in Python, but the protocol can be implemented in any language. It can also operate with other languages using webhooks.
The recommended message broker is RabbitMQ, but support for Redis and databases is also available.
Celery is easy to integrate with Django, Pylons and Flask, using the django-celery, celery-pylons and flask-celeryadd-on packages.This is a simple task adding two numbers:
from celery.decorators import task
@task
def add(x, y):
return x + y
You can execute the task in the background, or wait for it to finish:
>>> result = add.delay(8, 8)
>>> result.wait() # wait for and return the result
16
pip install -U celeryThere is a mailing-list available for general discussion.
For those craving real, human interaction, there is also an IRC channel
(#celery on irc.freenode.net).
Finally, if you find a bug or would like to request a feature, please submit an issue.
We're proud to announce the release of Celery 2.0. This version replaces the Django ORM with SQLAlchemy as the database result store. With this, Celery no longer depends on Django. Django integration is now available as a separate package called django-celery.
In addition there are a lot of new features: a curses monitor, time limits, complex crontab expressions, callbacks, simplified routing, and more. Everything is detailed in the changelog, so be sure to read it before upgrading.
RabbitMQ 1.8.0 came with stricter equivalence checks that broke the
AMQP result backend, this release resolves this. If you've already used
the AMQP backend you need to delete the previous declarations. For
instructions please read the full
changelog.
Download from PyPI,
or simply install the upgrade using pip install -U celery==1.0.6.
This release contains some important bugfixes related to shutdown and
broker connection loss, as well as some other minor fixes. Also
AbortableTask has been added to contrib. Please read the full changelog
before you upgrade. Download from PyPI,
or simply install the upgrade using pip install -U celery.
This release contains a drastic improvement in reliability and
performance. Please read the full changelog
before you upgrade. Download from PyPI,
or simply install the upgrade using pip install -U celery.
This is a bugfix release and has some important changes to the
shutdown procedure. Also improved compatibility with Windows and Python
2.4. Read the full Changelog
for more information. Download from PyPI,
or simply install the upgrade using pip install -U celery.
Celery 1.0 has finally been released! It is available on PyPI for
downloading. You can also install it via pip install
celery. You can read the announcement here.
1.0 is scheduled to be released this week! Please help us test the latest release candiate to make this happen. To upgrade from an earlier version, please read the changelog.
We finally got a home page. Big thanks to Jan Henrik Helmers