|
|
@@ -11,10 +11,13 @@ how to add Celery support for your application and library.
|
|
|
.. contents::
|
|
|
:local:
|
|
|
|
|
|
+Using Celery in your Application
|
|
|
+================================
|
|
|
+
|
|
|
.. _project-layout:
|
|
|
|
|
|
Our Project
|
|
|
-===========
|
|
|
+-----------
|
|
|
|
|
|
Project layout::
|
|
|
|
|
|
@@ -23,7 +26,7 @@ Project layout::
|
|
|
/tasks.py
|
|
|
|
|
|
:file:`proj/celery.py`
|
|
|
-----------------------
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
.. literalinclude:: ../../examples/next-steps/proj/celery.py
|
|
|
:language: python
|
|
|
@@ -53,18 +56,76 @@ you simply import this instance.
|
|
|
that the worker is able to find our tasks.
|
|
|
|
|
|
:file:`proj/tasks.py`
|
|
|
----------------------
|
|
|
+~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
.. literalinclude:: ../../examples/next-steps/proj/tasks.py
|
|
|
:language: python
|
|
|
|
|
|
|
|
|
Starting the worker
|
|
|
-===================
|
|
|
+-------------------
|
|
|
|
|
|
The :program:`celery` program can be used to start the worker::
|
|
|
|
|
|
- $ celery worker --app=proj -l info
|
|
|
+ $ celery worker --app=proj -l info
|
|
|
+
|
|
|
+When the worker starts you should see a banner and some messages::
|
|
|
+
|
|
|
+ -------------- celery@halcyon.local v2.6.0rc4
|
|
|
+ ---- **** -----
|
|
|
+ --- * *** * -- [Configuration]
|
|
|
+ -- * - **** --- . broker: amqp://guest@localhost:5672//
|
|
|
+ - ** ---------- . app: __main__:0x1012d8590
|
|
|
+ - ** ---------- . concurrency: 8 (processes)
|
|
|
+ - ** ---------- . events: OFF (enable -E to monitor this worker)
|
|
|
+ - ** ----------
|
|
|
+ - *** --- * --- [Queues]
|
|
|
+ -- ******* ---- . celery: exchange:celery(direct) binding:celery
|
|
|
+ --- ***** -----
|
|
|
+
|
|
|
+ [2012-06-08 16:23:51,078: WARNING/MainProcess] celery@halcyon.local has started.
|
|
|
+
|
|
|
+-- The *broker* is the URL you specifed in the broker argument in our ``celery``
|
|
|
+module, you can also specify a different broker on the command line by using
|
|
|
+the :option:`-b` option.
|
|
|
+
|
|
|
+-- *Concurrency* is the number of multiprocessing worker process used
|
|
|
+to process your tasks concurrently, when all of these are busy doing work
|
|
|
+new tasks will have to wait for one of the tasks to finish before
|
|
|
+it can be processed.
|
|
|
+
|
|
|
+The default concurrency number is the number of CPU's on that machine
|
|
|
+(including cores), you can specify a custom number using :option:`-c` option.
|
|
|
+There is no recommended value, as the optimal number depends on a number of
|
|
|
+factors, but if your tasks are mostly I/O-bound then you can try to increase
|
|
|
+it, experimentation has shown that adding more than twice the number
|
|
|
+of CPU's is rarely effective, and likely to degrade performance
|
|
|
+instead.
|
|
|
+
|
|
|
+Including the default multiprocessing pool, Celery also supports using
|
|
|
+Eventlet, Gevent, and threads (see :ref:`concurrency`).
|
|
|
+
|
|
|
+-- *Events* is an option that when enabled causes Celery to send
|
|
|
+monitoring messages (events) for actions occurring in the worker.
|
|
|
+These can be used by monitor programs like ``celery events``,
|
|
|
+celerymon and the Django-Celery admin monitor that you can read
|
|
|
+about in the :ref:`Monitoring and Management guide <guide-monitoring>`.
|
|
|
+
|
|
|
+-- *Queues* is the list of queues that the worker will consume
|
|
|
+tasks from. The worker can be told to consume from several queues
|
|
|
+at once, and this is used to route messages to specific workers
|
|
|
+as a means for Quality of Service, separation of concerns,
|
|
|
+and emulating priorities, all described in the :ref:`Routing Guide
|
|
|
+<guide-routing>`.
|
|
|
+
|
|
|
+You can get a complete list of command line arguments
|
|
|
+by passing in the `--help` flag::
|
|
|
+
|
|
|
+ $ celery worker --help
|
|
|
+
|
|
|
+These options are described in more detailed in the :ref:`Workers Guide <guide-worker>`.
|
|
|
+
|
|
|
+.. sidebar:: About the :option:`--app` argument
|
|
|
|
|
|
The :option:`--app` argument specifies the Celery app instance to use,
|
|
|
it must be in the form of ``module.path:celery``, where the part before the colon
|
|
|
@@ -79,8 +140,8 @@ This means that the following all results in the same::
|
|
|
$ celery --app=proj.celery:celery
|
|
|
|
|
|
|
|
|
-Subtasks
|
|
|
-========
|
|
|
+Designing Work-flows
|
|
|
+====================
|
|
|
|
|
|
A :func:`~celery.subtask` wraps the signature of a single task invocation:
|
|
|
arguments, keyword arguments and execution options.
|
Ask Solem