|
@@ -6,11 +6,12 @@
|
|
|
|
|
|
This document describes the configuration options available.
|
|
|
|
|
|
-If you're using the default loader, you must create the ``celeryconfig.py``
|
|
|
+If you're using the default loader, you must create the :file:`celeryconfig.py`
|
|
|
module and make sure it is available on the Python path.
|
|
|
|
|
|
.. contents::
|
|
|
:local:
|
|
|
+ :depth: 2
|
|
|
|
|
|
.. _conf-example:
|
|
|
|
|
@@ -18,7 +19,7 @@ Example configuration file
|
|
|
==========================
|
|
|
|
|
|
This is an example configuration file to get you started.
|
|
|
-It should contain all you need to run a basic celery set-up.
|
|
|
+It should contain all you need to run a basic Celery set-up.
|
|
|
|
|
|
.. code-block:: python
|
|
|
|
|
@@ -45,57 +46,78 @@ It should contain all you need to run a basic celery set-up.
|
|
|
# CELERYD_LOG_FILE = "celeryd.log"
|
|
|
# CELERYD_LOG_LEVEL = "INFO"
|
|
|
|
|
|
+
|
|
|
+Configuration Directives
|
|
|
+========================
|
|
|
+
|
|
|
.. _conf-concurrency:
|
|
|
|
|
|
Concurrency settings
|
|
|
-====================
|
|
|
+--------------------
|
|
|
+
|
|
|
+.. _CELERYD_CONCURRENCY:
|
|
|
+
|
|
|
+CELERYD_CONCURRENCY
|
|
|
+~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERYD_CONCURRENCY
|
|
|
+The number of concurrent worker processes, executing tasks simultaneously.
|
|
|
|
|
|
- The number of concurrent worker processes, executing tasks simultaneously.
|
|
|
+Defaults to the number of CPUs/cores available.
|
|
|
|
|
|
- Defaults to the number of CPUs/cores available.
|
|
|
+.. _CELERYD_PREFETCH_MULTIPLIER:
|
|
|
|
|
|
-* CELERYD_PREFETCH_MULTIPLIER
|
|
|
+CELERYD_PREFETCH_MULTIPLIER
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- How many messages to prefetch at a time multiplied by the number of
|
|
|
- concurrent processes. The default is 4 (four messages for each
|
|
|
- process). The default setting seems pretty good here. However, if you have
|
|
|
- very long running tasks waiting in the queue and you have to start the
|
|
|
- workers, note that the first worker to start will receive four times the
|
|
|
- number of messages initially. Thus the tasks may not be fairly balanced among the
|
|
|
- workers.
|
|
|
+How many messages to prefetch at a time multiplied by the number of
|
|
|
+concurrent processes. The default is 4 (four messages for each
|
|
|
+process). The default setting is usually a good choice, however -- if you
|
|
|
+have very long running tasks waiting in the queue and you have to start the
|
|
|
+workers, note that the first worker to start will receive four times the
|
|
|
+number of messages initially. Thus the tasks may not be fairly distributed
|
|
|
+to the workers.
|
|
|
|
|
|
.. _conf-result-backend:
|
|
|
|
|
|
Task result backend settings
|
|
|
-============================
|
|
|
+----------------------------
|
|
|
+
|
|
|
+.. _CELERY_RESULT_BACKEND:
|
|
|
|
|
|
-* CELERY_RESULT_BACKEND
|
|
|
+CELERY_RESULT_BACKEND
|
|
|
+~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- The backend used to store task results (tombstones).
|
|
|
- Can be one of the following:
|
|
|
+The backend used to store task results (tombstones).
|
|
|
+Can be one of the following:
|
|
|
|
|
|
- * database (default)
|
|
|
- Use a relational database supported by `SQLAlchemy`_.
|
|
|
+* database (default)
|
|
|
+ Use a relational database supported by `SQLAlchemy`_.
|
|
|
+ See :ref:`conf-database-result-backend`.
|
|
|
|
|
|
- * cache
|
|
|
- Use `memcached`_ to store the results.
|
|
|
+* cache
|
|
|
+ Use `memcached`_ to store the results.
|
|
|
+ See :ref:`conf-cache-result-backend`.
|
|
|
|
|
|
- * mongodb
|
|
|
- Use `MongoDB`_ to store the results.
|
|
|
+* mongodb
|
|
|
+ Use `MongoDB`_ to store the results.
|
|
|
+ See :ref:`conf-mongodb-result-backend`.
|
|
|
|
|
|
- * redis
|
|
|
- Use `Redis`_ to store the results.
|
|
|
+* redis
|
|
|
+ Use `Redis`_ to store the results.
|
|
|
+ See :ref:`conf-redis-result-backend`.
|
|
|
|
|
|
- * tyrant
|
|
|
- Use `Tokyo Tyrant`_ to store the results.
|
|
|
+* tyrant
|
|
|
+ Use `Tokyo Tyrant`_ to store the results.
|
|
|
+ See :ref:`conf-tyrant-result-backend`.
|
|
|
|
|
|
- * amqp
|
|
|
- Send results back as AMQP messages
|
|
|
- (**WARNING** While very fast, you must make sure you only
|
|
|
- receive the result once. See :doc:`userguide/executing`).
|
|
|
+* amqp
|
|
|
+ Send results back as AMQP messages
|
|
|
+ See :ref:`conf-amqp-result-backend`.
|
|
|
|
|
|
+.. warning:
|
|
|
+
|
|
|
+ While the AMQP result backend is very efficient, you must make sure
|
|
|
+ you only receive the same result once. See :doc:`userguide/executing`).
|
|
|
|
|
|
.. _`SQLAlchemy`: http://sqlalchemy.org
|
|
|
.. _`memcached`: http://memcached.org
|
|
@@ -106,7 +128,12 @@ Task result backend settings
|
|
|
.. _conf-database-result-backend:
|
|
|
|
|
|
Database backend settings
|
|
|
-=========================
|
|
|
+-------------------------
|
|
|
+
|
|
|
+.. _CELERY_RESULT_DBURI:
|
|
|
+
|
|
|
+CELERY_RESULT_DBURI
|
|
|
+~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
Please see `Supported Databases`_ for a table of supported databases.
|
|
|
To use this backend you need to configure it with an
|
|
@@ -129,6 +156,11 @@ To use this backend you need to configure it with an
|
|
|
See `Connection String`_ for more information about connection
|
|
|
strings.
|
|
|
|
|
|
+.. _CELERY_RESULT_ENGINE_OPTIONS:
|
|
|
+
|
|
|
+CELERY_RESULT_ENGINE_OPTIONS
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
To specify additional SQLAlchemy database engine options you can use
|
|
|
the :setting:`CELERY_RESULT_ENGINE_OPTIONS` setting::
|
|
|
|
|
@@ -142,7 +174,7 @@ the :setting:`CELERY_RESULT_ENGINE_OPTIONS` setting::
|
|
|
http://www.sqlalchemy.org/docs/dbengine.html#create-engine-url-arguments
|
|
|
|
|
|
Example configuration
|
|
|
----------------------
|
|
|
+~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
.. code-block:: python
|
|
|
|
|
@@ -152,42 +184,72 @@ Example configuration
|
|
|
.. _conf-amqp-result-backend:
|
|
|
|
|
|
AMQP backend settings
|
|
|
-=====================
|
|
|
+---------------------
|
|
|
+
|
|
|
+.. _CELERY_AMQP_TASK_RESULT_EXPIRES:
|
|
|
+
|
|
|
+CELERY_AMQP_TASK_RESULT_EXPIRES
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERY_RESULT_EXCHANGE
|
|
|
+The time in seconds of which the task result queues should expire.
|
|
|
|
|
|
- Name of the exchange to publish results in. Default is ``"celeryresults"``.
|
|
|
+.. note::
|
|
|
|
|
|
-* CELERY_RESULT_EXCHANGE_TYPE
|
|
|
+ AMQP result expiration requires RabbitMQ versions 2.1.0 and higher.
|
|
|
|
|
|
- The exchange type of the result exchange. Default is to use a ``direct``
|
|
|
- exchange.
|
|
|
+.. _CELERY_RESULT_EXCHANGE:
|
|
|
|
|
|
-* CELERY_RESULT_SERIALIZER
|
|
|
+CELERY_RESULT_EXCHANGE
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- Result message serialization format. Default is ``"pickle"``.
|
|
|
+Name of the exchange to publish results in. Default is ``"celeryresults"``.
|
|
|
|
|
|
-* CELERY_RESULTS_PERSISTENT
|
|
|
+.. _CELERY_RESULT_EXCHANGE_TYPE:
|
|
|
|
|
|
- If set to ``True``, result messages will be persistent. This means the
|
|
|
- messages will not be lost after a broker restart. The default is for the
|
|
|
- results to be transient.
|
|
|
+CELERY_RESULT_EXCHANGE_TYPE
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+The exchange type of the result exchange. Default is to use a ``direct``
|
|
|
+exchange.
|
|
|
+
|
|
|
+.. _CELERY_RESULT_SERIALIZER:
|
|
|
+
|
|
|
+CELERY_RESULT_SERIALIZER
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+Result message serialization format. Default is ``"pickle"``.
|
|
|
+
|
|
|
+.. _CELERY_RESULT_PERSISTENT:
|
|
|
+
|
|
|
+CELERY_RESULT_PERSISTENT
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+If set to :const:`True`, result messages will be persistent. This means the
|
|
|
+messages will not be lost after a broker restart. The default is for the
|
|
|
+results to be transient.
|
|
|
|
|
|
Example configuration
|
|
|
----------------------
|
|
|
+~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+.. code-block:: python
|
|
|
|
|
|
CELERY_RESULT_BACKEND = "amqp"
|
|
|
+ CELERY_AMQP_TASK_RESULT_EXPIRES = 18000 # 5 hours.
|
|
|
|
|
|
.. _conf-cache-result-backend:
|
|
|
|
|
|
Cache backend settings
|
|
|
-======================
|
|
|
+----------------------
|
|
|
|
|
|
-The cache backend supports the `pylibmc`_ and `python-memcached` libraries.
|
|
|
-The latter is used only if `pylibmc`_ is not installed.
|
|
|
+.. note::
|
|
|
|
|
|
-Example configuration
|
|
|
----------------------
|
|
|
+ The cache backend supports the `pylibmc`_ and `python-memcached`
|
|
|
+ libraries. The latter is used only if `pylibmc`_ is not installed.
|
|
|
+
|
|
|
+.. _CELERY_CACHE_BACKEND:
|
|
|
+
|
|
|
+CELERY_CACHE_BACKEND
|
|
|
+~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
Using a single memcached server:
|
|
|
|
|
@@ -202,7 +264,12 @@ Using multiple memcached servers:
|
|
|
CELERY_RESULT_BACKEND = "cache"
|
|
|
CELERY_CACHE_BACKEND = 'memcached://172.19.26.240:11211;172.19.26.242:11211/'
|
|
|
|
|
|
-You can set pylibmc options using the :setting:`ELERY_CACHE_BACKEND_OPTIONS`
|
|
|
+.. _CELERY_CACHE_BACKEND_OPTIONS:
|
|
|
+
|
|
|
+CELERY_CACHE_BACKEND_OPTIONS
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+You can set pylibmc options using the :setting:`CELERY_CACHE_BACKEND_OPTIONS`
|
|
|
setting:
|
|
|
|
|
|
.. code-block:: python
|
|
@@ -215,22 +282,32 @@ setting:
|
|
|
.. _conf-tyrant-result-backend:
|
|
|
|
|
|
Tokyo Tyrant backend settings
|
|
|
-=============================
|
|
|
+-----------------------------
|
|
|
+
|
|
|
+.. note::
|
|
|
|
|
|
-**NOTE** The Tokyo Tyrant backend requires the :mod:`pytyrant` library:
|
|
|
+ The Tokyo Tyrant backend requires the :mod:`pytyrant` library:
|
|
|
http://pypi.python.org/pypi/pytyrant/
|
|
|
|
|
|
This backend requires the following configuration directives to be set:
|
|
|
|
|
|
-* TT_HOST
|
|
|
- Hostname of the Tokyo Tyrant server.
|
|
|
+.. _TT_HOST:
|
|
|
+
|
|
|
+TT_HOST
|
|
|
+~~~~~~~
|
|
|
+
|
|
|
+Hostname of the Tokyo Tyrant server.
|
|
|
+
|
|
|
+.. _TT_PORT:
|
|
|
|
|
|
-* TT_PORT
|
|
|
- The port the Tokyo Tyrant server is listening to.
|
|
|
+TT_PORT
|
|
|
+~~~~~~~
|
|
|
+
|
|
|
+The port the Tokyo Tyrant server is listening to.
|
|
|
|
|
|
|
|
|
Example configuration
|
|
|
----------------------
|
|
|
+~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
.. code-block:: python
|
|
|
|
|
@@ -241,37 +318,49 @@ Example configuration
|
|
|
.. _conf-redis-result-backend:
|
|
|
|
|
|
Redis backend settings
|
|
|
-======================
|
|
|
+----------------------
|
|
|
|
|
|
-**NOTE** The Redis backend requires the :mod:`redis` library:
|
|
|
+.. note::
|
|
|
+
|
|
|
+ The Redis backend requires the :mod:`redis` library:
|
|
|
http://pypi.python.org/pypi/redis/0.5.5
|
|
|
|
|
|
-To install the redis package use ``pip`` or ``easy_install``::
|
|
|
+ To install the redis package use ``pip`` or ``easy_install``::
|
|
|
|
|
|
- $ pip install redis
|
|
|
+ $ pip install redis
|
|
|
|
|
|
-This backend requires the following configuration directives to be set:
|
|
|
+This backend requires the following configuration directives to be set.
|
|
|
|
|
|
-* REDIS_HOST
|
|
|
+.. _REDIS_HOST:
|
|
|
|
|
|
- Hostname of the Redis database server. e.g. ``"localhost"``.
|
|
|
+REDIS_HOST
|
|
|
+~~~~~~~~~~
|
|
|
|
|
|
-* REDIS_PORT
|
|
|
+Hostname of the Redis database server. e.g. ``"localhost"``.
|
|
|
|
|
|
- Port to the Redis database server. e.g. ``6379``.
|
|
|
+.. _REDIS_PORT:
|
|
|
|
|
|
-Also, the following optional configuration directives are available:
|
|
|
+REDIS_PORT
|
|
|
+~~~~~~~~~~
|
|
|
|
|
|
-* REDIS_DB
|
|
|
+Port to the Redis database server. e.g. ``6379``.
|
|
|
|
|
|
- Database number to use. Default is 0
|
|
|
+.. _REDIS_DB:
|
|
|
|
|
|
-* REDIS_PASSWORD
|
|
|
+REDIS_DB
|
|
|
+~~~~~~~~
|
|
|
|
|
|
- Password used to connect to the database.
|
|
|
+Database number to use. Default is 0
|
|
|
+
|
|
|
+.. _REDIS_PASSWORD:
|
|
|
+
|
|
|
+REDIS_PASSWORD
|
|
|
+~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+Password used to connect to the database.
|
|
|
|
|
|
Example configuration
|
|
|
----------------------
|
|
|
+~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
.. code-block:: python
|
|
|
|
|
@@ -284,37 +373,43 @@ Example configuration
|
|
|
.. _conf-mongodb-result-backend:
|
|
|
|
|
|
MongoDB backend settings
|
|
|
-========================
|
|
|
+------------------------
|
|
|
+
|
|
|
+.. note::
|
|
|
|
|
|
-**NOTE** The MongoDB backend requires the :mod:`pymongo` library:
|
|
|
+ The MongoDB backend requires the :mod:`pymongo` library:
|
|
|
http://github.com/mongodb/mongo-python-driver/tree/master
|
|
|
|
|
|
-* CELERY_MONGODB_BACKEND_SETTINGS
|
|
|
+.. _CELERY_MONGODB_BACKEND_SETTINGS:
|
|
|
|
|
|
- This is a dict supporting the following keys:
|
|
|
+CELERY_MONGODB_BACKEND_SETTINGS
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- * host
|
|
|
- Hostname of the MongoDB server. Defaults to "localhost".
|
|
|
+This is a dict supporting the following keys:
|
|
|
|
|
|
- * port
|
|
|
- The port the MongoDB server is listening to. Defaults to 27017.
|
|
|
+* host
|
|
|
+ Hostname of the MongoDB server. Defaults to "localhost".
|
|
|
|
|
|
- * user
|
|
|
- User name to authenticate to the MongoDB server as (optional).
|
|
|
+* port
|
|
|
+ The port the MongoDB server is listening to. Defaults to 27017.
|
|
|
|
|
|
- * password
|
|
|
- Password to authenticate to the MongoDB server (optional).
|
|
|
+* user
|
|
|
+ User name to authenticate to the MongoDB server as (optional).
|
|
|
|
|
|
- * database
|
|
|
- The database name to connect to. Defaults to "celery".
|
|
|
+* password
|
|
|
+ Password to authenticate to the MongoDB server (optional).
|
|
|
|
|
|
- * taskmeta_collection
|
|
|
- The collection name to store task meta data.
|
|
|
- Defaults to "celery_taskmeta".
|
|
|
+* database
|
|
|
+ The database name to connect to. Defaults to "celery".
|
|
|
|
|
|
+* taskmeta_collection
|
|
|
+ The collection name to store task meta data.
|
|
|
+ Defaults to "celery_taskmeta".
|
|
|
+
|
|
|
+.. _example-mongodb-result-config:
|
|
|
|
|
|
Example configuration
|
|
|
----------------------
|
|
|
+~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
.. code-block:: python
|
|
|
|
|
@@ -328,235 +423,338 @@ Example configuration
|
|
|
|
|
|
.. _conf-messaging:
|
|
|
|
|
|
-Messaging settings
|
|
|
-==================
|
|
|
+Message Routing
|
|
|
+---------------
|
|
|
|
|
|
.. _conf-messaging-routing:
|
|
|
|
|
|
-Routing
|
|
|
--------
|
|
|
+.. _CELERY_QUEUES:
|
|
|
+
|
|
|
+CELERY_QUEUES
|
|
|
+~~~~~~~~~~~~~
|
|
|
+
|
|
|
+The mapping of queues the worker consumes from. This is a dictionary
|
|
|
+of queue name/options. See :ref:`guide-routing` for more information.
|
|
|
+
|
|
|
+The default is a queue/exchange/binding key of ``"celery"``, with
|
|
|
+exchange type ``direct``.
|
|
|
+
|
|
|
+You don't have to care about this unless you want custom routing facilities.
|
|
|
+
|
|
|
+.. _CELERY_DEFAULT_QUEUE:
|
|
|
+
|
|
|
+CELERY_DEFAULT_QUEUE
|
|
|
+~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+The queue used by default, if no custom queue is specified. This queue must
|
|
|
+be listed in :setting:`CELERY_QUEUES`. The default is: ``celery``.
|
|
|
|
|
|
-* CELERY_QUEUES
|
|
|
- The mapping of queues the worker consumes from. This is a dictionary
|
|
|
- of queue name/options. See :doc:`userguide/routing` for more information.
|
|
|
+.. _CELERY_DEFAULT_EXCHANGE:
|
|
|
|
|
|
- The default is a queue/exchange/binding key of ``"celery"``, with
|
|
|
- exchange type ``direct``.
|
|
|
+CELERY_DEFAULT_EXCHANGE
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- You don't have to care about this unless you want custom routing facilities.
|
|
|
+Name of the default exchange to use when no custom exchange is
|
|
|
+specified. The default is: ``celery``.
|
|
|
|
|
|
-* CELERY_DEFAULT_QUEUE
|
|
|
- The queue used by default, if no custom queue is specified.
|
|
|
- This queue must be listed in :setting:`CELERY_QUEUES`.
|
|
|
- The default is: ``celery``.
|
|
|
+.. _CELERY_DEFAULT_EXCHANGE_TYPE:
|
|
|
|
|
|
-* CELERY_DEFAULT_EXCHANGE
|
|
|
- Name of the default exchange to use when no custom exchange
|
|
|
- is specified.
|
|
|
- The default is: ``celery``.
|
|
|
+CELERY_DEFAULT_EXCHANGE_TYPE
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+Default exchange type used when no custom exchange is specified.
|
|
|
+The default is: ``direct``.
|
|
|
+
|
|
|
+.. _CELERY_DEFAULT_ROUTING_KEY:
|
|
|
+
|
|
|
+CELERY_DEFAULT_ROUTING_KEY
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERY_DEFAULT_EXCHANGE_TYPE
|
|
|
- Default exchange type used when no custom exchange is specified.
|
|
|
- The default is: ``direct``.
|
|
|
+The default routing key used when sending tasks.
|
|
|
+The default is: ``celery``.
|
|
|
|
|
|
-* CELERY_DEFAULT_ROUTING_KEY
|
|
|
- The default routing key used when sending tasks.
|
|
|
- The default is: ``celery``.
|
|
|
+.. _CELERY_DEFAULT_DELIVERY_MODE:
|
|
|
|
|
|
-* CELERY_DEFAULT_DELIVERY_MODE
|
|
|
+CELERY_DEFAULT_DELIVERY_MODE
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- Can be ``transient`` or ``persistent``. Default is to send
|
|
|
- persistent messages.
|
|
|
+Can be ``transient`` or ``persistent``. The default is to send
|
|
|
+persistent messages.
|
|
|
|
|
|
.. _conf-broker-connection:
|
|
|
|
|
|
-Connection
|
|
|
-----------
|
|
|
+Broker Settings
|
|
|
+---------------
|
|
|
+
|
|
|
+.. _CELERY_BROKER_CONNECTION_TIMEOUT:
|
|
|
+
|
|
|
+CELERY_BROKER_CONNECTION_TIMEOUT
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+The default timeout in seconds before we give up establishing a connection
|
|
|
+to the AMQP server. Default is 4 seconds.
|
|
|
|
|
|
-* CELERY_BROKER_CONNECTION_TIMEOUT
|
|
|
- The timeout in seconds before we give up establishing a connection
|
|
|
- to the AMQP server. Default is 4 seconds.
|
|
|
+.. _CELERY_BROKER_CONNECTION_RETRY:
|
|
|
|
|
|
-* CELERY_BROKER_CONNECTION_RETRY
|
|
|
- Automatically try to re-establish the connection to the AMQP broker if
|
|
|
- it's lost.
|
|
|
+CELERY_BROKER_CONNECTION_RETRY
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- The time between retries is increased for each retry, and is
|
|
|
- not exhausted before :setting:`CELERY_BROKER_CONNECTION_MAX_RETRIES` is
|
|
|
- exceeded.
|
|
|
+Automatically try to re-establish the connection to the AMQP broker if lost.
|
|
|
|
|
|
- This behavior is on by default.
|
|
|
+The time between retries is increased for each retry, and is
|
|
|
+not exhausted before :setting:`CELERY_BROKER_CONNECTION_MAX_RETRIES` is
|
|
|
+exceeded.
|
|
|
|
|
|
-* CELERY_BROKER_CONNECTION_MAX_RETRIES
|
|
|
- Maximum number of retries before we give up re-establishing a connection
|
|
|
- to the AMQP broker.
|
|
|
+This behavior is on by default.
|
|
|
|
|
|
- If this is set to ``0`` or ``None``, we will retry forever.
|
|
|
+.. _CELERY_BROKER_CONNECTION_MAX_RETRIES:
|
|
|
|
|
|
- Default is 100 retries.
|
|
|
+CELERY_BROKER_CONNECTION_MAX_RETRIES
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+Maximum number of retries before we give up re-establishing a connection
|
|
|
+to the AMQP broker.
|
|
|
+
|
|
|
+If this is set to :const:`0` or :const:`None`, we will retry forever.
|
|
|
+
|
|
|
+Default is 100 retries.
|
|
|
|
|
|
.. _conf-task-execution:
|
|
|
|
|
|
Task execution settings
|
|
|
-=======================
|
|
|
+-----------------------
|
|
|
+
|
|
|
+.. _CELERY_ALWAYS_EAGER:
|
|
|
+
|
|
|
+CELERY_ALWAYS_EAGER
|
|
|
+~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+If this is :const:`True`, all tasks will be executed locally by blocking
|
|
|
+until it is finished. ``apply_async`` and ``Task.delay`` will return
|
|
|
+a :class:`~celery.result.EagerResult` which emulates the behavior of
|
|
|
+:class:`~celery.result.AsyncResult`, except the result has already
|
|
|
+been evaluated.
|
|
|
+
|
|
|
+Tasks will never be sent to the queue, but executed locally
|
|
|
+instead.
|
|
|
+
|
|
|
+.. _CELERY_EAGER_PROPAGATES_EXCEPTIONS:
|
|
|
+
|
|
|
+CELERY_EAGER_PROPAGATES_EXCEPTIONS
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+If this is :const:`True`, eagerly executed tasks (using ``.apply``, or with
|
|
|
+:setting:`CELERY_ALWAYS_EAGER` on), will raise exceptions.
|
|
|
+
|
|
|
+It's the same as always running ``apply`` with ``throw=True``.
|
|
|
+
|
|
|
+.. _CELERY_IGNORE_RESULT:
|
|
|
+
|
|
|
+CELERY_IGNORE_RESULT
|
|
|
+~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+Whether to store the task return values or not (tombstones).
|
|
|
+If you still want to store errors, just not successful return values,
|
|
|
+you can set :setting:`CELERY_STORE_ERRORS_EVEN_IF_IGNORED`.
|
|
|
+
|
|
|
+.. _CELERY_TASK_RESULT_EXPIRES:
|
|
|
+
|
|
|
+CELERY_TASK_RESULT_EXPIRES
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+Time (in seconds, or a :class:`~datetime.timedelta` object) for when after
|
|
|
+stored task tombstones will be deleted.
|
|
|
|
|
|
-* CELERY_ALWAYS_EAGER
|
|
|
- If this is ``True``, all tasks will be executed locally by blocking
|
|
|
- until it is finished. ``apply_async`` and ``Task.delay`` will return
|
|
|
- a :class:`celery.result.EagerResult` which emulates the behavior of
|
|
|
- :class:`celery.result.AsyncResult`, except the result has already
|
|
|
- been evaluated.
|
|
|
+A built-in periodic task will delete the results after this time
|
|
|
+(:class:`celery.task.builtins.backend_cleanup`).
|
|
|
|
|
|
- Tasks will never be sent to the queue, but executed locally
|
|
|
- instead.
|
|
|
+.. note::
|
|
|
|
|
|
-* CELERY_EAGER_PROPAGATES_EXCEPTIONS
|
|
|
+ For the moment this only works with the database, cache, redis and MongoDB
|
|
|
+ backends. For the AMQP backend see
|
|
|
+ :setting:`CELERY_AMQP_TASK_RESULT_EXPIRES`.
|
|
|
|
|
|
- If this is ``True``, eagerly executed tasks (using ``.apply``, or with
|
|
|
- :setting:`CELERY_ALWAYS_EAGER` on), will raise exceptions.
|
|
|
+ When using the database or MongoDB backends, ``celerybeat`` must be
|
|
|
+ running for the results to be expired.
|
|
|
|
|
|
- It's the same as always running ``apply`` with ``throw=True``.
|
|
|
|
|
|
-* CELERY_IGNORE_RESULT
|
|
|
+.. _CELERY_MAX_CACHED_RESULTS:
|
|
|
|
|
|
- Whether to store the task return values or not (tombstones).
|
|
|
- If you still want to store errors, just not successful return values,
|
|
|
- you can set :setting:`CELERY_STORE_ERRORS_EVEN_IF_IGNORED`.
|
|
|
+CELERY_MAX_CACHED_RESULTS
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERY_TASK_RESULT_EXPIRES
|
|
|
- Time (in seconds, or a :class:`datetime.timedelta` object) for when after
|
|
|
- stored task tombstones will be deleted.
|
|
|
+Total number of results to store before results are evicted from the
|
|
|
+result cache. The default is 5000.
|
|
|
|
|
|
- A built-in periodic task will delete the results after this time
|
|
|
- (:class:`celery.task.builtins.DeleteExpiredTaskMetaTask`).
|
|
|
+.. _CELERY_TRACK_STARTED:
|
|
|
|
|
|
- **NOTE**: For the moment this only works with the database, cache and MongoDB
|
|
|
- backends.
|
|
|
+CELERY_TRACK_STARTED
|
|
|
+~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- **NOTE**: ``celerybeat`` must be running for the results to be expired.
|
|
|
+If :const:`True` the task will report its status as "started" when the
|
|
|
+task is executed by a worker. The default value is :const:`False` as
|
|
|
+the normal behaviour is to not report that level of granularity. Tasks
|
|
|
+are either pending, finished, or waiting to be retried. Having a "started"
|
|
|
+state can be useful for when there are long running tasks and there is a
|
|
|
+need to report which task is currently running.
|
|
|
|
|
|
-* CELERY_MAX_CACHED_RESULTS
|
|
|
+.. _CELERY_TASK_SERIALIZER:
|
|
|
|
|
|
- Total number of results to store before results are evicted from the
|
|
|
- result cache. The default is ``5000``.
|
|
|
+CELERY_TASK_SERIALIZER
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERY_TRACK_STARTED
|
|
|
+A string identifying the default serialization method to use. Can be
|
|
|
+``pickle`` (default), ``json``, ``yaml``, or any custom serialization
|
|
|
+methods that have been registered with :mod:`carrot.serialization.registry`.
|
|
|
|
|
|
- If ``True`` the task will report its status as "started"
|
|
|
- when the task is executed by a worker.
|
|
|
- The default value is ``False`` as the normal behaviour is to not
|
|
|
- report that level of granularity. Tasks are either pending, finished,
|
|
|
- or waiting to be retried. Having a "started" status can be useful for
|
|
|
- when there are long running tasks and there is a need to report which
|
|
|
- task is currently running.
|
|
|
- backends.
|
|
|
+.. _CELERY_DEFAULT_RATE_LIMIT:
|
|
|
|
|
|
-* CELERY_TASK_SERIALIZER
|
|
|
- A string identifying the default serialization
|
|
|
- method to use. Can be ``pickle`` (default),
|
|
|
- ``json``, ``yaml``, or any custom serialization methods that have
|
|
|
- been registered with :mod:`carrot.serialization.registry`.
|
|
|
+CELERY_DEFAULT_RATE_LIMIT
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- Default is ``pickle``.
|
|
|
+The global default rate limit for tasks.
|
|
|
|
|
|
-* CELERY_DEFAULT_RATE_LIMIT
|
|
|
+This value is used for tasks that does not have a custom rate limit
|
|
|
+The default is no rate limit.
|
|
|
|
|
|
- The global default rate limit for tasks.
|
|
|
+.. _CELERY_DISABLE_RATE_LIMITS:
|
|
|
|
|
|
- This value is used for tasks that does not have a custom rate limit
|
|
|
- The default is no rate limit.
|
|
|
+CELERY_DISABLE_RATE_LIMITS
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERY_DISABLE_RATE_LIMITS
|
|
|
+Disable all rate limits, even if tasks has explicit rate limits set.
|
|
|
|
|
|
- Disable all rate limits, even if tasks has explicit rate limits set.
|
|
|
+.. _CELERY_ACKS_LATE:
|
|
|
|
|
|
-* CELERY_ACKS_LATE
|
|
|
+CELERY_ACKS_LATE
|
|
|
+~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- Late ack means the task messages will be acknowledged **after** the task
|
|
|
- has been executed, not *just before*, which is the default behavior.
|
|
|
+Late ack means the task messages will be acknowledged **after** the task
|
|
|
+has been executed, not *just before*, which is the default behavior.
|
|
|
|
|
|
- See http://ask.github.com/celery/faq.html#should-i-use-retry-or-acks-late
|
|
|
+.. seealso::
|
|
|
+
|
|
|
+ FAQ: :ref:`faq-acks_late-vs-retry`.
|
|
|
|
|
|
.. _conf-celeryd:
|
|
|
|
|
|
Worker: celeryd
|
|
|
-===============
|
|
|
+---------------
|
|
|
+
|
|
|
+.. _CELERY_IMPORTS:
|
|
|
+
|
|
|
+CELERY_IMPORTS
|
|
|
+~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+A sequence of modules to import when the celery daemon starts.
|
|
|
+
|
|
|
+This is used to specify the task modules to import, but also
|
|
|
+to import signal handlers and additional remote control commands, etc.
|
|
|
+
|
|
|
+.. _CELERYD_MAX_TASKS_PER_CHILD:
|
|
|
+
|
|
|
+CELERYD_MAX_TASKS_PER_CHILD
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+Maximum number of tasks a pool worker process can execute before
|
|
|
+it's replaced with a new one. Default is no limit.
|
|
|
|
|
|
-* CELERY_IMPORTS
|
|
|
+.. _CELERYD_TASK_TIME_LIMIT:
|
|
|
|
|
|
- A sequence of modules to import when the celery daemon starts.
|
|
|
+CELERYD_TASK_TIME_LIMIT
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- This is used to specify the task modules to import, but also
|
|
|
- to import signal handlers and additional remote control commands, etc.
|
|
|
+Task hard time limit in seconds. The worker processing the task will
|
|
|
+be killed and replaced with a new one when this is exceeded.
|
|
|
|
|
|
-* CELERYD_MAX_TASKS_PER_CHILD
|
|
|
+.. _CELERYD_SOFT_TASK_TIME_LIMIT:
|
|
|
|
|
|
- Maximum number of tasks a pool worker process can execute before
|
|
|
- it's replaced with a new one. Default is no limit.
|
|
|
+CELERYD_SOFT_TASK_TIME_LIMIT
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERYD_TASK_TIME_LIMIT
|
|
|
+Task soft time limit in seconds.
|
|
|
|
|
|
- Task hard time limit in seconds. The worker processing the task will
|
|
|
- be killed and replaced with a new one when this is exceeded.
|
|
|
+The :exc:`~celery.exceptions.SoftTimeLimitExceeded` exception will be
|
|
|
+raised when this is exceeded. The task can catch this to
|
|
|
+e.g. clean up before the hard time limit comes.
|
|
|
|
|
|
-* CELERYD_SOFT_TASK_TIME_LIMIT
|
|
|
+Example:
|
|
|
|
|
|
- Task soft time limit in seconds.
|
|
|
- The :exc:`celery.exceptions.SoftTimeLimitExceeded` exception will be
|
|
|
- raised when this is exceeded. The task can catch this to
|
|
|
- e.g. clean up before the hard time limit comes.
|
|
|
+.. code-block:: python
|
|
|
|
|
|
- .. code-block:: python
|
|
|
+ from celery.decorators import task
|
|
|
+ from celery.exceptions import SoftTimeLimitExceeded
|
|
|
|
|
|
- from celery.decorators import task
|
|
|
- from celery.exceptions import SoftTimeLimitExceeded
|
|
|
+ @task()
|
|
|
+ def mytask():
|
|
|
+ try:
|
|
|
+ return do_work()
|
|
|
+ except SoftTimeLimitExceeded:
|
|
|
+ cleanup_in_a_hurry()
|
|
|
|
|
|
- @task()
|
|
|
- def mytask():
|
|
|
- try:
|
|
|
- return do_work()
|
|
|
- except SoftTimeLimitExceeded:
|
|
|
- cleanup_in_a_hurry()
|
|
|
+.. _CELERY_STORE_ERRORS_EVEN_IF_IGNORED:
|
|
|
|
|
|
-* CELERY_STORE_ERRORS_EVEN_IF_IGNORED
|
|
|
+CELERY_STORE_ERRORS_EVEN_IF_IGNORED
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- If set, the worker stores all task errors in the result store even if
|
|
|
- ``Task.ignore_result`` is on.
|
|
|
+If set, the worker stores all task errors in the result store even if
|
|
|
+:attr:`Task.ignore_result <celery.task.base.Task.ignore_result>` is on.
|
|
|
|
|
|
.. _conf-error-mails:
|
|
|
|
|
|
Error E-Mails
|
|
|
-------------
|
|
|
|
|
|
-* CELERY_SEND_TASK_ERROR_EMAILS
|
|
|
+.. _CELERYD_SEND_TASK_ERROR_EMAILS:
|
|
|
+
|
|
|
+CELERY_SEND_TASK_ERROR_EMAILS
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+If set to ``True``, errors in tasks will be sent to admins by e-mail.
|
|
|
+
|
|
|
+.. _ADMINS:
|
|
|
+
|
|
|
+ADMINS
|
|
|
+~~~~~~
|
|
|
+
|
|
|
+List of ``(name, email_address)`` tuples for the admins that should
|
|
|
+receive error e-mails.
|
|
|
+
|
|
|
+.. _SERVER_EMAIL:
|
|
|
|
|
|
- If set to ``True``, errors in tasks will be sent to admins by e-mail.
|
|
|
+SERVER_EMAIL
|
|
|
+~~~~~~~~~~~~
|
|
|
|
|
|
-* ADMINS
|
|
|
+The e-mail address this worker sends e-mails from.
|
|
|
+Default is celery@localhost.
|
|
|
|
|
|
- List of ``(name, email_address)`` tuples for the admins that should
|
|
|
- receive error e-mails.
|
|
|
+.. _MAIL_HOST:
|
|
|
|
|
|
-* SERVER_EMAIL
|
|
|
+MAIL_HOST
|
|
|
+~~~~~~~~~
|
|
|
|
|
|
- The e-mail address this worker sends e-mails from.
|
|
|
- Default is ``"celery@localhost"``.
|
|
|
+The mail server to use. Default is ``"localhost"``.
|
|
|
|
|
|
-* MAIL_HOST
|
|
|
+.. _MAIL_HOST_USER:
|
|
|
|
|
|
- The mail server to use. Default is ``"localhost"``.
|
|
|
+MAIL_HOST_USER
|
|
|
+~~~~~~~~~~~~~~
|
|
|
|
|
|
-* MAIL_HOST_USER
|
|
|
+Username (if required) to log on to the mail server with.
|
|
|
|
|
|
- Username (if required) to log on to the mail server with.
|
|
|
+.. _MAIL_HOST_PASSWORD:
|
|
|
|
|
|
-* MAIL_HOST_PASSWORD
|
|
|
+MAIL_HOST_PASSWORD
|
|
|
+~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- Password (if required) to log on to the mail server with.
|
|
|
+Password (if required) to log on to the mail server with.
|
|
|
|
|
|
-* MAIL_PORT
|
|
|
+.. _MAIL_PORT:
|
|
|
|
|
|
- The port the mail server is listening on. Default is ``25``.
|
|
|
+MAIL_PORT
|
|
|
+~~~~~~~~~
|
|
|
+
|
|
|
+The port the mail server is listening on. Default is ``25``.
|
|
|
|
|
|
.. _conf-example-error-mail-config:
|
|
|
|
|
@@ -564,7 +762,7 @@ Example E-Mail configuration
|
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
This configuration enables the sending of error e-mails to
|
|
|
-``george@vandelay.com`` and ``kramer@vandelay.com``:
|
|
|
+george@vandelay.com and kramer@vandelay.com:
|
|
|
|
|
|
.. code-block:: python
|
|
|
|
|
@@ -591,168 +789,245 @@ This configuration enables the sending of error e-mails to
|
|
|
Events
|
|
|
------
|
|
|
|
|
|
-* CELERY_SEND_EVENTS
|
|
|
+.. _CELERY_SEND_EVENTS:
|
|
|
+
|
|
|
+CELERY_SEND_EVENTS
|
|
|
+~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+Send events so the worker can be monitored by tools like ``celerymon``.
|
|
|
+
|
|
|
+.. _CELERY_EVENT_EXCHANGE:
|
|
|
+
|
|
|
+CELERY_EVENT_EXCHANGE
|
|
|
+~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- Send events so the worker can be monitored by tools like ``celerymon``.
|
|
|
+Name of the exchange to send event messages to. Default is ``"celeryevent"``.
|
|
|
|
|
|
-* CELERY_EVENT_EXCHANGE
|
|
|
+.. _CELERY_EVENT_EXCHANGE_TYPE:
|
|
|
|
|
|
- Name of the exchange to send event messages to. Default is
|
|
|
- ``"celeryevent"``.
|
|
|
+CELERY_EVENT_EXCHANGE_TYPE
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERY_EVENT_EXCHANGE_TYPE
|
|
|
+The exchange type of the event exchange. Default is to use a ``"direct"``
|
|
|
+exchange.
|
|
|
|
|
|
- The exchange type of the event exchange. Default is to use a ``direct``
|
|
|
- exchange.
|
|
|
+.. _CELERY_EVENT_ROUTING_KEY:
|
|
|
|
|
|
-* CELERY_EVENT_ROUTING_KEY
|
|
|
+CELERY_EVENT_ROUTING_KEY
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- Routing key used when sending event messages. Default is
|
|
|
- ``"celeryevent"``.
|
|
|
+Routing key used when sending event messages. Default is ``"celeryevent"``.
|
|
|
|
|
|
-* CELERY_EVENT_SERIALIZER
|
|
|
+.. _CELERY_EVENT_SERIALIZER:
|
|
|
|
|
|
- Message serialization format used when sending event messages. Default is
|
|
|
- ``"json"``.
|
|
|
+CELERY_EVENT_SERIALIZER
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+Message serialization format used when sending event messages.
|
|
|
+Default is ``"json"``.
|
|
|
|
|
|
.. _conf-broadcast:
|
|
|
|
|
|
Broadcast Commands
|
|
|
------------------
|
|
|
|
|
|
-* CELERY_BROADCAST_QUEUE
|
|
|
+.. _CELERY_BROADCAST_QUEUE:
|
|
|
+
|
|
|
+CELERY_BROADCAST_QUEUE
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+Name prefix for the queue used when listening for broadcast messages.
|
|
|
+The workers hostname will be appended to the prefix to create the final
|
|
|
+queue name.
|
|
|
+
|
|
|
+Default is ``"celeryctl"``.
|
|
|
|
|
|
- Name prefix for the queue used when listening for
|
|
|
- broadcast messages. The workers hostname will be appended
|
|
|
- to the prefix to create the final queue name.
|
|
|
+.. _CELERY_BROADCASTS_EXCHANGE:
|
|
|
|
|
|
- Default is ``"celeryctl"``.
|
|
|
+CELERY_BROADCAST_EXCHANGE
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERY_BROADCAST_EXCHANGE
|
|
|
+Name of the exchange used for broadcast messages.
|
|
|
|
|
|
- Name of the exchange used for broadcast messages.
|
|
|
+Default is ``"celeryctl"``.
|
|
|
|
|
|
- Default is ``"celeryctl"``.
|
|
|
+.. _CELERY_BROADCAST_EXCHANGE_TYPE:
|
|
|
|
|
|
-* CELERY_BROADCAST_EXCHANGE_TYPE
|
|
|
+CELERY_BROADCAST_EXCHANGE_TYPE
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- Exchange type used for broadcast messages. Default is ``"fanout"``.
|
|
|
+Exchange type used for broadcast messages. Default is ``"fanout"``.
|
|
|
|
|
|
.. _conf-logging:
|
|
|
|
|
|
Logging
|
|
|
-------
|
|
|
|
|
|
-* CELERYD_LOG_FILE
|
|
|
- The default file name the worker daemon logs messages to, can be
|
|
|
- overridden using the `--logfile`` option to ``celeryd``.
|
|
|
+.. _CELERYD_LOG_FILE:
|
|
|
|
|
|
- The default is ``None`` (``stderr``)
|
|
|
- Can also be set via the ``--logfile`` argument.
|
|
|
+CELERYD_LOG_FILE
|
|
|
+~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERYD_LOG_LEVEL
|
|
|
+The default file name the worker daemon logs messages to. Can be overridden
|
|
|
+using the :option:`--logfile` option to :mod:`~celery.bin.celeryd`.
|
|
|
|
|
|
- Worker log level, can be any of ``DEBUG``, ``INFO``, ``WARNING``,
|
|
|
- ``ERROR``, ``CRITICAL``.
|
|
|
+The default is :const:`None` (``stderr``)
|
|
|
|
|
|
- Can also be set via the ``--loglevel`` argument.
|
|
|
+.. _CELERYD_LOG_LEVEL:
|
|
|
|
|
|
- See the :mod:`logging` module for more information.
|
|
|
+CELERYD_LOG_LEVEL
|
|
|
+~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERYD_LOG_FORMAT
|
|
|
+Worker log level, can be any of :const:`DEBUG`, :const:`INFO`, :const:`WARNING`,
|
|
|
+:const:`ERROR` or :const:`CRITICAL`.
|
|
|
|
|
|
- The format to use for log messages.
|
|
|
+Can also be set via the :option:`--loglevel` argument to
|
|
|
+:mod:`~celery.bin.celeryd`.
|
|
|
|
|
|
- Default is ``[%(asctime)s: %(levelname)s/%(processName)s] %(message)s``
|
|
|
+See the :mod:`logging` module for more information.
|
|
|
|
|
|
- See the Python :mod:`logging` module for more information about log
|
|
|
- formats.
|
|
|
+.. _CELERYD_LOG_FORMAT:
|
|
|
|
|
|
-* CELERYD_TASK_LOG_FORMAT
|
|
|
+CELERYD_LOG_FORMAT
|
|
|
+~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- The format to use for log messages logged in tasks. Can be overridden using
|
|
|
- the ``--loglevel`` option to ``celeryd``.
|
|
|
+The format to use for log messages.
|
|
|
|
|
|
- Default is::
|
|
|
+Default is ``[%(asctime)s: %(levelname)s/%(processName)s] %(message)s``
|
|
|
|
|
|
- [%(asctime)s: %(levelname)s/%(processName)s]
|
|
|
- [%(task_name)s(%(task_id)s)] %(message)s
|
|
|
+See the Python :mod:`logging` module for more information about log
|
|
|
+formats.
|
|
|
|
|
|
- See the Python :mod:`logging` module for more information about log
|
|
|
- formats.
|
|
|
+.. _CELERYD_TASK_LOG_FORMAT:
|
|
|
+
|
|
|
+CELERYD_TASK_LOG_FORMAT
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+The format to use for log messages logged in tasks. Can be overridden using
|
|
|
+the :option:`--loglevel` option to :mod:`~celery.bin.celeryd`.
|
|
|
+
|
|
|
+Default is::
|
|
|
+
|
|
|
+ [%(asctime)s: %(levelname)s/%(processName)s]
|
|
|
+ [%(task_name)s(%(task_id)s)] %(message)s
|
|
|
+
|
|
|
+See the Python :mod:`logging` module for more information about log
|
|
|
+formats.
|
|
|
|
|
|
.. _conf-custom-components:
|
|
|
|
|
|
Custom Component Classes (advanced)
|
|
|
-----------------------------------
|
|
|
|
|
|
-* CELERYD_POOL
|
|
|
+.. _CELERYD_POOL:
|
|
|
+
|
|
|
+CELERYD_POOL
|
|
|
+~~~~~~~~~~~~
|
|
|
+
|
|
|
+Name of the task pool class used by the worker.
|
|
|
+Default is :class:`celery.concurrency.processes.TaskPool`.
|
|
|
+
|
|
|
+.. _CELERYD_LISTENER:
|
|
|
|
|
|
- Name of the task pool class used by the worker.
|
|
|
- Default is ``"celery.concurrency.processes.TaskPool"``.
|
|
|
+CELERYD_LISTENER
|
|
|
+~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERYD_LISTENER
|
|
|
+Name of the listener class used by the worker.
|
|
|
+Default is :class:`celery.worker.listener.CarrotListener`.
|
|
|
|
|
|
- Name of the listener class used by the worker.
|
|
|
- Default is ``"celery.worker.listener.CarrotListener"``.
|
|
|
+.. _CELERYD_MEDIATOR:
|
|
|
|
|
|
-* CELERYD_MEDIATOR
|
|
|
+CELERYD_MEDIATOR
|
|
|
+~~~~~~~~~~~~~~~~
|
|
|
|
|
|
- Name of the mediator class used by the worker.
|
|
|
- Default is ``"celery.worker.controllers.Mediator"``.
|
|
|
+Name of the mediator class used by the worker.
|
|
|
+Default is :class:`celery.worker.controllers.Mediator`.
|
|
|
|
|
|
-* CELERYD_ETA_SCHEDULER
|
|
|
+.. _CELERYD_ETA_SCHEDULER:
|
|
|
|
|
|
- Name of the ETA scheduler class used by the worker.
|
|
|
- Default is ``"celery.worker.controllers.ScheduleController"``.
|
|
|
+CELERYD_ETA_SCHEDULER
|
|
|
+~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+Name of the ETA scheduler class used by the worker.
|
|
|
+Default is :class:`celery.worker.controllers.ScheduleController`.
|
|
|
|
|
|
.. _conf-celerybeat:
|
|
|
|
|
|
Periodic Task Server: celerybeat
|
|
|
-================================
|
|
|
+--------------------------------
|
|
|
+
|
|
|
+.. _CELERYBEAT_SCHEDULE:
|
|
|
+
|
|
|
+CELERYBEAT_SCHEDULE
|
|
|
+~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+The periodic task schedule used by :mod:`~celery.bin.celerybeat`.
|
|
|
+See :ref:`beat-entries`.
|
|
|
+
|
|
|
+.. _CELERYBEAT_SCHEDULE_FILENAME:
|
|
|
+
|
|
|
+CELERYBEAT_SCHEDULE_FILENAME
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+Name of the file used to stores the last run times.
|
|
|
+Can be a relative or absolute path, but be aware that the suffix ``.db``
|
|
|
+may be appended to the file name (depending on Python version).
|
|
|
|
|
|
-* CELERYBEAT_SCHEDULE_FILENAME
|
|
|
+Can also be set via the :option:`--schedule` argument to
|
|
|
+:mod:`~celery.bin.celerybeat`.
|
|
|
|
|
|
- Name of the file celerybeat stores the current schedule in.
|
|
|
- Can be a relative or absolute path, but be aware that the suffix ``.db``
|
|
|
- will be appended to the file name.
|
|
|
+.. _CELERYBEAT_MAX_LOOP_INTERVAL:
|
|
|
|
|
|
- Can also be set via the ``--schedule`` argument.
|
|
|
+CELERYBEAT_MAX_LOOP_INTERVAL
|
|
|
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+The maximum number of seconds :mod:`~celery.bin.celerybeat` can sleep
|
|
|
+between checking the schedule. Default is 300 seconds (5 minutes).
|
|
|
+
|
|
|
+.. _CELERYBEAT_LOG_FILE:
|
|
|
+
|
|
|
+CELERYBEAT_LOG_FILE
|
|
|
+~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERYBEAT_MAX_LOOP_INTERVAL
|
|
|
+The default file name to log messages to. Can be overridden using
|
|
|
+the `--logfile`` option to :mod:`~celery.bin.celerybeat`.
|
|
|
|
|
|
- The maximum number of seconds celerybeat can sleep between checking
|
|
|
- the schedule. Default is 300 seconds (5 minutes).
|
|
|
+The default is :const:`None` (``stderr``).
|
|
|
|
|
|
-* CELERYBEAT_LOG_FILE
|
|
|
- The default file name to log messages to, can be
|
|
|
- overridden using the `--logfile`` option.
|
|
|
+.. _CELERYBEAT_LOG_LEVEL:
|
|
|
|
|
|
- The default is ``None`` (``stderr``).
|
|
|
- Can also be set via the ``--logfile`` argument.
|
|
|
+CELERYBEAT_LOG_LEVEL
|
|
|
+~~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERYBEAT_LOG_LEVEL
|
|
|
- Logging level. Can be any of ``DEBUG``, ``INFO``, ``WARNING``,
|
|
|
- ``ERROR``, or ``CRITICAL``.
|
|
|
+Logging level. Can be any of :const:`DEBUG`, :const:`INFO`, :const:`WARNING`,
|
|
|
+:const:`ERROR`, or :const:`CRITICAL`.
|
|
|
|
|
|
- Can also be set via the ``--loglevel`` argument.
|
|
|
+Can also be set via the :option:`--loglevel` argument to
|
|
|
+:mod:`~celery.bin.celerybeat`.
|
|
|
|
|
|
- See the :mod:`logging` module for more information.
|
|
|
+See the :mod:`logging` module for more information.
|
|
|
|
|
|
.. _conf-celerymon:
|
|
|
|
|
|
Monitor Server: celerymon
|
|
|
-=========================
|
|
|
+-------------------------
|
|
|
+
|
|
|
+.. _CELERYMON_LOG_FILE:
|
|
|
+
|
|
|
+CELERYMON_LOG_FILE
|
|
|
+~~~~~~~~~~~~~~~~~~
|
|
|
+
|
|
|
+The default file name to log messages to. Can be overridden using
|
|
|
+the :option:`--logfile` argument to ``celerymon``.
|
|
|
+
|
|
|
+The default is :const:`None` (``stderr``)
|
|
|
|
|
|
-* CELERYMON_LOG_FILE
|
|
|
- The default file name to log messages to, can be
|
|
|
- overridden using the `--logfile`` option.
|
|
|
+.. _CELERYMON_LOG_LEVEL:
|
|
|
|
|
|
- The default is ``None`` (``stderr``)
|
|
|
- Can also be set via the ``--logfile`` argument.
|
|
|
+CELERYMON_LOG_LEVEL
|
|
|
+~~~~~~~~~~~~~~~~~~~
|
|
|
|
|
|
-* CELERYMON_LOG_LEVEL
|
|
|
- Logging level. Can be any of ``DEBUG``, ``INFO``, ``WARNING``,
|
|
|
- ``ERROR``, or ``CRITICAL``.
|
|
|
+Logging level. Can be any of :const:`DEBUG`, :const:`INFO`, :const:`WARNING`,
|
|
|
+:const:`ERROR`, or :const:`CRITICAL`.
|
|
|
|
|
|
- See the :mod:`logging` module for more information.
|
|
|
+See the :mod:`logging` module for more information.
|