configuration.rst 45 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077107810791080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104110511061107110811091110111111121113111411151116111711181119112011211122112311241125112611271128112911301131113211331134113511361137113811391140114111421143114411451146114711481149115011511152115311541155115611571158115911601161116211631164116511661167116811691170117111721173117411751176117711781179118011811182118311841185118611871188118911901191119211931194119511961197119811991200120112021203120412051206120712081209121012111212121312141215121612171218121912201221122212231224122512261227122812291230123112321233123412351236123712381239124012411242124312441245124612471248124912501251125212531254125512561257125812591260126112621263126412651266126712681269127012711272127312741275127612771278127912801281128212831284128512861287128812891290129112921293129412951296129712981299130013011302130313041305130613071308130913101311131213131314131513161317131813191320132113221323132413251326132713281329133013311332133313341335133613371338133913401341134213431344134513461347134813491350135113521353135413551356135713581359136013611362136313641365136613671368136913701371137213731374137513761377137813791380138113821383138413851386138713881389139013911392139313941395139613971398139914001401140214031404140514061407140814091410141114121413141414151416141714181419142014211422142314241425142614271428142914301431143214331434143514361437143814391440144114421443144414451446144714481449145014511452145314541455145614571458145914601461146214631464146514661467146814691470147114721473147414751476147714781479148014811482148314841485148614871488148914901491149214931494149514961497149814991500150115021503150415051506150715081509151015111512151315141515151615171518151915201521152215231524152515261527152815291530153115321533153415351536153715381539154015411542154315441545154615471548154915501551155215531554155515561557155815591560156115621563156415651566156715681569157015711572157315741575157615771578157915801581158215831584158515861587158815891590159115921593159415951596159715981599160016011602160316041605160616071608160916101611161216131614161516161617161816191620162116221623162416251626162716281629163016311632163316341635163616371638163916401641164216431644164516461647164816491650165116521653165416551656165716581659166016611662166316641665166616671668166916701671167216731674167516761677167816791680168116821683168416851686168716881689169016911692169316941695169616971698169917001701170217031704170517061707170817091710171117121713171417151716171717181719172017211722172317241725172617271728172917301731173217331734173517361737173817391740174117421743174417451746174717481749175017511752175317541755175617571758175917601761176217631764176517661767176817691770177117721773177417751776177717781779178017811782178317841785178617871788178917901791179217931794179517961797179817991800180118021803180418051806180718081809181018111812181318141815181618171818181918201821182218231824182518261827182818291830
  1. .. _configuration:
  2. ============================
  3. Configuration and defaults
  4. ============================
  5. This document describes the configuration options available.
  6. If you're using the default loader, you must create the :file:`celeryconfig.py`
  7. module and make sure it is available on the Python path.
  8. .. contents::
  9. :local:
  10. :depth: 2
  11. .. _conf-example:
  12. Example configuration file
  13. ==========================
  14. This is an example configuration file to get you started.
  15. It should contain all you need to run a basic Celery set-up.
  16. .. code-block:: python
  17. ## Broker settings.
  18. BROKER_URL = 'amqp://guest:guest@localhost:5672//'
  19. # List of modules to import when celery starts.
  20. CELERY_IMPORTS = ('myapp.tasks', )
  21. ## Using the database to store task state and results.
  22. CELERY_RESULT_BACKEND = 'db+sqlite:///results.db'
  23. CELERY_ANNOTATIONS = {'tasks.add': {'rate_limit': '10/s'}}
  24. Configuration Directives
  25. ========================
  26. .. _conf-datetime:
  27. Time and date settings
  28. ----------------------
  29. .. setting:: CELERY_ENABLE_UTC
  30. CELERY_ENABLE_UTC
  31. ~~~~~~~~~~~~~~~~~
  32. .. versionadded:: 2.5
  33. If enabled dates and times in messages will be converted to use
  34. the UTC timezone.
  35. Note that workers running Celery versions below 2.5 will assume a local
  36. timezone for all messages, so only enable if all workers have been
  37. upgraded.
  38. Enabled by default since version 3.0.
  39. .. setting:: CELERY_TIMEZONE
  40. CELERY_TIMEZONE
  41. ~~~~~~~~~~~~~~~
  42. Configure Celery to use a custom time zone.
  43. The timezone value can be any time zone supported by the `pytz`_
  44. library.
  45. If not set the UTC timezone is used. For backwards compatibility
  46. there is also a :setting:`CELERY_ENABLE_UTC` setting, and this is set
  47. to false the system local timezone is used instead.
  48. .. _`pytz`: http://pypi.python.org/pypi/pytz/
  49. .. _conf-tasks:
  50. Task settings
  51. -------------
  52. .. setting:: CELERY_ANNOTATIONS
  53. CELERY_ANNOTATIONS
  54. ~~~~~~~~~~~~~~~~~~
  55. This setting can be used to rewrite any task attribute from the
  56. configuration. The setting can be a dict, or a list of annotation
  57. objects that filter for tasks and return a map of attributes
  58. to change.
  59. This will change the ``rate_limit`` attribute for the ``tasks.add``
  60. task:
  61. .. code-block:: python
  62. CELERY_ANNOTATIONS = {'tasks.add': {'rate_limit': '10/s'}}
  63. or change the same for all tasks:
  64. .. code-block:: python
  65. CELERY_ANNOTATIONS = {'*': {'rate_limit': '10/s'}}
  66. You can change methods too, for example the ``on_failure`` handler:
  67. .. code-block:: python
  68. def my_on_failure(self, exc, task_id, args, kwargs, einfo):
  69. print('Oh no! Task failed: {0!r}'.format(exc))
  70. CELERY_ANNOTATIONS = {'*': {'on_failure': my_on_failure}}
  71. If you need more flexibility then you can use objects
  72. instead of a dict to choose which tasks to annotate:
  73. .. code-block:: python
  74. class MyAnnotate(object):
  75. def annotate(self, task):
  76. if task.name.startswith('tasks.'):
  77. return {'rate_limit': '10/s'}
  78. CELERY_ANNOTATIONS = (MyAnnotate(), {…})
  79. .. _conf-concurrency:
  80. Concurrency settings
  81. --------------------
  82. .. setting:: CELERYD_CONCURRENCY
  83. CELERYD_CONCURRENCY
  84. ~~~~~~~~~~~~~~~~~~~
  85. The number of concurrent worker processes/threads/green threads executing
  86. tasks.
  87. If you're doing mostly I/O you can have more processes,
  88. but if mostly CPU-bound, try to keep it close to the
  89. number of CPUs on your machine. If not set, the number of CPUs/cores
  90. on the host will be used.
  91. Defaults to the number of available CPUs.
  92. .. setting:: CELERYD_PREFETCH_MULTIPLIER
  93. CELERYD_PREFETCH_MULTIPLIER
  94. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  95. How many messages to prefetch at a time multiplied by the number of
  96. concurrent processes. The default is 4 (four messages for each
  97. process). The default setting is usually a good choice, however -- if you
  98. have very long running tasks waiting in the queue and you have to start the
  99. workers, note that the first worker to start will receive four times the
  100. number of messages initially. Thus the tasks may not be fairly distributed
  101. to the workers.
  102. .. note::
  103. Tasks with ETA/countdown are not affected by prefetch limits.
  104. .. _conf-result-backend:
  105. Task result backend settings
  106. ----------------------------
  107. .. setting:: CELERY_RESULT_BACKEND
  108. CELERY_RESULT_BACKEND
  109. ~~~~~~~~~~~~~~~~~~~~~
  110. :Deprecated aliases: ``CELERY_BACKEND``
  111. The backend used to store task results (tombstones).
  112. Disabled by default.
  113. Can be one of the following:
  114. * database
  115. Use a relational database supported by `SQLAlchemy`_.
  116. See :ref:`conf-database-result-backend`.
  117. * cache
  118. Use `memcached`_ to store the results.
  119. See :ref:`conf-cache-result-backend`.
  120. * mongodb
  121. Use `MongoDB`_ to store the results.
  122. See :ref:`conf-mongodb-result-backend`.
  123. * redis
  124. Use `Redis`_ to store the results.
  125. See :ref:`conf-redis-result-backend`.
  126. * amqp
  127. Send results back as AMQP messages
  128. See :ref:`conf-amqp-result-backend`.
  129. * cassandra
  130. Use `Cassandra`_ to store the results.
  131. See :ref:`conf-cassandra-result-backend`.
  132. * ironcache
  133. Use `IronCache`_ to store the results.
  134. See :ref:`conf-ironcache-result-backend`.
  135. * couchbase
  136. Use `Couchbase`_ to store the results.
  137. See :ref:`conf-couchbase-result-backend`.
  138. .. warning:
  139. While the AMQP result backend is very efficient, you must make sure
  140. you only receive the same result once. See :doc:`userguide/calling`).
  141. .. _`SQLAlchemy`: http://sqlalchemy.org
  142. .. _`memcached`: http://memcached.org
  143. .. _`MongoDB`: http://mongodb.org
  144. .. _`Redis`: http://redis.io
  145. .. _`Cassandra`: http://cassandra.apache.org/
  146. .. _`IronCache`: http://www.iron.io/cache
  147. .. _`Couchbase`: http://www.couchbase.com/
  148. .. setting:: CELERY_RESULT_SERIALIZER
  149. CELERY_RESULT_SERIALIZER
  150. ~~~~~~~~~~~~~~~~~~~~~~~~
  151. Result serialization format. Default is ``pickle``. See
  152. :ref:`calling-serializers` for information about supported
  153. serialization formats.
  154. .. _conf-database-result-backend:
  155. Database backend settings
  156. -------------------------
  157. Database URL Examples
  158. ~~~~~~~~~~~~~~~~~~~~~
  159. To use the database backend you have to configure the
  160. :setting:`CELERY_RESULT_BACKEND` setting with a connection URL and the ``db+``
  161. prefix:
  162. .. code-block:: python
  163. CELERY_RESULT_BACKEND = 'db+scheme://user:password@host:port/dbname'
  164. Examples:
  165. # sqlite (filename)
  166. CELERY_RESULT_BACKEND = 'db+sqlite:///results.sqlite'
  167. # mysql
  168. CELERY_RESULT_BACKEND = 'db+mysql://scott:tiger@localhost/foo'
  169. # postgresql
  170. CELERY_RESULT_BACKEND = 'db+postgresql://scott:tiger@localhost/mydatabase'
  171. # oracle
  172. CELERY_RESULT_BACKEND = 'db+oracle://scott:tiger@127.0.0.1:1521/sidname'
  173. .. code-block:: python
  174. Please see `Supported Databases`_ for a table of supported databases,
  175. and `Connection String`_ for more information about connection
  176. strings (which is the part of the URI that comes after the ``db+`` prefix).
  177. .. _`Supported Databases`:
  178. http://www.sqlalchemy.org/docs/core/engines.html#supported-databases
  179. .. _`Connection String`:
  180. http://www.sqlalchemy.org/docs/core/engines.html#database-urls
  181. .. setting:: CELERY_RESULT_DBURI
  182. CELERY_RESULT_DBURI
  183. ~~~~~~~~~~~~~~~~~~~
  184. This setting is no longer used as it's now possible to specify
  185. the database URL directly in the :setting:`CELERY_RESULT_BACKEND` setting.
  186. .. setting:: CELERY_RESULT_ENGINE_OPTIONS
  187. CELERY_RESULT_ENGINE_OPTIONS
  188. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  189. To specify additional SQLAlchemy database engine options you can use
  190. the :setting:`CELERY_RESULT_ENGINE_OPTIONS` setting::
  191. # echo enables verbose logging from SQLAlchemy.
  192. CELERY_RESULT_ENGINE_OPTIONS = {'echo': True}
  193. .. setting:: CELERY_RESULT_DB_SHORT_LIVED_SESSIONS
  194. CELERY_RESULT_DB_SHORT_LIVED_SESSIONS = True
  195. Short lived sessions are disabled by default. If enabled they can drastically reduce
  196. performance, especially on systems processing lots of tasks. This option is useful
  197. on low-traffic workers that experience errors as a result of cached database connections
  198. going stale through inactivity. For example, intermittent errors like
  199. `(OperationalError) (2006, 'MySQL server has gone away')` can be fixed by enabling
  200. short lived sessions. This option only affects the database backend.
  201. Specifying Table Names
  202. ~~~~~~~~~~~~~~~~~~~~~~
  203. .. setting:: CELERY_RESULT_DB_TABLENAMES
  204. When SQLAlchemy is configured as the result backend, Celery automatically
  205. creates two tables to store result metadata for tasks. This setting allows
  206. you to customize the table names:
  207. .. code-block:: python
  208. # use custom table names for the database result backend.
  209. CELERY_RESULT_DB_TABLENAMES = {
  210. 'task': 'myapp_taskmeta',
  211. 'group': 'myapp_groupmeta',
  212. }
  213. .. _conf-amqp-result-backend:
  214. AMQP backend settings
  215. ---------------------
  216. .. note::
  217. The AMQP backend requires RabbitMQ 1.1.0 or higher to automatically
  218. expire results. If you are running an older version of RabbitmQ
  219. you should disable result expiration like this:
  220. CELERY_TASK_RESULT_EXPIRES = None
  221. .. setting:: CELERY_RESULT_EXCHANGE
  222. CELERY_RESULT_EXCHANGE
  223. ~~~~~~~~~~~~~~~~~~~~~~
  224. Name of the exchange to publish results in. Default is `celeryresults`.
  225. .. setting:: CELERY_RESULT_EXCHANGE_TYPE
  226. CELERY_RESULT_EXCHANGE_TYPE
  227. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  228. The exchange type of the result exchange. Default is to use a `direct`
  229. exchange.
  230. .. setting:: CELERY_RESULT_PERSISTENT
  231. CELERY_RESULT_PERSISTENT
  232. ~~~~~~~~~~~~~~~~~~~~~~~~
  233. If set to :const:`True`, result messages will be persistent. This means the
  234. messages will not be lost after a broker restart. The default is for the
  235. results to be transient.
  236. Example configuration
  237. ~~~~~~~~~~~~~~~~~~~~~
  238. .. code-block:: python
  239. CELERY_RESULT_BACKEND = 'amqp'
  240. CELERY_TASK_RESULT_EXPIRES = 18000 # 5 hours.
  241. .. _conf-cache-result-backend:
  242. Cache backend settings
  243. ----------------------
  244. .. note::
  245. The cache backend supports the `pylibmc`_ and `python-memcached`
  246. libraries. The latter is used only if `pylibmc`_ is not installed.
  247. Using a single memcached server:
  248. .. code-block:: python
  249. CELERY_RESULT_BACKEND = 'cache+memcached://127.0.0.1:11211/'
  250. Using multiple memcached servers:
  251. .. code-block:: python
  252. CELERY_RESULT_BACKEND = """
  253. cache+memcached://172.19.26.240:11211;172.19.26.242:11211/
  254. """.strip()
  255. .. setting:: CELERY_CACHE_BACKEND_OPTIONS
  256. The "memory" backend stores the cache in memory only:
  257. CELERY_CACHE_BACKEND = 'memory'
  258. CELERY_CACHE_BACKEND_OPTIONS
  259. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  260. You can set pylibmc options using the :setting:`CELERY_CACHE_BACKEND_OPTIONS`
  261. setting:
  262. .. code-block:: python
  263. CELERY_CACHE_BACKEND_OPTIONS = {'binary': True,
  264. 'behaviors': {'tcp_nodelay': True}}
  265. .. _`pylibmc`: http://sendapatch.se/projects/pylibmc/
  266. .. setting:: CELERY_CACHE_BACKEND
  267. CELERY_CACHE_BACKEND
  268. ~~~~~~~~~~~~~~~~~~~~
  269. This setting is no longer used as it's now possible to specify
  270. the cache backend directly in the :setting:`CELERY_RESULT_BACKEND` setting.
  271. .. _conf-redis-result-backend:
  272. Redis backend settings
  273. ----------------------
  274. Configuring the backend URL
  275. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  276. .. note::
  277. The Redis backend requires the :mod:`redis` library:
  278. http://pypi.python.org/pypi/redis/
  279. To install the redis package use `pip` or `easy_install`:
  280. .. code-block:: bash
  281. $ pip install redis
  282. This backend requires the :setting:`CELERY_RESULT_BACKEND`
  283. setting to be set to a Redis URL::
  284. CELERY_RESULT_BACKEND = 'redis://:password@host:port/db'
  285. For example::
  286. CELERY_RESULT_BACKEND = 'redis://localhost/0'
  287. which is the same as::
  288. CELERY_RESULT_BACKEND = 'redis://'
  289. The fields of the URL is defined as folows:
  290. - *host*
  291. Host name or IP address of the Redis server. e.g. `localhost`.
  292. - *port*
  293. Port to the Redis server. Default is 6379.
  294. - *db*
  295. Database number to use. Default is 0.
  296. The db can include an optional leading slash.
  297. - *password*
  298. Password used to connect to the database.
  299. .. setting:: CELERY_REDIS_MAX_CONNECTIONS
  300. CELERY_REDIS_MAX_CONNECTIONS
  301. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  302. Maximum number of connections available in the Redis connection
  303. pool used for sending and retrieving results.
  304. .. _conf-mongodb-result-backend:
  305. MongoDB backend settings
  306. ------------------------
  307. .. note::
  308. The MongoDB backend requires the :mod:`pymongo` library:
  309. http://github.com/mongodb/mongo-python-driver/tree/master
  310. .. setting:: CELERY_MONGODB_BACKEND_SETTINGS
  311. CELERY_MONGODB_BACKEND_SETTINGS
  312. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  313. This is a dict supporting the following keys:
  314. * database
  315. The database name to connect to. Defaults to ``celery``.
  316. * taskmeta_collection
  317. The collection name to store task meta data.
  318. Defaults to ``celery_taskmeta``.
  319. * max_pool_size
  320. Passed as max_pool_size to PyMongo's Connection or MongoClient
  321. constructor. It is the maximum number of TCP connections to keep
  322. open to MongoDB at a given time. If there are more open connections
  323. than max_pool_size, sockets will be closed when they are released.
  324. Defaults to 10.
  325. * options
  326. Additional keyword arguments to pass to the mongodb connection
  327. constructor. See the :mod:`pymongo` docs to see a list of arguments
  328. supported.
  329. .. _example-mongodb-result-config:
  330. Example configuration
  331. ~~~~~~~~~~~~~~~~~~~~~
  332. .. code-block:: python
  333. CELERY_RESULT_BACKEND = 'mongodb://192.168.1.100:30000/'
  334. CELERY_MONGODB_BACKEND_SETTINGS = {
  335. 'database': 'mydb',
  336. 'taskmeta_collection': 'my_taskmeta_collection',
  337. }
  338. .. _conf-cassandra-result-backend:
  339. Cassandra backend settings
  340. --------------------------
  341. .. note::
  342. The Cassandra backend requires the :mod:`pycassa` library:
  343. http://pypi.python.org/pypi/pycassa/
  344. To install the pycassa package use `pip` or `easy_install`:
  345. .. code-block:: bash
  346. $ pip install pycassa
  347. This backend requires the following configuration directives to be set.
  348. .. setting:: CASSANDRA_SERVERS
  349. CASSANDRA_SERVERS
  350. ~~~~~~~~~~~~~~~~~
  351. List of ``host:port`` Cassandra servers. e.g.::
  352. CASSANDRA_SERVERS = ['localhost:9160']
  353. .. setting:: CASSANDRA_KEYSPACE
  354. CASSANDRA_KEYSPACE
  355. ~~~~~~~~~~~~~~~~~~
  356. The keyspace in which to store the results. e.g.::
  357. CASSANDRA_KEYSPACE = 'tasks_keyspace'
  358. .. setting:: CASSANDRA_COLUMN_FAMILY
  359. CASSANDRA_COLUMN_FAMILY
  360. ~~~~~~~~~~~~~~~~~~~~~~~
  361. The column family in which to store the results. e.g.::
  362. CASSANDRA_COLUMN_FAMILY = 'tasks'
  363. .. setting:: CASSANDRA_READ_CONSISTENCY
  364. CASSANDRA_READ_CONSISTENCY
  365. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  366. The read consistency used. Values can be ``ONE``, ``QUORUM`` or ``ALL``.
  367. .. setting:: CASSANDRA_WRITE_CONSISTENCY
  368. CASSANDRA_WRITE_CONSISTENCY
  369. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  370. The write consistency used. Values can be ``ONE``, ``QUORUM`` or ``ALL``.
  371. .. setting:: CASSANDRA_DETAILED_MODE
  372. CASSANDRA_DETAILED_MODE
  373. ~~~~~~~~~~~~~~~~~~~~~~~
  374. Enable or disable detailed mode. Default is :const:`False`.
  375. This mode allows to use the power of Cassandra wide columns to
  376. store all states for a task as a wide column, instead of only the last one.
  377. To use this mode, you need to configure your ColumnFamily to
  378. use the ``TimeUUID`` type as a comparator::
  379. create column family task_results with comparator = TimeUUIDType;
  380. CASSANDRA_OPTIONS
  381. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  382. Options to be passed to the `pycassa connection pool`_ (optional).
  383. .. _`pycassa connection pool`: http://pycassa.github.com/pycassa/api/pycassa/pool.html
  384. Example configuration
  385. ~~~~~~~~~~~~~~~~~~~~~
  386. .. code-block:: python
  387. CASSANDRA_SERVERS = ['localhost:9160']
  388. CASSANDRA_KEYSPACE = 'celery'
  389. CASSANDRA_COLUMN_FAMILY = 'task_results'
  390. CASSANDRA_READ_CONSISTENCY = 'ONE'
  391. CASSANDRA_WRITE_CONSISTENCY = 'ONE'
  392. CASSANDRA_DETAILED_MODE = True
  393. CASSANDRA_OPTIONS = {
  394. 'timeout': 300,
  395. 'max_retries': 10
  396. }
  397. .. _conf-ironcache-result-backend:
  398. IronCache backend settings
  399. --------------------------
  400. .. note::
  401. The IronCache backend requires the :mod:`iron_celery` library:
  402. http://pypi.python.org/pypi/iron_celery
  403. To install the iron_celery package use `pip` or `easy_install`:
  404. .. code-block:: bash
  405. $ pip install iron_celery
  406. IronCache is configured via the URL provided in :setting:`CELERY_RESULT_BACKEND`, for example::
  407. CELERY_RESULT_BACKEND = 'ironcache://project_id:token@'
  408. Or to change the cache name::
  409. ironcache:://project_id:token@/awesomecache
  410. For more information, see: https://github.com/iron-io/iron_celery
  411. .. _conf-couchbase-result-backend:
  412. Couchbase backend settings
  413. --------------------------
  414. .. note::
  415. The Couchbase backend requires the :mod:`couchbase` library:
  416. https://pypi.python.org/pypi/couchbase
  417. To install the couchbase package use `pip` or `easy_install`:
  418. .. code-block:: bash
  419. $ pip install couchbase
  420. This backend can be configured via the :setting:`CELERY_RESULT_BACKEND`
  421. set to a couchbase URL::
  422. CELERY_RESULT_BACKEND = 'couchbase://username:password@host:port/bucket'
  423. .. setting:: CELERY_COUCHBASE_BACKEND_SETTINGS
  424. CELERY_COUCHBASE_BACKEND_SETTINGS
  425. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  426. This is a dict supporting the following keys:
  427. * host
  428. Host name of the Couchbase server. Defaults to ``localhost``.
  429. * port
  430. The port the Couchbase server is listening to. Defaults to ``8091``.
  431. * bucket
  432. The default bucket the Couchbase server is writing to.
  433. Defaults to ``default``.
  434. * username
  435. User name to authenticate to the Couchbase server as (optional).
  436. * password
  437. Password to authenticate to the Couchbase server (optional).
  438. .. _conf-messaging:
  439. Message Routing
  440. ---------------
  441. .. _conf-messaging-routing:
  442. .. setting:: CELERY_QUEUES
  443. CELERY_QUEUES
  444. ~~~~~~~~~~~~~
  445. Most users will not want to specify this setting and should rather use
  446. the :ref:`automatic routing facilities <routing-automatic>`.
  447. If you really want to configure advanced routing, this setting should
  448. be a list of :class:`kombu.Queue` objects the worker will consume from.
  449. Note that workers can be overriden this setting via the `-Q` option,
  450. or individual queues from this list (by name) can be excluded using
  451. the `-X` option.
  452. Also see :ref:`routing-basics` for more information.
  453. The default is a queue/exchange/binding key of ``celery``, with
  454. exchange type ``direct``.
  455. .. setting:: CELERY_ROUTES
  456. CELERY_ROUTES
  457. ~~~~~~~~~~~~~
  458. A list of routers, or a single router used to route tasks to queues.
  459. When deciding the final destination of a task the routers are consulted
  460. in order. See :ref:`routers` for more information.
  461. .. setting:: CELERY_QUEUE_HA_POLICY
  462. CELERY_QUEUE_HA_POLICY
  463. ~~~~~~~~~~~~~~~~~~~~~~
  464. :brokers: RabbitMQ
  465. This will set the default HA policy for a queue, and the value
  466. can either be a string (usually ``all``):
  467. .. code-block:: python
  468. CELERY_QUEUE_HA_POLICY = 'all'
  469. Using 'all' will replicate the queue to all current nodes,
  470. Or you can give it a list of nodes to replicate to:
  471. .. code-block:: python
  472. CELERY_QUEUE_HA_POLICY = ['rabbit@host1', 'rabbit@host2']
  473. Using a list will implicitly set ``x-ha-policy`` to 'nodes' and
  474. ``x-ha-policy-params`` to the given list of nodes.
  475. See http://www.rabbitmq.com/ha.html for more information.
  476. .. setting:: CELERY_WORKER_DIRECT
  477. CELERY_WORKER_DIRECT
  478. ~~~~~~~~~~~~~~~~~~~~
  479. This option enables so that every worker has a dedicated queue,
  480. so that tasks can be routed to specific workers.
  481. The queue name for each worker is automatically generated based on
  482. the worker hostname and a ``.dq`` suffix, using the ``C.dq`` exchange.
  483. For example the queue name for the worker with node name ``w1@example.com``
  484. becomes::
  485. w1@example.com.dq
  486. Then you can route the task to the task by specifying the hostname
  487. as the routing key and the ``C.dq`` exchange::
  488. CELERY_ROUTES = {
  489. 'tasks.add': {'exchange': 'C.dq', 'routing_key': 'w1@example.com'}
  490. }
  491. .. setting:: CELERY_CREATE_MISSING_QUEUES
  492. CELERY_CREATE_MISSING_QUEUES
  493. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  494. If enabled (default), any queues specified that are not defined in
  495. :setting:`CELERY_QUEUES` will be automatically created. See
  496. :ref:`routing-automatic`.
  497. .. setting:: CELERY_DEFAULT_QUEUE
  498. CELERY_DEFAULT_QUEUE
  499. ~~~~~~~~~~~~~~~~~~~~
  500. The name of the default queue used by `.apply_async` if the message has
  501. no route or no custom queue has been specified.
  502. This queue must be listed in :setting:`CELERY_QUEUES`.
  503. If :setting:`CELERY_QUEUES` is not specified then it is automatically
  504. created containing one queue entry, where this name is used as the name of
  505. that queue.
  506. The default is: `celery`.
  507. .. seealso::
  508. :ref:`routing-changing-default-queue`
  509. .. setting:: CELERY_DEFAULT_EXCHANGE
  510. CELERY_DEFAULT_EXCHANGE
  511. ~~~~~~~~~~~~~~~~~~~~~~~
  512. Name of the default exchange to use when no custom exchange is
  513. specified for a key in the :setting:`CELERY_QUEUES` setting.
  514. The default is: `celery`.
  515. .. setting:: CELERY_DEFAULT_EXCHANGE_TYPE
  516. CELERY_DEFAULT_EXCHANGE_TYPE
  517. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  518. Default exchange type used when no custom exchange type is specified
  519. for a key in the :setting:`CELERY_QUEUES` setting.
  520. The default is: `direct`.
  521. .. setting:: CELERY_DEFAULT_ROUTING_KEY
  522. CELERY_DEFAULT_ROUTING_KEY
  523. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  524. The default routing key used when no custom routing key
  525. is specified for a key in the :setting:`CELERY_QUEUES` setting.
  526. The default is: `celery`.
  527. .. setting:: CELERY_DEFAULT_DELIVERY_MODE
  528. CELERY_DEFAULT_DELIVERY_MODE
  529. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  530. Can be `transient` or `persistent`. The default is to send
  531. persistent messages.
  532. .. _conf-broker-settings:
  533. Broker Settings
  534. ---------------
  535. .. setting:: CELERY_ACCEPT_CONTENT
  536. CELERY_ACCEPT_CONTENT
  537. ~~~~~~~~~~~~~~~~~~~~~
  538. A whitelist of content-types/serializers to allow.
  539. If a message is received that is not in this list then
  540. the message will be discarded with an error.
  541. By default any content type is enabled (including pickle and yaml)
  542. so make sure untrusted parties do not have access to your broker.
  543. See :ref:`guide-security` for more.
  544. Example::
  545. # using serializer name
  546. CELERY_ACCEPT_CONTENT = ['json']
  547. # or the actual content-type (MIME)
  548. CELERY_ACCEPT_CONTENT = ['application/json']
  549. .. setting:: BROKER_FAILOVER_STRATEGY
  550. BROKER_FAILOVER_STRATEGY
  551. ~~~~~~~~~~~~~~~~~~~~~~~~
  552. Default failover strategy for the broker Connection object. If supplied,
  553. may map to a key in 'kombu.connection.failover_strategies', or be a reference
  554. to any method that yields a single item from a supplied list.
  555. Example::
  556. # Random failover strategy
  557. def random_failover_strategy(servers):
  558. it = list(it) # don't modify callers list
  559. shuffle = random.shuffle
  560. for _ in repeat(None):
  561. shuffle(it)
  562. yield it[0]
  563. BROKER_FAILOVER_STRATEGY=random_failover_strategy
  564. .. setting:: BROKER_TRANSPORT
  565. BROKER_TRANSPORT
  566. ~~~~~~~~~~~~~~~~
  567. :Aliases: ``BROKER_BACKEND``
  568. :Deprecated aliases: ``CARROT_BACKEND``
  569. .. setting:: BROKER_URL
  570. BROKER_URL
  571. ~~~~~~~~~~
  572. Default broker URL. This must be an URL in the form of::
  573. transport://userid:password@hostname:port/virtual_host
  574. Only the scheme part (``transport://``) is required, the rest
  575. is optional, and defaults to the specific transports default values.
  576. The transport part is the broker implementation to use, and the
  577. default is ``amqp``, which uses ``librabbitmq`` by default or falls back to
  578. ``pyamqp`` if that is not installed. Also there are many other choices including
  579. ``redis``, ``beanstalk``, ``sqlalchemy``, ``django``, ``mongodb``,
  580. ``couchdb``.
  581. It can also be a fully qualified path to your own transport implementation.
  582. See :ref:`kombu:connection-urls` in the Kombu documentation for more
  583. information.
  584. .. setting:: BROKER_HEARTBEAT
  585. BROKER_HEARTBEAT
  586. ~~~~~~~~~~~~~~~~
  587. :transports supported: ``pyamqp``
  588. It's not always possible to detect connection loss in a timely
  589. manner using TCP/IP alone, so AMQP defines something called heartbeats
  590. that's is used both by the client and the broker to detect if
  591. a connection was closed.
  592. Hartbeats are disabled by default.
  593. If the heartbeat value is 10 seconds, then
  594. the heartbeat will be monitored at the interval specified
  595. by the :setting:`BROKER_HEARTBEAT_CHECKRATE` setting, which by default is
  596. double the rate of the heartbeat value
  597. (so for the default 10 seconds, the heartbeat is checked every 5 seconds).
  598. .. setting:: BROKER_HEARTBEAT_CHECKRATE
  599. BROKER_HEARTBEAT_CHECKRATE
  600. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  601. :transports supported: ``pyamqp``
  602. At intervals the worker will monitor that the broker has not missed
  603. too many heartbeats. The rate at which this is checked is calculated
  604. by dividing the :setting:`BROKER_HEARTBEAT` value with this value,
  605. so if the heartbeat is 10.0 and the rate is the default 2.0, the check
  606. will be performed every 5 seconds (twice the heartbeat sending rate).
  607. .. setting:: BROKER_USE_SSL
  608. BROKER_USE_SSL
  609. ~~~~~~~~~~~~~~
  610. Use SSL to connect to the broker. Off by default. This may not be supported
  611. by all transports.
  612. .. setting:: BROKER_POOL_LIMIT
  613. BROKER_POOL_LIMIT
  614. ~~~~~~~~~~~~~~~~~
  615. .. versionadded:: 2.3
  616. The maximum number of connections that can be open in the connection pool.
  617. The pool is enabled by default since version 2.5, with a default limit of ten
  618. connections. This number can be tweaked depending on the number of
  619. threads/greenthreads (eventlet/gevent) using a connection. For example
  620. running eventlet with 1000 greenlets that use a connection to the broker,
  621. contention can arise and you should consider increasing the limit.
  622. If set to :const:`None` or 0 the connection pool will be disabled and
  623. connections will be established and closed for every use.
  624. Default (since 2.5) is to use a pool of 10 connections.
  625. .. setting:: BROKER_CONNECTION_TIMEOUT
  626. BROKER_CONNECTION_TIMEOUT
  627. ~~~~~~~~~~~~~~~~~~~~~~~~~
  628. The default timeout in seconds before we give up establishing a connection
  629. to the AMQP server. Default is 4 seconds.
  630. .. setting:: BROKER_CONNECTION_RETRY
  631. BROKER_CONNECTION_RETRY
  632. ~~~~~~~~~~~~~~~~~~~~~~~
  633. Automatically try to re-establish the connection to the AMQP broker if lost.
  634. The time between retries is increased for each retry, and is
  635. not exhausted before :setting:`BROKER_CONNECTION_MAX_RETRIES` is
  636. exceeded.
  637. This behavior is on by default.
  638. .. setting:: BROKER_CONNECTION_MAX_RETRIES
  639. BROKER_CONNECTION_MAX_RETRIES
  640. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  641. Maximum number of retries before we give up re-establishing a connection
  642. to the AMQP broker.
  643. If this is set to :const:`0` or :const:`None`, we will retry forever.
  644. Default is 100 retries.
  645. .. setting:: BROKER_LOGIN_METHOD
  646. BROKER_LOGIN_METHOD
  647. ~~~~~~~~~~~~~~~~~~~
  648. Set custom amqp login method, default is ``AMQPLAIN``.
  649. .. setting:: BROKER_TRANSPORT_OPTIONS
  650. BROKER_TRANSPORT_OPTIONS
  651. ~~~~~~~~~~~~~~~~~~~~~~~~
  652. .. versionadded:: 2.2
  653. A dict of additional options passed to the underlying transport.
  654. See your transport user manual for supported options (if any).
  655. Example setting the visibility timeout (supported by Redis and SQS
  656. transports):
  657. .. code-block:: python
  658. BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 18000} # 5 hours
  659. .. _conf-task-execution:
  660. Task execution settings
  661. -----------------------
  662. .. setting:: CELERY_ALWAYS_EAGER
  663. CELERY_ALWAYS_EAGER
  664. ~~~~~~~~~~~~~~~~~~~
  665. If this is :const:`True`, all tasks will be executed locally by blocking until
  666. the task returns. ``apply_async()`` and ``Task.delay()`` will return
  667. an :class:`~celery.result.EagerResult` instance, which emulates the API
  668. and behavior of :class:`~celery.result.AsyncResult`, except the result
  669. is already evaluated.
  670. That is, tasks will be executed locally instead of being sent to
  671. the queue.
  672. .. setting:: CELERY_EAGER_PROPAGATES_EXCEPTIONS
  673. CELERY_EAGER_PROPAGATES_EXCEPTIONS
  674. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  675. If this is :const:`True`, eagerly executed tasks (applied by `task.apply()`,
  676. or when the :setting:`CELERY_ALWAYS_EAGER` setting is enabled), will
  677. propagate exceptions.
  678. It's the same as always running ``apply()`` with ``throw=True``.
  679. .. setting:: CELERY_IGNORE_RESULT
  680. CELERY_IGNORE_RESULT
  681. ~~~~~~~~~~~~~~~~~~~~
  682. Whether to store the task return values or not (tombstones).
  683. If you still want to store errors, just not successful return values,
  684. you can set :setting:`CELERY_STORE_ERRORS_EVEN_IF_IGNORED`.
  685. .. setting:: CELERY_MESSAGE_COMPRESSION
  686. CELERY_MESSAGE_COMPRESSION
  687. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  688. Default compression used for task messages.
  689. Can be ``gzip``, ``bzip2`` (if available), or any custom
  690. compression schemes registered in the Kombu compression registry.
  691. The default is to send uncompressed messages.
  692. .. setting:: CELERY_TASK_RESULT_EXPIRES
  693. CELERY_TASK_RESULT_EXPIRES
  694. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  695. Time (in seconds, or a :class:`~datetime.timedelta` object) for when after
  696. stored task tombstones will be deleted.
  697. A built-in periodic task will delete the results after this time
  698. (:class:`celery.task.backend_cleanup`).
  699. A value of :const:`None` or 0 means results will never expire (depending
  700. on backend specifications).
  701. Default is to expire after 1 day.
  702. .. note::
  703. For the moment this only works with the amqp, database, cache, redis and MongoDB
  704. backends.
  705. When using the database or MongoDB backends, `celery beat` must be
  706. running for the results to be expired.
  707. .. setting:: CELERY_MAX_CACHED_RESULTS
  708. CELERY_MAX_CACHED_RESULTS
  709. ~~~~~~~~~~~~~~~~~~~~~~~~~
  710. Result backends caches ready results used by the client.
  711. This is the total number of results to cache before older results are evicted.
  712. The default is 5000. 0 or None means no limit, and a value of :const:`-1`
  713. will disable the cache.
  714. .. setting:: CELERY_CHORD_PROPAGATES
  715. CELERY_CHORD_PROPAGATES
  716. ~~~~~~~~~~~~~~~~~~~~~~~
  717. .. versionadded:: 3.0.14
  718. This setting defines what happens when a task part of a chord raises an
  719. exception:
  720. - If propagate is True the chord callback will change state to FAILURE
  721. with the exception value set to a :exc:`~@ChordError`
  722. instance containing information about the error and the task that failed.
  723. This is the default behavior in Celery 3.1+
  724. - If propagate is False the exception value will instead be forwarded
  725. to the chord callback.
  726. This was the default behavior before version 3.1.
  727. .. setting:: CELERY_TRACK_STARTED
  728. CELERY_TRACK_STARTED
  729. ~~~~~~~~~~~~~~~~~~~~
  730. If :const:`True` the task will report its status as "started" when the
  731. task is executed by a worker. The default value is :const:`False` as
  732. the normal behaviour is to not report that level of granularity. Tasks
  733. are either pending, finished, or waiting to be retried. Having a "started"
  734. state can be useful for when there are long running tasks and there is a
  735. need to report which task is currently running.
  736. .. setting:: CELERY_TASK_SERIALIZER
  737. CELERY_TASK_SERIALIZER
  738. ~~~~~~~~~~~~~~~~~~~~~~
  739. A string identifying the default serialization method to use. Can be
  740. `pickle` (default), `json`, `yaml`, `msgpack` or any custom serialization
  741. methods that have been registered with :mod:`kombu.serialization.registry`.
  742. .. seealso::
  743. :ref:`calling-serializers`.
  744. .. setting:: CELERY_TASK_PUBLISH_RETRY
  745. CELERY_TASK_PUBLISH_RETRY
  746. ~~~~~~~~~~~~~~~~~~~~~~~~~
  747. .. versionadded:: 2.2
  748. Decides if publishing task messages will be retried in the case
  749. of connection loss or other connection errors.
  750. See also :setting:`CELERY_TASK_PUBLISH_RETRY_POLICY`.
  751. Enabled by default.
  752. .. setting:: CELERY_TASK_PUBLISH_RETRY_POLICY
  753. CELERY_TASK_PUBLISH_RETRY_POLICY
  754. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  755. .. versionadded:: 2.2
  756. Defines the default policy when retrying publishing a task message in
  757. the case of connection loss or other connection errors.
  758. See :ref:`calling-retry` for more information.
  759. .. setting:: CELERY_DEFAULT_RATE_LIMIT
  760. CELERY_DEFAULT_RATE_LIMIT
  761. ~~~~~~~~~~~~~~~~~~~~~~~~~
  762. The global default rate limit for tasks.
  763. This value is used for tasks that does not have a custom rate limit
  764. The default is no rate limit.
  765. .. setting:: CELERY_DISABLE_RATE_LIMITS
  766. CELERY_DISABLE_RATE_LIMITS
  767. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  768. Disable all rate limits, even if tasks has explicit rate limits set.
  769. .. setting:: CELERY_ACKS_LATE
  770. CELERY_ACKS_LATE
  771. ~~~~~~~~~~~~~~~~
  772. Late ack means the task messages will be acknowledged **after** the task
  773. has been executed, not *just before*, which is the default behavior.
  774. .. seealso::
  775. FAQ: :ref:`faq-acks_late-vs-retry`.
  776. .. _conf-worker:
  777. Worker
  778. ------
  779. .. setting:: CELERY_IMPORTS
  780. CELERY_IMPORTS
  781. ~~~~~~~~~~~~~~
  782. A sequence of modules to import when the worker starts.
  783. This is used to specify the task modules to import, but also
  784. to import signal handlers and additional remote control commands, etc.
  785. The modules will be imported in the original order.
  786. .. setting:: CELERY_INCLUDE
  787. CELERY_INCLUDE
  788. ~~~~~~~~~~~~~~
  789. Exact same semantics as :setting:`CELERY_IMPORTS`, but can be used as a means
  790. to have different import categories.
  791. The modules in this setting are imported after the modules in
  792. :setting:`CELERY_IMPORTS`.
  793. .. setting:: CELERYD_WORKER_LOST_WAIT
  794. CELERYD_WORKER_LOST_WAIT
  795. ~~~~~~~~~~~~~~~~~~~~~~~~
  796. In some cases a worker may be killed without proper cleanup,
  797. and the worker may have published a result before terminating.
  798. This value specifies how long we wait for any missing results before
  799. raising a :exc:`@WorkerLostError` exception.
  800. Default is 10.0
  801. .. setting:: CELERYD_MAX_TASKS_PER_CHILD
  802. CELERYD_MAX_TASKS_PER_CHILD
  803. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  804. Maximum number of tasks a pool worker process can execute before
  805. it's replaced with a new one. Default is no limit.
  806. .. setting:: CELERYD_TASK_TIME_LIMIT
  807. CELERYD_TASK_TIME_LIMIT
  808. ~~~~~~~~~~~~~~~~~~~~~~~
  809. Task hard time limit in seconds. The worker processing the task will
  810. be killed and replaced with a new one when this is exceeded.
  811. .. setting:: CELERYD_TASK_SOFT_TIME_LIMIT
  812. CELERYD_TASK_SOFT_TIME_LIMIT
  813. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  814. Task soft time limit in seconds.
  815. The :exc:`~@SoftTimeLimitExceeded` exception will be
  816. raised when this is exceeded. The task can catch this to
  817. e.g. clean up before the hard time limit comes.
  818. Example:
  819. .. code-block:: python
  820. from celery.exceptions import SoftTimeLimitExceeded
  821. @app.task
  822. def mytask():
  823. try:
  824. return do_work()
  825. except SoftTimeLimitExceeded:
  826. cleanup_in_a_hurry()
  827. .. setting:: CELERY_STORE_ERRORS_EVEN_IF_IGNORED
  828. CELERY_STORE_ERRORS_EVEN_IF_IGNORED
  829. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  830. If set, the worker stores all task errors in the result store even if
  831. :attr:`Task.ignore_result <celery.task.base.Task.ignore_result>` is on.
  832. .. setting:: CELERYD_STATE_DB
  833. CELERYD_STATE_DB
  834. ~~~~~~~~~~~~~~~~
  835. Name of the file used to stores persistent worker state (like revoked tasks).
  836. Can be a relative or absolute path, but be aware that the suffix `.db`
  837. may be appended to the file name (depending on Python version).
  838. Can also be set via the :option:`--statedb` argument to
  839. :mod:`~celery.bin.worker`.
  840. Not enabled by default.
  841. .. setting:: CELERYD_TIMER_PRECISION
  842. CELERYD_TIMER_PRECISION
  843. ~~~~~~~~~~~~~~~~~~~~~~~
  844. Set the maximum time in seconds that the ETA scheduler can sleep between
  845. rechecking the schedule. Default is 1 second.
  846. Setting this value to 1 second means the schedulers precision will
  847. be 1 second. If you need near millisecond precision you can set this to 0.1.
  848. .. setting:: CELERY_ENABLE_REMOTE_CONTROL
  849. CELERY_ENABLE_REMOTE_CONTROL
  850. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  851. Specify if remote control of the workers is enabled.
  852. Default is :const:`True`.
  853. .. _conf-error-mails:
  854. Error E-Mails
  855. -------------
  856. .. setting:: CELERY_SEND_TASK_ERROR_EMAILS
  857. CELERY_SEND_TASK_ERROR_EMAILS
  858. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  859. The default value for the `Task.send_error_emails` attribute, which if
  860. set to :const:`True` means errors occurring during task execution will be
  861. sent to :setting:`ADMINS` by email.
  862. Disabled by default.
  863. .. setting:: ADMINS
  864. ADMINS
  865. ~~~~~~
  866. List of `(name, email_address)` tuples for the administrators that should
  867. receive error emails.
  868. .. setting:: SERVER_EMAIL
  869. SERVER_EMAIL
  870. ~~~~~~~~~~~~
  871. The email address this worker sends emails from.
  872. Default is celery@localhost.
  873. .. setting:: EMAIL_HOST
  874. EMAIL_HOST
  875. ~~~~~~~~~~
  876. The mail server to use. Default is ``localhost``.
  877. .. setting:: EMAIL_HOST_USER
  878. EMAIL_HOST_USER
  879. ~~~~~~~~~~~~~~~
  880. User name (if required) to log on to the mail server with.
  881. .. setting:: EMAIL_HOST_PASSWORD
  882. EMAIL_HOST_PASSWORD
  883. ~~~~~~~~~~~~~~~~~~~
  884. Password (if required) to log on to the mail server with.
  885. .. setting:: EMAIL_PORT
  886. EMAIL_PORT
  887. ~~~~~~~~~~
  888. The port the mail server is listening on. Default is `25`.
  889. .. setting:: EMAIL_USE_SSL
  890. EMAIL_USE_SSL
  891. ~~~~~~~~~~~~~
  892. Use SSL when connecting to the SMTP server. Disabled by default.
  893. .. setting:: EMAIL_USE_TLS
  894. EMAIL_USE_TLS
  895. ~~~~~~~~~~~~~
  896. Use TLS when connecting to the SMTP server. Disabled by default.
  897. .. setting:: EMAIL_TIMEOUT
  898. EMAIL_TIMEOUT
  899. ~~~~~~~~~~~~~
  900. Timeout in seconds for when we give up trying to connect
  901. to the SMTP server when sending emails.
  902. The default is 2 seconds.
  903. .. _conf-example-error-mail-config:
  904. Example E-Mail configuration
  905. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  906. This configuration enables the sending of error emails to
  907. george@vandelay.com and kramer@vandelay.com:
  908. .. code-block:: python
  909. # Enables error emails.
  910. CELERY_SEND_TASK_ERROR_EMAILS = True
  911. # Name and email addresses of recipients
  912. ADMINS = (
  913. ('George Costanza', 'george@vandelay.com'),
  914. ('Cosmo Kramer', 'kosmo@vandelay.com'),
  915. )
  916. # Email address used as sender (From field).
  917. SERVER_EMAIL = 'no-reply@vandelay.com'
  918. # Mailserver configuration
  919. EMAIL_HOST = 'mail.vandelay.com'
  920. EMAIL_PORT = 25
  921. # EMAIL_HOST_USER = 'servers'
  922. # EMAIL_HOST_PASSWORD = 's3cr3t'
  923. .. _conf-events:
  924. Events
  925. ------
  926. .. setting:: CELERY_SEND_EVENTS
  927. CELERY_SEND_EVENTS
  928. ~~~~~~~~~~~~~~~~~~
  929. Send events so the worker can be monitored by tools like `celerymon`.
  930. .. setting:: CELERY_SEND_TASK_SENT_EVENT
  931. CELERY_SEND_TASK_SENT_EVENT
  932. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  933. .. versionadded:: 2.2
  934. If enabled, a :event:`task-sent` event will be sent for every task so tasks can be
  935. tracked before they are consumed by a worker.
  936. Disabled by default.
  937. .. setting:: CELERY_EVENT_QUEUE_TTL
  938. CELERY_EVENT_QUEUE_TTL
  939. ~~~~~~~~~~~~~~~~~~~~~~
  940. :transports supported: ``amqp``
  941. Message expiry time in seconds (int/float) for when messages sent to a monitor clients
  942. event queue is deleted (``x-message-ttl``)
  943. For example, if this value is set to 10 then a message delivered to this queue
  944. will be deleted after 10 seconds.
  945. Disabled by default.
  946. .. setting:: CELERY_EVENT_QUEUE_EXPIRES
  947. CELERY_EVENT_QUEUE_EXPIRES
  948. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  949. :transports supported: ``amqp``
  950. Expiry time in seconds (int/float) for when a monitor clients
  951. event queue will be deleted (``x-expires``).
  952. Default is never, relying on the queue autodelete setting.
  953. .. setting:: CELERY_EVENT_SERIALIZER
  954. CELERY_EVENT_SERIALIZER
  955. ~~~~~~~~~~~~~~~~~~~~~~~
  956. Message serialization format used when sending event messages.
  957. Default is ``json``. See :ref:`calling-serializers`.
  958. .. _conf-broadcast:
  959. Broadcast Commands
  960. ------------------
  961. .. setting:: CELERY_BROADCAST_QUEUE
  962. CELERY_BROADCAST_QUEUE
  963. ~~~~~~~~~~~~~~~~~~~~~~
  964. Name prefix for the queue used when listening for broadcast messages.
  965. The workers host name will be appended to the prefix to create the final
  966. queue name.
  967. Default is ``celeryctl``.
  968. .. setting:: CELERY_BROADCAST_EXCHANGE
  969. CELERY_BROADCAST_EXCHANGE
  970. ~~~~~~~~~~~~~~~~~~~~~~~~~
  971. Name of the exchange used for broadcast messages.
  972. Default is ``celeryctl``.
  973. .. setting:: CELERY_BROADCAST_EXCHANGE_TYPE
  974. CELERY_BROADCAST_EXCHANGE_TYPE
  975. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  976. Exchange type used for broadcast messages. Default is ``fanout``.
  977. .. _conf-logging:
  978. Logging
  979. -------
  980. .. setting:: CELERYD_HIJACK_ROOT_LOGGER
  981. CELERYD_HIJACK_ROOT_LOGGER
  982. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  983. .. versionadded:: 2.2
  984. By default any previously configured handlers on the root logger will be
  985. removed. If you want to customize your own logging handlers, then you
  986. can disable this behavior by setting
  987. `CELERYD_HIJACK_ROOT_LOGGER = False`.
  988. .. note::
  989. Logging can also be customized by connecting to the
  990. :signal:`celery.signals.setup_logging` signal.
  991. .. setting:: CELERYD_LOG_COLOR
  992. CELERYD_LOG_COLOR
  993. ~~~~~~~~~~~~~~~~~
  994. Enables/disables colors in logging output by the Celery apps.
  995. By default colors are enabled if
  996. 1) the app is logging to a real terminal, and not a file.
  997. 2) the app is not running on Windows.
  998. .. setting:: CELERYD_LOG_FORMAT
  999. CELERYD_LOG_FORMAT
  1000. ~~~~~~~~~~~~~~~~~~
  1001. The format to use for log messages.
  1002. Default is `[%(asctime)s: %(levelname)s/%(processName)s] %(message)s`
  1003. See the Python :mod:`logging` module for more information about log
  1004. formats.
  1005. .. setting:: CELERYD_TASK_LOG_FORMAT
  1006. CELERYD_TASK_LOG_FORMAT
  1007. ~~~~~~~~~~~~~~~~~~~~~~~
  1008. The format to use for log messages logged in tasks. Can be overridden using
  1009. the :option:`--loglevel` option to :mod:`~celery.bin.worker`.
  1010. Default is::
  1011. [%(asctime)s: %(levelname)s/%(processName)s]
  1012. [%(task_name)s(%(task_id)s)] %(message)s
  1013. See the Python :mod:`logging` module for more information about log
  1014. formats.
  1015. .. setting:: CELERY_REDIRECT_STDOUTS
  1016. CELERY_REDIRECT_STDOUTS
  1017. ~~~~~~~~~~~~~~~~~~~~~~~
  1018. If enabled `stdout` and `stderr` will be redirected
  1019. to the current logger.
  1020. Enabled by default.
  1021. Used by :program:`celery worker` and :program:`celery beat`.
  1022. .. setting:: CELERY_REDIRECT_STDOUTS_LEVEL
  1023. CELERY_REDIRECT_STDOUTS_LEVEL
  1024. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  1025. The log level output to `stdout` and `stderr` is logged as.
  1026. Can be one of :const:`DEBUG`, :const:`INFO`, :const:`WARNING`,
  1027. :const:`ERROR` or :const:`CRITICAL`.
  1028. Default is :const:`WARNING`.
  1029. .. _conf-security:
  1030. Security
  1031. --------
  1032. .. setting:: CELERY_SECURITY_KEY
  1033. CELERY_SECURITY_KEY
  1034. ~~~~~~~~~~~~~~~~~~~
  1035. .. versionadded:: 2.5
  1036. The relative or absolute path to a file containing the private key
  1037. used to sign messages when :ref:`message-signing` is used.
  1038. .. setting:: CELERY_SECURITY_CERTIFICATE
  1039. CELERY_SECURITY_CERTIFICATE
  1040. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  1041. .. versionadded:: 2.5
  1042. The relative or absolute path to an X.509 certificate file
  1043. used to sign messages when :ref:`message-signing` is used.
  1044. .. setting:: CELERY_SECURITY_CERT_STORE
  1045. CELERY_SECURITY_CERT_STORE
  1046. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  1047. .. versionadded:: 2.5
  1048. The directory containing X.509 certificates used for
  1049. :ref:`message-signing`. Can be a glob with wildcards,
  1050. (for example :file:`/etc/certs/*.pem`).
  1051. .. _conf-custom-components:
  1052. Custom Component Classes (advanced)
  1053. -----------------------------------
  1054. .. setting:: CELERYD_POOL
  1055. CELERYD_POOL
  1056. ~~~~~~~~~~~~
  1057. Name of the pool class used by the worker.
  1058. .. admonition:: Eventlet/Gevent
  1059. Never use this option to select the eventlet or gevent pool.
  1060. You must use the `-P` option instead, otherwise the monkey patching
  1061. will happen too late and things will break in strange and silent ways.
  1062. Default is ``celery.concurrency.prefork:TaskPool``.
  1063. .. setting:: CELERYD_POOL_RESTARTS
  1064. CELERYD_POOL_RESTARTS
  1065. ~~~~~~~~~~~~~~~~~~~~~
  1066. If enabled the worker pool can be restarted using the
  1067. :control:`pool_restart` remote control command.
  1068. Disabled by default.
  1069. .. setting:: CELERYD_AUTOSCALER
  1070. CELERYD_AUTOSCALER
  1071. ~~~~~~~~~~~~~~~~~~
  1072. .. versionadded:: 2.2
  1073. Name of the autoscaler class to use.
  1074. Default is ``celery.worker.autoscale:Autoscaler``.
  1075. .. setting:: CELERYD_AUTORELOADER
  1076. CELERYD_AUTORELOADER
  1077. ~~~~~~~~~~~~~~~~~~~~
  1078. Name of the autoreloader class used by the worker to reload
  1079. Python modules and files that have changed.
  1080. Default is: ``celery.worker.autoreload:Autoreloader``.
  1081. .. setting:: CELERYD_CONSUMER
  1082. CELERYD_CONSUMER
  1083. ~~~~~~~~~~~~~~~~
  1084. Name of the consumer class used by the worker.
  1085. Default is :class:`celery.worker.consumer.Consumer`
  1086. .. setting:: CELERYD_TIMER
  1087. CELERYD_TIMER
  1088. ~~~~~~~~~~~~~~~~~~~~~
  1089. Name of the ETA scheduler class used by the worker.
  1090. Default is :class:`celery.utils.timer2.Timer`, or one overrided
  1091. by the pool implementation.
  1092. .. _conf-celerybeat:
  1093. Periodic Task Server: celery beat
  1094. ---------------------------------
  1095. .. setting:: CELERYBEAT_SCHEDULE
  1096. CELERYBEAT_SCHEDULE
  1097. ~~~~~~~~~~~~~~~~~~~
  1098. The periodic task schedule used by :mod:`~celery.bin.beat`.
  1099. See :ref:`beat-entries`.
  1100. .. setting:: CELERYBEAT_SCHEDULER
  1101. CELERYBEAT_SCHEDULER
  1102. ~~~~~~~~~~~~~~~~~~~~
  1103. The default scheduler class. Default is ``celery.beat:PersistentScheduler``.
  1104. Can also be set via the :option:`-S` argument to
  1105. :mod:`~celery.bin.beat`.
  1106. .. setting:: CELERYBEAT_SCHEDULE_FILENAME
  1107. CELERYBEAT_SCHEDULE_FILENAME
  1108. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  1109. Name of the file used by `PersistentScheduler` to store the last run times
  1110. of periodic tasks. Can be a relative or absolute path, but be aware that the
  1111. suffix `.db` may be appended to the file name (depending on Python version).
  1112. Can also be set via the :option:`--schedule` argument to
  1113. :mod:`~celery.bin.beat`.
  1114. .. setting:: CELERYBEAT_SYNC_EVERY
  1115. CELERYBEAT_SYNC_EVERY
  1116. ~~~~~~~~~~~~~~~~~~~~~
  1117. The number of periodic tasks that can be called before another database sync
  1118. is issued.
  1119. Defaults to 0 (sync based on timing - default of 3 minutes as determined by
  1120. scheduler.sync_every). If set to 1, beat will call sync after every task
  1121. message sent.
  1122. .. setting:: CELERYBEAT_MAX_LOOP_INTERVAL
  1123. CELERYBEAT_MAX_LOOP_INTERVAL
  1124. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  1125. The maximum number of seconds :mod:`~celery.bin.beat` can sleep
  1126. between checking the schedule.
  1127. The default for this value is scheduler specific.
  1128. For the default celery beat scheduler the value is 300 (5 minutes),
  1129. but for e.g. the django-celery database scheduler it is 5 seconds
  1130. because the schedule may be changed externally, and so it must take
  1131. changes to the schedule into account.
  1132. Also when running celery beat embedded (:option:`-B`) on Jython as a thread
  1133. the max interval is overridden and set to 1 so that it's possible
  1134. to shut down in a timely manner.
  1135. .. _conf-celerymon:
  1136. Monitor Server: celerymon
  1137. -------------------------
  1138. .. setting:: CELERYMON_LOG_FORMAT
  1139. CELERYMON_LOG_FORMAT
  1140. ~~~~~~~~~~~~~~~~~~~~
  1141. The format to use for log messages.
  1142. Default is `[%(asctime)s: %(levelname)s/%(processName)s] %(message)s`
  1143. See the Python :mod:`logging` module for more information about log
  1144. formats.