configuration.rst 39 KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619
  1. .. _configuration:
  2. ============================
  3. Configuration and defaults
  4. ============================
  5. This document describes the configuration options available.
  6. If you're using the default loader, you must create the :file:`celeryconfig.py`
  7. module and make sure it is available on the Python path.
  8. .. contents::
  9. :local:
  10. :depth: 2
  11. .. _conf-example:
  12. Example configuration file
  13. ==========================
  14. This is an example configuration file to get you started.
  15. It should contain all you need to run a basic Celery set-up.
  16. .. code-block:: python
  17. ## Broker settings.
  18. BROKER_URL = "amqp://guest:guest@localhost:5672//"
  19. # List of modules to import when celery starts.
  20. CELERY_IMPORTS = ("myapp.tasks", )
  21. ## Using the database to store task state and results.
  22. CELERY_RESULT_BACKEND = "database"
  23. CELERY_RESULT_DBURI = "sqlite:///mydatabase.db"
  24. CELERY_ANNOTATIONS = {"tasks.add": {"rate_limit": "10/s"}}
  25. Configuration Directives
  26. ========================
  27. .. _conf-datetime:
  28. Time and date settings
  29. ----------------------
  30. .. setting:: CELERY_ENABLE_UTC
  31. CELERY_ENABLE_UTC
  32. ~~~~~~~~~~~~~~~~~
  33. .. versionadded:: 2.5
  34. If enabled dates and times in messages will be converted to use
  35. the UTC timezone.
  36. Note that workers running Celery versions below 2.5 will assume a local
  37. timezone for all messages, so only enable if all workers have been
  38. upgraded.
  39. Enabled by default since version 3.0.
  40. .. setting:: CELERY_TIMEZONE
  41. CELERY_TIMEZONE
  42. ---------------
  43. Configure Celery to use a custom time zone.
  44. The timezone value can be any time zone supported by the :mod:`pytz`
  45. library.
  46. If not set then the UTC timezone is used if :setting:`CELERY_ENABLE_UTC` is
  47. enabled, otherwise it falls back to the local timezone.
  48. .. _conf-tasks:
  49. Task settings
  50. -------------
  51. .. setting:: CELERY_ANNOTATIONS
  52. CELERY_ANNOTATIONS
  53. ~~~~~~~~~~~~~~~~~~
  54. This setting can be used to rewrite any task attribute from the
  55. configuration. The setting can be a dict, or a list of annotation
  56. objects that filter for tasks and return a map of attributes
  57. to change.
  58. This will change the ``rate_limit`` attribute for the ``tasks.add``
  59. task:
  60. .. code-block:: python
  61. CELERY_ANNOTATIONS = {"tasks.add": {"rate_limit": "10/s"}}
  62. or change the same for all tasks:
  63. .. code-block:: python
  64. CELERY_ANNOTATIONS = {"*": {"rate_limit": "10/s"}}
  65. You can change methods too, for example the ``on_failure`` handler:
  66. .. code-block:: python
  67. def my_on_failure(self, exc, task_id, args, kwargs, einfo):
  68. print("Oh no! Task failed: {0!r}".format(exc))
  69. CELERY_ANNOTATIONS = {"*": {"on_failure": my_on_failure}}
  70. If you need more flexibility then you can use objects
  71. instead of a dict to choose which tasks to annotate:
  72. .. code-block:: python
  73. class MyAnnotate(object):
  74. def annotate(self, task):
  75. if task.name.startswith("tasks."):
  76. return {"rate_limit": "10/s"}
  77. CELERY_ANNOTATIONS = (MyAnnotate(), {...})
  78. .. _conf-concurrency:
  79. Concurrency settings
  80. --------------------
  81. .. setting:: CELERYD_CONCURRENCY
  82. CELERYD_CONCURRENCY
  83. ~~~~~~~~~~~~~~~~~~~
  84. The number of concurrent worker processes/threads/green threads executing
  85. tasks.
  86. If you're doing mostly I/O you can have more processes,
  87. but if mostly CPU-bound, try to keep it close to the
  88. number of CPUs on your machine. If not set, the number of CPUs/cores
  89. on the host will be used.
  90. Defaults to the number of available CPUs.
  91. .. setting:: CELERYD_PREFETCH_MULTIPLIER
  92. CELERYD_PREFETCH_MULTIPLIER
  93. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  94. How many messages to prefetch at a time multiplied by the number of
  95. concurrent processes. The default is 4 (four messages for each
  96. process). The default setting is usually a good choice, however -- if you
  97. have very long running tasks waiting in the queue and you have to start the
  98. workers, note that the first worker to start will receive four times the
  99. number of messages initially. Thus the tasks may not be fairly distributed
  100. to the workers.
  101. .. _conf-result-backend:
  102. Task result backend settings
  103. ----------------------------
  104. .. setting:: CELERY_RESULT_BACKEND
  105. CELERY_RESULT_BACKEND
  106. ~~~~~~~~~~~~~~~~~~~~~
  107. :Deprecated aliases: ``CELERY_BACKEND``
  108. The backend used to store task results (tombstones).
  109. Disabled by default.
  110. Can be one of the following:
  111. * database
  112. Use a relational database supported by `SQLAlchemy`_.
  113. See :ref:`conf-database-result-backend`.
  114. * cache
  115. Use `memcached`_ to store the results.
  116. See :ref:`conf-cache-result-backend`.
  117. * mongodb
  118. Use `MongoDB`_ to store the results.
  119. See :ref:`conf-mongodb-result-backend`.
  120. * redis
  121. Use `Redis`_ to store the results.
  122. See :ref:`conf-redis-result-backend`.
  123. * amqp
  124. Send results back as AMQP messages
  125. See :ref:`conf-amqp-result-backend`.
  126. * cassandra
  127. Use `Cassandra`_ to store the results.
  128. See :ref:`conf-cassandra-result-backend`.
  129. .. warning:
  130. While the AMQP result backend is very efficient, you must make sure
  131. you only receive the same result once. See :doc:`userguide/calling`).
  132. .. _`SQLAlchemy`: http://sqlalchemy.org
  133. .. _`memcached`: http://memcached.org
  134. .. _`MongoDB`: http://mongodb.org
  135. .. _`Redis`: http://code.google.com/p/redis/
  136. .. _`Cassandra`: http://cassandra.apache.org/
  137. .. setting:: CELERY_RESULT_SERIALIZER
  138. CELERY_RESULT_SERIALIZER
  139. ~~~~~~~~~~~~~~~~~~~~~~~~
  140. Result serialization format. Default is `"pickle"`. See
  141. :ref:`calling-serializers` for information about supported
  142. serialization formats.
  143. .. _conf-database-result-backend:
  144. Database backend settings
  145. -------------------------
  146. .. setting:: CELERY_RESULT_DBURI
  147. CELERY_RESULT_DBURI
  148. ~~~~~~~~~~~~~~~~~~~
  149. Please see `Supported Databases`_ for a table of supported databases.
  150. To use this backend you need to configure it with an
  151. `Connection String`_, some examples include:
  152. .. code-block:: python
  153. # sqlite (filename)
  154. CELERY_RESULT_DBURI = "sqlite:///celerydb.sqlite"
  155. # mysql
  156. CELERY_RESULT_DBURI = "mysql://scott:tiger@localhost/foo"
  157. # postgresql
  158. CELERY_RESULT_DBURI = "postgresql://scott:tiger@localhost/mydatabase"
  159. # oracle
  160. CELERY_RESULT_DBURI = "oracle://scott:tiger@127.0.0.1:1521/sidname"
  161. See `Connection String`_ for more information about connection
  162. strings.
  163. .. setting:: CELERY_RESULT_ENGINE_OPTIONS
  164. CELERY_RESULT_ENGINE_OPTIONS
  165. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  166. To specify additional SQLAlchemy database engine options you can use
  167. the :setting:`CELERY_RESULT_ENGINE_OPTIONS` setting::
  168. # echo enables verbose logging from SQLAlchemy.
  169. CELERY_RESULT_ENGINE_OPTIONS = {"echo": True}
  170. .. setting:: CELERY_RESULT_DB_SHORT_LIVED_SESSIONS
  171. CELERY_RESULT_DB_SHORT_LIVED_SESSIONS = True
  172. Short lived sessions are disabled by default. If enabled they can drastically reduce
  173. performance, especially on systems processing lots of tasks. This option is useful
  174. on low-traffic workers that experience errors as a result of cached database connections
  175. going stale through inactivity. For example, intermittent errors like
  176. `(OperationalError) (2006, 'MySQL server has gone away')` can be fixed by enabling
  177. short lived sessions. This option only affects the database backend.
  178. .. _`Supported Databases`:
  179. http://www.sqlalchemy.org/docs/core/engines.html#supported-databases
  180. .. _`Connection String`:
  181. http://www.sqlalchemy.org/docs/core/engines.html#database-urls
  182. Example configuration
  183. ~~~~~~~~~~~~~~~~~~~~~
  184. .. code-block:: python
  185. CELERY_RESULT_BACKEND = "database"
  186. CELERY_RESULT_DBURI = "mysql://user:password@host/dbname"
  187. .. _conf-amqp-result-backend:
  188. AMQP backend settings
  189. ---------------------
  190. .. note::
  191. The AMQP backend requires RabbitMQ 1.1.0 or higher to automatically
  192. expire results. If you are running an older version of RabbitmQ
  193. you should disable result expiration like this:
  194. CELERY_TASK_RESULT_EXPIRES = None
  195. .. setting:: CELERY_RESULT_EXCHANGE
  196. CELERY_RESULT_EXCHANGE
  197. ~~~~~~~~~~~~~~~~~~~~~~
  198. Name of the exchange to publish results in. Default is `"celeryresults"`.
  199. .. setting:: CELERY_RESULT_EXCHANGE_TYPE
  200. CELERY_RESULT_EXCHANGE_TYPE
  201. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  202. The exchange type of the result exchange. Default is to use a `direct`
  203. exchange.
  204. .. setting:: CELERY_RESULT_PERSISTENT
  205. CELERY_RESULT_PERSISTENT
  206. ~~~~~~~~~~~~~~~~~~~~~~~~
  207. If set to :const:`True`, result messages will be persistent. This means the
  208. messages will not be lost after a broker restart. The default is for the
  209. results to be transient.
  210. Example configuration
  211. ~~~~~~~~~~~~~~~~~~~~~
  212. .. code-block:: python
  213. CELERY_RESULT_BACKEND = "amqp"
  214. CELERY_TASK_RESULT_EXPIRES = 18000 # 5 hours.
  215. .. _conf-cache-result-backend:
  216. Cache backend settings
  217. ----------------------
  218. .. note::
  219. The cache backend supports the `pylibmc`_ and `python-memcached`
  220. libraries. The latter is used only if `pylibmc`_ is not installed.
  221. .. setting:: CELERY_CACHE_BACKEND
  222. CELERY_CACHE_BACKEND
  223. ~~~~~~~~~~~~~~~~~~~~
  224. Using a single memcached server:
  225. .. code-block:: python
  226. CELERY_CACHE_BACKEND = 'memcached://127.0.0.1:11211/'
  227. Using multiple memcached servers:
  228. .. code-block:: python
  229. CELERY_RESULT_BACKEND = "cache"
  230. CELERY_CACHE_BACKEND = 'memcached://172.19.26.240:11211;172.19.26.242:11211/'
  231. .. setting:: CELERY_CACHE_BACKEND_OPTIONS
  232. The "dummy" backend stores the cache in memory only:
  233. CELERY_CACHE_BACKEND = "dummy"
  234. CELERY_CACHE_BACKEND_OPTIONS
  235. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  236. You can set pylibmc options using the :setting:`CELERY_CACHE_BACKEND_OPTIONS`
  237. setting:
  238. .. code-block:: python
  239. CELERY_CACHE_BACKEND_OPTIONS = {"binary": True,
  240. "behaviors": {"tcp_nodelay": True}}
  241. .. _`pylibmc`: http://sendapatch.se/projects/pylibmc/
  242. .. _conf-redis-result-backend:
  243. Redis backend settings
  244. ----------------------
  245. Configuring the backend URL
  246. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  247. .. note::
  248. The Redis backend requires the :mod:`redis` library:
  249. http://pypi.python.org/pypi/redis/
  250. To install the redis package use `pip` or `easy_install`:
  251. .. code-block:: bash
  252. $ pip install redis
  253. This backend requires the :setting:`CELERY_RESULT_BACKEND`
  254. setting to be set to a Redis URL::
  255. CELERY_RESULT_BACKEND = "redis://:password@host:port/db"
  256. For example::
  257. CELERY_RESULT_BACKEND = "redis://localhost/0"
  258. which is the same as::
  259. CELERY_RESULT_BACKEND = "redis://"
  260. The fields of the URL is defined as folows:
  261. - *host*
  262. Host name or IP address of the Redis server. e.g. `"localhost"`.
  263. - *port*
  264. Port to the Redis server. Default is 6379.
  265. - *db*
  266. Database number to use. Default is 0.
  267. The db can include an optional leading slash.
  268. - *password*
  269. Password used to connect to the database.
  270. .. setting:: CELERY_REDIS_MAX_CONNECTIONS
  271. CELERY_REDIS_MAX_CONNECTIONS
  272. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  273. Maximum number of connections available in the Redis connection
  274. pool used for sending and retrieving results.
  275. .. _conf-mongodb-result-backend:
  276. MongoDB backend settings
  277. ------------------------
  278. .. note::
  279. The MongoDB backend requires the :mod:`pymongo` library:
  280. http://github.com/mongodb/mongo-python-driver/tree/master
  281. .. setting:: CELERY_MONGODB_BACKEND_SETTINGS
  282. CELERY_MONGODB_BACKEND_SETTINGS
  283. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  284. This is a dict supporting the following keys:
  285. * host
  286. Host name of the MongoDB server. Defaults to "localhost".
  287. * port
  288. The port the MongoDB server is listening to. Defaults to 27017.
  289. * user
  290. User name to authenticate to the MongoDB server as (optional).
  291. * password
  292. Password to authenticate to the MongoDB server (optional).
  293. * database
  294. The database name to connect to. Defaults to "celery".
  295. * taskmeta_collection
  296. The collection name to store task meta data.
  297. Defaults to "celery_taskmeta".
  298. * max_pool_size
  299. Passed as max_pool_size to PyMongo's Connection or MongoClient
  300. constructor. It is the maximum number of TCP connections to keep
  301. open to MongoDB at a given time. If there are more open connections
  302. than max_pool_size, sockets will be closed when they are released.
  303. Defaults to 10.
  304. .. _example-mongodb-result-config:
  305. Example configuration
  306. ~~~~~~~~~~~~~~~~~~~~~
  307. .. code-block:: python
  308. CELERY_RESULT_BACKEND = "mongodb"
  309. CELERY_MONGODB_BACKEND_SETTINGS = {
  310. "host": "192.168.1.100",
  311. "port": 30000,
  312. "database": "mydb",
  313. "taskmeta_collection": "my_taskmeta_collection",
  314. }
  315. .. _conf-cassandra-result-backend:
  316. Cassandra backend settings
  317. --------------------------
  318. .. note::
  319. The Cassandra backend requires the :mod:`pycassa` library:
  320. http://pypi.python.org/pypi/pycassa/
  321. To install the pycassa package use `pip` or `easy_install`:
  322. .. code-block:: bash
  323. $ pip install pycassa
  324. This backend requires the following configuration directives to be set.
  325. .. setting:: CASSANDRA_SERVERS
  326. CASSANDRA_SERVERS
  327. ~~~~~~~~~~~~~~~~~
  328. List of ``host:port`` Cassandra servers. e.g. ``["localhost:9160]"``.
  329. .. setting:: CASSANDRA_KEYSPACE
  330. CASSANDRA_KEYSPACE
  331. ~~~~~~~~~~~~~~~~~~
  332. The keyspace in which to store the results. e.g. ``"tasks_keyspace"``.
  333. .. setting:: CASSANDRA_COLUMN_FAMILY
  334. CASSANDRA_COLUMN_FAMILY
  335. ~~~~~~~~~~~~~~~~~~~~~~~
  336. The column family in which to store the results. eg ``"tasks"``
  337. .. setting:: CASSANDRA_READ_CONSISTENCY
  338. CASSANDRA_READ_CONSISTENCY
  339. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  340. The read consistency used. Values can be ``"ONE"``, ``"QUORUM"`` or ``"ALL"``.
  341. .. setting:: CASSANDRA_WRITE_CONSISTENCY
  342. CASSANDRA_WRITE_CONSISTENCY
  343. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  344. The write consistency used. Values can be ``"ONE"``, ``"QUORUM"`` or ``"ALL"``.
  345. .. setting:: CASSANDRA_DETAILED_MODE
  346. CASSANDRA_DETAILED_MODE
  347. ~~~~~~~~~~~~~~~~~~~~~~~
  348. Enable or disable detailed mode. Default is :const:`False`.
  349. This mode allows to use the power of Cassandra wide columns to
  350. store all states for a task as a wide column, instead of only the last one.
  351. To use this mode, you need to configure your ColumnFamily to
  352. use the ``TimeUUID`` type as a comparator::
  353. create column family task_results with comparator = TimeUUIDType;
  354. CASSANDRA_OPTIONS
  355. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  356. Options to be passed to the `pycassa connection pool`_ (optional).
  357. .. _`pycassa connection pool`: http://pycassa.github.com/pycassa/api/pycassa/pool.html
  358. Example configuration
  359. ~~~~~~~~~~~~~~~~~~~~~
  360. .. code-block:: python
  361. CASSANDRA_SERVERS = ["localhost:9160"]
  362. CASSANDRA_KEYSPACE = "celery"
  363. CASSANDRA_COLUMN_FAMILY = "task_results"
  364. CASSANDRA_READ_CONSISTENCY = "ONE"
  365. CASSANDRA_WRITE_CONSISTENCY = "ONE"
  366. CASSANDRA_DETAILED_MODE = True
  367. CASSANDRA_OPTIONS = {
  368. 'timeout': 300,
  369. 'max_retries': 10
  370. }
  371. .. _conf-messaging:
  372. Message Routing
  373. ---------------
  374. .. _conf-messaging-routing:
  375. .. setting:: CELERY_QUEUES
  376. CELERY_QUEUES
  377. ~~~~~~~~~~~~~
  378. The mapping of queues the worker consumes from. This is a dictionary
  379. of queue name/options. See :ref:`guide-routing` for more information.
  380. The default is a queue/exchange/binding key of `"celery"`, with
  381. exchange type `direct`.
  382. You don't have to care about this unless you want custom routing facilities.
  383. .. setting:: CELERY_ROUTES
  384. CELERY_ROUTES
  385. ~~~~~~~~~~~~~
  386. A list of routers, or a single router used to route tasks to queues.
  387. When deciding the final destination of a task the routers are consulted
  388. in order. See :ref:`routers` for more information.
  389. .. setting:: CELERY_QUEUE_HA_POLICY
  390. CELERY_QUEUE_HA_POLICY
  391. ~~~~~~~~~~~~~~~~~~~~~~
  392. :brokers: RabbitMQ
  393. This will set the default HA policy for a queue, and the value
  394. can either be a string (usually ``all``):
  395. .. code-block:: python
  396. CELERY_QUEUE_HA_POLICY = 'all'
  397. Using 'all' will replicate the queue to all current nodes,
  398. Or you can give it a list of nodes to replicate to:
  399. .. code-block:: python
  400. CELERY_QUEUE_HA_POLICY = ['rabbit@host1', 'rabbit@host2']
  401. Using a list will implicitly set ``x-ha-policy`` to 'nodes' and
  402. ``x-ha-policy-params`` to the given list of nodes.
  403. See http://www.rabbitmq.com/ha.html for more information.
  404. .. setting:: CELERY_WORKER_DIRECT
  405. CELERY_WORKER_DIRECT
  406. ~~~~~~~~~~~~~~~~~~~~
  407. This option enables so that every worker has a dedicated queue,
  408. so that tasks can be routed to specific workers.
  409. The queue name for each worker is automatically generated based on
  410. the worker hostname and a ``.dq`` suffix, using the ``C.dq`` exchange.
  411. For example the queue name for the worker with hostname ``w1.example.com``
  412. becomes::
  413. w1.example.com.dq
  414. Then you can route the task to the task by specifying the hostname
  415. as the routung key and the ``C.dq`` exchange::
  416. CELERY_ROUTES = {
  417. 'tasks.add': {'exchange': 'C.dq', 'routing_key': 'w1.example.com'}
  418. }
  419. This setting is mandatory if you want to use the ``move_to_worker`` features
  420. of :mod:`celery.contrib.migrate`.
  421. .. setting:: CELERY_CREATE_MISSING_QUEUES
  422. CELERY_CREATE_MISSING_QUEUES
  423. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  424. If enabled (default), any queues specified that is not defined in
  425. :setting:`CELERY_QUEUES` will be automatically created. See
  426. :ref:`routing-automatic`.
  427. .. setting:: CELERY_DEFAULT_QUEUE
  428. CELERY_DEFAULT_QUEUE
  429. ~~~~~~~~~~~~~~~~~~~~
  430. The name of the default queue used by `.apply_async` if the message has
  431. no route or no custom queue has been specified.
  432. This queue must be listed in :setting:`CELERY_QUEUES`.
  433. If :setting:`CELERY_QUEUES` is not specified then it this automatically
  434. created containing one queue entry, where this name is used as the name of
  435. that queue.
  436. The default is: `celery`.
  437. .. seealso::
  438. :ref:`routing-changing-default-queue`
  439. .. setting:: CELERY_DEFAULT_EXCHANGE
  440. CELERY_DEFAULT_EXCHANGE
  441. ~~~~~~~~~~~~~~~~~~~~~~~
  442. Name of the default exchange to use when no custom exchange is
  443. specified for a key in the :setting:`CELERY_QUEUES` setting.
  444. The default is: `celery`.
  445. .. setting:: CELERY_DEFAULT_EXCHANGE_TYPE
  446. CELERY_DEFAULT_EXCHANGE_TYPE
  447. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  448. Default exchange type used when no custom exchange type is specified.
  449. for a key in the :setting:`CELERY_QUEUES` setting.
  450. The default is: `direct`.
  451. .. setting:: CELERY_DEFAULT_ROUTING_KEY
  452. CELERY_DEFAULT_ROUTING_KEY
  453. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  454. The default routing key used when no custom routing key
  455. is specified for a key in the :setting:`CELERY_QUEUES` setting.
  456. The default is: `celery`.
  457. .. setting:: CELERY_DEFAULT_DELIVERY_MODE
  458. CELERY_DEFAULT_DELIVERY_MODE
  459. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  460. Can be `transient` or `persistent`. The default is to send
  461. persistent messages.
  462. .. _conf-broker-settings:
  463. Broker Settings
  464. ---------------
  465. .. setting:: BROKER_TRANSPORT
  466. BROKER_TRANSPORT
  467. ~~~~~~~~~~~~~~~~
  468. :Aliases: ``BROKER_BACKEND``
  469. :Deprecated aliases: ``CARROT_BACKEND``
  470. .. setting:: BROKER_URL
  471. BROKER_URL
  472. ~~~~~~~~~~
  473. Default broker URL. This must be an URL in the form of::
  474. transport://userid:password@hostname:port/virtual_host
  475. Only the scheme part (``transport://``) is required, the rest
  476. is optional, and defaults to the specific transports default values.
  477. The transport part is the broker implementation to use, and the
  478. default is ``amqp``, which uses ``librabbitmq`` by default or falls back to
  479. ``pyamqp`` if that is not installed. Also there are many other choices including
  480. ``redis``, ``beanstalk``, ``sqlalchemy``, ``django``, ``mongodb``,
  481. ``couchdb``.
  482. It can also be a fully qualified path to your own transport implementation.
  483. See the Kombu documentation for more information about broker URLs.
  484. .. setting:: BROKER_HEARTBEAT
  485. BROKER_HEARTBEAT
  486. ~~~~~~~~~~~~~~~~
  487. :transports supported: ``pyamqp``
  488. It's not always possible to detect connection loss in a timely
  489. manner using TCP/IP alone, so AMQP defines something called heartbeats
  490. that's is used both by the client and the broker to detect if
  491. a connection was closed.
  492. Heartbeats are currently only supported by the ``pyamqp://`` transport,
  493. and this requires the :mod:`amqp` module:
  494. .. code-block:: bash
  495. $ pip install amqp
  496. The default heartbeat value is 10 seconds,
  497. the heartbeat will then be monitored at double the rate of the heartbeat value
  498. (so for the default 10 seconds, the heartbeat is checked every 5 seconds).
  499. .. setting:: BROKER_USE_SSL
  500. BROKER_USE_SSL
  501. ~~~~~~~~~~~~~~
  502. Use SSL to connect to the broker. Off by default. This may not be supported
  503. by all transports.
  504. .. setting:: BROKER_POOL_LIMIT
  505. BROKER_POOL_LIMIT
  506. ~~~~~~~~~~~~~~~~~
  507. .. versionadded:: 2.3
  508. The maximum number of connections that can be open in the connection pool.
  509. The pool is enabled by default since version 2.5, with a default limit of ten
  510. connections. This number can be tweaked depending on the number of
  511. threads/greenthreads (eventlet/gevent) using a connection. For example
  512. running eventlet with 1000 greenlets that use a connection to the broker,
  513. contention can arise and you should consider increasing the limit.
  514. If set to :const:`None` or 0 the connection pool will be disabled and
  515. connections will be established and closed for every use.
  516. Default (since 2.5) is to use a pool of 10 connections.
  517. .. setting:: BROKER_CONNECTION_TIMEOUT
  518. BROKER_CONNECTION_TIMEOUT
  519. ~~~~~~~~~~~~~~~~~~~~~~~~~
  520. The default timeout in seconds before we give up establishing a connection
  521. to the AMQP server. Default is 4 seconds.
  522. .. setting:: BROKER_CONNECTION_RETRY
  523. BROKER_CONNECTION_RETRY
  524. ~~~~~~~~~~~~~~~~~~~~~~~
  525. Automatically try to re-establish the connection to the AMQP broker if lost.
  526. The time between retries is increased for each retry, and is
  527. not exhausted before :setting:`BROKER_CONNECTION_MAX_RETRIES` is
  528. exceeded.
  529. This behavior is on by default.
  530. .. setting:: BROKER_CONNECTION_MAX_RETRIES
  531. BROKER_CONNECTION_MAX_RETRIES
  532. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  533. Maximum number of retries before we give up re-establishing a connection
  534. to the AMQP broker.
  535. If this is set to :const:`0` or :const:`None`, we will retry forever.
  536. Default is 100 retries.
  537. .. setting:: BROKER_TRANSPORT_OPTIONS
  538. BROKER_TRANSPORT_OPTIONS
  539. ~~~~~~~~~~~~~~~~~~~~~~~~
  540. .. versionadded:: 2.2
  541. A dict of additional options passed to the underlying transport.
  542. See your transport user manual for supported options (if any).
  543. Example setting the visibility timeout (supported by Redis and SQS
  544. transports):
  545. .. code-block:: python
  546. BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 18000} # 5 hours
  547. .. _conf-task-execution:
  548. Task execution settings
  549. -----------------------
  550. .. setting:: CELERY_ALWAYS_EAGER
  551. CELERY_ALWAYS_EAGER
  552. ~~~~~~~~~~~~~~~~~~~
  553. If this is :const:`True`, all tasks will be executed locally by blocking until
  554. the task returns. ``apply_async()`` and ``Task.delay()`` will return
  555. an :class:`~celery.result.EagerResult` instance, which emulates the API
  556. and behavior of :class:`~celery.result.AsyncResult`, except the result
  557. is already evaluated.
  558. That is, tasks will be executed locally instead of being sent to
  559. the queue.
  560. .. setting:: CELERY_EAGER_PROPAGATES_EXCEPTIONS
  561. CELERY_EAGER_PROPAGATES_EXCEPTIONS
  562. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  563. If this is :const:`True`, eagerly executed tasks (applied by `task.apply()`,
  564. or when the :setting:`CELERY_ALWAYS_EAGER` setting is enabled), will
  565. propagate exceptions.
  566. It's the same as always running ``apply()`` with ``throw=True``.
  567. .. setting:: CELERY_IGNORE_RESULT
  568. CELERY_IGNORE_RESULT
  569. ~~~~~~~~~~~~~~~~~~~~
  570. Whether to store the task return values or not (tombstones).
  571. If you still want to store errors, just not successful return values,
  572. you can set :setting:`CELERY_STORE_ERRORS_EVEN_IF_IGNORED`.
  573. .. setting:: CELERY_MESSAGE_COMPRESSION
  574. CELERY_MESSAGE_COMPRESSION
  575. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  576. Default compression used for task messages.
  577. Can be ``"gzip"``, ``"bzip2"`` (if available), or any custom
  578. compression schemes registered in the Kombu compression registry.
  579. The default is to send uncompressed messages.
  580. .. setting:: CELERY_TASK_RESULT_EXPIRES
  581. CELERY_TASK_RESULT_EXPIRES
  582. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  583. Time (in seconds, or a :class:`~datetime.timedelta` object) for when after
  584. stored task tombstones will be deleted.
  585. A built-in periodic task will delete the results after this time
  586. (:class:`celery.task.backend_cleanup`).
  587. Default is to expire after 1 day.
  588. .. note::
  589. For the moment this only works with the amqp, database, cache, redis and MongoDB
  590. backends.
  591. When using the database or MongoDB backends, `celery beat` must be
  592. running for the results to be expired.
  593. .. setting:: CELERY_MAX_CACHED_RESULTS
  594. CELERY_MAX_CACHED_RESULTS
  595. ~~~~~~~~~~~~~~~~~~~~~~~~~
  596. Result backends caches ready results used by the client.
  597. This is the total number of results to cache before older results are evicted.
  598. The default is 5000.
  599. .. setting:: CELERY_TRACK_STARTED
  600. CELERY_TRACK_STARTED
  601. ~~~~~~~~~~~~~~~~~~~~
  602. If :const:`True` the task will report its status as "started" when the
  603. task is executed by a worker. The default value is :const:`False` as
  604. the normal behaviour is to not report that level of granularity. Tasks
  605. are either pending, finished, or waiting to be retried. Having a "started"
  606. state can be useful for when there are long running tasks and there is a
  607. need to report which task is currently running.
  608. .. setting:: CELERY_TASK_SERIALIZER
  609. CELERY_TASK_SERIALIZER
  610. ~~~~~~~~~~~~~~~~~~~~~~
  611. A string identifying the default serialization method to use. Can be
  612. `pickle` (default), `json`, `yaml`, `msgpack` or any custom serialization
  613. methods that have been registered with :mod:`kombu.serialization.registry`.
  614. .. seealso::
  615. :ref:`calling-serializers`.
  616. .. setting:: CELERY_TASK_PUBLISH_RETRY
  617. CELERY_TASK_PUBLISH_RETRY
  618. ~~~~~~~~~~~~~~~~~~~~~~~~~
  619. .. versionadded:: 2.2
  620. Decides if publishing task messages will be retried in the case
  621. of connection loss or other connection errors.
  622. See also :setting:`CELERY_TASK_PUBLISH_RETRY_POLICY`.
  623. Enabled by default.
  624. .. setting:: CELERY_TASK_PUBLISH_RETRY_POLICY
  625. CELERY_TASK_PUBLISH_RETRY_POLICY
  626. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  627. .. versionadded:: 2.2
  628. Defines the default policy when retrying publishing a task message in
  629. the case of connection loss or other connection errors.
  630. See :ref:`calling-retry` for more information.
  631. .. setting:: CELERY_DEFAULT_RATE_LIMIT
  632. CELERY_DEFAULT_RATE_LIMIT
  633. ~~~~~~~~~~~~~~~~~~~~~~~~~
  634. The global default rate limit for tasks.
  635. This value is used for tasks that does not have a custom rate limit
  636. The default is no rate limit.
  637. .. setting:: CELERY_DISABLE_RATE_LIMITS
  638. CELERY_DISABLE_RATE_LIMITS
  639. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  640. Disable all rate limits, even if tasks has explicit rate limits set.
  641. .. setting:: CELERY_ACKS_LATE
  642. CELERY_ACKS_LATE
  643. ~~~~~~~~~~~~~~~~
  644. Late ack means the task messages will be acknowledged **after** the task
  645. has been executed, not *just before*, which is the default behavior.
  646. .. seealso::
  647. FAQ: :ref:`faq-acks_late-vs-retry`.
  648. .. _conf-worker:
  649. Worker
  650. ------
  651. .. setting:: CELERY_IMPORTS
  652. CELERY_IMPORTS
  653. ~~~~~~~~~~~~~~
  654. A sequence of modules to import when the worker starts.
  655. This is used to specify the task modules to import, but also
  656. to import signal handlers and additional remote control commands, etc.
  657. .. setting:: CELERY_INCLUDE
  658. CELERY_INCLUDE
  659. ~~~~~~~~~~~~~~
  660. Exact same semantics as :setting:`CELERY_IMPORTS`, but can be used as a means
  661. to have different import categories.
  662. .. setting:: CELERYD_FORCE_EXECV
  663. CELERYD_FORCE_EXECV
  664. ~~~~~~~~~~~~~~~~~~~
  665. On Unix the processes pool will fork, so that child processes
  666. start with the same memory as the parent process.
  667. This can cause problems as there is a known deadlock condition
  668. with pthread locking primitives when `fork()` is combined with threads.
  669. You should enable this setting if you are experiencing hangs (deadlocks),
  670. especially in combination with time limits or having a max tasks per child limit.
  671. This option will be enabled by default in a later version.
  672. This is not a problem on Windows, as it does not have `fork()`.
  673. .. setting:: CELERYD_WORKER_LOST_WAIT
  674. CELERYD_WORKER_LOST_WAIT
  675. ~~~~~~~~~~~~~~~~~~~~~~~~
  676. In some cases a worker may be killed without proper cleanup,
  677. and the worker may have published a result before terminating.
  678. This value specifies how long we wait for any missing results before
  679. raising a :exc:`@WorkerLostError` exception.
  680. Default is 10.0
  681. .. setting:: CELERYD_MAX_TASKS_PER_CHILD
  682. CELERYD_MAX_TASKS_PER_CHILD
  683. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  684. Maximum number of tasks a pool worker process can execute before
  685. it's replaced with a new one. Default is no limit.
  686. .. setting:: CELERYD_TASK_TIME_LIMIT
  687. CELERYD_TASK_TIME_LIMIT
  688. ~~~~~~~~~~~~~~~~~~~~~~~
  689. Task hard time limit in seconds. The worker processing the task will
  690. be killed and replaced with a new one when this is exceeded.
  691. .. setting:: CELERYD_TASK_SOFT_TIME_LIMIT
  692. CELERYD_TASK_SOFT_TIME_LIMIT
  693. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  694. Task soft time limit in seconds.
  695. The :exc:`~@SoftTimeLimitExceeded` exception will be
  696. raised when this is exceeded. The task can catch this to
  697. e.g. clean up before the hard time limit comes.
  698. Example:
  699. .. code-block:: python
  700. from celery.exceptions import SoftTimeLimitExceeded
  701. @celery.task
  702. def mytask():
  703. try:
  704. return do_work()
  705. except SoftTimeLimitExceeded:
  706. cleanup_in_a_hurry()
  707. .. setting:: CELERY_STORE_ERRORS_EVEN_IF_IGNORED
  708. CELERY_STORE_ERRORS_EVEN_IF_IGNORED
  709. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  710. If set, the worker stores all task errors in the result store even if
  711. :attr:`Task.ignore_result <celery.task.base.Task.ignore_result>` is on.
  712. .. setting:: CELERYD_STATE_DB
  713. CELERYD_STATE_DB
  714. ~~~~~~~~~~~~~~~~
  715. Name of the file used to stores persistent worker state (like revoked tasks).
  716. Can be a relative or absolute path, but be aware that the suffix `.db`
  717. may be appended to the file name (depending on Python version).
  718. Can also be set via the :option:`--statedb` argument to
  719. :mod:`~celery.bin.worker`.
  720. Not enabled by default.
  721. .. setting:: CELERYD_TIMER_PRECISION
  722. CELERYD_TIMER_PRECISION
  723. ~~~~~~~~~~~~~~~~~~~~~~~
  724. Set the maximum time in seconds that the ETA scheduler can sleep between
  725. rechecking the schedule. Default is 1 second.
  726. Setting this value to 1 second means the schedulers precision will
  727. be 1 second. If you need near millisecond precision you can set this to 0.1.
  728. .. _conf-error-mails:
  729. Error E-Mails
  730. -------------
  731. .. setting:: CELERY_SEND_TASK_ERROR_EMAILS
  732. CELERY_SEND_TASK_ERROR_EMAILS
  733. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  734. The default value for the `Task.send_error_emails` attribute, which if
  735. set to :const:`True` means errors occurring during task execution will be
  736. sent to :setting:`ADMINS` by email.
  737. Disabled by default.
  738. .. setting:: ADMINS
  739. ADMINS
  740. ~~~~~~
  741. List of `(name, email_address)` tuples for the administrators that should
  742. receive error emails.
  743. .. setting:: SERVER_EMAIL
  744. SERVER_EMAIL
  745. ~~~~~~~~~~~~
  746. The email address this worker sends emails from.
  747. Default is celery@localhost.
  748. .. setting:: EMAIL_HOST
  749. EMAIL_HOST
  750. ~~~~~~~~~~
  751. The mail server to use. Default is `"localhost"`.
  752. .. setting:: EMAIL_HOST_USER
  753. EMAIL_HOST_USER
  754. ~~~~~~~~~~~~~~~
  755. User name (if required) to log on to the mail server with.
  756. .. setting:: EMAIL_HOST_PASSWORD
  757. EMAIL_HOST_PASSWORD
  758. ~~~~~~~~~~~~~~~~~~~
  759. Password (if required) to log on to the mail server with.
  760. .. setting:: EMAIL_PORT
  761. EMAIL_PORT
  762. ~~~~~~~~~~
  763. The port the mail server is listening on. Default is `25`.
  764. .. setting:: EMAIL_USE_SSL
  765. EMAIL_USE_SSL
  766. ~~~~~~~~~~~~~
  767. Use SSL when connecting to the SMTP server. Disabled by default.
  768. .. setting:: EMAIL_USE_TLS
  769. EMAIL_USE_TLS
  770. ~~~~~~~~~~~~~
  771. Use TLS when connecting to the SMTP server. Disabled by default.
  772. .. setting:: EMAIL_TIMEOUT
  773. EMAIL_TIMEOUT
  774. ~~~~~~~~~~~~~
  775. Timeout in seconds for when we give up trying to connect
  776. to the SMTP server when sending emails.
  777. The default is 2 seconds.
  778. .. _conf-example-error-mail-config:
  779. Example E-Mail configuration
  780. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  781. This configuration enables the sending of error emails to
  782. george@vandelay.com and kramer@vandelay.com:
  783. .. code-block:: python
  784. # Enables error emails.
  785. CELERY_SEND_TASK_ERROR_EMAILS = True
  786. # Name and email addresses of recipients
  787. ADMINS = (
  788. ("George Costanza", "george@vandelay.com"),
  789. ("Cosmo Kramer", "kosmo@vandelay.com"),
  790. )
  791. # Email address used as sender (From field).
  792. SERVER_EMAIL = "no-reply@vandelay.com"
  793. # Mailserver configuration
  794. EMAIL_HOST = "mail.vandelay.com"
  795. EMAIL_PORT = 25
  796. # EMAIL_HOST_USER = "servers"
  797. # EMAIL_HOST_PASSWORD = "s3cr3t"
  798. .. _conf-events:
  799. Events
  800. ------
  801. .. setting:: CELERY_SEND_EVENTS
  802. CELERY_SEND_EVENTS
  803. ~~~~~~~~~~~~~~~~~~
  804. Send events so the worker can be monitored by tools like `celerymon`.
  805. .. setting:: CELERY_SEND_TASK_SENT_EVENT
  806. CELERY_SEND_TASK_SENT_EVENT
  807. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  808. .. versionadded:: 2.2
  809. If enabled, a :event:`task-sent` event will be sent for every task so tasks can be
  810. tracked before they are consumed by a worker.
  811. Disabled by default.
  812. .. setting:: CELERY_EVENT_SERIALIZER
  813. CELERY_EVENT_SERIALIZER
  814. ~~~~~~~~~~~~~~~~~~~~~~~
  815. Message serialization format used when sending event messages.
  816. Default is `"json"`. See :ref:`calling-serializers`.
  817. .. _conf-broadcast:
  818. Broadcast Commands
  819. ------------------
  820. .. setting:: CELERY_BROADCAST_QUEUE
  821. CELERY_BROADCAST_QUEUE
  822. ~~~~~~~~~~~~~~~~~~~~~~
  823. Name prefix for the queue used when listening for broadcast messages.
  824. The workers host name will be appended to the prefix to create the final
  825. queue name.
  826. Default is `"celeryctl"`.
  827. .. setting:: CELERY_BROADCAST_EXCHANGE
  828. CELERY_BROADCAST_EXCHANGE
  829. ~~~~~~~~~~~~~~~~~~~~~~~~~
  830. Name of the exchange used for broadcast messages.
  831. Default is `"celeryctl"`.
  832. .. setting:: CELERY_BROADCAST_EXCHANGE_TYPE
  833. CELERY_BROADCAST_EXCHANGE_TYPE
  834. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  835. Exchange type used for broadcast messages. Default is `"fanout"`.
  836. .. _conf-logging:
  837. Logging
  838. -------
  839. .. setting:: CELERYD_HIJACK_ROOT_LOGGER
  840. CELERYD_HIJACK_ROOT_LOGGER
  841. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  842. .. versionadded:: 2.2
  843. By default any previously configured logging options will be reset,
  844. because the Celery programs "hijacks" the root logger.
  845. If you want to customize your own logging then you can disable
  846. this behavior.
  847. .. note::
  848. Logging can also be customized by connecting to the
  849. :signal:`celery.signals.setup_logging` signal.
  850. .. setting:: CELERYD_LOG_COLOR
  851. CELERYD_LOG_COLOR
  852. ~~~~~~~~~~~~~~~~~
  853. Enables/disables colors in logging output by the Celery apps.
  854. By default colors are enabled if
  855. 1) the app is logging to a real terminal, and not a file.
  856. 2) the app is not running on Windows.
  857. .. setting:: CELERYD_LOG_FORMAT
  858. CELERYD_LOG_FORMAT
  859. ~~~~~~~~~~~~~~~~~~
  860. The format to use for log messages.
  861. Default is `[%(asctime)s: %(levelname)s/%(processName)s] %(message)s`
  862. See the Python :mod:`logging` module for more information about log
  863. formats.
  864. .. setting:: CELERYD_TASK_LOG_FORMAT
  865. CELERYD_TASK_LOG_FORMAT
  866. ~~~~~~~~~~~~~~~~~~~~~~~
  867. The format to use for log messages logged in tasks. Can be overridden using
  868. the :option:`--loglevel` option to :mod:`~celery.bin.worker`.
  869. Default is::
  870. [%(asctime)s: %(levelname)s/%(processName)s]
  871. [%(task_name)s(%(task_id)s)] %(message)s
  872. See the Python :mod:`logging` module for more information about log
  873. formats.
  874. .. setting:: CELERY_REDIRECT_STDOUTS
  875. CELERY_REDIRECT_STDOUTS
  876. ~~~~~~~~~~~~~~~~~~~~~~~
  877. If enabled `stdout` and `stderr` will be redirected
  878. to the current logger.
  879. Enabled by default.
  880. Used by :program:`celery worker` and :program:`celery beat`.
  881. .. setting:: CELERY_REDIRECT_STDOUTS_LEVEL
  882. CELERY_REDIRECT_STDOUTS_LEVEL
  883. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  884. The log level output to `stdout` and `stderr` is logged as.
  885. Can be one of :const:`DEBUG`, :const:`INFO`, :const:`WARNING`,
  886. :const:`ERROR` or :const:`CRITICAL`.
  887. Default is :const:`WARNING`.
  888. .. _conf-security:
  889. Security
  890. --------
  891. .. setting:: CELERY_SECURITY_KEY
  892. CELERY_SECURITY_KEY
  893. ~~~~~~~~~~~~~~~~~~~
  894. .. versionadded:: 2.5
  895. The relative or absolute path to a file containing the private key
  896. used to sign messages when :ref:`message-signing` is used.
  897. .. setting:: CELERY_SECURITY_CERTIFICATE
  898. CELERY_SECURITY_CERTIFICATE
  899. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  900. .. versionadded:: 2.5
  901. The relative or absolute path to an X.509 certificate file
  902. used to sign messages when :ref:`message-signing` is used.
  903. .. setting:: CELERY_SECURITY_CERT_STORE
  904. CELERY_SECURITY_CERT_STORE
  905. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  906. .. versionadded:: 2.5
  907. The directory containing X.509 certificates used for
  908. :ref:`message-signing`. Can be a glob with wildcards,
  909. (for example :file:`/etc/certs/*.pem`).
  910. .. _conf-custom-components:
  911. Custom Component Classes (advanced)
  912. -----------------------------------
  913. .. setting:: CELERYD_BOOTSTEPS
  914. CELERYD_BOOTSTEPS
  915. ~~~~~~~~~~~~~~~~~
  916. This setting enables you to add additional components to the worker process.
  917. It should be a list of module names with
  918. :class:`celery.bootsteps.Step`
  919. classes, that augments functionality in the worker.
  920. .. setting:: CELERYD_CONSUMER_BOOTSTEPS
  921. CELERYD_CONSUMER_BOOTSTEPS
  922. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  923. This setting enables you to add additional components to the workers consumer.
  924. It should be a list of module names with
  925. :class:`celery.bootsteps.Step`` classes, that augments
  926. functionality in the consumer.
  927. .. setting:: CELERYD_POOL
  928. CELERYD_POOL
  929. ~~~~~~~~~~~~
  930. Name of the pool class used by the worker.
  931. You can use a custom pool class name, or select one of
  932. the built-in aliases: ``processes``, ``eventlet``, ``gevent``.
  933. Default is ``processes``.
  934. .. setting:: CELERYD_POOL_RESTARTS
  935. CELERYD_POOL_RESTARTS
  936. ~~~~~~~~~~~~~~~~~~~~~
  937. If enabled the worker pool can be restarted using the
  938. :control:`pool_restart` remote control command.
  939. Disabled by default.
  940. .. setting:: CELERYD_AUTOSCALER
  941. CELERYD_AUTOSCALER
  942. ~~~~~~~~~~~~~~~~~~
  943. .. versionadded:: 2.2
  944. Name of the autoscaler class to use.
  945. Default is ``"celery.worker.autoscale.Autoscaler"``.
  946. .. setting:: CELERYD_AUTORELOADER
  947. CELERYD_AUTORELOADER
  948. ~~~~~~~~~~~~~~~~~~~~
  949. Name of the autoreloader class used by the worker to reload
  950. Python modules and files that have changed.
  951. Default is: ``"celery.worker.autoreload.Autoreloader"``.
  952. .. setting:: CELERYD_CONSUMER
  953. CELERYD_CONSUMER
  954. ~~~~~~~~~~~~~~~~
  955. Name of the consumer class used by the worker.
  956. Default is :class:`celery.worker.consumer.Consumer`
  957. .. setting:: CELERYD_MEDIATOR
  958. CELERYD_MEDIATOR
  959. ~~~~~~~~~~~~~~~~
  960. Name of the mediator class used by the worker.
  961. Default is :class:`celery.worker.controllers.Mediator`.
  962. .. setting:: CELERYD_TIMER
  963. CELERYD_TIMER
  964. ~~~~~~~~~~~~~~~~~~~~~
  965. Name of the ETA scheduler class used by the worker.
  966. Default is :class:`celery.utils.timer2.Timer`, or one overrided
  967. by the pool implementation.
  968. .. _conf-celerybeat:
  969. Periodic Task Server: celery beat
  970. ---------------------------------
  971. .. setting:: CELERYBEAT_SCHEDULE
  972. CELERYBEAT_SCHEDULE
  973. ~~~~~~~~~~~~~~~~~~~
  974. The periodic task schedule used by :mod:`~celery.bin.beat`.
  975. See :ref:`beat-entries`.
  976. .. setting:: CELERYBEAT_SCHEDULER
  977. CELERYBEAT_SCHEDULER
  978. ~~~~~~~~~~~~~~~~~~~~
  979. The default scheduler class. Default is
  980. `"celery.beat.PersistentScheduler"`.
  981. Can also be set via the :option:`-S` argument to
  982. :mod:`~celery.bin.beat`.
  983. .. setting:: CELERYBEAT_SCHEDULE_FILENAME
  984. CELERYBEAT_SCHEDULE_FILENAME
  985. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  986. Name of the file used by `PersistentScheduler` to store the last run times
  987. of periodic tasks. Can be a relative or absolute path, but be aware that the
  988. suffix `.db` may be appended to the file name (depending on Python version).
  989. Can also be set via the :option:`--schedule` argument to
  990. :mod:`~celery.bin.beat`.
  991. .. setting:: CELERYBEAT_MAX_LOOP_INTERVAL
  992. CELERYBEAT_MAX_LOOP_INTERVAL
  993. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  994. The maximum number of seconds :mod:`~celery.bin.beat` can sleep
  995. between checking the schedule.
  996. The default for this value is scheduler specific.
  997. For the default celery beat scheduler the value is 300 (5 minutes),
  998. but for e.g. the django-celery database scheduler it is 5 seconds
  999. because the schedule may be changed externally, and so it must take
  1000. changes to the schedule into account.
  1001. Also when running celery beat embedded (:option:`-B`) on Jython as a thread
  1002. the max interval is overridden and set to 1 so that it's possible
  1003. to shut down in a timely manner.
  1004. .. _conf-celerymon:
  1005. Monitor Server: celerymon
  1006. -------------------------
  1007. .. setting:: CELERYMON_LOG_FORMAT
  1008. CELERYMON_LOG_FORMAT
  1009. ~~~~~~~~~~~~~~~~~~~~
  1010. The format to use for log messages.
  1011. Default is `[%(asctime)s: %(levelname)s/%(processName)s] %(message)s`
  1012. See the Python :mod:`logging` module for more information about log
  1013. formats.