configuration.rst 28 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077107810791080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104110511061107110811091110111111121113111411151116111711181119112011211122112311241125112611271128112911301131113211331134113511361137113811391140114111421143114411451146114711481149115011511152115311541155115611571158115911601161116211631164116511661167116811691170117111721173117411751176117711781179118011811182118311841185118611871188118911901191119211931194119511961197
  1. .. _configuration:
  2. ============================
  3. Configuration and defaults
  4. ============================
  5. This document describes the configuration options available.
  6. If you're using the default loader, you must create the :file:`celeryconfig.py`
  7. module and make sure it is available on the Python path.
  8. .. contents::
  9. :local:
  10. :depth: 2
  11. .. _conf-example:
  12. Example configuration file
  13. ==========================
  14. This is an example configuration file to get you started.
  15. It should contain all you need to run a basic Celery set-up.
  16. .. code-block:: python
  17. # List of modules to import when celery starts.
  18. CELERY_IMPORTS = ("myapp.tasks", )
  19. ## Result store settings.
  20. CELERY_RESULT_BACKEND = "database"
  21. CELERY_RESULT_DBURI = "sqlite:///mydatabase.db"
  22. ## Broker settings.
  23. BROKER_HOST = "localhost"
  24. BROKER_PORT = 5672
  25. BROKER_VHOST = "/"
  26. BROKER_USER = "guest"
  27. BROKER_PASSWORD = "guest"
  28. ## Worker settings
  29. ## If you're doing mostly I/O you can have more processes,
  30. ## but if mostly spending CPU, try to keep it close to the
  31. ## number of CPUs on your machine. If not set, the number of CPUs/cores
  32. ## available will be used.
  33. CELERYD_CONCURRENCY = 10
  34. # CELERYD_LOG_FILE = "celeryd.log"
  35. # CELERYD_LOG_LEVEL = "INFO"
  36. Configuration Directives
  37. ========================
  38. .. _conf-concurrency:
  39. Concurrency settings
  40. --------------------
  41. .. setting:: CELERYD_CONCURRENCY
  42. CELERYD_CONCURRENCY
  43. ~~~~~~~~~~~~~~~~~~~
  44. The number of concurrent worker processes, executing tasks simultaneously.
  45. Defaults to the number of CPUs/cores available.
  46. .. setting:: CELERYD_PREFETCH_MULTIPLIER
  47. CELERYD_PREFETCH_MULTIPLIER
  48. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  49. How many messages to prefetch at a time multiplied by the number of
  50. concurrent processes. The default is 4 (four messages for each
  51. process). The default setting is usually a good choice, however -- if you
  52. have very long running tasks waiting in the queue and you have to start the
  53. workers, note that the first worker to start will receive four times the
  54. number of messages initially. Thus the tasks may not be fairly distributed
  55. to the workers.
  56. .. _conf-result-backend:
  57. Task result backend settings
  58. ----------------------------
  59. .. setting:: CELERY_RESULT_BACKEND
  60. CELERY_RESULT_BACKEND
  61. ~~~~~~~~~~~~~~~~~~~~~
  62. The backend used to store task results (tombstones).
  63. Can be one of the following:
  64. * database (default)
  65. Use a relational database supported by `SQLAlchemy`_.
  66. See :ref:`conf-database-result-backend`.
  67. * cache
  68. Use `memcached`_ to store the results.
  69. See :ref:`conf-cache-result-backend`.
  70. * mongodb
  71. Use `MongoDB`_ to store the results.
  72. See :ref:`conf-mongodb-result-backend`.
  73. * redis
  74. Use `Redis`_ to store the results.
  75. See :ref:`conf-redis-result-backend`.
  76. * tyrant
  77. Use `Tokyo Tyrant`_ to store the results.
  78. See :ref:`conf-tyrant-result-backend`.
  79. * amqp
  80. Send results back as AMQP messages
  81. See :ref:`conf-amqp-result-backend`.
  82. .. warning:
  83. While the AMQP result backend is very efficient, you must make sure
  84. you only receive the same result once. See :doc:`userguide/executing`).
  85. .. _`SQLAlchemy`: http://sqlalchemy.org
  86. .. _`memcached`: http://memcached.org
  87. .. _`MongoDB`: http://mongodb.org
  88. .. _`Redis`: http://code.google.com/p/redis/
  89. .. _`Tokyo Tyrant`: http://1978th.net/tokyotyrant/
  90. .. _conf-database-result-backend:
  91. Database backend settings
  92. -------------------------
  93. .. setting:: CELERY_RESULT_DBURI
  94. CELERY_RESULT_DBURI
  95. ~~~~~~~~~~~~~~~~~~~
  96. Please see `Supported Databases`_ for a table of supported databases.
  97. To use this backend you need to configure it with an
  98. `Connection String`_, some examples include:
  99. .. code-block:: python
  100. # sqlite (filename)
  101. CELERY_RESULT_DBURI = "sqlite:///celerydb.sqlite"
  102. # mysql
  103. CELERY_RESULT_DBURI = "mysql://scott:tiger@localhost/foo"
  104. # postgresql
  105. CELERY_RESULT_DBURI = "postgresql://scott:tiger@localhost/mydatabase"
  106. # oracle
  107. CELERY_RESULT_DBURI = "oracle://scott:tiger@127.0.0.1:1521/sidname"
  108. See `Connection String`_ for more information about connection
  109. strings.
  110. .. setting:: CELERY_RESULT_ENGINE_OPTIONS
  111. CELERY_RESULT_ENGINE_OPTIONS
  112. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  113. To specify additional SQLAlchemy database engine options you can use
  114. the :setting:`CELERY_RESULT_ENGINE_OPTIONS` setting::
  115. # echo enables verbose logging from SQLAlchemy.
  116. CELERY_RESULT_ENGINE_OPTIONS = {"echo": True}
  117. .. _`Supported Databases`:
  118. http://www.sqlalchemy.org/docs/dbengine.html#supported-databases
  119. .. _`Connection String`:
  120. http://www.sqlalchemy.org/docs/dbengine.html#create-engine-url-arguments
  121. Example configuration
  122. ~~~~~~~~~~~~~~~~~~~~~
  123. .. code-block:: python
  124. CELERY_RESULT_BACKEND = "database"
  125. CELERY_RESULT_DBURI = "mysql://user:password@host/dbname"
  126. .. _conf-amqp-result-backend:
  127. AMQP backend settings
  128. ---------------------
  129. .. setting:: CELERY_AMQP_TASK_RESULT_EXPIRES
  130. CELERY_AMQP_TASK_RESULT_EXPIRES
  131. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  132. The time in seconds of which the task result queues should expire.
  133. .. note::
  134. AMQP result expiration requires RabbitMQ versions 2.1.0 and higher.
  135. .. setting:: CELERY_RESULT_EXCHANGE
  136. CELERY_RESULT_EXCHANGE
  137. ~~~~~~~~~~~~~~~~~~~~~~
  138. Name of the exchange to publish results in. Default is `"celeryresults"`.
  139. .. setting:: CELERY_RESULT_EXCHANGE_TYPE
  140. CELERY_RESULT_EXCHANGE_TYPE
  141. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  142. The exchange type of the result exchange. Default is to use a `direct`
  143. exchange.
  144. .. setting:: CELERY_RESULT_SERIALIZER
  145. CELERY_RESULT_SERIALIZER
  146. ~~~~~~~~~~~~~~~~~~~~~~~~
  147. Result message serialization format. Default is `"pickle"`. See
  148. :ref:`executing-serializers`.
  149. .. setting:: CELERY_RESULT_PERSISTENT
  150. CELERY_RESULT_PERSISTENT
  151. ~~~~~~~~~~~~~~~~~~~~~~~~
  152. If set to :const:`True`, result messages will be persistent. This means the
  153. messages will not be lost after a broker restart. The default is for the
  154. results to be transient.
  155. Example configuration
  156. ~~~~~~~~~~~~~~~~~~~~~
  157. .. code-block:: python
  158. CELERY_RESULT_BACKEND = "amqp"
  159. CELERY_AMQP_TASK_RESULT_EXPIRES = 18000 # 5 hours.
  160. .. _conf-cache-result-backend:
  161. Cache backend settings
  162. ----------------------
  163. .. note::
  164. The cache backend supports the `pylibmc`_ and `python-memcached`
  165. libraries. The latter is used only if `pylibmc`_ is not installed.
  166. .. setting:: CELERY_CACHE_BACKEND
  167. CELERY_CACHE_BACKEND
  168. ~~~~~~~~~~~~~~~~~~~~
  169. Using a single memcached server:
  170. .. code-block:: python
  171. CELERY_CACHE_BACKEND = 'memcached://127.0.0.1:11211/'
  172. Using multiple memcached servers:
  173. .. code-block:: python
  174. CELERY_RESULT_BACKEND = "cache"
  175. CELERY_CACHE_BACKEND = 'memcached://172.19.26.240:11211;172.19.26.242:11211/'
  176. .. setting:: CELERY_CACHE_BACKEND_OPTIONS
  177. CELERY_CACHE_BACKEND_OPTIONS
  178. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  179. You can set pylibmc options using the :setting:`CELERY_CACHE_BACKEND_OPTIONS`
  180. setting:
  181. .. code-block:: python
  182. CELERY_CACHE_BACKEND_OPTIONS = {"binary": True,
  183. "behaviors": {"tcp_nodelay": True}}
  184. .. _`pylibmc`: http://sendapatch.se/projects/pylibmc/
  185. .. _conf-tyrant-result-backend:
  186. Tokyo Tyrant backend settings
  187. -----------------------------
  188. .. note::
  189. The Tokyo Tyrant backend requires the :mod:`pytyrant` library:
  190. http://pypi.python.org/pypi/pytyrant/
  191. This backend requires the following configuration directives to be set:
  192. .. setting:: TT_HOST
  193. TT_HOST
  194. ~~~~~~~
  195. Host name of the Tokyo Tyrant server.
  196. .. setting:: TT_PORT
  197. TT_PORT
  198. ~~~~~~~
  199. The port the Tokyo Tyrant server is listening to.
  200. Example configuration
  201. ~~~~~~~~~~~~~~~~~~~~~
  202. .. code-block:: python
  203. CELERY_RESULT_BACKEND = "tyrant"
  204. TT_HOST = "localhost"
  205. TT_PORT = 1978
  206. .. _conf-redis-result-backend:
  207. Redis backend settings
  208. ----------------------
  209. .. note::
  210. The Redis backend requires the :mod:`redis` library:
  211. http://pypi.python.org/pypi/redis/0.5.5
  212. To install the redis package use `pip` or `easy_install`::
  213. $ pip install redis
  214. This backend requires the following configuration directives to be set.
  215. .. setting:: REDIS_HOST
  216. REDIS_HOST
  217. ~~~~~~~~~~
  218. Host name of the Redis database server. e.g. `"localhost"`.
  219. .. setting:: REDIS_PORT
  220. REDIS_PORT
  221. ~~~~~~~~~~
  222. Port to the Redis database server. e.g. `6379`.
  223. .. setting:: REDIS_DB
  224. REDIS_DB
  225. ~~~~~~~~
  226. Database number to use. Default is 0
  227. .. setting:: REDIS_PASSWORD
  228. REDIS_PASSWORD
  229. ~~~~~~~~~~~~~~
  230. Password used to connect to the database.
  231. Example configuration
  232. ~~~~~~~~~~~~~~~~~~~~~
  233. .. code-block:: python
  234. CELERY_RESULT_BACKEND = "redis"
  235. REDIS_HOST = "localhost"
  236. REDIS_PORT = 6379
  237. REDIS_DB = 0
  238. REDIS_CONNECT_RETRY = True
  239. .. _conf-mongodb-result-backend:
  240. MongoDB backend settings
  241. ------------------------
  242. .. note::
  243. The MongoDB backend requires the :mod:`pymongo` library:
  244. http://github.com/mongodb/mongo-python-driver/tree/master
  245. .. setting:: CELERY_MONGODB_BACKEND_SETTINGS
  246. CELERY_MONGODB_BACKEND_SETTINGS
  247. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  248. This is a dict supporting the following keys:
  249. * host
  250. Host name of the MongoDB server. Defaults to "localhost".
  251. * port
  252. The port the MongoDB server is listening to. Defaults to 27017.
  253. * user
  254. User name to authenticate to the MongoDB server as (optional).
  255. * password
  256. Password to authenticate to the MongoDB server (optional).
  257. * database
  258. The database name to connect to. Defaults to "celery".
  259. * taskmeta_collection
  260. The collection name to store task meta data.
  261. Defaults to "celery_taskmeta".
  262. .. _example-mongodb-result-config:
  263. Example configuration
  264. ~~~~~~~~~~~~~~~~~~~~~
  265. .. code-block:: python
  266. CELERY_RESULT_BACKEND = "mongodb"
  267. CELERY_MONGODB_BACKEND_SETTINGS = {
  268. "host": "192.168.1.100",
  269. "port": 30000,
  270. "database": "mydb",
  271. "taskmeta_collection": "my_taskmeta_collection",
  272. }
  273. .. _conf-messaging:
  274. Message Routing
  275. ---------------
  276. .. _conf-messaging-routing:
  277. .. setting:: CELERY_QUEUES
  278. CELERY_QUEUES
  279. ~~~~~~~~~~~~~
  280. The mapping of queues the worker consumes from. This is a dictionary
  281. of queue name/options. See :ref:`guide-routing` for more information.
  282. The default is a queue/exchange/binding key of `"celery"`, with
  283. exchange type `direct`.
  284. You don't have to care about this unless you want custom routing facilities.
  285. .. setting:: CELERY_ROUTES
  286. CELERY_ROUTES
  287. ~~~~~~~~~~~~~
  288. A list of routers, or a single router used to route tasks to queues.
  289. When deciding the final destination of a task the routers are consulted
  290. in order. See :ref:`routers` for more information.
  291. .. setting:: CELERY_CREATE_MISSING_QUEUES
  292. CELERY_CREATE_MISSING_QUEUES
  293. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  294. If enabled (default), any queues specified that is not defined in
  295. :setting:`CELERY_QUEUES` will be automatically created. See
  296. :ref:`routing-automatic`.
  297. .. setting:: CELERY_DEFAULT_QUEUE
  298. CELERY_DEFAULT_QUEUE
  299. ~~~~~~~~~~~~~~~~~~~~
  300. The queue used by default, if no custom queue is specified. This queue must
  301. be listed in :setting:`CELERY_QUEUES`. The default is: `celery`.
  302. .. seealso::
  303. :ref:`routing-changing-default-queue`
  304. .. setting:: CELERY_DEFAULT_EXCHANGE
  305. CELERY_DEFAULT_EXCHANGE
  306. ~~~~~~~~~~~~~~~~~~~~~~~
  307. Name of the default exchange to use when no custom exchange is
  308. specified. The default is: `celery`.
  309. .. setting:: CELERY_DEFAULT_EXCHANGE_TYPE
  310. CELERY_DEFAULT_EXCHANGE_TYPE
  311. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  312. Default exchange type used when no custom exchange is specified.
  313. The default is: `direct`.
  314. .. setting:: CELERY_DEFAULT_ROUTING_KEY
  315. CELERY_DEFAULT_ROUTING_KEY
  316. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  317. The default routing key used when sending tasks.
  318. The default is: `celery`.
  319. .. setting:: CELERY_DEFAULT_DELIVERY_MODE
  320. CELERY_DEFAULT_DELIVERY_MODE
  321. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  322. Can be `transient` or `persistent`. The default is to send
  323. persistent messages.
  324. .. _conf-broker-connection:
  325. Broker Settings
  326. ---------------
  327. .. setting:: BROKER_BACKEND
  328. BROKER_BACKEND
  329. ~~~~~~~~~~~~~~
  330. The messaging backend to use. Default is `"amqplib"`.
  331. .. setting:: BROKER_HOST
  332. BROKER_HOST
  333. ~~~~~~~~~~~
  334. Hostname of the broker.
  335. .. setting:: BROKER_PORT
  336. BROKER_PORT
  337. ~~~~~~~~~~~
  338. Custom port of the broker. Default is to use the default port for the
  339. selected backend.
  340. .. setting:: BROKER_USER
  341. BROKER_USER
  342. ~~~~~~~~~~~
  343. Username to connect as.
  344. .. setting:: BROKER_PASSWORD
  345. BROKER_PASSWORD
  346. ~~~~~~~~~~~~~~~
  347. Password to connect with.
  348. .. setting:: BROKER_VHOST
  349. BROKER_VHOST
  350. ~~~~~~~~~~~~
  351. Virtual host. Default is `"/"`.
  352. .. setting:: BROKER_USE_SSL
  353. BROKER_USE_SSL
  354. ~~~~~~~~~~~~~~
  355. Use SSL to connect to the broker. Off by default. This may not be supported
  356. by all transports.
  357. .. setting:: BROKER_CONNECTION_TIMEOUT
  358. BROKER_CONNECTION_TIMEOUT
  359. ~~~~~~~~~~~~~~~~~~~~~~~~~
  360. The default timeout in seconds before we give up establishing a connection
  361. to the AMQP server. Default is 4 seconds.
  362. .. setting:: CELERY_BROKER_CONNECTION_RETRY
  363. BROKER_CONNECTION_RETRY
  364. ~~~~~~~~~~~~~~~~~~~~~~~
  365. Automatically try to re-establish the connection to the AMQP broker if lost.
  366. The time between retries is increased for each retry, and is
  367. not exhausted before :setting:`CELERY_BROKER_CONNECTION_MAX_RETRIES` is
  368. exceeded.
  369. This behavior is on by default.
  370. .. setting:: CELERY_BROKER_CONNECTION_MAX_RETRIES
  371. CELERY_BROKER_CONNECTION_MAX_RETRIES
  372. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  373. Maximum number of retries before we give up re-establishing a connection
  374. to the AMQP broker.
  375. If this is set to :const:`0` or :const:`None`, we will retry forever.
  376. Default is 100 retries.
  377. .. _conf-task-execution:
  378. Task execution settings
  379. -----------------------
  380. .. setting:: CELERY_ALWAYS_EAGER
  381. CELERY_ALWAYS_EAGER
  382. ~~~~~~~~~~~~~~~~~~~
  383. If this is :const:`True`, all tasks will be executed locally by blocking
  384. until it is finished. `apply_async` and `Task.delay` will return
  385. a :class:`~celery.result.EagerResult` which emulates the behavior of
  386. :class:`~celery.result.AsyncResult`, except the result has already
  387. been evaluated.
  388. Tasks will never be sent to the queue, but executed locally
  389. instead.
  390. .. setting:: CELERY_EAGER_PROPAGATES_EXCEPTIONS
  391. CELERY_EAGER_PROPAGATES_EXCEPTIONS
  392. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  393. If this is :const:`True`, eagerly executed tasks (using `.apply`, or with
  394. :setting:`CELERY_ALWAYS_EAGER` on), will raise exceptions.
  395. It's the same as always running `apply` with `throw=True`.
  396. .. setting:: CELERY_EAGER_RERAISES_EXCEPTIONS
  397. CELERY_EAGER_RERAISES_EXCEPTIONS
  398. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  399. If this is :const:`True`, eagerly executed tasks (using `.apply`, or with
  400. :setting:`CELERY_ALWAYS_EAGER` on), will re-raise exceptions with a full
  401. application traceback.
  402. It's the same as always running `apply` with `reraise_exceptions=True`.
  403. .. setting:: CELERY_IGNORE_RESULT
  404. CELERY_IGNORE_RESULT
  405. ~~~~~~~~~~~~~~~~~~~~
  406. Whether to store the task return values or not (tombstones).
  407. If you still want to store errors, just not successful return values,
  408. you can set :setting:`CELERY_STORE_ERRORS_EVEN_IF_IGNORED`.
  409. .. setting:: CELERY_TASK_RESULT_EXPIRES
  410. CELERY_TASK_RESULT_EXPIRES
  411. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  412. Time (in seconds, or a :class:`~datetime.timedelta` object) for when after
  413. stored task tombstones will be deleted.
  414. A built-in periodic task will delete the results after this time
  415. (:class:`celery.task.builtins.backend_cleanup`).
  416. .. note::
  417. For the moment this only works with the database, cache, redis and MongoDB
  418. backends. For the AMQP backend see
  419. :setting:`CELERY_AMQP_TASK_RESULT_EXPIRES`.
  420. When using the database or MongoDB backends, `celerybeat` must be
  421. running for the results to be expired.
  422. .. setting:: CELERY_MAX_CACHED_RESULTS
  423. CELERY_MAX_CACHED_RESULTS
  424. ~~~~~~~~~~~~~~~~~~~~~~~~~
  425. Total number of results to store before results are evicted from the
  426. result cache. The default is 5000.
  427. .. setting:: CELERY_TRACK_STARTED
  428. CELERY_TRACK_STARTED
  429. ~~~~~~~~~~~~~~~~~~~~
  430. If :const:`True` the task will report its status as "started" when the
  431. task is executed by a worker. The default value is :const:`False` as
  432. the normal behaviour is to not report that level of granularity. Tasks
  433. are either pending, finished, or waiting to be retried. Having a "started"
  434. state can be useful for when there are long running tasks and there is a
  435. need to report which task is currently running.
  436. .. setting:: CELERY_TASK_SERIALIZER
  437. CELERY_TASK_SERIALIZER
  438. ~~~~~~~~~~~~~~~~~~~~~~
  439. A string identifying the default serialization method to use. Can be
  440. `pickle` (default), `json`, `yaml`, `msgpack` or any custom serialization
  441. methods that have been registered with :mod:`kombu.serialization.registry`.
  442. .. seealso::
  443. :ref:`executing-serializers`.
  444. .. setting:: CELERY_DEFAULT_RATE_LIMIT
  445. CELERY_DEFAULT_RATE_LIMIT
  446. ~~~~~~~~~~~~~~~~~~~~~~~~~
  447. The global default rate limit for tasks.
  448. This value is used for tasks that does not have a custom rate limit
  449. The default is no rate limit.
  450. .. setting:: CELERY_DISABLE_RATE_LIMITS
  451. CELERY_DISABLE_RATE_LIMITS
  452. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  453. Disable all rate limits, even if tasks has explicit rate limits set.
  454. .. setting:: CELERY_ACKS_LATE
  455. CELERY_ACKS_LATE
  456. ~~~~~~~~~~~~~~~~
  457. Late ack means the task messages will be acknowledged **after** the task
  458. has been executed, not *just before*, which is the default behavior.
  459. .. seealso::
  460. FAQ: :ref:`faq-acks_late-vs-retry`.
  461. .. _conf-celeryd:
  462. Worker: celeryd
  463. ---------------
  464. .. setting:: CELERY_IMPORTS
  465. CELERY_IMPORTS
  466. ~~~~~~~~~~~~~~
  467. A sequence of modules to import when the celery daemon starts.
  468. This is used to specify the task modules to import, but also
  469. to import signal handlers and additional remote control commands, etc.
  470. .. setting:: CELERYD_MAX_TASKS_PER_CHILD
  471. CELERYD_MAX_TASKS_PER_CHILD
  472. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  473. Maximum number of tasks a pool worker process can execute before
  474. it's replaced with a new one. Default is no limit.
  475. .. setting:: CELERYD_TASK_TIME_LIMIT
  476. CELERYD_TASK_TIME_LIMIT
  477. ~~~~~~~~~~~~~~~~~~~~~~~
  478. Task hard time limit in seconds. The worker processing the task will
  479. be killed and replaced with a new one when this is exceeded.
  480. .. setting:: CELERYD_TASK_SOFT_TIME_LIMIT
  481. CELERYD_TASK_SOFT_TIME_LIMIT
  482. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  483. Task soft time limit in seconds.
  484. The :exc:`~celery.exceptions.SoftTimeLimitExceeded` exception will be
  485. raised when this is exceeded. The task can catch this to
  486. e.g. clean up before the hard time limit comes.
  487. Example:
  488. .. code-block:: python
  489. from celery.decorators import task
  490. from celery.exceptions import SoftTimeLimitExceeded
  491. @task()
  492. def mytask():
  493. try:
  494. return do_work()
  495. except SoftTimeLimitExceeded:
  496. cleanup_in_a_hurry()
  497. .. setting:: CELERY_STORE_ERRORS_EVEN_IF_IGNORED
  498. CELERY_STORE_ERRORS_EVEN_IF_IGNORED
  499. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  500. If set, the worker stores all task errors in the result store even if
  501. :attr:`Task.ignore_result <celery.task.base.Task.ignore_result>` is on.
  502. .. setting:: CELERYD_STATE_DB
  503. CELERYD_STATE_DB
  504. ~~~~~~~~~~~~~~~~
  505. Name of the file used to stores persistent worker state (like revoked tasks).
  506. Can be a relative or absolute path, but be aware that the suffix `.db`
  507. may be appended to the file name (depending on Python version).
  508. Can also be set via the :option:`--statedb` argument to
  509. :mod:`~celery.bin.celeryd`.
  510. Not enabled by default.
  511. .. setting:: CELERYD_ETA_SCHEDULER_PRECISION
  512. CELERYD_ETA_SCHEDULER_PRECISION
  513. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  514. Set the maximum time in seconds that the ETA scheduler can sleep between
  515. rechecking the schedule. Default is 1 second.
  516. Setting this value to 1 second means the schedulers precision will
  517. be 1 second. If you need near millisecond precision you can set this to 0.1.
  518. .. _conf-error-mails:
  519. Error E-Mails
  520. -------------
  521. .. setting:: CELERYD_SEND_TASK_ERROR_EMAILS
  522. CELERY_SEND_TASK_ERROR_EMAILS
  523. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  524. The default value for the `Task.send_error_emails` attribute, which if
  525. set to :const:`True` means errors occurring during task execution will be
  526. sent to :setting:`ADMINS` by e-mail.
  527. .. setting:: CELERY_TASK_ERROR_WHITELIST
  528. CELERY_TASK_ERROR_WHITELIST
  529. ~~~~~~~~~~~~~~~~~~~~~~~~~~~
  530. A white list of exceptions to send error e-mails for.
  531. .. setting:: ADMINS
  532. ADMINS
  533. ~~~~~~
  534. List of `(name, email_address)` tuples for the administrators that should
  535. receive error e-mails.
  536. .. setting:: SERVER_EMAIL
  537. SERVER_EMAIL
  538. ~~~~~~~~~~~~
  539. The e-mail address this worker sends e-mails from.
  540. Default is celery@localhost.
  541. .. setting:: MAIL_HOST
  542. MAIL_HOST
  543. ~~~~~~~~~
  544. The mail server to use. Default is `"localhost"`.
  545. .. setting:: MAIL_HOST_USER
  546. MAIL_HOST_USER
  547. ~~~~~~~~~~~~~~
  548. User name (if required) to log on to the mail server with.
  549. .. setting:: MAIL_HOST_PASSWORD
  550. MAIL_HOST_PASSWORD
  551. ~~~~~~~~~~~~~~~~~~
  552. Password (if required) to log on to the mail server with.
  553. .. setting:: MAIL_PORT
  554. MAIL_PORT
  555. ~~~~~~~~~
  556. The port the mail server is listening on. Default is `25`.
  557. .. _conf-example-error-mail-config:
  558. Example E-Mail configuration
  559. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  560. This configuration enables the sending of error e-mails to
  561. george@vandelay.com and kramer@vandelay.com:
  562. .. code-block:: python
  563. # Enables error e-mails.
  564. CELERY_SEND_TASK_ERROR_EMAILS = True
  565. # Name and e-mail addresses of recipients
  566. ADMINS = (
  567. ("George Costanza", "george@vandelay.com"),
  568. ("Cosmo Kramer", "kosmo@vandelay.com"),
  569. )
  570. # E-mail address used as sender (From field).
  571. SERVER_EMAIL = "no-reply@vandelay.com"
  572. # Mailserver configuration
  573. EMAIL_HOST = "mail.vandelay.com"
  574. EMAIL_PORT = 25
  575. # EMAIL_HOST_USER = "servers"
  576. # EMAIL_HOST_PASSWORD = "s3cr3t"
  577. .. _conf-events:
  578. Events
  579. ------
  580. .. setting:: CELERY_SEND_EVENTS
  581. CELERY_SEND_EVENTS
  582. ~~~~~~~~~~~~~~~~~~
  583. Send events so the worker can be monitored by tools like `celerymon`.
  584. .. setting:: CELERY_EVENT_QUEUE
  585. CELERY_EVENT_QUEUE
  586. ~~~~~~~~~~~~~~~~~~
  587. Name of the queue to consume event messages from. Default is
  588. `"celeryevent"`.
  589. .. setting:: CELERY_EVENT_EXCHANGE
  590. CELERY_EVENT_EXCHANGE
  591. ~~~~~~~~~~~~~~~~~~~~~
  592. Name of the exchange to send event messages to. Default is `"celeryevent"`.
  593. .. setting:: CELERY_EVENT_EXCHANGE_TYPE
  594. CELERY_EVENT_EXCHANGE_TYPE
  595. ~~~~~~~~~~~~~~~~~~~~~~~~~~
  596. The exchange type of the event exchange. Default is to use a `"direct"`
  597. exchange.
  598. .. setting:: CELERY_EVENT_ROUTING_KEY
  599. CELERY_EVENT_ROUTING_KEY
  600. ~~~~~~~~~~~~~~~~~~~~~~~~
  601. Routing key used when sending event messages. Default is `"celeryevent"`.
  602. .. setting:: CELERY_EVENT_SERIALIZER
  603. CELERY_EVENT_SERIALIZER
  604. ~~~~~~~~~~~~~~~~~~~~~~~
  605. Message serialization format used when sending event messages.
  606. Default is `"json"`. See :ref:`executing-serializers`.
  607. .. _conf-broadcast:
  608. Broadcast Commands
  609. ------------------
  610. .. setting:: CELERY_BROADCAST_QUEUE
  611. CELERY_BROADCAST_QUEUE
  612. ~~~~~~~~~~~~~~~~~~~~~~
  613. Name prefix for the queue used when listening for broadcast messages.
  614. The workers host name will be appended to the prefix to create the final
  615. queue name.
  616. Default is `"celeryctl"`.
  617. .. setting:: CELERY_BROADCASTS_EXCHANGE
  618. CELERY_BROADCAST_EXCHANGE
  619. ~~~~~~~~~~~~~~~~~~~~~~~~~
  620. Name of the exchange used for broadcast messages.
  621. Default is `"celeryctl"`.
  622. .. setting:: CELERY_BROADCAST_EXCHANGE_TYPE
  623. CELERY_BROADCAST_EXCHANGE_TYPE
  624. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  625. Exchange type used for broadcast messages. Default is `"fanout"`.
  626. .. _conf-logging:
  627. Logging
  628. -------
  629. .. setting:: CELERYD_LOG_FILE
  630. CELERYD_LOG_FILE
  631. ~~~~~~~~~~~~~~~~
  632. The default file name the worker daemon logs messages to. Can be overridden
  633. using the :option:`--logfile` option to :mod:`~celery.bin.celeryd`.
  634. The default is :const:`None` (`stderr`)
  635. .. setting:: CELERYD_LOG_LEVEL
  636. CELERYD_LOG_LEVEL
  637. ~~~~~~~~~~~~~~~~~
  638. Worker log level, can be one of :const:`DEBUG`, :const:`INFO`, :const:`WARNING`,
  639. :const:`ERROR` or :const:`CRITICAL`.
  640. Can also be set via the :option:`--loglevel` argument to
  641. :mod:`~celery.bin.celeryd`.
  642. See the :mod:`logging` module for more information.
  643. .. setting:: CELERYD_LOG_FORMAT
  644. CELERYD_LOG_FORMAT
  645. ~~~~~~~~~~~~~~~~~~
  646. The format to use for log messages.
  647. Default is `[%(asctime)s: %(levelname)s/%(processName)s] %(message)s`
  648. See the Python :mod:`logging` module for more information about log
  649. formats.
  650. .. setting:: CELERYD_TASK_LOG_FORMAT
  651. CELERYD_TASK_LOG_FORMAT
  652. ~~~~~~~~~~~~~~~~~~~~~~~
  653. The format to use for log messages logged in tasks. Can be overridden using
  654. the :option:`--loglevel` option to :mod:`~celery.bin.celeryd`.
  655. Default is::
  656. [%(asctime)s: %(levelname)s/%(processName)s]
  657. [%(task_name)s(%(task_id)s)] %(message)s
  658. See the Python :mod:`logging` module for more information about log
  659. formats.
  660. .. setting:: CELERY_REDIRECT_STDOUTS
  661. CELERY_REDIRECT_STDOUTS
  662. ~~~~~~~~~~~~~~~~~~~~~~~
  663. If enabled `stdout` and `stderr` will be redirected
  664. to the current logger.
  665. Enabled by default.
  666. Used by :program:`celeryd` and :program:`celerybeat`.
  667. .. setting:: CELERY_REDIRECT_STDOUTS_LEVEL
  668. CELERY_REDIRECT_STDOUTS_LEVEL
  669. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  670. The log level output to `stdout` and `stderr` is logged as.
  671. Can be one of :const:`DEBUG`, :const:`INFO`, :const:`WARNING`,
  672. :const:`ERROR` or :const:`CRITICAL`.
  673. Default is :const:`WARNING`.
  674. .. _conf-custom-components:
  675. Custom Component Classes (advanced)
  676. -----------------------------------
  677. .. setting:: CELERYD_POOL
  678. CELERYD_POOL
  679. ~~~~~~~~~~~~
  680. Name of the task pool class used by the worker.
  681. Default is :class:`celery.concurrency.processes.TaskPool`.
  682. .. setting:: CELERYD_CONSUMER
  683. CELERYD_CONSUMER
  684. ~~~~~~~~~~~~~~~~
  685. Name of the consumer class used by the worker.
  686. Default is :class:`celery.worker.consumer.Consumer`
  687. .. setting:: CELERYD_MEDIATOR
  688. CELERYD_MEDIATOR
  689. ~~~~~~~~~~~~~~~~
  690. Name of the mediator class used by the worker.
  691. Default is :class:`celery.worker.controllers.Mediator`.
  692. .. setting:: CELERYD_ETA_SCHEDULER
  693. CELERYD_ETA_SCHEDULER
  694. ~~~~~~~~~~~~~~~~~~~~~
  695. Name of the ETA scheduler class used by the worker.
  696. Default is :class:`celery.worker.controllers.ScheduleController`.
  697. .. _conf-celerybeat:
  698. Periodic Task Server: celerybeat
  699. --------------------------------
  700. .. setting:: CELERYBEAT_SCHEDULE
  701. CELERYBEAT_SCHEDULE
  702. ~~~~~~~~~~~~~~~~~~~
  703. The periodic task schedule used by :mod:`~celery.bin.celerybeat`.
  704. See :ref:`beat-entries`.
  705. .. setting:: CELERYBEAT_SCHEDULER
  706. CELERYBEAT_SCHEDULER
  707. ~~~~~~~~~~~~~~~~~~~~
  708. The default scheduler class. Default is
  709. `"celery.beat.PersistentScheduler"`.
  710. Can also be set via the :option:`-S` argument to
  711. :mod:`~celery.bin.celerybeat`.
  712. .. setting:: CELERYBEAT_SCHEDULE_FILENAME
  713. CELERYBEAT_SCHEDULE_FILENAME
  714. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  715. Name of the file used by `PersistentScheduler` to store the last run times
  716. of periodic tasks. Can be a relative or absolute path, but be aware that the
  717. suffix `.db` may be appended to the file name (depending on Python version).
  718. Can also be set via the :option:`--schedule` argument to
  719. :mod:`~celery.bin.celerybeat`.
  720. .. setting:: CELERYBEAT_MAX_LOOP_INTERVAL
  721. CELERYBEAT_MAX_LOOP_INTERVAL
  722. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  723. The maximum number of seconds :mod:`~celery.bin.celerybeat` can sleep
  724. between checking the schedule. Default is 300 seconds (5 minutes).
  725. .. setting:: CELERYBEAT_LOG_FILE
  726. CELERYBEAT_LOG_FILE
  727. ~~~~~~~~~~~~~~~~~~~
  728. The default file name to log messages to. Can be overridden using
  729. the `--logfile` option to :mod:`~celery.bin.celerybeat`.
  730. The default is :const:`None` (`stderr`).
  731. .. setting:: CELERYBEAT_LOG_LEVEL
  732. CELERYBEAT_LOG_LEVEL
  733. ~~~~~~~~~~~~~~~~~~~~
  734. Logging level. Can be any of :const:`DEBUG`, :const:`INFO`, :const:`WARNING`,
  735. :const:`ERROR`, or :const:`CRITICAL`.
  736. Can also be set via the :option:`--loglevel` argument to
  737. :mod:`~celery.bin.celerybeat`.
  738. See the :mod:`logging` module for more information.
  739. .. _conf-celerymon:
  740. Monitor Server: celerymon
  741. -------------------------
  742. .. setting:: CELERYMON_LOG_FILE
  743. CELERYMON_LOG_FILE
  744. ~~~~~~~~~~~~~~~~~~
  745. The default file name to log messages to. Can be overridden using
  746. the :option:`--logfile` argument to `celerymon`.
  747. The default is :const:`None` (`stderr`)
  748. .. setting:: CELERYMON_LOG_LEVEL
  749. CELERYMON_LOG_LEVEL
  750. ~~~~~~~~~~~~~~~~~~~
  751. Logging level. Can be any of :const:`DEBUG`, :const:`INFO`, :const:`WARNING`,
  752. :const:`ERROR`, or :const:`CRITICAL`.
  753. See the :mod:`logging` module for more information.