periodic-tasks.rst 11 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275
  1. .. _guide-beat:
  2. ================
  3. Periodic Tasks
  4. ================
  5. .. contents::
  6. :local:
  7. Introduction
  8. ============
  9. :program:`celery beat` is a scheduler. It kicks off tasks at regular intervals,
  10. which are then executed by the worker nodes available in the cluster.
  11. By default the entries are taken from the :setting:`CELERYBEAT_SCHEDULE` setting,
  12. but custom stores can also be used, like storing the entries
  13. in an SQL database.
  14. You have to ensure only a single scheduler is running for a schedule
  15. at a time, otherwise you would end up with duplicate tasks. Using
  16. a centralized approach means the schedule does not have to be synchronized,
  17. and the service can operate without using locks.
  18. .. _beat-timezones:
  19. Time Zones
  20. ==========
  21. The periodic task schedules uses the UTC time zone by default,
  22. but you can change the time zone used using the :setting:`CELERY_TIMEZONE`
  23. setting.
  24. An example time zone could be `Europe/London`:
  25. .. code-block:: python
  26. CELERY_TIMEZONE = 'Europe/London'
  27. The default scheduler (storing the schedule in the :file:`celerybeat-schedule`
  28. file) will automatically detect that the time zone has changed, and so will
  29. reset the schedule itself, but other schedulers may not be so smart (e.g. the
  30. Django database scheduler, see below) and in that case you will have to reset the
  31. schedule manually.
  32. .. admonition:: Django Users
  33. Celery recommends and is compatible with the new ``USE_TZ`` setting introduced
  34. in Django 1.4.
  35. For Django users the time zone specified in the ``TIME_ZONE`` setting
  36. will be used, or you can specify a custom time zone for Celery alone
  37. by using the :setting:`CELERY_TIMEZONE` setting.
  38. The database scheduler will not reset when timezone related settings
  39. change, so you must do this manually:
  40. .. code-block:: bash
  41. $ python manage.py shell
  42. >>> from djcelery.models import PeriodicTask
  43. >>> PeriodicTask.objects.update(last_run_at=None)
  44. .. _beat-entries:
  45. Entries
  46. =======
  47. To schedule a task periodically you have to add an entry to the
  48. :setting:`CELERYBEAT_SCHEDULE` setting.
  49. Example: Run the `tasks.add` task every 30 seconds.
  50. .. code-block:: python
  51. from datetime import timedelta
  52. CELERYBEAT_SCHEDULE = {
  53. 'add-every-30-seconds': {
  54. 'task': 'tasks.add',
  55. 'schedule': timedelta(seconds=30),
  56. 'args': (16, 16)
  57. },
  58. }
  59. CELERY_TIMEZONE = 'UTC'
  60. Using a :class:`~datetime.timedelta` for the schedule means the task will
  61. be sent in 30 second intervals (the first task will be sent 30 seconds
  62. after `celery beat` starts, and then every 30 seconds
  63. after the last run).
  64. A crontab like schedule also exists, see the section on `Crontab schedules`_.
  65. Like with ``cron``, the tasks may overlap if the first task does not complete
  66. before the next. If that is a concern you should use a locking
  67. strategy to ensure only one instance can run at a time (see for example
  68. :ref:`cookbook-task-serial`).
  69. .. _beat-entry-fields:
  70. Available Fields
  71. ----------------
  72. * `task`
  73. The name of the task to execute.
  74. * `schedule`
  75. The frequency of execution.
  76. This can be the number of seconds as an integer, a
  77. :class:`~datetime.timedelta`, or a :class:`~celery.schedules.crontab`.
  78. You can also define your own custom schedule types, by extending the
  79. interface of :class:`~celery.schedules.schedule`.
  80. * `args`
  81. Positional arguments (:class:`list` or :class:`tuple`).
  82. * `kwargs`
  83. Keyword arguments (:class:`dict`).
  84. * `options`
  85. Execution options (:class:`dict`).
  86. This can be any argument supported by
  87. :meth:`~celery.task.base.Task.apply_async`,
  88. e.g. `exchange`, `routing_key`, `expires`, and so on.
  89. * `relative`
  90. By default :class:`~datetime.timedelta` schedules are scheduled
  91. "by the clock". This means the frequency is rounded to the nearest
  92. second, minute, hour or day depending on the period of the timedelta.
  93. If `relative` is true the frequency is not rounded and will be
  94. relative to the time when :program:`celery beat` was started.
  95. .. _beat-crontab:
  96. Crontab schedules
  97. =================
  98. If you want more control over when the task is executed, for
  99. example, a particular time of day or day of the week, you can use
  100. the :class:`~celery.schedules.crontab` schedule type:
  101. .. code-block:: python
  102. from celery.schedules import crontab
  103. CELERYBEAT_SCHEDULE = {
  104. # Executes every Monday morning at 7:30 A.M
  105. 'add-every-monday-morning': {
  106. 'task': 'tasks.add',
  107. 'schedule': crontab(hour=7, minute=30, day_of_week=1),
  108. 'args': (16, 16),
  109. },
  110. }
  111. The syntax of these crontab expressions are very flexible. Some examples:
  112. +-----------------------------------------+--------------------------------------------+
  113. | **Example** | **Meaning** |
  114. +-----------------------------------------+--------------------------------------------+
  115. | ``crontab()`` | Execute every minute. |
  116. +-----------------------------------------+--------------------------------------------+
  117. | ``crontab(minute=0, hour=0)`` | Execute daily at midnight. |
  118. +-----------------------------------------+--------------------------------------------+
  119. | ``crontab(minute=0, hour='*/3')`` | Execute every three hours: |
  120. | | 3am, 6am, 9am, noon, 3pm, 6pm, 9pm. |
  121. +-----------------------------------------+--------------------------------------------+
  122. | ``crontab(minute=0,`` | Same as previous. |
  123. | ``hour='0,3,6,9,12,15,18,21')`` | |
  124. +-----------------------------------------+--------------------------------------------+
  125. | ``crontab(minute='*/15')`` | Execute every 15 minutes. |
  126. +-----------------------------------------+--------------------------------------------+
  127. | ``crontab(day_of_week='sunday')`` | Execute every minute (!) at Sundays. |
  128. +-----------------------------------------+--------------------------------------------+
  129. | ``crontab(minute='*',`` | Same as previous. |
  130. | ``hour='*',`` | |
  131. | ``day_of_week='sun')`` | |
  132. +-----------------------------------------+--------------------------------------------+
  133. | ``crontab(minute='*/10',`` | Execute every ten minutes, but only |
  134. | ``hour='3,17,22',`` | between 3-4 am, 5-6 pm and 10-11 pm on |
  135. | ``day_of_week='thu,fri')`` | Thursdays or Fridays. |
  136. +-----------------------------------------+--------------------------------------------+
  137. | ``crontab(minute=0, hour='*/2,*/3')`` | Execute every even hour, and every hour |
  138. | | divisible by three. This means: |
  139. | | at every hour *except*: 1am, |
  140. | | 5am, 7am, 11am, 1pm, 5pm, 7pm, |
  141. | | 11pm |
  142. +-----------------------------------------+--------------------------------------------+
  143. | ``crontab(minute=0, hour='*/5')`` | Execute hour divisible by 5. This means |
  144. | | that it is triggered at 3pm, not 5pm |
  145. | | (since 3pm equals the 24-hour clock |
  146. | | value of "15", which is divisible by 5). |
  147. +-----------------------------------------+--------------------------------------------+
  148. | ``crontab(minute=0, hour='*/3,8-17')`` | Execute every hour divisible by 3, and |
  149. | | every hour during office hours (8am-5pm). |
  150. +-----------------------------------------+--------------------------------------------+
  151. | ``crontab(day_of_month='2')`` | Execute on the second day of every month. |
  152. | | |
  153. +-----------------------------------------+--------------------------------------------+
  154. | ``crontab(day_of_month='2-30/3')`` | Execute on every even numbered day. |
  155. | | |
  156. +-----------------------------------------+--------------------------------------------+
  157. | ``crontab(day_of_month='1-7,15-21')`` | Execute on the first and third weeks of |
  158. | | the month. |
  159. +-----------------------------------------+--------------------------------------------+
  160. | ``crontab(day_of_month='11',`` | Execute on 11th of May every year. |
  161. | ``month_of_year='5')`` | |
  162. +-----------------------------------------+--------------------------------------------+
  163. | ``crontab(month_of_year='*/3')`` | Execute on the first month of every |
  164. | | quarter. |
  165. +-----------------------------------------+--------------------------------------------+
  166. See :class:`celery.schedules.crontab` for more documentation.
  167. .. _beat-starting:
  168. Starting the Scheduler
  169. ======================
  170. To start the :program:`celery beat` service:
  171. .. code-block:: bash
  172. $ celery beat
  173. You can also start embed `beat` inside the worker by enabling
  174. workers `-B` option, this is convenient if you only intend to
  175. use one worker node:
  176. .. code-block:: bash
  177. $ celery worker -B
  178. Beat needs to store the last run times of the tasks in a local database
  179. file (named `celerybeat-schedule` by default), so it needs access to
  180. write in the current directory, or alternatively you can specify a custom
  181. location for this file:
  182. .. code-block:: bash
  183. $ celery beat -s /home/celery/var/run/celerybeat-schedule
  184. .. note::
  185. To daemonize beat see :ref:`daemonizing`.
  186. .. _beat-custom-schedulers:
  187. Using custom scheduler classes
  188. ------------------------------
  189. Custom scheduler classes can be specified on the command-line (the `-S`
  190. argument). The default scheduler is :class:`celery.beat.PersistentScheduler`,
  191. which is simply keeping track of the last run times in a local database file
  192. (a :mod:`shelve`).
  193. `django-celery` also ships with a scheduler that stores the schedule in the
  194. Django database:
  195. .. code-block:: bash
  196. $ celery beat -S djcelery.schedulers.DatabaseScheduler
  197. Using `django-celery`'s scheduler you can add, modify and remove periodic
  198. tasks from the Django Admin.