Changelog 31 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000100110021003100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033
  1. .. _changelog:
  2. ================
  3. Change history
  4. ================
  5. .. contents::
  6. :local:
  7. If you're looking for versions prior to 3.0.x you should go to :ref:`history`.
  8. .. _version-3.0.14:
  9. 3.0.14
  10. ======
  11. :release-date: 2013-02-08 05:00:00 P.M UTC
  12. - Now depends on Kombu 2.5.6
  13. - Now depends on billiard 2.7.3.20
  14. - Fixed a bug with how chords handles failures in subtasks (Issue #1172).
  15. Before if a task part of a chord raised an exception,
  16. that exception value would be included as part of the callback
  17. argument. This was not intended and the new behavior has been
  18. documented in :ref:`chord-errors`.
  19. - ``execv`` is now disabled by default.
  20. It was causing too many problems for users, you can still enable
  21. it using the :setting:`CELERY_FORCE_EXECV` setting.
  22. - ``GroupResult.join_native`` now respects the ``propagate`` argument.
  23. - ``subtask.id`` added as an alias to ``subtask['options'].id``
  24. .. code-block:: python
  25. >>> s = add.s(2, 2)
  26. >>> s.id = 'my-id'
  27. >>> s['options']
  28. {'task_id': 'my-id'}
  29. >>> s.id
  30. 'my-id'
  31. - worker: Fixed error `Could not start worker processes` occurring
  32. when restarting after connection failure (Issue #1118).
  33. - Adds new signal :signal:`task-retried` (Issue #1169).
  34. - `celery events --dumper` now handles connection loss.
  35. - Will now retry sending the task-sent event in case of connection failure.
  36. - amqp backend: Now uses ``Message.requeue`` instead of republishing
  37. the message after poll.
  38. - New :setting:`BROKER_HEARTBEAT_CHECKRATE` setting introduced to modify the
  39. rate at which broker connection heartbeats are monitored.
  40. - The stats broadcast command now includes the workers pid.
  41. Contributed by Mher Movsisyan.
  42. - New ``conf`` remote control command to get a workers current configuration.
  43. Contributed by Mher Movsisyan.
  44. - Adds the ability to modify the chord unlock task's countdown
  45. argument (Issue #1146).
  46. Contributed by Jun Sakai
  47. - beat: The scheduler now uses the `now()`` method of the schedule,
  48. so that schedules can provide a custom way to get the current date and time.
  49. Contributed by Raphaël Slinckx
  50. - Fixed pickling of configuration modules on Windows or when execv is used
  51. (Issue #1126).
  52. - Multiprocessing logger is now configured with loglevel ``ERROR``
  53. by default.
  54. You can still set the :envvar:`MP_LOG` environment variable
  55. to set it to value of the `--loglevel` argument.e
  56. - worker: Fixes bug with ignored and retried tasks.
  57. The ``on_chord_part_return`` and ``Task.after_return`` callbacks,
  58. nor the ``task_postrun`` signal should be called when the task was
  59. retried/ignored.
  60. Fix contributed by Vlad.
  61. .. _version-3.0.13:
  62. 3.0.13
  63. ======
  64. :release-date: 2013-01-07 04:00:00 P.M UTC
  65. - Now depends on Kombu 2.5
  66. - py-amqp has replaced amqplib as the default transport,
  67. gaining support for AMQP 0.9, and the RabbitMQ extensions
  68. including Consumer Cancel Notifications and heartbeats.
  69. - support for multiple connection URLs for failover.
  70. - Read more in the `Kombu 2.5 changelog`_.
  71. .. _`Kombu 2.5 changelog`:
  72. http://kombu.readthedocs.org/en/latest/changelog.html#version-2-5-0
  73. - Now depends on billiard 2.7.3.19
  74. - Fixed a deadlock issue that could occur when the producer pool
  75. inherited the connection pool instance of the parent process.
  76. - The :option:`--loader` option now works again (Issue #1066).
  77. - :program:`celery` umbrella command: All subcommands now supports
  78. the :option:`--workdir` option (Issue #1063).
  79. - Groups included in chains now give GroupResults (Issue #1057)
  80. Previously it would incorrectly add a regular result instead of a group
  81. result, but now this works:
  82. .. code-block:: python
  83. # [4 + 4, 4 + 8, 16 + 8]
  84. >>> res = (add.s(2, 2) | group(add.s(4), add.s(8), add.s(16)))()
  85. >>> res
  86. <GroupResult: a0acf905-c704-499e-b03a-8d445e6398f7 [
  87. 4346501c-cb99-4ad8-8577-12256c7a22b1,
  88. b12ead10-a622-4d44-86e9-3193a778f345,
  89. 26c7a420-11f3-4b33-8fac-66cd3b62abfd]>
  90. - Chains can now chain other chains and use partial arguments (Issue #1057).
  91. Example:
  92. .. code-block:: python
  93. >>> c1 = (add.s(2) | add.s(4))
  94. >>> c2 = (add.s(8) | add.s(16))
  95. >>> c3 = (c1 | c2)
  96. # 8 + 2 + 4 + 8 + 16
  97. >>> assert c3(8).get() == 38
  98. - Subtasks can now be used with unregistered tasks.
  99. You can specify subtasks even if you just have the name::
  100. >>> s = subtask(task_name, args=(), kwargs=())
  101. >>> s.delay()
  102. - The :program:`celery shell` command now always adds the current
  103. directory to the module path.
  104. - The worker will now properly handle the :exc:`pytz.AmbiguousTimeError`
  105. exception raised when an ETA/countdown is prepared while being in DST
  106. transition (Issue #1061).
  107. - force_execv: Now makes sure that task symbols in the original
  108. task modules will always use the correct app instance (Issue #1072).
  109. - AMQP Backend: Now republishes result messages that have been polled
  110. (using ``result.ready()`` and friends, ``result.get()`` will not do this
  111. in this version).
  112. - Crontab schedule values can now "wrap around"
  113. This means that values like ``11-1`` translates to ``[11, 12, 1]``.
  114. Contributed by Loren Abrams.
  115. - multi stopwait command now shows the pid of processes.
  116. Contributed by Loren Abrams.
  117. - Handling of ETA/countdown fixed when the :setting:`CELERY_ENABLE_UTC`
  118. setting is disabled (Issue #1065).
  119. - A number of uneeded properties were included in messages,
  120. caused by accidentally passing ``Queue.as_dict`` as message properties.
  121. - Rate limit values can now be float
  122. This also extends the string format so that values like ``"0.5/s"`` works.
  123. Contributed by Christoph Krybus
  124. - Fixed a typo in the broadcast routing documentation (Issue #1026).
  125. - Rewrote confusing section about idempotence in the task user guide.
  126. - Fixed typo in the daemonization tutorial (Issue #1055).
  127. - Fixed several typos in the documentation.
  128. Contributed by Marius Gedminas.
  129. - Batches: Now works when using the eventlet pool.
  130. Fix contributed by Thomas Grainger.
  131. - Batches: Added example sending results to :mod:`celery.contrib.batches`.
  132. Contributed by Thomas Grainger.
  133. - Mongodb backend: Connection ``max_pool_size`` can now be set in
  134. :setting:`CELERY_MONGODB_BACKEND_SETTINGS`.
  135. Contributed by Craig Younkins.
  136. - Fixed problem when using earlier versions of :mod:`pytz`.
  137. Fix contributed by Vlad.
  138. - Docs updated to include the default value for the
  139. :setting:`CELERY_TASK_RESULT_EXPIRES` setting.
  140. - Improvements to the django-celery tutorial.
  141. Contributed by Locker537.
  142. - The ``add_consumer`` control command did not properly persist
  143. the addition of new queues so that they survived connection failure
  144. (Issue #1079).
  145. 3.0.12
  146. ======
  147. :release-date: 2012-11-06 02:00 P.M UTC
  148. - Now depends on kombu 2.4.8
  149. - [Redis] New and improved fair queue cycle algorithm (Kevin McCarthy).
  150. - [Redis] Now uses a Redis-based mutex when restoring messages.
  151. - [Redis] Number of messages that can be restored in one interval is no
  152. longer limited (but can be set using the
  153. ``unacked_restore_limit``
  154. :setting:`transport option <BROKER_TRANSPORT_OPTIONS>`.)
  155. - Heartbeat value can be specified in broker URLs (Mher Movsisyan).
  156. - Fixed problem with msgpack on Python 3 (Jasper Bryant-Greene).
  157. - Now depends on billiard 2.7.3.18
  158. - Celery can now be used with static analysis tools like PyDev/PyCharm/pylint
  159. etc.
  160. - Development documentation has moved to Read The Docs.
  161. The new URL is: http://docs.celeryproject.org/en/master
  162. - New :setting:`CELERY_QUEUE_HA_POLICY` setting used to set the default
  163. HA policy for queues when using RabbitMQ.
  164. - New method ``Task.subtask_from_request`` returns a subtask using the current
  165. request.
  166. - Results get_many method did not respect timeout argument.
  167. Fix contributed by Remigiusz Modrzejewski
  168. - generic_init.d scripts now support setting :envvar:`CELERY_CREATE_DIRS` to
  169. always create log and pid directories (Issue #1045).
  170. This can be set in your :file:`/etc/default/celeryd`.
  171. - Fixed strange kombu import problem on Python 3.2 (Issue #1034).
  172. - Worker: ETA scheduler now uses millisecond precision (Issue #1040).
  173. - The ``--config`` argument to programs is now supported by all loaders.
  174. - The :setting:`CASSANDRA_OPTIONS` setting has now been documented.
  175. Contributed by Jared Biel.
  176. - Task methods (:mod:`celery.contrib.methods`) cannot be used with the old
  177. task base class, the task decorator in that module now inherits from the new.
  178. - An optimization was too eager and caused some logging messages to never emit.
  179. - :mod:`celery.contrib.batches` now works again.
  180. - Fixed missing whitespace in ``bdist_rpm`` requirements (Issue #1046).
  181. - Event state's ``tasks_by_name`` applied limit before filtering by name.
  182. Fix contributed by Alexander A. Sosnovskiy.
  183. .. _version-3.0.11:
  184. 3.0.11
  185. ======
  186. :release-date: 2012-09-26 04:00 P.M UTC
  187. - [security:low] generic-init.d scripts changed permissions of /var/log & /var/run
  188. In the daemonization tutorial the recommended directories were as follows:
  189. .. code-block:: bash
  190. CELERYD_LOG_FILE="/var/log/celery/%n.log"
  191. CELERYD_PID_FILE="/var/run/celery/%n.pid"
  192. But in the scripts themselves the default files were ``/var/log/celery%n.log``
  193. and ``/var/run/celery%n.pid``, so if the user did not change the location
  194. by configuration, the directories ``/var/log`` and ``/var/run`` would be
  195. created - and worse have their permissions and owners changed.
  196. This change means that:
  197. - Default pid file is ``/var/run/celery/%n.pid``
  198. - Default log file is ``/var/log/celery/%n.log``
  199. - The directories are only created and have their permissions
  200. changed if *no custom locations are set*.
  201. Users can force paths to be created by calling the ``create-paths``
  202. subcommand:
  203. .. code-block:: bash
  204. $ sudo /etc/init.d/celeryd create-paths
  205. .. admonition:: Upgrading Celery will not update init scripts
  206. To update the init scripts you have to re-download
  207. the files from source control and update them manually.
  208. You can find the init scripts for version 3.0.x at:
  209. http://github.com/celery/celery/tree/3.0/extra/generic-init.d
  210. - Now depends on billiard 2.7.3.17
  211. - Fixes request stack protection when app is initialized more than
  212. once (Issue #1003).
  213. - ETA tasks now properly works when system timezone is not the same
  214. as the configured timezone (Issue #1004).
  215. - Terminating a task now works if the task has been sent to the
  216. pool but not yet acknowledged by a pool process (Issue #1007).
  217. Fix contributed by Alexey Zatelepin
  218. - Terminating a task now properly updates the state of the task to revoked,
  219. and sends a ``task-revoked`` event.
  220. - Generic worker init script now waits for workers to shutdown by default.
  221. - Multi: No longer parses --app option (Issue #1008).
  222. - Multi: stop_verify command renamed to stopwait.
  223. - Daemonization: Now delays trying to create pidfile/logfile until after
  224. the working directory has been changed into.
  225. - :program:`celery worker` and :program:`celery beat` commands now respects
  226. the :option:`--no-color` option (Issue #999).
  227. - Fixed typos in eventlet examples (Issue #1000)
  228. Fix contributed by Bryan Bishop.
  229. Congratulations on opening bug #1000!
  230. - Tasks that raise :exc:`~celery.exceptions.Ignore` are now acknowledged.
  231. - Beat: Now shows the name of the entry in ``sending due task`` logs.
  232. .. _version-3.0.10:
  233. 3.0.10
  234. ======
  235. :release-date: 2012-09-20 05:30 P.M BST
  236. - Now depends on kombu 2.4.7
  237. - Now depends on billiard 2.7.3.14
  238. - Fixes crash at startup when using Django and pre-1.4 projects
  239. (setup_environ).
  240. - Hard time limits now sends the KILL signal shortly after TERM,
  241. to terminate processes that have signal handlers blocked by C extensions.
  242. - Billiard now installs even if the C extension cannot be built.
  243. It's still recommended to build the C extension if you are using
  244. a transport other than rabbitmq/redis (or use forced execv for some
  245. other reason).
  246. - Pool now sets a ``current_process().index`` attribute that can be used to create
  247. as many log files as there are processes in the pool.
  248. - Canvas: chord/group/chain no longer modifies the state when called
  249. Previously calling a chord/group/chain would modify the ids of subtasks
  250. so that:
  251. .. code-block:: python
  252. >>> c = chord([add.s(2, 2), add.s(4, 4)], xsum.s())
  253. >>> c()
  254. >>> c() <-- call again
  255. at the second time the ids for the tasks would be the same as in the
  256. previous invocation. This is now fixed, so that calling a subtask
  257. won't mutate any options.
  258. - Canvas: Chaining a chord to another task now works (Issue #965).
  259. - Worker: Fixed a bug where the request stack could be corrupted if
  260. relative imports are used.
  261. Problem usually manifested itself as an exception while trying to
  262. send a failed task result (``NoneType does not have id attribute``).
  263. Fix contributed by Sam Cooke.
  264. - Tasks can now raise :exc:`~celery.exceptions.Ignore` to skip updating states
  265. or events after return.
  266. Example:
  267. .. code-block:: python
  268. from celery.exceptions import Ignore
  269. @task
  270. def custom_revokes():
  271. if redis.sismember('tasks.revoked', custom_revokes.request.id):
  272. raise Ignore()
  273. - The worker now makes sure the request/task stacks are not modified
  274. by the initial ``Task.__call__``.
  275. This would previously be a problem if a custom task class defined
  276. ``__call__`` and also called ``super()``.
  277. - Because of problems the fast local optimization has been disabled,
  278. and can only be enabled by setting the :envvar:`USE_FAST_LOCALS` attribute.
  279. - Worker: Now sets a default socket timeout of 5 seconds at shutdown
  280. so that broken socket reads do not hinder proper shutdown (Issue #975).
  281. - More fixes related to late eventlet/gevent patching.
  282. - Documentation for settings out of sync with reality:
  283. - :setting:`CELERY_TASK_PUBLISH_RETRY`
  284. Documented as disabled by default, but it was enabled by default
  285. since 2.5 as stated by the 2.5 changelog.
  286. - :setting:`CELERY_TASK_PUBLISH_RETRY_POLICY`
  287. The default max_retries had been set to 100, but documented as being
  288. 3, and the interval_max was set to 1 but documented as 0.2.
  289. The default setting are now set to 3 and 0.2 as it was originally
  290. documented.
  291. Fix contributed by Matt Long.
  292. - Worker: Log messages when connection established and lost have been improved.
  293. - The repr of a crontab schedule value of '0' should be '*' (Issue #972).
  294. - Revoked tasks are now removed from reserved/active state in the worker
  295. (Issue #969)
  296. Fix contributed by Alexey Zatelepin.
  297. - gevent: Now supports hard time limits using ``gevent.Timeout``.
  298. - Documentation: Links to init scripts now point to the 3.0 branch instead
  299. of the development branch (master).
  300. - Documentation: Fixed typo in signals user guide (Issue #986).
  301. ``instance.app.queues`` -> ``instance.app.amqp.queues``.
  302. - Eventlet/gevent: The worker did not properly set the custom app
  303. for new greenlets.
  304. - Eventlet/gevent: Fixed a bug where the worker could not recover
  305. from connection loss (Issue #959).
  306. Also, because of a suspected bug in gevent the
  307. :setting:`BROKER_CONNECTION_TIMEOUT` setting has been disabled
  308. when using gevent
  309. 3.0.9
  310. =====
  311. :release-date: 2012-08-31 06:00 P.M BST
  312. - Important note for users of Django and the database scheduler!
  313. Recently a timezone issue has been fixed for periodic tasks,
  314. but erroneous timezones could have already been stored in the
  315. database, so for the fix to work you need to reset
  316. the ``last_run_at`` fields.
  317. You can do this by executing the following command:
  318. .. code-block:: bash
  319. $ python manage.py shell
  320. >>> from djcelery.models import PeriodicTask
  321. >>> PeriodicTask.objects.update(last_run_at=None)
  322. You also have to do this if you change the timezone or
  323. :setting:`CELERY_ENABLE_UTC` setting.
  324. - Note about the :setting:`CELERY_ENABLE_UTC` setting.
  325. If you previously disabled this just to force periodic tasks to work with
  326. your timezone, then you are now *encouraged to re-enable it*.
  327. - Now depends on Kombu 2.4.5 which fixes PyPy + Jython installation.
  328. - Fixed bug with timezones when :setting:`CELERY_ENABLE_UTC` is disabled
  329. (Issue #952).
  330. - Fixed a typo in the celerybeat upgrade mechanism (Issue #951).
  331. - Make sure the `exc_info` argument to logging is resolved (Issue #899).
  332. - Fixed problem with Python 3.2 and thread join timeout overflow (Issue #796).
  333. - A test case was occasionally broken for Python 2.5.
  334. - Unit test suite now passes for PyPy 1.9.
  335. - App instances now supports the with statement.
  336. This calls the new :meth:`~celery.Celery.close` method at exit, which
  337. cleans up after the app like closing pool connections.
  338. Note that this is only necessary when dynamically creating apps,
  339. e.g. for "temporary" apps.
  340. - Support for piping a subtask to a chain.
  341. For example:
  342. .. code-block:: python
  343. pipe = sometask.s() | othertask.s()
  344. new_pipe = mytask.s() | pipe
  345. Contributed by Steve Morin.
  346. - Fixed problem with group results on non-pickle serializers.
  347. Fix contributed by Steeve Morin.
  348. .. _version-3.0.8:
  349. 3.0.8
  350. =====
  351. :release-date: 2012-08-29 05:00 P.M BST
  352. - Now depends on Kombu 2.4.4
  353. - Fixed problem with amqplib and receiving larger message payloads
  354. (Issue #922).
  355. The problem would manifest itself as either the worker hanging,
  356. or occasionally a ``Framing error`` exception appearing.
  357. Users of the new ``pyamqp://`` transport must upgrade to
  358. :mod:`amqp` 0.9.3.
  359. - Beat: Fixed another timezone bug with interval and crontab schedules
  360. (Issue #943).
  361. - Beat: The schedule file is now automatically cleared if the timezone
  362. is changed.
  363. The schedule is also cleared when you upgrade to 3.0.8 from an earlier
  364. version, this to register the initial timezone info.
  365. - Events: The :event:`worker-heartbeat` event now include processed and active
  366. count fields.
  367. Contributed by Mher Movsisyan.
  368. - Fixed error with error email and new task classes (Issue #931).
  369. - ``BaseTask.__call__`` is no longer optimized away if it has been monkey
  370. patched.
  371. - Fixed shutdown issue when using gevent (Issue #911 & Issue #936).
  372. Fix contributed by Thomas Meson.
  373. .. _version-3.0.7:
  374. 3.0.7
  375. =====
  376. :release-date: 2012-08-24 05:00 P.M BST
  377. - Fixes several problems with periodic tasks and timezones (Issue #937).
  378. - Now depends on kombu 2.4.2
  379. - Redis: Fixes a race condition crash
  380. - Fixes an infinite loop that could happen when retrying establishing
  381. the broker connection.
  382. - Daemons now redirect standard file descriptors to :file:`/dev/null`
  383. Though by default the standard outs are also redirected
  384. to the logger instead, but you can disable this by changing
  385. the :setting:`CELERY_REDIRECT_STDOUTS` setting.
  386. - Fixes possible problems when eventlet/gevent is patched too late.
  387. - ``LoggingProxy`` no longer defines ``fileno()`` (Issue #928).
  388. - Results are now ignored for the chord unlock task.
  389. Fix contributed by Steeve Morin.
  390. - Cassandra backend now works if result expiry is disabled.
  391. Fix contributed by Steeve Morin.
  392. - The traceback object is now passed to signal handlers instead
  393. of the string representation.
  394. Fix contributed by Adam DePue.
  395. - Celery command: Extensions are now sorted by name.
  396. - A regression caused the :event:`task-failed` event to be sent
  397. with the exception object instead of its string representation.
  398. - The worker daemon would try to create the pid file before daemonizing
  399. to catch errors, but this file was not immediately released (Issue #923).
  400. - Fixes Jython compatibility.
  401. - ``billiard.forking_enable`` was called by all pools not just the
  402. processes pool, which would result in a useless warning if the billiard
  403. C extensions were not installed.
  404. .. _version-3.0.6:
  405. 3.0.6
  406. =====
  407. :release-date: 2012-08-17 11:00 P.M BST
  408. - Now depends on kombu 2.4.0
  409. - Now depends on billiard 2.7.3.12
  410. - Redis: Celery now tries to restore messages whenever there are no messages
  411. in the queue.
  412. - Crontab schedules now properly respects :setting:`CELERY_TIMEZONE` setting.
  413. It's important to note that crontab schedules uses UTC time by default
  414. unless this setting is set.
  415. Issue #904 and django-celery #150.
  416. - ``billiard.enable_forking`` is now only set by the processes pool.
  417. - The transport is now properly shown by :program:`celery report`
  418. (Issue #913).
  419. - The `--app` argument now works if the last part is a module name
  420. (Issue #921).
  421. - Fixed problem with unpickleable exceptions (billiard #12).
  422. - Adds ``task_name`` attribute to ``EagerResult`` which is always
  423. :const:`None` (Issue #907).
  424. - Old Task class in :mod:`celery.task` no longer accepts magic kwargs by
  425. default (Issue #918).
  426. A regression long ago disabled magic kwargs for these, and since
  427. no one has complained about it we don't have any incentive to fix it now.
  428. - The ``inspect reserved`` control command did not work properly.
  429. - Should now play better with static analyzation tools by explicitly
  430. specifying dynamically created attributes in the :mod:`celery` and
  431. :mod:`celery.task` modules.
  432. - Terminating a task now results in
  433. :exc:`~celery.exceptions.RevokedTaskError` instead of a ``WorkerLostError``.
  434. - ``AsyncResult.revoke`` now accepts ``terminate`` and ``signal`` arguments.
  435. - The :event:`task-revoked` event now includes new fields: ``terminated``,
  436. ``signum``, and ``expired``.
  437. - The argument to :class:`~celery.exceptions.TaskRevokedError` is now one
  438. of the reasons ``revoked``, ``expired`` or ``terminated``.
  439. - Old Task class does no longer use classmethods for push_request and
  440. pop_request (Issue #912).
  441. - ``GroupResult`` now supports the ``children`` attribute (Issue #916).
  442. - ``AsyncResult.collect`` now respects the ``intermediate`` argument
  443. (Issue #917).
  444. - Fixes example task in documentation (Issue #902).
  445. - Eventlet fixed so that the environment is patched as soon as possible.
  446. - eventlet: Now warns if celery related modules that depends on threads
  447. are imported before eventlet is patched.
  448. - Improved event and camera examples in the monitoring guide.
  449. - Disables celery command setuptools entrypoints if the command can't be
  450. loaded.
  451. - Fixed broken ``dump_request`` example in the tasks guide.
  452. .. _version-3.0.5:
  453. 3.0.5
  454. =====
  455. :release-date: 2012-08-01 04:00 P.M BST
  456. - Now depends on kombu 2.3.1 + billiard 2.7.3.11
  457. - Fixed a bug with the -B option (``cannot pickle thread.lock objects``)
  458. (Issue #894 + Issue #892, + django-celery #154).
  459. - The :control:`restart_pool` control command now requires the
  460. :setting:`CELERYD_POOL_RESTARTS` setting to be enabled
  461. This change was necessary as the multiprocessing event that the restart
  462. command depends on is responsible for creating many semaphores/file
  463. descriptors, resulting in problems in some environments.
  464. - ``chain.apply`` now passes args to the first task (Issue #889).
  465. - Documented previously secret options to the Django-Celery monitor
  466. in the monitoring userguide (Issue #396).
  467. - Old changelog are now organized in separate documents for each series,
  468. see :ref:`history`.
  469. .. _version-3.0.4:
  470. 3.0.4
  471. =====
  472. :release-date: 2012-07-26 07:00 P.M BST
  473. - Now depends on Kombu 2.3
  474. - New experimental standalone Celery monitor: Flower
  475. See :ref:`monitoring-flower` to read more about it!
  476. Contributed by Mher Movsisyan.
  477. - Now supports AMQP heartbeats if using the new ``pyamqp://`` transport.
  478. - The py-amqp transport requires the :mod:`amqp` library to be installed::
  479. $ pip install amqp
  480. - Then you need to set the transport URL prefix to ``pyamqp://``.
  481. - The default heartbeat value is 10 seconds, but this can be changed using
  482. the :setting:`BROKER_HEARTBEAT` setting::
  483. BROKER_HEARTBEAT = 5.0
  484. - If the broker heartbeat is set to 10 seconds, the heartbeats will be
  485. monitored every 5 seconds (double the hertbeat rate).
  486. See the `Kombu 2.3 changelog`_ for more information.
  487. .. _`Kombu 2.3 changelog`:
  488. http://kombu.readthedocs.org/en/latest/changelog.html#version-2-3-0
  489. - Now supports RabbitMQ Consumer Cancel Notifications, using the ``pyamqp://``
  490. transport.
  491. This is essential when running RabbitMQ in a cluster.
  492. See the `Kombu 2.3 changelog`_ for more information.
  493. - Delivery info is no longer passed directly through.
  494. It was discovered that the SQS transport adds objects that can't
  495. be pickled to the delivery info mapping, so we had to go back
  496. to using the whitelist again.
  497. Fixing this bug also means that the SQS transport is now working again.
  498. - The semaphore was not properly released when a task was revoked (Issue #877).
  499. This could lead to tasks being swallowed and not released until a worker
  500. restart.
  501. Thanks to Hynek Schlawack for debugging the issue.
  502. - Retrying a task now also forwards any linked tasks.
  503. This means that if a task is part of a chain (or linked in some other
  504. way) and that even if the task is retried, then the next task in the chain
  505. will be executed when the retry succeeds.
  506. - Chords: Now supports setting the interval and other keyword arguments
  507. to the chord unlock task.
  508. - The interval can now be set as part of the chord subtasks kwargs::
  509. chord(header)(body, interval=10.0)
  510. - In addition the chord unlock task now honors the Task.default_retry_delay
  511. option, used when none is specified, which also means that the default
  512. interval can also be changed using annotations:
  513. .. code-block:: python
  514. CELERY_ANNOTATIONS = {
  515. 'celery.chord_unlock': {
  516. 'default_retry_delay': 10.0,
  517. }
  518. }
  519. - New :meth:`@Celery.add_defaults` method can add new default configuration
  520. dicts to the applications configuration.
  521. For example::
  522. config = {'FOO': 10}
  523. celery.add_defaults(config)
  524. is the same as ``celery.conf.update(config)`` except that data will not be
  525. copied, and that it will not be pickled when the worker spawns child
  526. processes.
  527. In addition the method accepts a callable::
  528. def initialize_config():
  529. # insert heavy stuff that can't be done at import time here.
  530. celery.add_defaults(initialize_config)
  531. which means the same as the above except that it will not happen
  532. until the celery configuration is actually used.
  533. As an example, Celery can lazily use the configuration of a Flask app::
  534. flask_app = Flask()
  535. celery = Celery()
  536. celery.add_defaults(lambda: flask_app.config)
  537. - Revoked tasks were not marked as revoked in the result backend (Issue #871).
  538. Fix contributed by Hynek Schlawack.
  539. - Eventloop now properly handles the case when the epoll poller object
  540. has been closed (Issue #882).
  541. - Fixed syntax error in ``funtests/test_leak.py``
  542. Fix contributed by Catalin Iacob.
  543. - group/chunks: Now accepts empty task list (Issue #873).
  544. - New method names:
  545. - ``Celery.default_connection()`` ➠ :meth:`~@Celery.connection_or_acquire`.
  546. - ``Celery.default_producer()`` ➠ :meth:`~@Celery.producer_or_acquire`.
  547. The old names still work for backward compatibility.
  548. .. _version-3.0.3:
  549. 3.0.3
  550. =====
  551. :release-date: 2012-07-20 09:17 P.M BST
  552. :by: Ask Solem
  553. - amqplib passes the channel object as part of the delivery_info
  554. and it's not pickleable, so we now remove it.
  555. .. _version-3.0.2:
  556. 3.0.2
  557. =====
  558. :release-date: 2012-07-20 04:00 P.M BST
  559. :by: Ask Solem
  560. - A bug caused the following task options to not take defaults from the
  561. configuration (Issue #867 + Issue #858)
  562. The following settings were affected:
  563. - :setting:`CELERY_IGNORE_RESULT`
  564. - :setting:`CELERYD_SEND_TASK_ERROR_EMAILS`
  565. - :setting:`CELERY_TRACK_STARTED`
  566. - :setting:`CElERY_STORE_ERRORS_EVEN_IF_IGNORED`
  567. Fix contributed by John Watson.
  568. - Task Request: ``delivery_info`` is now passed through as-is (Issue #807).
  569. - The eta argument now supports datetime's with a timezone set (Issue #855).
  570. - The worker's banner displayed the autoscale settings in the wrong order
  571. (Issue #859).
  572. - Extension commands are now loaded after concurrency is set up
  573. so that they don't interfere with e.g. eventlet patching.
  574. - Fixed bug in the threaded pool (Issue #863)
  575. - The task failure handler mixed up the fields in :func:`sys.exc_info`.
  576. Fix contributed by Rinat Shigapov.
  577. - Fixed typos and wording in the docs.
  578. Fix contributed by Paul McMillan
  579. - New setting: :setting:`CELERY_WORKER_DIRECT`
  580. If enabled each worker will consume from their own dedicated queue
  581. which can be used to route tasks to specific workers.
  582. - Fixed several edge case bugs in the add consumer remote control command.
  583. - :mod:`~celery.contrib.migrate`: Can now filter and move tasks to specific
  584. workers if :setting:`CELERY_WORKER_DIRECT` is enabled.
  585. Among other improvements, the following functions have been added:
  586. * ``move_direct(filterfun, **opts)``
  587. * ``move_direct_by_id(task_id, worker_hostname, **opts)``
  588. * ``move_direct_by_idmap({task_id: worker_hostname, ...}, **opts)``
  589. * ``move_direct_by_taskmap({task_name: worker_hostname, ...}, **opts)``
  590. - :meth:`~celery.Celery.default_connection` now accepts a pool argument that
  591. if set to false causes a new connection to be created instead of acquiring
  592. one from the pool.
  593. - New signal: :signal:`celeryd_after_setup`.
  594. - Default loader now keeps lowercase attributes from the configuration module.
  595. .. _version-3.0.1:
  596. 3.0.1
  597. =====
  598. :release-date: 2012-07-10 06:00 P.M BST
  599. :by: Ask Solem
  600. - Now depends on kombu 2.2.5
  601. - inspect now supports limit argument::
  602. myapp.control.inspect(limit=1).ping()
  603. - Beat: now works with timezone aware datetime's.
  604. - Task classes inheriting ``from celery import Task``
  605. mistakingly enabled ``accept_magic_kwargs``.
  606. - Fixed bug in ``inspect scheduled`` (Issue #829).
  607. - Beat: Now resets the schedule to upgrade to UTC.
  608. - The :program:`celery worker` command now works with eventlet/gevent.
  609. Previously it would not patch the environment early enough.
  610. - The :program:`celery` command now supports extension commands
  611. using setuptools entry-points.
  612. Libraries can add additional commands to the :program:`celery`
  613. command by adding an entry-point like::
  614. setup(
  615. entry_points=[
  616. 'celery.commands': [
  617. 'foo = my.module:Command',
  618. ],
  619. ],
  620. ...)
  621. The command must then support the interface of
  622. :class:`celery.bin.base.Command`.
  623. - contrib.migrate: New utilities to move tasks from one queue to another.
  624. - :func:`~celery.contrib.migrate.move_tasks`
  625. - :func:`~celery.contrib.migrate.move_task_by_id`
  626. - The :event:`task-sent` event now contains ``exchange`` and ``routing_key``
  627. fields.
  628. - Fixes bug with installing on Python 3.
  629. Fix contributed by Jed Smith.
  630. .. _version-3.0.0:
  631. 3.0.0 (Chiastic Slide)
  632. ======================
  633. :release-date: 2012-07-07 01:30 P.M BST
  634. :by: Ask Solem
  635. See :ref:`whatsnew-3.0`.