changelog-3.0.rst 81 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077107810791080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104110511061107110811091110111111121113111411151116111711181119112011211122112311241125112611271128112911301131113211331134113511361137113811391140114111421143114411451146114711481149115011511152115311541155115611571158115911601161116211631164116511661167116811691170117111721173117411751176117711781179118011811182118311841185118611871188118911901191119211931194119511961197119811991200120112021203120412051206120712081209121012111212121312141215121612171218121912201221122212231224122512261227122812291230123112321233123412351236123712381239124012411242124312441245124612471248124912501251125212531254125512561257125812591260126112621263126412651266126712681269127012711272127312741275127612771278127912801281128212831284128512861287128812891290129112921293129412951296129712981299130013011302130313041305130613071308130913101311131213131314131513161317131813191320132113221323132413251326132713281329133013311332133313341335133613371338133913401341134213431344134513461347134813491350135113521353135413551356135713581359136013611362136313641365136613671368136913701371137213731374137513761377137813791380138113821383138413851386138713881389139013911392139313941395139613971398139914001401140214031404140514061407140814091410141114121413141414151416141714181419142014211422142314241425142614271428142914301431143214331434143514361437143814391440144114421443144414451446144714481449145014511452145314541455145614571458145914601461146214631464146514661467146814691470147114721473147414751476147714781479148014811482148314841485148614871488148914901491149214931494149514961497149814991500150115021503150415051506150715081509151015111512151315141515151615171518151915201521152215231524152515261527152815291530153115321533153415351536153715381539154015411542154315441545154615471548154915501551155215531554155515561557155815591560156115621563156415651566156715681569157015711572157315741575157615771578157915801581158215831584158515861587158815891590159115921593159415951596159715981599160016011602160316041605160616071608160916101611161216131614161516161617161816191620162116221623162416251626162716281629163016311632163316341635163616371638163916401641164216431644164516461647164816491650165116521653165416551656165716581659166016611662166316641665166616671668166916701671167216731674167516761677167816791680168116821683168416851686168716881689169016911692169316941695169616971698169917001701170217031704170517061707170817091710171117121713171417151716171717181719172017211722172317241725172617271728172917301731173217331734173517361737173817391740174117421743174417451746174717481749175017511752175317541755175617571758175917601761176217631764176517661767176817691770177117721773177417751776177717781779178017811782178317841785178617871788178917901791179217931794179517961797179817991800180118021803180418051806180718081809181018111812181318141815181618171818181918201821182218231824182518261827182818291830183118321833183418351836183718381839184018411842184318441845184618471848184918501851185218531854185518561857185818591860186118621863186418651866186718681869187018711872187318741875187618771878187918801881188218831884188518861887188818891890189118921893189418951896189718981899190019011902190319041905190619071908190919101911191219131914191519161917191819191920192119221923192419251926192719281929193019311932193319341935193619371938193919401941194219431944194519461947194819491950195119521953195419551956195719581959196019611962196319641965196619671968196919701971197219731974197519761977197819791980198119821983198419851986198719881989199019911992199319941995199619971998199920002001200220032004200520062007200820092010201120122013201420152016201720182019202020212022202320242025202620272028202920302031203220332034203520362037203820392040204120422043204420452046204720482049205020512052205320542055205620572058205920602061206220632064206520662067206820692070207120722073207420752076207720782079208020812082208320842085208620872088208920902091209220932094209520962097209820992100210121022103210421052106210721082109211021112112211321142115211621172118211921202121212221232124212521262127212821292130213121322133213421352136213721382139214021412142214321442145214621472148214921502151215221532154215521562157215821592160216121622163216421652166216721682169217021712172217321742175217621772178217921802181218221832184218521862187218821892190219121922193219421952196219721982199220022012202220322042205220622072208220922102211221222132214221522162217221822192220222122222223222422252226222722282229223022312232223322342235223622372238223922402241224222432244224522462247224822492250225122522253225422552256225722582259226022612262226322642265226622672268226922702271227222732274227522762277227822792280228122822283228422852286228722882289229022912292229322942295229622972298229923002301230223032304230523062307230823092310231123122313231423152316231723182319232023212322232323242325232623272328232923302331233223332334233523362337233823392340234123422343234423452346234723482349235023512352235323542355235623572358235923602361236223632364236523662367236823692370237123722373237423752376237723782379238023812382238323842385238623872388238923902391239223932394239523962397239823992400240124022403240424052406240724082409241024112412241324142415241624172418241924202421242224232424242524262427242824292430243124322433243424352436243724382439244024412442244324442445244624472448244924502451245224532454245524562457245824592460246124622463246424652466246724682469247024712472247324742475247624772478247924802481248224832484248524862487248824892490249124922493249424952496249724982499250025012502250325042505250625072508250925102511251225132514251525162517251825192520252125222523252425252526252725282529253025312532253325342535253625372538253925402541254225432544254525462547254825492550255125522553255425552556255725582559256025612562256325642565256625672568256925702571257225732574257525762577257825792580258125822583258425852586258725882589259025912592259325942595259625972598259926002601260226032604260526062607260826092610261126122613261426152616261726182619262026212622262326242625262626272628262926302631263226332634263526362637263826392640264126422643264426452646264726482649265026512652265326542655265626572658265926602661266226632664266526662667266826692670267126722673
  1. .. _changelog-3.0:
  2. ================
  3. Change history
  4. ================
  5. .. contents::
  6. :local:
  7. If you're looking for versions prior to 3.0.x you should go to :ref:`history`.
  8. .. _version-3.0.21:
  9. 3.0.21
  10. ======
  11. :release-date: 2013-07-05 16:30 P.M BST
  12. - Now depends on :mod:`billiard` 2.7.3.31.
  13. This version fixed a bug when running without the billiard C extension.
  14. - 3.0.20 broke eventlet/gevent support (worker not starting).
  15. - Fixed memory leak problem when MongoDB result backend was used with the
  16. gevent pool.
  17. Fix contributed by Ross Lawley.
  18. .. _version-3.0.20:
  19. 3.0.20
  20. ======
  21. :release-date: 2013-06-26 16:00 P.M BST
  22. - Now depends on :ref:`Kombu 2.5.11 <kombu:version-2.5.11>`.
  23. - ``--loader`` argument no longer supported importing loaders from the
  24. current directory.
  25. - [Worker] Fixed memory leak when restarting after connection lost
  26. (Issue #1325).
  27. - [Worker] Fixed unicode decode error at startup (Issue #1373).
  28. Fix contributed by Jessica Tallon.
  29. - [Worker] Now properly rewrites unpickleable exceptions again.
  30. - Fixed possible race condition when evicting items from the revoked task set.
  31. - [generic-init.d] Fixed compatibility with Ubuntu's minimal Dash
  32. shell (Issue #1387).
  33. Fix contributed by monkut.
  34. - ``Task.apply``/``ALWAYS_EAGER`` now also executes callbacks and errbacks
  35. (Issue #1336).
  36. - [Worker] The :signal:`worker-shutdown` signal was no longer being dispatched
  37. (Issue #1339).
  38. - [Python 3] Fixed problem with threading.Event.
  39. Fix contributed by Xavier Ordoquy.
  40. - [Python 3] Now handles ``io.UnsupportedOperation`` that may be raised
  41. by ``file.fileno()`` in Python 3.
  42. - [MongoDB backend] No longer uses deprecated ``safe`` parameter.
  43. Fix contributed by rfkrocktk
  44. - [Canvas] Fixed regression where immutable chord members may receive
  45. arguments (Issue #1340).
  46. Fix contributed by Peter Brook.
  47. - [Canvas] chain now accepts generator argument again (Issue #1319).
  48. - ``celery.migrate`` command now consumes from all queues if no queues
  49. specified.
  50. Fix contributed by John Watson.
  51. .. _version-3.0.19:
  52. 3.0.19
  53. ======
  54. :release-date: 2013-04-17 04:30:00 P.M BST
  55. - Now depends on :mod:`billiard` 2.7.3.28
  56. - A Python 3 related fix managed to disable the deadlock fix
  57. announced in 3.0.18.
  58. Tests have been added to make sure this does not happen again.
  59. - Task retry policy: Default max_retries is now 3.
  60. This ensures clients will not be hanging while the broker is down.
  61. .. note::
  62. You can set a longer retry for the worker by
  63. using the :signal:`celeryd_after_setup` signal:
  64. .. code-block:: python
  65. from celery.signals import celeryd_after_setup
  66. @celeryd_after_setup.connect
  67. def configure_worker(instance, conf, **kwargs):
  68. conf.CELERY_TASK_PUBLISH_RETRY_POLICY = {
  69. 'max_retries': 100,
  70. 'interval_start': 0,
  71. 'interval_max': 1,
  72. 'interval_step': 0.2,
  73. }
  74. - Worker: Will now properly display message body in error messages
  75. even if the body is a buffer instance.
  76. - 3.0.18 broke the MongoDB result backend (Issue #1303).
  77. .. _version-3.0.18:
  78. 3.0.18
  79. ======
  80. :release-date: 2013-04-12 05:00:00 P.M BST
  81. - Now depends on :mod:`kombu` 2.5.10.
  82. See the :ref:`kombu changelog <kombu:version-2.5.10>`.
  83. - Now depends on :mod:`billiard` 2.7.3.26.
  84. - Can now specify a whitelist of accepted serializers using
  85. the new :setting:`CELERY_ACCEPT_CONTENT` setting.
  86. This means that you can force the worker to discard messages
  87. serialized with pickle and other untrusted serializers.
  88. For example to only allow JSON serialized messages use::
  89. CELERY_ACCEPT_CONTENT = ['json']
  90. you can also specify MIME types in the whitelist::
  91. CELERY_ACCEPT_CONTENT = ['application/json']
  92. - Fixed deadlock in multiprocessing's pool caused by the
  93. semaphore not being released when terminated by signal.
  94. - Processes Pool: It's now possible to debug pool processes using GDB.
  95. - ``celery report`` now censors possibly secret settings, like passwords
  96. and secret tokens.
  97. You should still check the output before pasting anything
  98. on the internet.
  99. - Connection URLs now ignore multiple '+' tokens.
  100. - Worker/statedb: Now uses pickle protocol 2 (Py2.5+)
  101. - Fixed Python 3 compatibility issues.
  102. - Worker: A warning is now given if a worker is started with the
  103. same node name as an existing worker.
  104. - Worker: Fixed a deadlock that could occur while revoking tasks (Issue #1297).
  105. - Worker: The :sig:`HUP` handler now closes all open file descriptors
  106. before restarting to ensure file descriptors does not leak (Issue #1270).
  107. - Worker: Optimized storing/loading the revoked tasks list (Issue #1289).
  108. After this change the ``--statedb`` file will take up more disk space,
  109. but loading from and storing the revoked tasks will be considerably
  110. faster (what before took 5 minutes will now take less than a second).
  111. - Celery will now suggest alternatives if there's a typo in the
  112. broker transport name (e.g. ``ampq`` -> ``amqp``).
  113. - Worker: The auto-reloader would cause a crash if a monitored file
  114. was unlinked.
  115. Fix contributed by Agris Ameriks.
  116. - Fixed AsyncResult pickling error.
  117. Fix contributed by Thomas Minor.
  118. - Fixed handling of Unicode in logging output when using log colors
  119. (Issue #427).
  120. - :class:`~celery.app.utils.ConfigurationView` is now a ``MutableMapping``.
  121. Contributed by Aaron Harnly.
  122. - Fixed memory leak in LRU cache implementation.
  123. Fix contributed by Romuald Brunet.
  124. - ``celery.contrib.rdb``: Now works when sockets are in non-blocking mode.
  125. Fix contributed by Theo Spears.
  126. - The `inspect reserved` remote control command included active (started) tasks
  127. with the reserved tasks (Issue #1030).
  128. - The :signal:`task_failure` signal received a modified traceback object
  129. meant for pickling purposes, this has been fixed so that it now
  130. receives the real traceback instead.
  131. - The ``@task`` decorator silently ignored positional arguments,
  132. it now raises the expected :exc:`TypeError` instead (Issue #1125).
  133. - The worker will now properly handle messages with invalid
  134. eta/expires fields (Issue #1232).
  135. - The ``pool_restart`` remote control command now reports
  136. an error if the :setting:`CELERYD_POOL_RESTARTS` setting is not set.
  137. - ``celery.conf.add_defaults`` can now be used with non-dict objects.
  138. - Fixed compatibility problems in the Proxy class (Issue #1087).
  139. The class attributes ``__module__``, ``__name__`` and ``__doc__``
  140. are now meaningful string objects.
  141. Thanks to Marius Gedminas.
  142. - MongoDB Backend: The :setting:`MONGODB_BACKEND_SETTINGS` setting
  143. now accepts a ``option`` key that lets you forward arbitrary kwargs
  144. to the underlying ``pymongo.Connection`` object (Issue #1015).
  145. - Beat: The daily backend cleanup task is no longer enabled
  146. for result backends that support automatic result expiration (Issue #1031).
  147. - Canvas list operations now takes application instance from the first
  148. task in the list, instead of depending on the ``current_app`` (Issue #1249).
  149. - Worker: Message decoding error log message now includes traceback
  150. information.
  151. - Worker: The startup banner now includes system platform.
  152. - ``celery inspect|status|control`` now gives an error if used
  153. with an SQL based broker transport.
  154. .. _version-3.0.17:
  155. 3.0.17
  156. ======
  157. :release-date: 2013-03-22 04:00:00 P.M UTC
  158. - Now depends on kombu 2.5.8
  159. - Now depends on billiard 2.7.3.23
  160. - RabbitMQ/Redis: thread-less and lock-free rate-limit implementation.
  161. This means that rate limits pose minimal overhead when used with
  162. RabbitMQ/Redis or future transports using the eventloop,
  163. and that the rate-limit implementation is now thread-less and lock-free.
  164. The thread-based transports will still use the old implementation for
  165. now, but the plan is to use the timer also for other
  166. broker transports in Celery 3.1.
  167. - Rate limits now works with eventlet/gevent if using RabbitMQ/Redis as the
  168. broker.
  169. - A regression caused ``task.retry`` to ignore additional keyword arguments.
  170. Extra keyword arguments are now used as execution options again.
  171. Fix contributed by Simon Engledew.
  172. - Windows: Fixed problem with the worker trying to pickle the Django settings
  173. module at worker startup.
  174. - generic-init.d: No longer double quotes ``$CELERYD_CHDIR`` (Issue #1235).
  175. - generic-init.d: Removes bash-specific syntax.
  176. Fix contributed by Pär Wieslander.
  177. - Cassandra Result Backend: Now handles the
  178. :exc:`~pycassa.AllServersUnavailable` error (Issue #1010).
  179. Fix contributed by Jared Biel.
  180. - Result: Now properly forwards apps to GroupResults when deserializing
  181. (Issue #1249).
  182. Fix contributed by Charles-Axel Dein.
  183. - ``GroupResult.revoke`` now supports the ``terminate`` and ``signal``
  184. keyword arguments.
  185. - Worker: Multiprocessing pool workers now import task modules/configuration
  186. before setting up the logging system so that logging signals can be
  187. connected before they're dispatched.
  188. - chord: The ``AsyncResult`` instance returned now has its ``parent``
  189. attribute set to the header ``GroupResult``.
  190. This is consistent with how ``chain`` works.
  191. .. _version-3.0.16:
  192. 3.0.16
  193. ======
  194. :release-date: 2013-03-07 04:00:00 P.M UTC
  195. - Happy International Women's Day!
  196. We have a long way to go, so this is a chance for you to get involved in one
  197. of the organizations working for making our communities more
  198. diverse.
  199. - PyLadies — http://pyladies.com
  200. - Girls Who Code — http://www.girlswhocode.com
  201. - Women Who Code — http://www.meetup.com/Women-Who-Code-SF/
  202. - Now depends on :mod:`kombu` version 2.5.7
  203. - Now depends on :mod:`billiard` version 2.7.3.22
  204. - AMQP heartbeats are now disabled by default.
  205. Some users experiences issues with heartbeats enabled,
  206. and it's not strictly necessary to use them.
  207. If you're experiencing problems detecting connection failures,
  208. you can re-enable heartbeats by configuring the :setting:`BROKER_HEARTBEAT`
  209. setting.
  210. - Worker: Now propagates connection errors occurring in multiprocessing
  211. callbacks, so that the connection can be reset (Issue #1226).
  212. - Worker: Now propagates connection errors occurring in timer callbacks,
  213. so that the connection can be reset.
  214. - The modules in :setting:`CELERY_IMPORTS` and :setting:`CELERY_INCLUDE`
  215. are now imported in the original order (Issue #1161).
  216. The modules in :setting:`CELERY_IMPORTS` will be imported first,
  217. then continued by :setting:`CELERY_INCLUDE`.
  218. Thanks to Joey Wilhelm.
  219. - New bash completion for ``celery`` available in the git repository:
  220. https://github.com/celery/celery/tree/3.0/extra/bash-completion
  221. You can source this file or put it in ``bash_completion.d`` to
  222. get auto-completion for the ``celery`` command-line utility.
  223. - The node name of a worker can now include unicode characters (Issue #1186).
  224. - The repr of a ``crontab`` object now displays correctly (Issue #972).
  225. - ``events.State`` no longer modifies the original event dictionary.
  226. - No longer uses ``Logger.warn`` deprecated in Python 3.
  227. - Cache Backend: Now works with chords again (Issue #1094).
  228. - Chord unlock now handles errors occurring while calling the callback.
  229. - Generic worker init.d script: Status check is now performed by
  230. querying the pid of the instance instead of sending messages.
  231. Contributed by Milen Pavlov.
  232. - Improved init scripts for CentOS.
  233. - Updated to support celery 3.x conventions.
  234. - Now uses CentOS built-in ``status`` and ``killproc``
  235. - Support for multi-node / multi-pid worker services.
  236. - Standard color-coded CentOS service-init output.
  237. - A test suite.
  238. Contributed by Milen Pavlov.
  239. - ``ResultSet.join`` now always works with empty result set (Issue #1219).
  240. - A ``group`` consisting of a single task is now supported (Issue #1219).
  241. - Now supports the ``pycallgraph`` program (Issue #1051).
  242. - Fixed Jython compatibility problems.
  243. - Django tutorial: Now mentions that the example app must be added to
  244. ``INSTALLED_APPS`` (Issue #1192).
  245. .. _version-3.0.15:
  246. 3.0.15
  247. ======
  248. :release-date: 2013-02-11 04:30:00 P.M UTC
  249. - Now depends on billiard 2.7.3.21 which fixed a syntax error crash.
  250. - Fixed bug with :setting:`CELERY_SEND_TASK_SENT_EVENT`.
  251. .. _version-3.0.14:
  252. 3.0.14
  253. ======
  254. :release-date: 2013-02-08 05:00:00 P.M UTC
  255. - Now depends on Kombu 2.5.6
  256. - Now depends on billiard 2.7.3.20
  257. - ``execv`` is now disabled by default.
  258. It was causing too many problems for users, you can still enable
  259. it using the :setting:`CELERYD_FORCE_EXECV` setting.
  260. execv was only enabled when transports other than amqp/redis was used,
  261. and it's there to prevent deadlocks caused by mutexes not being released
  262. before the process forks. Sadly it also changes the environment
  263. introducing many corner case bugs that is hard to fix without adding
  264. horrible hacks. Deadlock issues are reported far less often than the
  265. bugs that execv are causing, so we now disable it by default.
  266. Work is in motion to create non-blocking versions of these transports
  267. so that execv is not necessary (which is the situation with the amqp
  268. and redis broker transports)
  269. - Chord exception behavior defined (Issue #1172).
  270. From Celery 3.1 the chord callback will change state to FAILURE
  271. when a task part of a chord raises an exception.
  272. It was never documented what happens in this case,
  273. and the actual behavior was very unsatisfactory, indeed
  274. it will just forward the exception value to the chord callback.
  275. For backward compatibility reasons we do not change to the new
  276. behavior in a bugfix release, even if the current behavior was
  277. never documented. Instead you can enable the
  278. :setting:`CELERY_CHORD_PROPAGATES` setting to get the new behavior
  279. that will be default from Celery 3.1.
  280. See more at :ref:`chord-errors`.
  281. - worker: Fixes bug with ignored and retried tasks.
  282. The ``on_chord_part_return`` and ``Task.after_return`` callbacks,
  283. nor the ``task_postrun`` signal should be called when the task was
  284. retried/ignored.
  285. Fix contributed by Vlad.
  286. - ``GroupResult.join_native`` now respects the ``propagate`` argument.
  287. - ``subtask.id`` added as an alias to ``subtask['options'].id``
  288. .. code-block:: python
  289. >>> s = add.s(2, 2)
  290. >>> s.id = 'my-id'
  291. >>> s['options']
  292. {'task_id': 'my-id'}
  293. >>> s.id
  294. 'my-id'
  295. - worker: Fixed error `Could not start worker processes` occurring
  296. when restarting after connection failure (Issue #1118).
  297. - Adds new signal :signal:`task-retried` (Issue #1169).
  298. - `celery events --dumper` now handles connection loss.
  299. - Will now retry sending the task-sent event in case of connection failure.
  300. - amqp backend: Now uses ``Message.requeue`` instead of republishing
  301. the message after poll.
  302. - New :setting:`BROKER_HEARTBEAT_CHECKRATE` setting introduced to modify the
  303. rate at which broker connection heartbeats are monitored.
  304. The default value was also changed from 3.0 to 2.0.
  305. - :class:`celery.events.state.State` is now pickleable.
  306. Fix contributed by Mher Movsisyan.
  307. - :class:`celery.datastructures.LRUCache` is now pickleable.
  308. Fix contributed by Mher Movsisyan.
  309. - The stats broadcast command now includes the workers pid.
  310. Contributed by Mher Movsisyan.
  311. - New ``conf`` remote control command to get a workers current configuration.
  312. Contributed by Mher Movsisyan.
  313. - Adds the ability to modify the chord unlock task's countdown
  314. argument (Issue #1146).
  315. Contributed by Jun Sakai
  316. - beat: The scheduler now uses the `now()`` method of the schedule,
  317. so that schedules can provide a custom way to get the current date and time.
  318. Contributed by Raphaël Slinckx
  319. - Fixed pickling of configuration modules on Windows or when execv is used
  320. (Issue #1126).
  321. - Multiprocessing logger is now configured with loglevel ``ERROR``
  322. by default.
  323. Since 3.0 the multiprocessing loggers were disabled by default
  324. (only configured when the :envvar:`MP_LOG` environment variable was set).
  325. .. _version-3.0.13:
  326. 3.0.13
  327. ======
  328. :release-date: 2013-01-07 04:00:00 P.M UTC
  329. - Now depends on Kombu 2.5
  330. - py-amqp has replaced amqplib as the default transport,
  331. gaining support for AMQP 0.9, and the RabbitMQ extensions
  332. including Consumer Cancel Notifications and heartbeats.
  333. - support for multiple connection URLs for failover.
  334. - Read more in the `Kombu 2.5 changelog`_.
  335. .. _`Kombu 2.5 changelog`:
  336. http://kombu.readthedocs.org/en/latest/changelog.html#version-2-5-0
  337. - Now depends on billiard 2.7.3.19
  338. - Fixed a deadlock issue that could occur when the producer pool
  339. inherited the connection pool instance of the parent process.
  340. - The :option:`--loader` option now works again (Issue #1066).
  341. - :program:`celery` umbrella command: All subcommands now supports
  342. the :option:`--workdir` option (Issue #1063).
  343. - Groups included in chains now give GroupResults (Issue #1057)
  344. Previously it would incorrectly add a regular result instead of a group
  345. result, but now this works:
  346. .. code-block:: python
  347. # [4 + 4, 4 + 8, 16 + 8]
  348. >>> res = (add.s(2, 2) | group(add.s(4), add.s(8), add.s(16)))()
  349. >>> res
  350. <GroupResult: a0acf905-c704-499e-b03a-8d445e6398f7 [
  351. 4346501c-cb99-4ad8-8577-12256c7a22b1,
  352. b12ead10-a622-4d44-86e9-3193a778f345,
  353. 26c7a420-11f3-4b33-8fac-66cd3b62abfd]>
  354. - Chains can now chain other chains and use partial arguments (Issue #1057).
  355. Example:
  356. .. code-block:: python
  357. >>> c1 = (add.s(2) | add.s(4))
  358. >>> c2 = (add.s(8) | add.s(16))
  359. >>> c3 = (c1 | c2)
  360. # 8 + 2 + 4 + 8 + 16
  361. >>> assert c3(8).get() == 38
  362. - Subtasks can now be used with unregistered tasks.
  363. You can specify subtasks even if you just have the name::
  364. >>> s = subtask(task_name, args=(), kwargs=())
  365. >>> s.delay()
  366. - The :program:`celery shell` command now always adds the current
  367. directory to the module path.
  368. - The worker will now properly handle the :exc:`pytz.AmbiguousTimeError`
  369. exception raised when an ETA/countdown is prepared while being in DST
  370. transition (Issue #1061).
  371. - force_execv: Now makes sure that task symbols in the original
  372. task modules will always use the correct app instance (Issue #1072).
  373. - AMQP Backend: Now republishes result messages that have been polled
  374. (using ``result.ready()`` and friends, ``result.get()`` will not do this
  375. in this version).
  376. - Crontab schedule values can now "wrap around"
  377. This means that values like ``11-1`` translates to ``[11, 12, 1]``.
  378. Contributed by Loren Abrams.
  379. - multi stopwait command now shows the pid of processes.
  380. Contributed by Loren Abrams.
  381. - Handling of ETA/countdown fixed when the :setting:`CELERY_ENABLE_UTC`
  382. setting is disabled (Issue #1065).
  383. - A number of uneeded properties were included in messages,
  384. caused by accidentally passing ``Queue.as_dict`` as message properties.
  385. - Rate limit values can now be float
  386. This also extends the string format so that values like ``"0.5/s"`` works.
  387. Contributed by Christoph Krybus
  388. - Fixed a typo in the broadcast routing documentation (Issue #1026).
  389. - Rewrote confusing section about idempotence in the task user guide.
  390. - Fixed typo in the daemonization tutorial (Issue #1055).
  391. - Fixed several typos in the documentation.
  392. Contributed by Marius Gedminas.
  393. - Batches: Now works when using the eventlet pool.
  394. Fix contributed by Thomas Grainger.
  395. - Batches: Added example sending results to :mod:`celery.contrib.batches`.
  396. Contributed by Thomas Grainger.
  397. - Mongodb backend: Connection ``max_pool_size`` can now be set in
  398. :setting:`CELERY_MONGODB_BACKEND_SETTINGS`.
  399. Contributed by Craig Younkins.
  400. - Fixed problem when using earlier versions of :mod:`pytz`.
  401. Fix contributed by Vlad.
  402. - Docs updated to include the default value for the
  403. :setting:`CELERY_TASK_RESULT_EXPIRES` setting.
  404. - Improvements to the django-celery tutorial.
  405. Contributed by Locker537.
  406. - The ``add_consumer`` control command did not properly persist
  407. the addition of new queues so that they survived connection failure
  408. (Issue #1079).
  409. 3.0.12
  410. ======
  411. :release-date: 2012-11-06 02:00 P.M UTC
  412. - Now depends on kombu 2.4.8
  413. - [Redis] New and improved fair queue cycle algorithm (Kevin McCarthy).
  414. - [Redis] Now uses a Redis-based mutex when restoring messages.
  415. - [Redis] Number of messages that can be restored in one interval is no
  416. longer limited (but can be set using the
  417. ``unacked_restore_limit``
  418. :setting:`transport option <BROKER_TRANSPORT_OPTIONS>`.)
  419. - Heartbeat value can be specified in broker URLs (Mher Movsisyan).
  420. - Fixed problem with msgpack on Python 3 (Jasper Bryant-Greene).
  421. - Now depends on billiard 2.7.3.18
  422. - Celery can now be used with static analysis tools like PyDev/PyCharm/pylint
  423. etc.
  424. - Development documentation has moved to Read The Docs.
  425. The new URL is: http://docs.celeryproject.org/en/master
  426. - New :setting:`CELERY_QUEUE_HA_POLICY` setting used to set the default
  427. HA policy for queues when using RabbitMQ.
  428. - New method ``Task.subtask_from_request`` returns a subtask using the current
  429. request.
  430. - Results get_many method did not respect timeout argument.
  431. Fix contributed by Remigiusz Modrzejewski
  432. - generic_init.d scripts now support setting :envvar:`CELERY_CREATE_DIRS` to
  433. always create log and pid directories (Issue #1045).
  434. This can be set in your :file:`/etc/default/celeryd`.
  435. - Fixed strange kombu import problem on Python 3.2 (Issue #1034).
  436. - Worker: ETA scheduler now uses millisecond precision (Issue #1040).
  437. - The ``--config`` argument to programs is now supported by all loaders.
  438. - The :setting:`CASSANDRA_OPTIONS` setting has now been documented.
  439. Contributed by Jared Biel.
  440. - Task methods (:mod:`celery.contrib.methods`) cannot be used with the old
  441. task base class, the task decorator in that module now inherits from the new.
  442. - An optimization was too eager and caused some logging messages to never emit.
  443. - :mod:`celery.contrib.batches` now works again.
  444. - Fixed missing whitespace in ``bdist_rpm`` requirements (Issue #1046).
  445. - Event state's ``tasks_by_name`` applied limit before filtering by name.
  446. Fix contributed by Alexander A. Sosnovskiy.
  447. .. _version-3.0.11:
  448. 3.0.11
  449. ======
  450. :release-date: 2012-09-26 04:00 P.M UTC
  451. - [security:low] generic-init.d scripts changed permissions of /var/log & /var/run
  452. In the daemonization tutorial the recommended directories were as follows:
  453. .. code-block:: bash
  454. CELERYD_LOG_FILE="/var/log/celery/%n.log"
  455. CELERYD_PID_FILE="/var/run/celery/%n.pid"
  456. But in the scripts themselves the default files were ``/var/log/celery%n.log``
  457. and ``/var/run/celery%n.pid``, so if the user did not change the location
  458. by configuration, the directories ``/var/log`` and ``/var/run`` would be
  459. created - and worse have their permissions and owners changed.
  460. This change means that:
  461. - Default pid file is ``/var/run/celery/%n.pid``
  462. - Default log file is ``/var/log/celery/%n.log``
  463. - The directories are only created and have their permissions
  464. changed if *no custom locations are set*.
  465. Users can force paths to be created by calling the ``create-paths``
  466. subcommand:
  467. .. code-block:: bash
  468. $ sudo /etc/init.d/celeryd create-paths
  469. .. admonition:: Upgrading Celery will not update init scripts
  470. To update the init scripts you have to re-download
  471. the files from source control and update them manually.
  472. You can find the init scripts for version 3.0.x at:
  473. http://github.com/celery/celery/tree/3.0/extra/generic-init.d
  474. - Now depends on billiard 2.7.3.17
  475. - Fixes request stack protection when app is initialized more than
  476. once (Issue #1003).
  477. - ETA tasks now properly works when system timezone is not the same
  478. as the configured timezone (Issue #1004).
  479. - Terminating a task now works if the task has been sent to the
  480. pool but not yet acknowledged by a pool process (Issue #1007).
  481. Fix contributed by Alexey Zatelepin
  482. - Terminating a task now properly updates the state of the task to revoked,
  483. and sends a ``task-revoked`` event.
  484. - Generic worker init script now waits for workers to shutdown by default.
  485. - Multi: No longer parses --app option (Issue #1008).
  486. - Multi: stop_verify command renamed to stopwait.
  487. - Daemonization: Now delays trying to create pidfile/logfile until after
  488. the working directory has been changed into.
  489. - :program:`celery worker` and :program:`celery beat` commands now respects
  490. the :option:`--no-color` option (Issue #999).
  491. - Fixed typos in eventlet examples (Issue #1000)
  492. Fix contributed by Bryan Bishop.
  493. Congratulations on opening bug #1000!
  494. - Tasks that raise :exc:`~celery.exceptions.Ignore` are now acknowledged.
  495. - Beat: Now shows the name of the entry in ``sending due task`` logs.
  496. .. _version-3.0.10:
  497. 3.0.10
  498. ======
  499. :release-date: 2012-09-20 05:30 P.M BST
  500. - Now depends on kombu 2.4.7
  501. - Now depends on billiard 2.7.3.14
  502. - Fixes crash at startup when using Django and pre-1.4 projects
  503. (setup_environ).
  504. - Hard time limits now sends the KILL signal shortly after TERM,
  505. to terminate processes that have signal handlers blocked by C extensions.
  506. - Billiard now installs even if the C extension cannot be built.
  507. It's still recommended to build the C extension if you are using
  508. a transport other than rabbitmq/redis (or use forced execv for some
  509. other reason).
  510. - Pool now sets a ``current_process().index`` attribute that can be used to create
  511. as many log files as there are processes in the pool.
  512. - Canvas: chord/group/chain no longer modifies the state when called
  513. Previously calling a chord/group/chain would modify the ids of subtasks
  514. so that:
  515. .. code-block:: python
  516. >>> c = chord([add.s(2, 2), add.s(4, 4)], xsum.s())
  517. >>> c()
  518. >>> c() <-- call again
  519. at the second time the ids for the tasks would be the same as in the
  520. previous invocation. This is now fixed, so that calling a subtask
  521. won't mutate any options.
  522. - Canvas: Chaining a chord to another task now works (Issue #965).
  523. - Worker: Fixed a bug where the request stack could be corrupted if
  524. relative imports are used.
  525. Problem usually manifested itself as an exception while trying to
  526. send a failed task result (``NoneType does not have id attribute``).
  527. Fix contributed by Sam Cooke.
  528. - Tasks can now raise :exc:`~celery.exceptions.Ignore` to skip updating states
  529. or events after return.
  530. Example:
  531. .. code-block:: python
  532. from celery.exceptions import Ignore
  533. @task
  534. def custom_revokes():
  535. if redis.sismember('tasks.revoked', custom_revokes.request.id):
  536. raise Ignore()
  537. - The worker now makes sure the request/task stacks are not modified
  538. by the initial ``Task.__call__``.
  539. This would previously be a problem if a custom task class defined
  540. ``__call__`` and also called ``super()``.
  541. - Because of problems the fast local optimization has been disabled,
  542. and can only be enabled by setting the :envvar:`USE_FAST_LOCALS` attribute.
  543. - Worker: Now sets a default socket timeout of 5 seconds at shutdown
  544. so that broken socket reads do not hinder proper shutdown (Issue #975).
  545. - More fixes related to late eventlet/gevent patching.
  546. - Documentation for settings out of sync with reality:
  547. - :setting:`CELERY_TASK_PUBLISH_RETRY`
  548. Documented as disabled by default, but it was enabled by default
  549. since 2.5 as stated by the 2.5 changelog.
  550. - :setting:`CELERY_TASK_PUBLISH_RETRY_POLICY`
  551. The default max_retries had been set to 100, but documented as being
  552. 3, and the interval_max was set to 1 but documented as 0.2.
  553. The default setting are now set to 3 and 0.2 as it was originally
  554. documented.
  555. Fix contributed by Matt Long.
  556. - Worker: Log messages when connection established and lost have been improved.
  557. - The repr of a crontab schedule value of '0' should be '*' (Issue #972).
  558. - Revoked tasks are now removed from reserved/active state in the worker
  559. (Issue #969)
  560. Fix contributed by Alexey Zatelepin.
  561. - gevent: Now supports hard time limits using ``gevent.Timeout``.
  562. - Documentation: Links to init scripts now point to the 3.0 branch instead
  563. of the development branch (master).
  564. - Documentation: Fixed typo in signals user guide (Issue #986).
  565. ``instance.app.queues`` -> ``instance.app.amqp.queues``.
  566. - Eventlet/gevent: The worker did not properly set the custom app
  567. for new greenlets.
  568. - Eventlet/gevent: Fixed a bug where the worker could not recover
  569. from connection loss (Issue #959).
  570. Also, because of a suspected bug in gevent the
  571. :setting:`BROKER_CONNECTION_TIMEOUT` setting has been disabled
  572. when using gevent
  573. 3.0.9
  574. =====
  575. :release-date: 2012-08-31 06:00 P.M BST
  576. - Important note for users of Django and the database scheduler!
  577. Recently a timezone issue has been fixed for periodic tasks,
  578. but erroneous timezones could have already been stored in the
  579. database, so for the fix to work you need to reset
  580. the ``last_run_at`` fields.
  581. You can do this by executing the following command:
  582. .. code-block:: bash
  583. $ python manage.py shell
  584. >>> from djcelery.models import PeriodicTask
  585. >>> PeriodicTask.objects.update(last_run_at=None)
  586. You also have to do this if you change the timezone or
  587. :setting:`CELERY_ENABLE_UTC` setting.
  588. - Note about the :setting:`CELERY_ENABLE_UTC` setting.
  589. If you previously disabled this just to force periodic tasks to work with
  590. your timezone, then you are now *encouraged to re-enable it*.
  591. - Now depends on Kombu 2.4.5 which fixes PyPy + Jython installation.
  592. - Fixed bug with timezones when :setting:`CELERY_ENABLE_UTC` is disabled
  593. (Issue #952).
  594. - Fixed a typo in the celerybeat upgrade mechanism (Issue #951).
  595. - Make sure the `exc_info` argument to logging is resolved (Issue #899).
  596. - Fixed problem with Python 3.2 and thread join timeout overflow (Issue #796).
  597. - A test case was occasionally broken for Python 2.5.
  598. - Unit test suite now passes for PyPy 1.9.
  599. - App instances now supports the with statement.
  600. This calls the new :meth:`~celery.Celery.close` method at exit, which
  601. cleans up after the app like closing pool connections.
  602. Note that this is only necessary when dynamically creating apps,
  603. e.g. for "temporary" apps.
  604. - Support for piping a subtask to a chain.
  605. For example:
  606. .. code-block:: python
  607. pipe = sometask.s() | othertask.s()
  608. new_pipe = mytask.s() | pipe
  609. Contributed by Steve Morin.
  610. - Fixed problem with group results on non-pickle serializers.
  611. Fix contributed by Steeve Morin.
  612. .. _version-3.0.8:
  613. 3.0.8
  614. =====
  615. :release-date: 2012-08-29 05:00 P.M BST
  616. - Now depends on Kombu 2.4.4
  617. - Fixed problem with amqplib and receiving larger message payloads
  618. (Issue #922).
  619. The problem would manifest itself as either the worker hanging,
  620. or occasionally a ``Framing error`` exception appearing.
  621. Users of the new ``pyamqp://`` transport must upgrade to
  622. :mod:`amqp` 0.9.3.
  623. - Beat: Fixed another timezone bug with interval and crontab schedules
  624. (Issue #943).
  625. - Beat: The schedule file is now automatically cleared if the timezone
  626. is changed.
  627. The schedule is also cleared when you upgrade to 3.0.8 from an earlier
  628. version, this to register the initial timezone info.
  629. - Events: The :event:`worker-heartbeat` event now include processed and active
  630. count fields.
  631. Contributed by Mher Movsisyan.
  632. - Fixed error with error email and new task classes (Issue #931).
  633. - ``BaseTask.__call__`` is no longer optimized away if it has been monkey
  634. patched.
  635. - Fixed shutdown issue when using gevent (Issue #911 & Issue #936).
  636. Fix contributed by Thomas Meson.
  637. .. _version-3.0.7:
  638. 3.0.7
  639. =====
  640. :release-date: 2012-08-24 05:00 P.M BST
  641. - Fixes several problems with periodic tasks and timezones (Issue #937).
  642. - Now depends on kombu 2.4.2
  643. - Redis: Fixes a race condition crash
  644. - Fixes an infinite loop that could happen when retrying establishing
  645. the broker connection.
  646. - Daemons now redirect standard file descriptors to :file:`/dev/null`
  647. Though by default the standard outs are also redirected
  648. to the logger instead, but you can disable this by changing
  649. the :setting:`CELERY_REDIRECT_STDOUTS` setting.
  650. - Fixes possible problems when eventlet/gevent is patched too late.
  651. - ``LoggingProxy`` no longer defines ``fileno()`` (Issue #928).
  652. - Results are now ignored for the chord unlock task.
  653. Fix contributed by Steeve Morin.
  654. - Cassandra backend now works if result expiry is disabled.
  655. Fix contributed by Steeve Morin.
  656. - The traceback object is now passed to signal handlers instead
  657. of the string representation.
  658. Fix contributed by Adam DePue.
  659. - Celery command: Extensions are now sorted by name.
  660. - A regression caused the :event:`task-failed` event to be sent
  661. with the exception object instead of its string representation.
  662. - The worker daemon would try to create the pid file before daemonizing
  663. to catch errors, but this file was not immediately released (Issue #923).
  664. - Fixes Jython compatibility.
  665. - ``billiard.forking_enable`` was called by all pools not just the
  666. processes pool, which would result in a useless warning if the billiard
  667. C extensions were not installed.
  668. .. _version-3.0.6:
  669. 3.0.6
  670. =====
  671. :release-date: 2012-08-17 11:00 P.M BST
  672. - Now depends on kombu 2.4.0
  673. - Now depends on billiard 2.7.3.12
  674. - Redis: Celery now tries to restore messages whenever there are no messages
  675. in the queue.
  676. - Crontab schedules now properly respects :setting:`CELERY_TIMEZONE` setting.
  677. It's important to note that crontab schedules uses UTC time by default
  678. unless this setting is set.
  679. Issue #904 and django-celery #150.
  680. - ``billiard.enable_forking`` is now only set by the processes pool.
  681. - The transport is now properly shown by :program:`celery report`
  682. (Issue #913).
  683. - The `--app` argument now works if the last part is a module name
  684. (Issue #921).
  685. - Fixed problem with unpickleable exceptions (billiard #12).
  686. - Adds ``task_name`` attribute to ``EagerResult`` which is always
  687. :const:`None` (Issue #907).
  688. - Old Task class in :mod:`celery.task` no longer accepts magic kwargs by
  689. default (Issue #918).
  690. A regression long ago disabled magic kwargs for these, and since
  691. no one has complained about it we don't have any incentive to fix it now.
  692. - The ``inspect reserved`` control command did not work properly.
  693. - Should now play better with static analyzation tools by explicitly
  694. specifying dynamically created attributes in the :mod:`celery` and
  695. :mod:`celery.task` modules.
  696. - Terminating a task now results in
  697. :exc:`~celery.exceptions.RevokedTaskError` instead of a ``WorkerLostError``.
  698. - ``AsyncResult.revoke`` now accepts ``terminate`` and ``signal`` arguments.
  699. - The :event:`task-revoked` event now includes new fields: ``terminated``,
  700. ``signum``, and ``expired``.
  701. - The argument to :class:`~celery.exceptions.TaskRevokedError` is now one
  702. of the reasons ``revoked``, ``expired`` or ``terminated``.
  703. - Old Task class does no longer use classmethods for push_request and
  704. pop_request (Issue #912).
  705. - ``GroupResult`` now supports the ``children`` attribute (Issue #916).
  706. - ``AsyncResult.collect`` now respects the ``intermediate`` argument
  707. (Issue #917).
  708. - Fixes example task in documentation (Issue #902).
  709. - Eventlet fixed so that the environment is patched as soon as possible.
  710. - eventlet: Now warns if celery related modules that depends on threads
  711. are imported before eventlet is patched.
  712. - Improved event and camera examples in the monitoring guide.
  713. - Disables celery command setuptools entrypoints if the command can't be
  714. loaded.
  715. - Fixed broken ``dump_request`` example in the tasks guide.
  716. .. _version-3.0.5:
  717. 3.0.5
  718. =====
  719. :release-date: 2012-08-01 04:00 P.M BST
  720. - Now depends on kombu 2.3.1 + billiard 2.7.3.11
  721. - Fixed a bug with the -B option (``cannot pickle thread.lock objects``)
  722. (Issue #894 + Issue #892, + django-celery #154).
  723. - The :control:`restart_pool` control command now requires the
  724. :setting:`CELERYD_POOL_RESTARTS` setting to be enabled
  725. This change was necessary as the multiprocessing event that the restart
  726. command depends on is responsible for creating many semaphores/file
  727. descriptors, resulting in problems in some environments.
  728. - ``chain.apply`` now passes args to the first task (Issue #889).
  729. - Documented previously secret options to the Django-Celery monitor
  730. in the monitoring userguide (Issue #396).
  731. - Old changelog are now organized in separate documents for each series,
  732. see :ref:`history`.
  733. .. _version-3.0.4:
  734. 3.0.4
  735. =====
  736. :release-date: 2012-07-26 07:00 P.M BST
  737. - Now depends on Kombu 2.3
  738. - New experimental standalone Celery monitor: Flower
  739. See :ref:`monitoring-flower` to read more about it!
  740. Contributed by Mher Movsisyan.
  741. - Now supports AMQP heartbeats if using the new ``pyamqp://`` transport.
  742. - The py-amqp transport requires the :mod:`amqp` library to be installed::
  743. $ pip install amqp
  744. - Then you need to set the transport URL prefix to ``pyamqp://``.
  745. - The default heartbeat value is 10 seconds, but this can be changed using
  746. the :setting:`BROKER_HEARTBEAT` setting::
  747. BROKER_HEARTBEAT = 5.0
  748. - If the broker heartbeat is set to 10 seconds, the heartbeats will be
  749. monitored every 5 seconds (double the hertbeat rate).
  750. See the `Kombu 2.3 changelog`_ for more information.
  751. .. _`Kombu 2.3 changelog`:
  752. http://kombu.readthedocs.org/en/latest/changelog.html#version-2-3-0
  753. - Now supports RabbitMQ Consumer Cancel Notifications, using the ``pyamqp://``
  754. transport.
  755. This is essential when running RabbitMQ in a cluster.
  756. See the `Kombu 2.3 changelog`_ for more information.
  757. - Delivery info is no longer passed directly through.
  758. It was discovered that the SQS transport adds objects that can't
  759. be pickled to the delivery info mapping, so we had to go back
  760. to using the whitelist again.
  761. Fixing this bug also means that the SQS transport is now working again.
  762. - The semaphore was not properly released when a task was revoked (Issue #877).
  763. This could lead to tasks being swallowed and not released until a worker
  764. restart.
  765. Thanks to Hynek Schlawack for debugging the issue.
  766. - Retrying a task now also forwards any linked tasks.
  767. This means that if a task is part of a chain (or linked in some other
  768. way) and that even if the task is retried, then the next task in the chain
  769. will be executed when the retry succeeds.
  770. - Chords: Now supports setting the interval and other keyword arguments
  771. to the chord unlock task.
  772. - The interval can now be set as part of the chord subtasks kwargs::
  773. chord(header)(body, interval=10.0)
  774. - In addition the chord unlock task now honors the Task.default_retry_delay
  775. option, used when none is specified, which also means that the default
  776. interval can also be changed using annotations:
  777. .. code-block:: python
  778. CELERY_ANNOTATIONS = {
  779. 'celery.chord_unlock': {
  780. 'default_retry_delay': 10.0,
  781. }
  782. }
  783. - New :meth:`@Celery.add_defaults` method can add new default configuration
  784. dicts to the applications configuration.
  785. For example::
  786. config = {'FOO': 10}
  787. celery.add_defaults(config)
  788. is the same as ``celery.conf.update(config)`` except that data will not be
  789. copied, and that it will not be pickled when the worker spawns child
  790. processes.
  791. In addition the method accepts a callable::
  792. def initialize_config():
  793. # insert heavy stuff that can't be done at import time here.
  794. celery.add_defaults(initialize_config)
  795. which means the same as the above except that it will not happen
  796. until the celery configuration is actually used.
  797. As an example, Celery can lazily use the configuration of a Flask app::
  798. flask_app = Flask()
  799. celery = Celery()
  800. celery.add_defaults(lambda: flask_app.config)
  801. - Revoked tasks were not marked as revoked in the result backend (Issue #871).
  802. Fix contributed by Hynek Schlawack.
  803. - Eventloop now properly handles the case when the epoll poller object
  804. has been closed (Issue #882).
  805. - Fixed syntax error in ``funtests/test_leak.py``
  806. Fix contributed by Catalin Iacob.
  807. - group/chunks: Now accepts empty task list (Issue #873).
  808. - New method names:
  809. - ``Celery.default_connection()`` ➠ :meth:`~@Celery.connection_or_acquire`.
  810. - ``Celery.default_producer()`` ➠ :meth:`~@Celery.producer_or_acquire`.
  811. The old names still work for backward compatibility.
  812. .. _version-3.0.3:
  813. 3.0.3
  814. =====
  815. :release-date: 2012-07-20 09:17 P.M BST
  816. :by: Ask Solem
  817. - amqplib passes the channel object as part of the delivery_info
  818. and it's not pickleable, so we now remove it.
  819. .. _version-3.0.2:
  820. 3.0.2
  821. =====
  822. :release-date: 2012-07-20 04:00 P.M BST
  823. :by: Ask Solem
  824. - A bug caused the following task options to not take defaults from the
  825. configuration (Issue #867 + Issue #858)
  826. The following settings were affected:
  827. - :setting:`CELERY_IGNORE_RESULT`
  828. - :setting:`CELERYD_SEND_TASK_ERROR_EMAILS`
  829. - :setting:`CELERY_TRACK_STARTED`
  830. - :setting:`CElERY_STORE_ERRORS_EVEN_IF_IGNORED`
  831. Fix contributed by John Watson.
  832. - Task Request: ``delivery_info`` is now passed through as-is (Issue #807).
  833. - The eta argument now supports datetime's with a timezone set (Issue #855).
  834. - The worker's banner displayed the autoscale settings in the wrong order
  835. (Issue #859).
  836. - Extension commands are now loaded after concurrency is set up
  837. so that they don't interfere with e.g. eventlet patching.
  838. - Fixed bug in the threaded pool (Issue #863)
  839. - The task failure handler mixed up the fields in :func:`sys.exc_info`.
  840. Fix contributed by Rinat Shigapov.
  841. - Fixed typos and wording in the docs.
  842. Fix contributed by Paul McMillan
  843. - New setting: :setting:`CELERY_WORKER_DIRECT`
  844. If enabled each worker will consume from their own dedicated queue
  845. which can be used to route tasks to specific workers.
  846. - Fixed several edge case bugs in the add consumer remote control command.
  847. - :mod:`~celery.contrib.migrate`: Can now filter and move tasks to specific
  848. workers if :setting:`CELERY_WORKER_DIRECT` is enabled.
  849. Among other improvements, the following functions have been added:
  850. * ``move_direct(filterfun, **opts)``
  851. * ``move_direct_by_id(task_id, worker_hostname, **opts)``
  852. * ``move_direct_by_idmap({task_id: worker_hostname, ...}, **opts)``
  853. * ``move_direct_by_taskmap({task_name: worker_hostname, ...}, **opts)``
  854. - :meth:`~celery.Celery.default_connection` now accepts a pool argument that
  855. if set to false causes a new connection to be created instead of acquiring
  856. one from the pool.
  857. - New signal: :signal:`celeryd_after_setup`.
  858. - Default loader now keeps lowercase attributes from the configuration module.
  859. .. _version-3.0.1:
  860. 3.0.1
  861. =====
  862. :release-date: 2012-07-10 06:00 P.M BST
  863. :by: Ask Solem
  864. - Now depends on kombu 2.2.5
  865. - inspect now supports limit argument::
  866. myapp.control.inspect(limit=1).ping()
  867. - Beat: now works with timezone aware datetime's.
  868. - Task classes inheriting ``from celery import Task``
  869. mistakingly enabled ``accept_magic_kwargs``.
  870. - Fixed bug in ``inspect scheduled`` (Issue #829).
  871. - Beat: Now resets the schedule to upgrade to UTC.
  872. - The :program:`celery worker` command now works with eventlet/gevent.
  873. Previously it would not patch the environment early enough.
  874. - The :program:`celery` command now supports extension commands
  875. using setuptools entry-points.
  876. Libraries can add additional commands to the :program:`celery`
  877. command by adding an entry-point like::
  878. setup(
  879. entry_points=[
  880. 'celery.commands': [
  881. 'foo = my.module:Command',
  882. ],
  883. ],
  884. ...)
  885. The command must then support the interface of
  886. :class:`celery.bin.base.Command`.
  887. - contrib.migrate: New utilities to move tasks from one queue to another.
  888. - :func:`~celery.contrib.migrate.move_tasks`
  889. - :func:`~celery.contrib.migrate.move_task_by_id`
  890. - The :event:`task-sent` event now contains ``exchange`` and ``routing_key``
  891. fields.
  892. - Fixes bug with installing on Python 3.
  893. Fix contributed by Jed Smith.
  894. .. _version-3.0.0:
  895. 3.0.0 (Chiastic Slide)
  896. ======================
  897. :release-date: 2012-07-07 01:30 P.M BST
  898. :by: Ask Solem
  899. See :ref:`whatsnew-3.0`.
  900. >>>>>>> 3.0
  901. .. _version-3.0.17:
  902. 3.0.17
  903. ======
  904. :release-date: 2013-03-22 04:00:00 P.M UTC
  905. - Now depends on kombu 2.5.8
  906. - Now depends on billiard 2.7.3.23
  907. - RabbitMQ/Redis: thread-less and lock-free rate-limit implementation.
  908. This means that rate limits pose minimal overhead when used with
  909. RabbitMQ/Redis or future transports using the eventloop,
  910. and that the rate-limit implementation is now thread-less and lock-free.
  911. The thread-based transports will still use the old implementation for
  912. now, but the plan is to use the timer also for other
  913. broker transports in Celery 3.1.
  914. - Rate limits now works with eventlet/gevent if using RabbitMQ/Redis as the
  915. broker.
  916. - A regression caused ``task.retry`` to ignore additional keyword arguments.
  917. Extra keyword arguments are now used as execution options again.
  918. Fix contributed by Simon Engledew.
  919. - Windows: Fixed problem with the worker trying to pickle the Django settings
  920. module at worker startup.
  921. - generic-init.d: No longer double quotes ``$CELERYD_CHDIR`` (Issue #1235).
  922. - generic-init.d: Removes bash-specific syntax.
  923. Fix contributed by Pär Wieslander.
  924. - Cassandra Result Backend: Now handles the
  925. :exc:`~pycassa.AllServersUnavailable` error (Issue #1010).
  926. Fix contributed by Jared Biel.
  927. - Result: Now properly forwards apps to GroupResults when deserializing
  928. (Issue #1249).
  929. Fix contributed by Charles-Axel Dein.
  930. - ``GroupResult.revoke`` now supports the ``terminate`` and ``signal``
  931. keyword arguments.
  932. - Worker: Multiprocessing pool workers now import task modules/configuration
  933. before setting up the logging system so that logging signals can be
  934. connected before they're dispatched.
  935. - chord: The ``AsyncResult`` instance returned now has its ``parent``
  936. attribute set to the header ``GroupResult``.
  937. This is consistent with how ``chain`` works.
  938. .. _version-3.0.16:
  939. 3.0.16
  940. ======
  941. :release-date: 2013-03-07 04:00:00 P.M UTC
  942. - Happy International Women's Day!
  943. We have a long way to go, so this is a chance for you to get involved in one
  944. of the organizations working for making our communities more
  945. diverse.
  946. - PyLadies — http://pyladies.com
  947. - Girls Who Code — http://www.girlswhocode.com
  948. - Women Who Code — http://www.meetup.com/Women-Who-Code-SF/
  949. - Now depends on :mod:`kombu` version 2.5.7
  950. - Now depends on :mod:`billiard` version 2.7.3.22
  951. - AMQP heartbeats are now disabled by default.
  952. Some users experiences issues with heartbeats enabled,
  953. and it's not strictly necessary to use them.
  954. If you're experiencing problems detecting connection failures,
  955. you can re-enable heartbeats by configuring the :setting:`BROKER_HEARTBEAT`
  956. setting.
  957. - Worker: Now propagates connection errors occurring in multiprocessing
  958. callbacks, so that the connection can be reset (Issue #1226).
  959. - Worker: Now propagates connection errors occurring in timer callbacks,
  960. so that the connection can be reset.
  961. - The modules in :setting:`CELERY_IMPORTS` and :setting:`CELERY_INCLUDE`
  962. are now imported in the original order (Issue #1161).
  963. The modules in :setting:`CELERY_IMPORTS` will be imported first,
  964. then continued by :setting:`CELERY_INCLUDE`.
  965. Thanks to Joey Wilhelm.
  966. - New bash completion for ``celery`` available in the git repository:
  967. https://github.com/celery/celery/tree/3.0/extra/bash-completion
  968. You can source this file or put it in ``bash_completion.d`` to
  969. get auto-completion for the ``celery`` command-line utility.
  970. - The node name of a worker can now include unicode characters (Issue #1186).
  971. - The repr of a ``crontab`` object now displays correctly (Issue #972).
  972. - ``events.State`` no longer modifies the original event dictionary.
  973. - No longer uses ``Logger.warn`` deprecated in Python 3.
  974. - Cache Backend: Now works with chords again (Issue #1094).
  975. - Chord unlock now handles errors occurring while calling the callback.
  976. - Generic worker init.d script: Status check is now performed by
  977. querying the pid of the instance instead of sending messages.
  978. Contributed by Milen Pavlov.
  979. - Improved init scripts for CentOS.
  980. - Updated to support celery 3.x conventions.
  981. - Now uses CentOS built-in ``status`` and ``killproc``
  982. - Support for multi-node / multi-pid worker services.
  983. - Standard color-coded CentOS service-init output.
  984. - A test suite.
  985. Contributed by Milen Pavlov.
  986. - ``ResultSet.join`` now always works with empty result set (Issue #1219).
  987. - A ``group`` consisting of a single task is now supported (Issue #1219).
  988. - Now supports the ``pycallgraph`` program (Issue #1051).
  989. - Fixed Jython compatibility problems.
  990. - Django tutorial: Now mentions that the example app must be added to
  991. ``INSTALLED_APPS`` (Issue #1192).
  992. .. _version-3.0.15:
  993. 3.0.15
  994. ======
  995. :release-date: 2013-02-11 04:30:00 P.M UTC
  996. - Now depends on billiard 2.7.3.21 which fixed a syntax error crash.
  997. - Fixed bug with :setting:`CELERY_SEND_TASK_SENT_EVENT`.
  998. .. _version-3.0.14:
  999. 3.0.14
  1000. ======
  1001. :release-date: 2013-02-08 05:00:00 P.M UTC
  1002. - Now depends on Kombu 2.5.6
  1003. - Now depends on billiard 2.7.3.20
  1004. - ``execv`` is now disabled by default.
  1005. It was causing too many problems for users, you can still enable
  1006. it using the :setting:`CELERYD_FORCE_EXECV` setting.
  1007. execv was only enabled when transports other than amqp/redis was used,
  1008. and it's there to prevent deadlocks caused by mutexes not being released
  1009. before the process forks. Sadly it also changes the environment
  1010. introducing many corner case bugs that is hard to fix without adding
  1011. horrible hacks. Deadlock issues are reported far less often than the
  1012. bugs that execv are causing, so we now disable it by default.
  1013. Work is in motion to create non-blocking versions of these transports
  1014. so that execv is not necessary (which is the situation with the amqp
  1015. and redis broker transports)
  1016. - Chord exception behavior defined (Issue #1172).
  1017. From Celery 3.1 the chord callback will change state to FAILURE
  1018. when a task part of a chord raises an exception.
  1019. It was never documented what happens in this case,
  1020. and the actual behavior was very unsatisfactory, indeed
  1021. it will just forward the exception value to the chord callback.
  1022. For backward compatibility reasons we do not change to the new
  1023. behavior in a bugfix release, even if the current behavior was
  1024. never documented. Instead you can enable the
  1025. :setting:`CELERY_CHORD_PROPAGATES` setting to get the new behavior
  1026. that will be default from Celery 3.1.
  1027. See more at :ref:`chord-errors`.
  1028. - worker: Fixes bug with ignored and retried tasks.
  1029. The ``on_chord_part_return`` and ``Task.after_return`` callbacks,
  1030. nor the ``task_postrun`` signal should be called when the task was
  1031. retried/ignored.
  1032. Fix contributed by Vlad.
  1033. - ``GroupResult.join_native`` now respects the ``propagate`` argument.
  1034. - ``subtask.id`` added as an alias to ``subtask['options'].id``
  1035. .. code-block:: python
  1036. >>> s = add.s(2, 2)
  1037. >>> s.id = 'my-id'
  1038. >>> s['options']
  1039. {'task_id': 'my-id'}
  1040. >>> s.id
  1041. 'my-id'
  1042. - worker: Fixed error `Could not start worker processes` occurring
  1043. when restarting after connection failure (Issue #1118).
  1044. - Adds new signal :signal:`task-retried` (Issue #1169).
  1045. - `celery events --dumper` now handles connection loss.
  1046. - Will now retry sending the task-sent event in case of connection failure.
  1047. - amqp backend: Now uses ``Message.requeue`` instead of republishing
  1048. the message after poll.
  1049. - New :setting:`BROKER_HEARTBEAT_CHECKRATE` setting introduced to modify the
  1050. rate at which broker connection heartbeats are monitored.
  1051. The default value was also changed from 3.0 to 2.0.
  1052. - :class:`celery.events.state.State` is now pickleable.
  1053. Fix contributed by Mher Movsisyan.
  1054. - :class:`celery.datastructures.LRUCache` is now pickleable.
  1055. Fix contributed by Mher Movsisyan.
  1056. - The stats broadcast command now includes the workers pid.
  1057. Contributed by Mher Movsisyan.
  1058. - New ``conf`` remote control command to get a workers current configuration.
  1059. Contributed by Mher Movsisyan.
  1060. - Adds the ability to modify the chord unlock task's countdown
  1061. argument (Issue #1146).
  1062. Contributed by Jun Sakai
  1063. - beat: The scheduler now uses the `now()`` method of the schedule,
  1064. so that schedules can provide a custom way to get the current date and time.
  1065. Contributed by Raphaël Slinckx
  1066. - Fixed pickling of configuration modules on Windows or when execv is used
  1067. (Issue #1126).
  1068. - Multiprocessing logger is now configured with loglevel ``ERROR``
  1069. by default.
  1070. Since 3.0 the multiprocessing loggers were disabled by default
  1071. (only configured when the :envvar:`MP_LOG` environment variable was set).
  1072. .. _version-3.0.13:
  1073. 3.0.13
  1074. ======
  1075. :release-date: 2013-01-07 04:00:00 P.M UTC
  1076. - Now depends on Kombu 2.5
  1077. - py-amqp has replaced amqplib as the default transport,
  1078. gaining support for AMQP 0.9, and the RabbitMQ extensions
  1079. including Consumer Cancel Notifications and heartbeats.
  1080. - support for multiple connection URLs for failover.
  1081. - Read more in the `Kombu 2.5 changelog`_.
  1082. .. _`Kombu 2.5 changelog`:
  1083. http://kombu.readthedocs.org/en/latest/changelog.html#version-2-5-0
  1084. - Now depends on billiard 2.7.3.19
  1085. - Fixed a deadlock issue that could occur when the producer pool
  1086. inherited the connection pool instance of the parent process.
  1087. - The :option:`--loader` option now works again (Issue #1066).
  1088. - :program:`celery` umbrella command: All subcommands now supports
  1089. the :option:`--workdir` option (Issue #1063).
  1090. - Groups included in chains now give GroupResults (Issue #1057)
  1091. Previously it would incorrectly add a regular result instead of a group
  1092. result, but now this works:
  1093. .. code-block:: python
  1094. # [4 + 4, 4 + 8, 16 + 8]
  1095. >>> res = (add.s(2, 2) | group(add.s(4), add.s(8), add.s(16)))()
  1096. >>> res
  1097. <GroupResult: a0acf905-c704-499e-b03a-8d445e6398f7 [
  1098. 4346501c-cb99-4ad8-8577-12256c7a22b1,
  1099. b12ead10-a622-4d44-86e9-3193a778f345,
  1100. 26c7a420-11f3-4b33-8fac-66cd3b62abfd]>
  1101. - Chains can now chain other chains and use partial arguments (Issue #1057).
  1102. Example:
  1103. .. code-block:: python
  1104. >>> c1 = (add.s(2) | add.s(4))
  1105. >>> c2 = (add.s(8) | add.s(16))
  1106. >>> c3 = (c1 | c2)
  1107. # 8 + 2 + 4 + 8 + 16
  1108. >>> assert c3(8).get() == 38
  1109. - Subtasks can now be used with unregistered tasks.
  1110. You can specify subtasks even if you just have the name::
  1111. >>> s = subtask(task_name, args=(), kwargs=())
  1112. >>> s.delay()
  1113. - The :program:`celery shell` command now always adds the current
  1114. directory to the module path.
  1115. - The worker will now properly handle the :exc:`pytz.AmbiguousTimeError`
  1116. exception raised when an ETA/countdown is prepared while being in DST
  1117. transition (Issue #1061).
  1118. - force_execv: Now makes sure that task symbols in the original
  1119. task modules will always use the correct app instance (Issue #1072).
  1120. - AMQP Backend: Now republishes result messages that have been polled
  1121. (using ``result.ready()`` and friends, ``result.get()`` will not do this
  1122. in this version).
  1123. - Crontab schedule values can now "wrap around"
  1124. This means that values like ``11-1`` translates to ``[11, 12, 1]``.
  1125. Contributed by Loren Abrams.
  1126. - multi stopwait command now shows the pid of processes.
  1127. Contributed by Loren Abrams.
  1128. - Handling of ETA/countdown fixed when the :setting:`CELERY_ENABLE_UTC`
  1129. setting is disabled (Issue #1065).
  1130. - A number of uneeded properties were included in messages,
  1131. caused by accidentally passing ``Queue.as_dict`` as message properties.
  1132. - Rate limit values can now be float
  1133. This also extends the string format so that values like ``"0.5/s"`` works.
  1134. Contributed by Christoph Krybus
  1135. - Fixed a typo in the broadcast routing documentation (Issue #1026).
  1136. - Rewrote confusing section about idempotence in the task user guide.
  1137. - Fixed typo in the daemonization tutorial (Issue #1055).
  1138. - Fixed several typos in the documentation.
  1139. Contributed by Marius Gedminas.
  1140. - Batches: Now works when using the eventlet pool.
  1141. Fix contributed by Thomas Grainger.
  1142. - Batches: Added example sending results to :mod:`celery.contrib.batches`.
  1143. Contributed by Thomas Grainger.
  1144. - Mongodb backend: Connection ``max_pool_size`` can now be set in
  1145. :setting:`CELERY_MONGODB_BACKEND_SETTINGS`.
  1146. Contributed by Craig Younkins.
  1147. - Fixed problem when using earlier versions of :mod:`pytz`.
  1148. Fix contributed by Vlad.
  1149. - Docs updated to include the default value for the
  1150. :setting:`CELERY_TASK_RESULT_EXPIRES` setting.
  1151. - Improvements to the django-celery tutorial.
  1152. Contributed by Locker537.
  1153. - The ``add_consumer`` control command did not properly persist
  1154. the addition of new queues so that they survived connection failure
  1155. (Issue #1079).
  1156. 3.0.12
  1157. ======
  1158. :release-date: 2012-11-06 02:00 P.M UTC
  1159. - Now depends on kombu 2.4.8
  1160. - [Redis] New and improved fair queue cycle algorithm (Kevin McCarthy).
  1161. - [Redis] Now uses a Redis-based mutex when restoring messages.
  1162. - [Redis] Number of messages that can be restored in one interval is no
  1163. longer limited (but can be set using the
  1164. ``unacked_restore_limit``
  1165. :setting:`transport option <BROKER_TRANSPORT_OPTIONS>`.)
  1166. - Heartbeat value can be specified in broker URLs (Mher Movsisyan).
  1167. - Fixed problem with msgpack on Python 3 (Jasper Bryant-Greene).
  1168. - Now depends on billiard 2.7.3.18
  1169. - Celery can now be used with static analysis tools like PyDev/PyCharm/pylint
  1170. etc.
  1171. - Development documentation has moved to Read The Docs.
  1172. The new URL is: http://docs.celeryproject.org/en/master
  1173. - New :setting:`CELERY_QUEUE_HA_POLICY` setting used to set the default
  1174. HA policy for queues when using RabbitMQ.
  1175. - New method ``Task.subtask_from_request`` returns a subtask using the current
  1176. request.
  1177. - Results get_many method did not respect timeout argument.
  1178. Fix contributed by Remigiusz Modrzejewski
  1179. - generic_init.d scripts now support setting :envvar:`CELERY_CREATE_DIRS` to
  1180. always create log and pid directories (Issue #1045).
  1181. This can be set in your :file:`/etc/default/celeryd`.
  1182. - Fixed strange kombu import problem on Python 3.2 (Issue #1034).
  1183. - Worker: ETA scheduler now uses millisecond precision (Issue #1040).
  1184. - The ``--config`` argument to programs is now supported by all loaders.
  1185. - The :setting:`CASSANDRA_OPTIONS` setting has now been documented.
  1186. Contributed by Jared Biel.
  1187. - Task methods (:mod:`celery.contrib.methods`) cannot be used with the old
  1188. task base class, the task decorator in that module now inherits from the new.
  1189. - An optimization was too eager and caused some logging messages to never emit.
  1190. - :mod:`celery.contrib.batches` now works again.
  1191. - Fixed missing whitespace in ``bdist_rpm`` requirements (Issue #1046).
  1192. - Event state's ``tasks_by_name`` applied limit before filtering by name.
  1193. Fix contributed by Alexander A. Sosnovskiy.
  1194. .. _version-3.0.11:
  1195. 3.0.11
  1196. ======
  1197. :release-date: 2012-09-26 04:00 P.M UTC
  1198. - [security:low] generic-init.d scripts changed permissions of /var/log & /var/run
  1199. In the daemonization tutorial the recommended directories were as follows:
  1200. .. code-block:: bash
  1201. CELERYD_LOG_FILE="/var/log/celery/%n.log"
  1202. CELERYD_PID_FILE="/var/run/celery/%n.pid"
  1203. But in the scripts themselves the default files were ``/var/log/celery%n.log``
  1204. and ``/var/run/celery%n.pid``, so if the user did not change the location
  1205. by configuration, the directories ``/var/log`` and ``/var/run`` would be
  1206. created - and worse have their permissions and owners changed.
  1207. This change means that:
  1208. - Default pid file is ``/var/run/celery/%n.pid``
  1209. - Default log file is ``/var/log/celery/%n.log``
  1210. - The directories are only created and have their permissions
  1211. changed if *no custom locations are set*.
  1212. Users can force paths to be created by calling the ``create-paths``
  1213. subcommand:
  1214. .. code-block:: bash
  1215. $ sudo /etc/init.d/celeryd create-paths
  1216. .. admonition:: Upgrading Celery will not update init scripts
  1217. To update the init scripts you have to re-download
  1218. the files from source control and update them manually.
  1219. You can find the init scripts for version 3.0.x at:
  1220. http://github.com/celery/celery/tree/3.0/extra/generic-init.d
  1221. - Now depends on billiard 2.7.3.17
  1222. - Fixes request stack protection when app is initialized more than
  1223. once (Issue #1003).
  1224. - ETA tasks now properly works when system timezone is not the same
  1225. as the configured timezone (Issue #1004).
  1226. - Terminating a task now works if the task has been sent to the
  1227. pool but not yet acknowledged by a pool process (Issue #1007).
  1228. Fix contributed by Alexey Zatelepin
  1229. - Terminating a task now properly updates the state of the task to revoked,
  1230. and sends a ``task-revoked`` event.
  1231. - Generic worker init script now waits for workers to shutdown by default.
  1232. - Multi: No longer parses --app option (Issue #1008).
  1233. - Multi: stop_verify command renamed to stopwait.
  1234. - Daemonization: Now delays trying to create pidfile/logfile until after
  1235. the working directory has been changed into.
  1236. - :program:`celery worker` and :program:`celery beat` commands now respects
  1237. the :option:`--no-color` option (Issue #999).
  1238. - Fixed typos in eventlet examples (Issue #1000)
  1239. Fix contributed by Bryan Bishop.
  1240. Congratulations on opening bug #1000!
  1241. - Tasks that raise :exc:`~celery.exceptions.Ignore` are now acknowledged.
  1242. - Beat: Now shows the name of the entry in ``sending due task`` logs.
  1243. .. _version-3.0.10:
  1244. 3.0.10
  1245. ======
  1246. :release-date: 2012-09-20 05:30 P.M BST
  1247. - Now depends on kombu 2.4.7
  1248. - Now depends on billiard 2.7.3.14
  1249. - Fixes crash at startup when using Django and pre-1.4 projects
  1250. (setup_environ).
  1251. - Hard time limits now sends the KILL signal shortly after TERM,
  1252. to terminate processes that have signal handlers blocked by C extensions.
  1253. - Billiard now installs even if the C extension cannot be built.
  1254. It's still recommended to build the C extension if you are using
  1255. a transport other than rabbitmq/redis (or use forced execv for some
  1256. other reason).
  1257. - Pool now sets a ``current_process().index`` attribute that can be used to create
  1258. as many log files as there are processes in the pool.
  1259. - Canvas: chord/group/chain no longer modifies the state when called
  1260. Previously calling a chord/group/chain would modify the ids of subtasks
  1261. so that:
  1262. .. code-block:: python
  1263. >>> c = chord([add.s(2, 2), add.s(4, 4)], xsum.s())
  1264. >>> c()
  1265. >>> c() <-- call again
  1266. at the second time the ids for the tasks would be the same as in the
  1267. previous invocation. This is now fixed, so that calling a subtask
  1268. won't mutate any options.
  1269. - Canvas: Chaining a chord to another task now works (Issue #965).
  1270. - Worker: Fixed a bug where the request stack could be corrupted if
  1271. relative imports are used.
  1272. Problem usually manifested itself as an exception while trying to
  1273. send a failed task result (``NoneType does not have id attribute``).
  1274. Fix contributed by Sam Cooke.
  1275. - Tasks can now raise :exc:`~celery.exceptions.Ignore` to skip updating states
  1276. or events after return.
  1277. Example:
  1278. .. code-block:: python
  1279. from celery.exceptions import Ignore
  1280. @task
  1281. def custom_revokes():
  1282. if redis.sismember('tasks.revoked', custom_revokes.request.id):
  1283. raise Ignore()
  1284. - The worker now makes sure the request/task stacks are not modified
  1285. by the initial ``Task.__call__``.
  1286. This would previously be a problem if a custom task class defined
  1287. ``__call__`` and also called ``super()``.
  1288. - Because of problems the fast local optimization has been disabled,
  1289. and can only be enabled by setting the :envvar:`USE_FAST_LOCALS` attribute.
  1290. - Worker: Now sets a default socket timeout of 5 seconds at shutdown
  1291. so that broken socket reads do not hinder proper shutdown (Issue #975).
  1292. - More fixes related to late eventlet/gevent patching.
  1293. - Documentation for settings out of sync with reality:
  1294. - :setting:`CELERY_TASK_PUBLISH_RETRY`
  1295. Documented as disabled by default, but it was enabled by default
  1296. since 2.5 as stated by the 2.5 changelog.
  1297. - :setting:`CELERY_TASK_PUBLISH_RETRY_POLICY`
  1298. The default max_retries had been set to 100, but documented as being
  1299. 3, and the interval_max was set to 1 but documented as 0.2.
  1300. The default setting are now set to 3 and 0.2 as it was originally
  1301. documented.
  1302. Fix contributed by Matt Long.
  1303. - Worker: Log messages when connection established and lost have been improved.
  1304. - The repr of a crontab schedule value of '0' should be '*' (Issue #972).
  1305. - Revoked tasks are now removed from reserved/active state in the worker
  1306. (Issue #969)
  1307. Fix contributed by Alexey Zatelepin.
  1308. - gevent: Now supports hard time limits using ``gevent.Timeout``.
  1309. - Documentation: Links to init scripts now point to the 3.0 branch instead
  1310. of the development branch (master).
  1311. - Documentation: Fixed typo in signals user guide (Issue #986).
  1312. ``instance.app.queues`` -> ``instance.app.amqp.queues``.
  1313. - Eventlet/gevent: The worker did not properly set the custom app
  1314. for new greenlets.
  1315. - Eventlet/gevent: Fixed a bug where the worker could not recover
  1316. from connection loss (Issue #959).
  1317. Also, because of a suspected bug in gevent the
  1318. :setting:`BROKER_CONNECTION_TIMEOUT` setting has been disabled
  1319. when using gevent
  1320. 3.0.9
  1321. =====
  1322. :release-date: 2012-08-31 06:00 P.M BST
  1323. - Important note for users of Django and the database scheduler!
  1324. Recently a timezone issue has been fixed for periodic tasks,
  1325. but erroneous timezones could have already been stored in the
  1326. database, so for the fix to work you need to reset
  1327. the ``last_run_at`` fields.
  1328. You can do this by executing the following command:
  1329. .. code-block:: bash
  1330. $ python manage.py shell
  1331. >>> from djcelery.models import PeriodicTask
  1332. >>> PeriodicTask.objects.update(last_run_at=None)
  1333. You also have to do this if you change the timezone or
  1334. :setting:`CELERY_ENABLE_UTC` setting.
  1335. - Note about the :setting:`CELERY_ENABLE_UTC` setting.
  1336. If you previously disabled this just to force periodic tasks to work with
  1337. your timezone, then you are now *encouraged to re-enable it*.
  1338. - Now depends on Kombu 2.4.5 which fixes PyPy + Jython installation.
  1339. - Fixed bug with timezones when :setting:`CELERY_ENABLE_UTC` is disabled
  1340. (Issue #952).
  1341. - Fixed a typo in the celerybeat upgrade mechanism (Issue #951).
  1342. - Make sure the `exc_info` argument to logging is resolved (Issue #899).
  1343. - Fixed problem with Python 3.2 and thread join timeout overflow (Issue #796).
  1344. - A test case was occasionally broken for Python 2.5.
  1345. - Unit test suite now passes for PyPy 1.9.
  1346. - App instances now supports the with statement.
  1347. This calls the new :meth:`~celery.Celery.close` method at exit, which
  1348. cleans up after the app like closing pool connections.
  1349. Note that this is only necessary when dynamically creating apps,
  1350. e.g. for "temporary" apps.
  1351. - Support for piping a subtask to a chain.
  1352. For example:
  1353. .. code-block:: python
  1354. pipe = sometask.s() | othertask.s()
  1355. new_pipe = mytask.s() | pipe
  1356. Contributed by Steve Morin.
  1357. - Fixed problem with group results on non-pickle serializers.
  1358. Fix contributed by Steeve Morin.
  1359. .. _version-3.0.8:
  1360. 3.0.8
  1361. =====
  1362. :release-date: 2012-08-29 05:00 P.M BST
  1363. - Now depends on Kombu 2.4.4
  1364. - Fixed problem with amqplib and receiving larger message payloads
  1365. (Issue #922).
  1366. The problem would manifest itself as either the worker hanging,
  1367. or occasionally a ``Framing error`` exception appearing.
  1368. Users of the new ``pyamqp://`` transport must upgrade to
  1369. :mod:`amqp` 0.9.3.
  1370. - Beat: Fixed another timezone bug with interval and crontab schedules
  1371. (Issue #943).
  1372. - Beat: The schedule file is now automatically cleared if the timezone
  1373. is changed.
  1374. The schedule is also cleared when you upgrade to 3.0.8 from an earlier
  1375. version, this to register the initial timezone info.
  1376. - Events: The :event:`worker-heartbeat` event now include processed and active
  1377. count fields.
  1378. Contributed by Mher Movsisyan.
  1379. - Fixed error with error email and new task classes (Issue #931).
  1380. - ``BaseTask.__call__`` is no longer optimized away if it has been monkey
  1381. patched.
  1382. - Fixed shutdown issue when using gevent (Issue #911 & Issue #936).
  1383. Fix contributed by Thomas Meson.
  1384. .. _version-3.0.7:
  1385. 3.0.7
  1386. =====
  1387. :release-date: 2012-08-24 05:00 P.M BST
  1388. - Fixes several problems with periodic tasks and timezones (Issue #937).
  1389. - Now depends on kombu 2.4.2
  1390. - Redis: Fixes a race condition crash
  1391. - Fixes an infinite loop that could happen when retrying establishing
  1392. the broker connection.
  1393. - Daemons now redirect standard file descriptors to :file:`/dev/null`
  1394. Though by default the standard outs are also redirected
  1395. to the logger instead, but you can disable this by changing
  1396. the :setting:`CELERY_REDIRECT_STDOUTS` setting.
  1397. - Fixes possible problems when eventlet/gevent is patched too late.
  1398. - ``LoggingProxy`` no longer defines ``fileno()`` (Issue #928).
  1399. - Results are now ignored for the chord unlock task.
  1400. Fix contributed by Steeve Morin.
  1401. - Cassandra backend now works if result expiry is disabled.
  1402. Fix contributed by Steeve Morin.
  1403. - The traceback object is now passed to signal handlers instead
  1404. of the string representation.
  1405. Fix contributed by Adam DePue.
  1406. - Celery command: Extensions are now sorted by name.
  1407. - A regression caused the :event:`task-failed` event to be sent
  1408. with the exception object instead of its string representation.
  1409. - The worker daemon would try to create the pid file before daemonizing
  1410. to catch errors, but this file was not immediately released (Issue #923).
  1411. - Fixes Jython compatibility.
  1412. - ``billiard.forking_enable`` was called by all pools not just the
  1413. processes pool, which would result in a useless warning if the billiard
  1414. C extensions were not installed.
  1415. .. _version-3.0.6:
  1416. 3.0.6
  1417. =====
  1418. :release-date: 2012-08-17 11:00 P.M BST
  1419. - Now depends on kombu 2.4.0
  1420. - Now depends on billiard 2.7.3.12
  1421. - Redis: Celery now tries to restore messages whenever there are no messages
  1422. in the queue.
  1423. - Crontab schedules now properly respects :setting:`CELERY_TIMEZONE` setting.
  1424. It's important to note that crontab schedules uses UTC time by default
  1425. unless this setting is set.
  1426. Issue #904 and django-celery #150.
  1427. - ``billiard.enable_forking`` is now only set by the processes pool.
  1428. - The transport is now properly shown by :program:`celery report`
  1429. (Issue #913).
  1430. - The `--app` argument now works if the last part is a module name
  1431. (Issue #921).
  1432. - Fixed problem with unpickleable exceptions (billiard #12).
  1433. - Adds ``task_name`` attribute to ``EagerResult`` which is always
  1434. :const:`None` (Issue #907).
  1435. - Old Task class in :mod:`celery.task` no longer accepts magic kwargs by
  1436. default (Issue #918).
  1437. A regression long ago disabled magic kwargs for these, and since
  1438. no one has complained about it we don't have any incentive to fix it now.
  1439. - The ``inspect reserved`` control command did not work properly.
  1440. - Should now play better with static analyzation tools by explicitly
  1441. specifying dynamically created attributes in the :mod:`celery` and
  1442. :mod:`celery.task` modules.
  1443. - Terminating a task now results in
  1444. :exc:`~celery.exceptions.RevokedTaskError` instead of a ``WorkerLostError``.
  1445. - ``AsyncResult.revoke`` now accepts ``terminate`` and ``signal`` arguments.
  1446. - The :event:`task-revoked` event now includes new fields: ``terminated``,
  1447. ``signum``, and ``expired``.
  1448. - The argument to :class:`~celery.exceptions.TaskRevokedError` is now one
  1449. of the reasons ``revoked``, ``expired`` or ``terminated``.
  1450. - Old Task class does no longer use classmethods for push_request and
  1451. pop_request (Issue #912).
  1452. - ``GroupResult`` now supports the ``children`` attribute (Issue #916).
  1453. - ``AsyncResult.collect`` now respects the ``intermediate`` argument
  1454. (Issue #917).
  1455. - Fixes example task in documentation (Issue #902).
  1456. - Eventlet fixed so that the environment is patched as soon as possible.
  1457. - eventlet: Now warns if celery related modules that depends on threads
  1458. are imported before eventlet is patched.
  1459. - Improved event and camera examples in the monitoring guide.
  1460. - Disables celery command setuptools entrypoints if the command can't be
  1461. loaded.
  1462. - Fixed broken ``dump_request`` example in the tasks guide.
  1463. .. _version-3.0.5:
  1464. 3.0.5
  1465. =====
  1466. :release-date: 2012-08-01 04:00 P.M BST
  1467. - Now depends on kombu 2.3.1 + billiard 2.7.3.11
  1468. - Fixed a bug with the -B option (``cannot pickle thread.lock objects``)
  1469. (Issue #894 + Issue #892, + django-celery #154).
  1470. - The :control:`restart_pool` control command now requires the
  1471. :setting:`CELERYD_POOL_RESTARTS` setting to be enabled
  1472. This change was necessary as the multiprocessing event that the restart
  1473. command depends on is responsible for creating many semaphores/file
  1474. descriptors, resulting in problems in some environments.
  1475. - ``chain.apply`` now passes args to the first task (Issue #889).
  1476. - Documented previously secret options to the Django-Celery monitor
  1477. in the monitoring userguide (Issue #396).
  1478. - Old changelog are now organized in separate documents for each series,
  1479. see :ref:`history`.
  1480. .. _version-3.0.4:
  1481. 3.0.4
  1482. =====
  1483. :release-date: 2012-07-26 07:00 P.M BST
  1484. - Now depends on Kombu 2.3
  1485. - New experimental standalone Celery monitor: Flower
  1486. See :ref:`monitoring-flower` to read more about it!
  1487. Contributed by Mher Movsisyan.
  1488. - Now supports AMQP heartbeats if using the new ``pyamqp://`` transport.
  1489. - The py-amqp transport requires the :mod:`amqp` library to be installed::
  1490. $ pip install amqp
  1491. - Then you need to set the transport URL prefix to ``pyamqp://``.
  1492. - The default heartbeat value is 10 seconds, but this can be changed using
  1493. the :setting:`BROKER_HEARTBEAT` setting::
  1494. BROKER_HEARTBEAT = 5.0
  1495. - If the broker heartbeat is set to 10 seconds, the heartbeats will be
  1496. monitored every 5 seconds (double the hertbeat rate).
  1497. See the `Kombu 2.3 changelog`_ for more information.
  1498. .. _`Kombu 2.3 changelog`:
  1499. http://kombu.readthedocs.org/en/latest/changelog.html#version-2-3-0
  1500. - Now supports RabbitMQ Consumer Cancel Notifications, using the ``pyamqp://``
  1501. transport.
  1502. This is essential when running RabbitMQ in a cluster.
  1503. See the `Kombu 2.3 changelog`_ for more information.
  1504. - Delivery info is no longer passed directly through.
  1505. It was discovered that the SQS transport adds objects that can't
  1506. be pickled to the delivery info mapping, so we had to go back
  1507. to using the whitelist again.
  1508. Fixing this bug also means that the SQS transport is now working again.
  1509. - The semaphore was not properly released when a task was revoked (Issue #877).
  1510. This could lead to tasks being swallowed and not released until a worker
  1511. restart.
  1512. Thanks to Hynek Schlawack for debugging the issue.
  1513. - Retrying a task now also forwards any linked tasks.
  1514. This means that if a task is part of a chain (or linked in some other
  1515. way) and that even if the task is retried, then the next task in the chain
  1516. will be executed when the retry succeeds.
  1517. - Chords: Now supports setting the interval and other keyword arguments
  1518. to the chord unlock task.
  1519. - The interval can now be set as part of the chord subtasks kwargs::
  1520. chord(header)(body, interval=10.0)
  1521. - In addition the chord unlock task now honors the Task.default_retry_delay
  1522. option, used when none is specified, which also means that the default
  1523. interval can also be changed using annotations:
  1524. .. code-block:: python
  1525. CELERY_ANNOTATIONS = {
  1526. 'celery.chord_unlock': {
  1527. 'default_retry_delay': 10.0,
  1528. }
  1529. }
  1530. - New :meth:`@Celery.add_defaults` method can add new default configuration
  1531. dicts to the applications configuration.
  1532. For example::
  1533. config = {'FOO': 10}
  1534. celery.add_defaults(config)
  1535. is the same as ``celery.conf.update(config)`` except that data will not be
  1536. copied, and that it will not be pickled when the worker spawns child
  1537. processes.
  1538. In addition the method accepts a callable::
  1539. def initialize_config():
  1540. # insert heavy stuff that can't be done at import time here.
  1541. celery.add_defaults(initialize_config)
  1542. which means the same as the above except that it will not happen
  1543. until the celery configuration is actually used.
  1544. As an example, Celery can lazily use the configuration of a Flask app::
  1545. flask_app = Flask()
  1546. celery = Celery()
  1547. celery.add_defaults(lambda: flask_app.config)
  1548. - Revoked tasks were not marked as revoked in the result backend (Issue #871).
  1549. Fix contributed by Hynek Schlawack.
  1550. - Eventloop now properly handles the case when the epoll poller object
  1551. has been closed (Issue #882).
  1552. - Fixed syntax error in ``funtests/test_leak.py``
  1553. Fix contributed by Catalin Iacob.
  1554. - group/chunks: Now accepts empty task list (Issue #873).
  1555. - New method names:
  1556. - ``Celery.default_connection()`` ➠ :meth:`~@Celery.connection_or_acquire`.
  1557. - ``Celery.default_producer()`` ➠ :meth:`~@Celery.producer_or_acquire`.
  1558. The old names still work for backward compatibility.
  1559. .. _version-3.0.3:
  1560. 3.0.3
  1561. =====
  1562. :release-date: 2012-07-20 09:17 P.M BST
  1563. :by: Ask Solem
  1564. - amqplib passes the channel object as part of the delivery_info
  1565. and it's not pickleable, so we now remove it.
  1566. .. _version-3.0.2:
  1567. 3.0.2
  1568. =====
  1569. :release-date: 2012-07-20 04:00 P.M BST
  1570. :by: Ask Solem
  1571. - A bug caused the following task options to not take defaults from the
  1572. configuration (Issue #867 + Issue #858)
  1573. The following settings were affected:
  1574. - :setting:`CELERY_IGNORE_RESULT`
  1575. - :setting:`CELERYD_SEND_TASK_ERROR_EMAILS`
  1576. - :setting:`CELERY_TRACK_STARTED`
  1577. - :setting:`CElERY_STORE_ERRORS_EVEN_IF_IGNORED`
  1578. Fix contributed by John Watson.
  1579. - Task Request: ``delivery_info`` is now passed through as-is (Issue #807).
  1580. - The eta argument now supports datetime's with a timezone set (Issue #855).
  1581. - The worker's banner displayed the autoscale settings in the wrong order
  1582. (Issue #859).
  1583. - Extension commands are now loaded after concurrency is set up
  1584. so that they don't interfere with e.g. eventlet patching.
  1585. - Fixed bug in the threaded pool (Issue #863)
  1586. - The task failure handler mixed up the fields in :func:`sys.exc_info`.
  1587. Fix contributed by Rinat Shigapov.
  1588. - Fixed typos and wording in the docs.
  1589. Fix contributed by Paul McMillan
  1590. - New setting: :setting:`CELERY_WORKER_DIRECT`
  1591. If enabled each worker will consume from their own dedicated queue
  1592. which can be used to route tasks to specific workers.
  1593. - Fixed several edge case bugs in the add consumer remote control command.
  1594. - :mod:`~celery.contrib.migrate`: Can now filter and move tasks to specific
  1595. workers if :setting:`CELERY_WORKER_DIRECT` is enabled.
  1596. Among other improvements, the following functions have been added:
  1597. * ``move_direct(filterfun, **opts)``
  1598. * ``move_direct_by_id(task_id, worker_hostname, **opts)``
  1599. * ``move_direct_by_idmap({task_id: worker_hostname, ...}, **opts)``
  1600. * ``move_direct_by_taskmap({task_name: worker_hostname, ...}, **opts)``
  1601. - :meth:`~celery.Celery.default_connection` now accepts a pool argument that
  1602. if set to false causes a new connection to be created instead of acquiring
  1603. one from the pool.
  1604. - New signal: :signal:`celeryd_after_setup`.
  1605. - Default loader now keeps lowercase attributes from the configuration module.
  1606. .. _version-3.0.1:
  1607. 3.0.1
  1608. =====
  1609. :release-date: 2012-07-10 06:00 P.M BST
  1610. :by: Ask Solem
  1611. - Now depends on kombu 2.2.5
  1612. - inspect now supports limit argument::
  1613. myapp.control.inspect(limit=1).ping()
  1614. - Beat: now works with timezone aware datetime's.
  1615. - Task classes inheriting ``from celery import Task``
  1616. mistakingly enabled ``accept_magic_kwargs``.
  1617. - Fixed bug in ``inspect scheduled`` (Issue #829).
  1618. - Beat: Now resets the schedule to upgrade to UTC.
  1619. - The :program:`celery worker` command now works with eventlet/gevent.
  1620. Previously it would not patch the environment early enough.
  1621. - The :program:`celery` command now supports extension commands
  1622. using setuptools entry-points.
  1623. Libraries can add additional commands to the :program:`celery`
  1624. command by adding an entry-point like::
  1625. setup(
  1626. entry_points=[
  1627. 'celery.commands': [
  1628. 'foo = my.module:Command',
  1629. ],
  1630. ],
  1631. ...)
  1632. The command must then support the interface of
  1633. :class:`celery.bin.base.Command`.
  1634. - contrib.migrate: New utilities to move tasks from one queue to another.
  1635. - :func:`~celery.contrib.migrate.move_tasks`
  1636. - :func:`~celery.contrib.migrate.move_task_by_id`
  1637. - The :event:`task-sent` event now contains ``exchange`` and ``routing_key``
  1638. fields.
  1639. - Fixes bug with installing on Python 3.
  1640. Fix contributed by Jed Smith.
  1641. .. _version-3.0.0:
  1642. 3.0.0 (Chiastic Slide)
  1643. ======================
  1644. :release-date: 2012-07-07 01:30 P.M BST
  1645. :by: Ask Solem
  1646. See :ref:`whatsnew-3.0`.