changelog-2.3.rst 11 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371
  1. .. _changelog-2.3:
  2. ===============================
  3. Change history for Celery 2.3
  4. ===============================
  5. .. contents::
  6. :local:
  7. .. _version-2.3.4:
  8. 2.3.4
  9. =====
  10. :release-date: 2011-11-25 04:00 P.M GMT
  11. :release-by: Ask Solem
  12. .. _v234-security-fixes:
  13. Security Fixes
  14. --------------
  15. * [Security: `CELERYSA-0001`_] Daemons would set effective id's rather than
  16. real id's when the :option:`--uid <celery --uid>`/
  17. :option:`--gid <celery --gid>` arguments to :program:`celery multi`,
  18. :program:`celeryd_detach`, :program:`celery beat` and
  19. :program:`celery events` were used.
  20. This means privileges weren't properly dropped, and that it would
  21. be possible to regain supervisor privileges later.
  22. .. _`CELERYSA-0001`:
  23. https://github.com/celery/celery/tree/master/docs/sec/CELERYSA-0001.txt
  24. Fixes
  25. -----
  26. * Backported fix for #455 from 2.4 to 2.3.
  27. * StateDB was not saved at shutdown.
  28. * Fixes worker sometimes hanging when hard time limit exceeded.
  29. .. _version-2.3.3:
  30. 2.3.3
  31. =====
  32. :release-date: 2011-16-09 05:00 P.M BST
  33. :release-by: Mher Movsisyan
  34. * Monkey patching :attr:`sys.stdout` could result in the worker
  35. crashing if the replacing object did not define :meth:`isatty`
  36. (Issue #477).
  37. * ``CELERYD`` option in :file:`/etc/default/celeryd` should not
  38. be used with generic init-scripts.
  39. .. _version-2.3.2:
  40. 2.3.2
  41. =====
  42. :release-date: 2011-10-07 05:00 P.M BST
  43. :release-by: Ask Solem
  44. .. _v232-news:
  45. News
  46. ----
  47. * Improved Contributing guide.
  48. If you'd like to contribute to Celery you should read the
  49. :ref:`Contributing Gudie <contributing>`.
  50. We are looking for contributors at all skill levels, so don't
  51. hesitate!
  52. * Now depends on Kombu 1.3.1
  53. * ``Task.request`` now contains the current worker host name (Issue #460).
  54. Available as ``task.request.hostname``.
  55. * It is now easier for app subclasses to extend how they are pickled.
  56. (see :class:`celery.app.AppPickler`).
  57. .. _v232-fixes:
  58. Fixes
  59. -----
  60. * `purge/discard_all` was not working correctly (Issue #455).
  61. * The coloring of log messages didn't handle non-ASCII data well
  62. (Issue #427).
  63. * [Windows] the multiprocessing pool tried to import ``os.kill``
  64. even though this is not available there (Issue #450).
  65. * Fixes case where the worker could become unresponsive because of tasks
  66. exceeding the hard time limit.
  67. * The :event:`task-sent` event was missing from the event reference.
  68. * ``ResultSet.iterate`` now returns results as they finish (Issue #459).
  69. This was not the case previously, even though the documentation
  70. states this was the expected behavior.
  71. * Retries will no longer be performed when tasks are called directly
  72. (using ``__call__``).
  73. Instead the exception passed to ``retry`` will be re-raised.
  74. * Eventlet no longer crashes if autoscale is enabled.
  75. growing and shrinking eventlet pools is still not supported.
  76. * ``py24`` target removed from :file:`tox.ini`.
  77. .. _version-2.3.1:
  78. 2.3.1
  79. =====
  80. :release-date: 2011-08-07 08:00 P.M BST
  81. :release-by: Ask Solem
  82. Fixes
  83. -----
  84. * The :setting:`CELERY_AMQP_TASK_RESULT_EXPIRES` setting did not work,
  85. resulting in an AMQP related error about not being able to serialize
  86. floats while trying to publish task states (Issue #446).
  87. .. _version-2.3.0:
  88. 2.3.0
  89. =====
  90. :release-date: 2011-08-05 12:00 P.M BST
  91. :tested: CPython: 2.5, 2.6, 2.7; PyPy: 1.5; Jython: 2.5.2
  92. :release-by: Ask Solem
  93. .. _v230-important:
  94. Important Notes
  95. ---------------
  96. * Now requires Kombu 1.2.1
  97. * Results are now disabled by default.
  98. The AMQP backend was not a good default because often the users were
  99. not consuming the results, resulting in thousands of queues.
  100. While the queues can be configured to expire if left unused, it was not
  101. possible to enable this by default because this was only available in
  102. recent RabbitMQ versions (2.1.1+)
  103. With this change enabling a result backend will be a conscious choice,
  104. which will hopefully lead the user to read the documentation and be aware
  105. of any common pitfalls with the particular backend.
  106. The default backend is now a dummy backend
  107. (:class:`celery.backends.base.DisabledBackend`). Saving state is simply an
  108. no-op, and AsyncResult.wait(), .result, .state, etc. will raise
  109. a :exc:`NotImplementedError` telling the user to configure the result backend.
  110. For help choosing a backend please see :ref:`task-result-backends`.
  111. If you depend on the previous default which was the AMQP backend, then
  112. you have to set this explicitly before upgrading::
  113. CELERY_RESULT_BACKEND = 'amqp'
  114. .. note::
  115. For :pypi:`django-celery` users the default backend is
  116. still ``database``, and results are not disabled by default.
  117. * The Debian init-scripts have been deprecated in favor of the generic-init.d
  118. init-scripts.
  119. In addition generic init-scripts for ``celerybeat`` and ``celeryev`` has
  120. been added.
  121. .. _v230-news:
  122. News
  123. ----
  124. * Automatic connection pool support.
  125. The pool is used by everything that requires a broker connection. For
  126. example calling tasks, sending broadcast commands, retrieving results
  127. with the AMQP result backend, and so on.
  128. The pool is disabled by default, but you can enable it by configuring the
  129. :setting:`BROKER_POOL_LIMIT` setting::
  130. BROKER_POOL_LIMIT = 10
  131. A limit of 10 means a maximum of 10 simultaneous connections can co-exist.
  132. Only a single connection will ever be used in a single-thread
  133. environment, but in a concurrent environment (threads, greenlets, etc., but
  134. not processes) when the limit has been exceeded, any try to acquire a
  135. connection will block the thread and wait for a connection to be released.
  136. This is something to take into consideration when choosing a limit.
  137. A limit of :const:`None` or 0 means no limit, and connections will be
  138. established and closed every time.
  139. * Introducing Chords (taskset callbacks).
  140. A chord is a task that only executes after all of the tasks in a taskset
  141. has finished executing. It's a fancy term for "taskset callbacks"
  142. adopted from
  143. `Cω <http://research.microsoft.com/en-us/um/cambridge/projects/comega/>`_).
  144. It works with all result backends, but the best implementation is
  145. currently provided by the Redis result backend.
  146. Here's an example chord::
  147. >>> chord(add.subtask((i, i))
  148. ... for i in xrange(100))(tsum.subtask()).get()
  149. 9900
  150. Please read the :ref:`Chords section in the user guide <canvas-chord>`, if you
  151. want to know more.
  152. * Time limits can now be set for individual tasks.
  153. To set the soft and hard time limits for a task use the ``time_limit``
  154. and ``soft_time_limit`` attributes:
  155. .. code-block:: python
  156. import time
  157. @task(time_limit=60, soft_time_limit=30)
  158. def sleeptask(seconds):
  159. time.sleep(seconds)
  160. If the attributes are not set, then the workers default time limits
  161. will be used.
  162. New in this version you can also change the time limits for a task
  163. at runtime using the :func:`time_limit` remote control command::
  164. >>> from celery.task import control
  165. >>> control.time_limit('tasks.sleeptask',
  166. ... soft=60, hard=120, reply=True)
  167. [{'worker1.example.com': {'ok': 'time limits set successfully'}}]
  168. Only tasks that starts executing after the time limit change will be affected.
  169. .. note::
  170. Soft time limits will still not work on Windows or other platforms
  171. that do not have the ``SIGUSR1`` signal.
  172. * Redis backend configuration directive names changed to include the
  173. ``CELERY_`` prefix.
  174. ===================================== ===================================
  175. **Old setting name** **Replace with**
  176. ===================================== ===================================
  177. `REDIS_HOST` `CELERY_REDIS_HOST`
  178. `REDIS_PORT` `CELERY_REDIS_PORT`
  179. `REDIS_DB` `CELERY_REDIS_DB`
  180. `REDIS_PASSWORD` `CELERY_REDIS_PASSWORD`
  181. ===================================== ===================================
  182. The old names are still supported but pending deprecation.
  183. * PyPy: The default pool implementation used is now multiprocessing
  184. if running on PyPy 1.5.
  185. * multi: now supports "pass through" options.
  186. Pass through options makes it easier to use celery without a
  187. configuration file, or just add last-minute options on the command
  188. line.
  189. Example use:
  190. .. code-block:: console
  191. $ celery multi start 4 -c 2 -- broker.host=amqp.example.com \
  192. broker.vhost=/ \
  193. celery.disable_rate_limits=yes
  194. * ``celerybeat``: Now retries establishing the connection (Issue #419).
  195. * ``celeryctl``: New ``list bindings`` command.
  196. Lists the current or all available bindings, depending on the
  197. broker transport used.
  198. * Heartbeat is now sent every 30 seconds (previously every 2 minutes).
  199. * ``ResultSet.join_native()`` and ``iter_native()`` is now supported by
  200. the Redis and Cache result backends.
  201. This is an optimized version of ``join()`` using the underlying
  202. backends ability to fetch multiple results at once.
  203. * Can now use SSL when sending error e-mails by enabling the
  204. :setting:`EMAIL_USE_SSL` setting.
  205. * ``events.default_dispatcher()``: Context manager to easily obtain
  206. an event dispatcher instance using the connection pool.
  207. * Import errors in the configuration module will not be silenced anymore.
  208. * ResultSet.iterate: Now supports the ``timeout``, ``propagate`` and
  209. ``interval`` arguments.
  210. * ``with_default_connection`` -> ``with default_connection``
  211. * TaskPool.apply_async: Keyword arguments ``callbacks`` and ``errbacks``
  212. has been renamed to ``callback`` and ``errback`` and take a single scalar
  213. value instead of a list.
  214. * No longer propagates errors occurring during process cleanup (Issue #365)
  215. * Added ``TaskSetResult.delete()``, which will delete a previously
  216. saved taskset result.
  217. * ``celerybeat`` now syncs every 3 minutes instead of only at
  218. shutdown (Issue #382).
  219. * Monitors now properly handles unknown events, so user-defined events
  220. are displayed.
  221. * Terminating a task on Windows now also terminates all of the tasks child
  222. processes (Issue #384).
  223. * worker: ``-I|--include`` option now always searches the current directory
  224. to import the specified modules.
  225. * Cassandra backend: Now expires results by using TTLs.
  226. * Functional test suite in ``funtests`` is now actually working properly, and
  227. passing tests.
  228. .. _v230-fixes:
  229. Fixes
  230. -----
  231. * ``celeryev`` was trying to create the pidfile twice.
  232. * celery.contrib.batches: Fixed problem where tasks failed
  233. silently (Issue #393).
  234. * Fixed an issue where logging objects would give "<Unrepresentable",
  235. even though the objects were.
  236. * ``CELERY_TASK_ERROR_WHITE_LIST`` is now properly initialized
  237. in all loaders.
  238. * ``celeryd_detach`` now passes through command line configuration.
  239. * Remote control command ``add_consumer`` now does nothing if the
  240. queue is already being consumed from.