Updates whatsnew document

docs/whatsnew-3.1.rst

@@ -59,106 +59,364 @@ Highlights
 Important Notes
 ===============
 
-XXX
----
+No longer supports Python 2.5
+-----------------------------
 
-YYY
+Celery now requires Python 2.6 or later.
+
+We now have a dual codebase that runs on both Python 2 and 3 without
+using the ``2to3`` porting tool.
+
+Last version to use Pickle.
+---------------------------
+
+Starting from Celery 3.2 the default serializer will be json.
+
+If you depend on pickle being accepted you should be prepared
+for this change by explicitly allowing your worker
+to consume pickled messages using the :setting:`CELERY_ACCEPT_CONTENT``
+setting::
+
+    CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']
+
+Make sure you select only the serialization formats that you will actually be using,
+and make sure you have properly secured your broker from unwanted access
+(see the :ref:`guide-security` guide).
+
+The worker will show a deprecation warning if you don't define this setting.
 
 .. _v310-news:
 
 News
 ====
 
-XXX
----
+Now supports Django out of the box.
+-----------------------------------
 
-YYY
+The fixes and improvements applied by the django-celery library is now
+automatically applied by core Celery when it detects that
+the :envvar:`DJANGO_SETTINGS_MODULE` environment variable is set.
 
-In Other News
--------------
+The distribution ships with a new example project using Django
+in :file:`examples/django`:
 
-- No longer supports Python 2.5
+http://github.com/celery/celery/tree/master/examples/django
 
-    From this version Celery requires Python 2.6 or later.
+Some features still require the :mod:`django-celery` library:
 
-    Insteaad of using the 2to3 porting tool we now have
-    a dual codebase that runs on both Python 2 and Python 3.
+    - Celery does not implement the Django database or cache result backends.
+    - Celery does not ship with the database-based periodic task
+        scheduler.
 
-- Now depends on :ref:`Kombu 3.0 <kombu:version-3.0.0>`.
+.. note::
 
-- Now depends on :mod:`billiard` version 3.3.
+    If you are using django-celery then it is crucial that you have
+    ``djcelery.setup_loader()`` in your settings module, as this
+    no longer happens as a side-effect of importing the :mod:`djcelery`
+    module.
 
-- No longer depends on ``python-dateutil``
+Multiprocessing Pool improvements
 
-    Instead a dependency on :mod:`pytz` has been added, which was already
-    recommended in the documentation for accurate timezone support.
+:mod:`pytz` replaces ``python-dateutil`` dependency.
+----------------------------------------------------
 
-    This also means that dependencies are on the same on both Python 2 and
-    Python 3, and that the :file:`requirements/default-py3k.txt` file has
-    been removed.
+Celery no longer depends on the ``python-dateutil`` library,
+but instead a new dependency on the :mod:`pytz` library was added.
 
-- Time limits can now be set by the client for individual tasks (Issue #802).
+The :mod:`pytz` library was already recommended for accurate timezone support.
 
-    You can set both hard and soft time limits using the ``timeout`` and
-    ``soft_timeout`` calling options:
+This also means that dependencies are the same for both Python 2 and
+Python 3, and that the :file:`requirements/default-py3k.txt` file has
+been removed.
 
-    .. code-block:: python
+Old command-line programs removed and deprecated
+------------------------------------------------
 
-        >>> res = add.apply_async((2, 2), timeout=10, soft_timeout=8)
+The goal is that everyone should move the new :program:`celery` umbrella
+command, so with this version we deprecate the old command names,
+and remove commands that are not used in init scripts.
 
-        >>> res = add.subtask((2, 2), timeout=10, soft_timeout=8)()
++-------------------+--------------+-------------------------------------+
+| Program           | New Status   | Replacement                         |
++===================+==============+=====================================+
+| ``celeryd``       | *DEPRECATED* | :program:`celery worker`            |
++-------------------+--------------+-------------------------------------+
+| ``celerybeat``    | *DEPRECATED* | :program:`celery beat`              |
++-------------------+--------------+-------------------------------------+
+| ``celeryd-multi`` | *DEPRECATED* | :program:`celery multi`             |
++-------------------+--------------+-------------------------------------+
+| ``celeryctl``     | **REMOVED**  | :program:`celery`                   |
++-------------------+--------------+-------------------------------------+
+| ``celeryev``      | **REMOVED**  | :program:`celery events`            |
++-------------------+--------------+-------------------------------------+
+| ``camqadm``       | **REMOVED**  | :program:`celery amqp`              |
++-------------------+--------------+-------------------------------------+
 
-        >>> res = add.s(2, 2).set(timeout=10, soft_timeout=8)()
+Please see :program:`celery --help` for help using the umbrella command.
 
-    Contributed by Mher Movsisyan.
+Bootsteps: Extending the worker
+-------------------------------
 
-- Old command-line programs removed and deprecated
+By writing bootsteps you can now easily extend the consumer part
+of the worker to add additional features, or even message consumers.
 
-    The goal is that everyone should move the new :program:`celery` umbrella
-    command, so with this version we deprecate the old command names,
-    and remove commands that are not used in init scripts.
+The worker has been using bootsteps for some time, but these were never
+documented.  In this version the consumer part of the worker
+has also been rewritten to use bootsteps and the new :ref:`guide-extending`
+guide documents examples extending the worker, including adding
+custom message consumers.
 
-    +-------------------+--------------+-------------------------------------+
-    | Program           | New Status   | Replacement                         |
-    +===================+==============+=====================================+
-    | ``celeryd``       | *DEPRECATED* | :program:`celery worker`            |
-    +-------------------+--------------+-------------------------------------+
-    | ``celerybeat``    | *DEPRECATED* | :program:`celery beat`              |
-    +-------------------+--------------+-------------------------------------+
-    | ``celeryd-multi`` | *DEPRECATED* | :program:`celery multi`             |
-    +-------------------+--------------+-------------------------------------+
-    | ``celeryctl``     | **REMOVED**  | :program:`celery`                   |
-    +-------------------+--------------+-------------------------------------+
-    | ``celeryev``      | **REMOVED**  | :program:`celery events`            |
-    +-------------------+--------------+-------------------------------------+
-    | ``camqadm``       | **REMOVED**  | :program:`celery amqp`              |
-    +-------------------+--------------+-------------------------------------+
+See the :ref:`guide-extending` guide for more information.
 
-    Please see :program:`celery --help` for help using the umbrella command.
+.. note::
 
-- Celery now support Django out of the box.
+    Bootsteps written for older versions will not be compatible
+    with this version, as the API has changed significantly.
 
-    The fixes and improvements applied by the django-celery library is now
-    automatically applied by core Celery when it detects that
-    the :envvar:`DJANGO_SETTINGS_MODULE` environment setting is set.
+    The old API was experimental and internal so hopefully no one
+    is depending.  Should you happen to be using it then please
+    contact the mailing-list and we will help you port to the new version.
 
-    The distribution ships with a new example project using Django
-    in :file:`examples/django`:
+New result backend with RPC semantics
+-------------------------------------
 
-    http://github.com/celery/celery/tree/master/examples/django
+This version of the ``amqp`` result backend is a very good alternative
+to use in classical RPC scenarios, where the process that initiates
+the task is always the process to retrieve the result.
 
-    There are cases where you would want to use django-celery still
-    as:
+It uses Kombu to send and retrieve results, and each client
+will create a unique queue for replies to be sent to. Avoiding
+the significant overhead of the original amqp backend which creates
+one queue per task.
 
-        - Celery does not implement the Django database or cache backends.
-        - Celery does not automatically read configuration from Django settings.
-        - Celery does not ship with the database-based periodic task
-          scheduler.
+Results sent using this backend is not persistent, and so will
+not survive a broker restart, but you can set
+the :setting:`CELERY_RESULT_PERSISTENT` setting to change that.
 
-    If you are using django-celery then it is crucial that you have
-    ``djcelery.setup_loader()`` in your settings module, as this
-    no longer happens as a side-effect of importing the :mod:`djcelery`
-    module.
+.. code-block:: python
+
+    CELERY_RESULT_BACKEND = 'rpc'
+
+Note that chords are currently not supported by the RPC backend.
+
+Time limits can now be set by the client.
+-----------------------------------------
+
+You can set both hard and soft time limits using the ``time_limit`` and
+``soft_time_limit`` calling options:
+
+.. code-block:: python
+
+    >>> res = add.apply_async((2, 2), time_limit=10, soft_time_limit=8)
+
+    >>> res = add.subtask((2, 2), time_limit=10, soft_time_limit=8).delay()
+
+    >>> res = add.s(2, 2).set(time_limit=10, soft_time_limit=8).delay()
+
+Contributed by Mher Movsisyan.
+
+Redis: Separate broadcast messages by virtual host
+---------------------------------------------------------------------------
+
+Broadcast messages are seen by all virtual hosts when using the Redis
+transport.  You can fix this by enabling a prefix to all channels
+so that the messages are separated by virtual host::
+
+    BROKER_TRANSPORT_OPTIONS = {'fanout_prefix': True}
+
+Note that you will not be able to communicate with workers running older
+versions or workers that does not have this setting enabled.
+
+This setting will be the default in the future, so better to migrate
+sooner rather than later.
+
+Related to Issue #1490.
+
+Events are now ordered using logical time.
+------------------------------------------
+
+Timestamps are not a reliable way to order events in a distributed system,
+for one the floating point value does not have enough precision, but
+also it's impossible to keep physical clocks in sync.
+
+Celery event messages have included a logical clock value for some time,
+but starting with this version that field is also used to order them
+(that is if the monitor is using :mod:`celery.events.state`).
+
+The logical clock is currently implemented using Lamport timestamps,
+which does not have a high degree of accuracy, but should be good
+enough to casually order the events.
+
+Also, events now records timezone information for better timestamp
+accuracy, where a new ``utcoffset`` field is included in the event.
+This is a signed integer telling the difference from UTC time in hours,
+so e.g. an even sent from the Europe/London timezone in daylight savings
+time will have an offset of 1.
+
+:class:`@events.Receiver` will automatically convert the timestamps
+to the destination timezone.
+
+.. note::
+
+    The logical clock is synchronized with other nodes
+    in the same cluster (neighbors), so this means that the logical
+    epoch will start at the point when the first worker in the cluster
+    starts.
+
+    If all of the workers are shutdown the clock value will be lost
+    and reset to 0, to protect against this you should specify
+    a :option:`--statedb` so that the worker can persist the clock
+    value at shutdown.
+
+    You may notice that the logical clock is an integer value and
+    increases very rapidly.  Do not worry about the value overflowing
+    though, as even in the most busy clusters it may take several
+    millennia before the clock exceeds a 64 bits value.
+
+New worker node name format (``name@host``).
+-------------------------------------------------------------------------
+
+Node names are now constructed by a node name and a hostname separated by '@'.
+
+This change was made to more easily identify multiple instances running
+on the same machine.
+
+If a custom name is not specified then the
+worker will use the name 'celery' by default, resulting in a
+fully qualified node name of 'celery@hostname':
+
+.. code-block:: bash
+
+    $ celery worker -n example.com
+    celery@example.com
+
+To also set the name you must include the @:
+
+.. code-block:: bash
+
+    $ celery worker -n worker1@example.com
+    worker1@example.com
+
+The worker will identify itself using the fully qualified
+node name in events and broadcast messages, so where before
+a worker would identify as 'worker1.example.com', it will now
+use 'celery@worker1.example.com'.
+
+Remember that the ``-n`` argument also supports simple variable
+substitutions, so if the current hostname is *jerry.example.com*
+then ``%h`` will expand into that:
+
+.. code-block:: bash
+
+    $ celery worker -n worker1@%h
+    worker1@jerry.example.com
+
+The available substitutions are as follows:
+
++---------------+---------------------------------------+
+| Variable      | Substitution                          |
++===============+=======================================+
+| ``%h``        | Full hostname (including domain name) |
++---------------+---------------------------------------+
+| ``%d``        | Domain name only                      |
++---------------+---------------------------------------+
+| ``%n``        | Hostname only (without domain name)   |
++---------------+---------------------------------------+
+| ``%%``        | The character ``%``                   |
++---------------+---------------------------------------+
+
+Bound tasks
+-----------
+
+The task decorator can now created "bound tasks", which means that the
+task will receive the ``self`` argument.
+
+.. code-block:: python
+
+    @app.task(bind=True)
+    def send_twitter_status(self, oauth, tweet):
+        try:
+            twitter = Twitter(oauth)
+            twitter.update_status(tweet)
+        except (Twitter.FailWhaleError, Twitter.LoginError) as exc:
+            raise self.retry(exc=exc)
+
+Using *bound tasks* is now the recommended approach whenever
+you need access to the current task or request context.
+Previously one would have to refer to the name of the task
+instead (``send_twitter_status.retry``), but this could lead to problems
+in some instances where the registered task was no longer the same
+object.
+
+Gossip: Worker <-> Worker communication.
+----------------------------------------
+
+Workers now synchronizes revoked tasks with its neighbors.
+
+This happens at startup and causes a one second startup delay
+to collect broadcast responses from other workers.
+
+Now supports Setuptools extra requirements.
+-------------------------------------------
+
+Pip now supports installing setuptools extra requirements
+so we have deprecated the old bundles, replacing them with these
+little creatures, which are more convenient since you can easily
+specify multiple extras (e.g. ``pip install celery[redis,mongodb]``).
+
++-------------+-------------------------+---------------------------+
+| Extension   | Requirement entry       | Type                      |
++=============+=========================+===========================+
+| Redis       | ``celery[redis]``       | transport, result backend |
++-------------+-------------------------+---------------------------+
+| MongoDB``   | ``celery[mongodb]``     | transport, result backend |
++-------------+-------------------------+---------------------------+
+| CouchDB     | ``celery[couchdb]``     | transport                 |
++-------------+-------------------------+---------------------------+
+| Beanstalk   | ``celery[beanstalk]``   | transport                 |
++-------------+-------------------------+---------------------------+
+| ZeroMQ      | ``celery[zeromq]``      | transport                 |
++-------------+-------------------------+---------------------------+
+| Zookeeper   | ``celery[zookeeper]``   | transport                 |
++-------------+-------------------------+---------------------------+
+| SQLAlchemy  | ``celery[sqlalchemy]``  | transport, result backend |
++-------------+-------------------------+---------------------------+
+| librabbitmq | ``celery[librabbitmq]`` | transport (C amqp client) |
++-------------+-------------------------+---------------------------+
+
+There are more examples in the :ref:`bundles` section.
+
+Calling a subtask will now execute the task directly
+----------------------------------------------------
+
+A misunderstanding led to ``Signature.__call__`` being an alias of
+``.delay`` but this does not conform to the calling API of ``Task`` which
+should call the underlying task method.
+
+This means that:
+
+.. code-block:: python
+
+    @app.task
+    def add(x, y):
+        return x + y
+
+    add.s(2, 2)()
+
+now does the same as calling the task directly:
+
+.. code-block:: python
+
+    add(2, 2)
+
+In Other News
+-------------
+
+- Now depends on :ref:`Kombu 3.0 <kombu:version-3.0.0>`.
+
+- Now depends on :mod:`billiard` version 3.3.
+
+- Worker will now crash if running as the root user with pickle enabled.
 
 - Canvas: ``group.apply_async`` and ``chain.apply_async`` no longer starts
   separate task.
@@ -168,23 +426,10 @@ In Other News
     users.  If you still want this behavior you can create a task to do it
     for you.
 
-- Redis: Option to separate broadcast messages by virtual host (Issue #1490).
-
-    Broadcast messages are seen by all virtual hosts when using the Redis
-    transport.  You can fix this by enabling a prefix to all channels
-    so that the messages are separated by virtual host::
-
-        BROKER_TRANSPORT_OPTIONS = {'fanout_prefix': True}
+- New method ``Signature.freeze()`` can be used to "finalize"
+  signatures/subtask.
 
-    Note that you will not be able to communicate with workers running older
-    versions or workers that does not have this setting enabled.
-
-    This setting will be the default in the future, so better to migrate
-    sooner rather than later.
-
-- ``Signature.freeze()`` can now be used to "finalize" subtasks
-
-    Regular subtask:
+    Regular signature:
 
     .. code-block:: python
 
@@ -207,50 +452,8 @@ In Other News
         >>> g()
         <GroupResult: e1094b1d-08fc-4e14-838e-6d601b99da6d [70c0fb3d-b60e-4b22-8df7-aa25b9abc86d, 58fcd260-2e32-4308-a2ea-f5be4a24f7f4]>
 
-
-
-- The consumer part of the worker has been rewritten to use Bootsteps.
-
-    By writing bootsteps you can now easily extend the consumer part
-    of the worker to add additional features, or even message consumers.
-
-    See the :ref:`guide-extending` guide for more information.
-
-- New Bootsteps implementation.
-
-    The bootsteps and namespaces have been refactored for the better,
-    sadly this means that bootsteps written for older versions will
-    not be compatible with this version.
-
-    Bootsteps were never publicly documented and was considered
-    experimental, so chances are no one has ever implemented custom
-    bootsteps, but if you did please contact the mailing-list
-    and we'll help you port them.
-
-    - Module ``celery.worker.bootsteps`` renamed to :mod:`celery.bootsteps`
-    - The name of a bootstep no longer contain the name of the namespace.
-    - A bootstep can now be part of multiple namespaces.
-    - Namespaces must instantiate individual bootsteps, and
-      there's no global registry of bootsteps.
-
-- New result backend with RPC semantics (``rpc``).
-
-    This version of the ``amqp`` result backend is a very good alternative
-    to use in classical RPC scenarios, where the process that initiates
-    the task is always the process to retrieve the result.
-
-    It uses Kombu to send and retrieve results, and each client
-    will create a unique queue for replies to be sent to. Avoiding
-    the significant overhead of the original amqp backend which creates
-    one queue per task, but it's important to consider that it will
-    not be possible to retrieve the result from another process,
-    and that results sent using this backend is not persistent and so will
-    not survive a broker restart.
-
-    It has only been tested with the AMQP and Redis transports.
-
-- App instances can now add additional command line options
-  to the worker and beat programs.
+-  New ability to specify additional command line options
+   to the worker and beat programs.
 
     The :attr:`@Celery.user_options` attribute can be used
     to add additional command-line arguments, and expects
@@ -266,35 +469,11 @@ In Other News
             Option('--my-argument'),
         )
 
-    See :ref:`guide-extending` for more information.
-
-- Events are now ordered using logical time.
-
-    Timestamps are not a reliable way to order events in a distributed system,
-    for one the floating point value does not have enough precision, but
-    also it's impossible to keep physical clocks in sync.
-
-    Celery event messages have included a logical clock value for some time,
-    but starting with this version that field is also used to order them
-    (that is if the monitor is using :mod:`celery.events.state`).
-
-    The logical clock is currently implemented using Lamport timestamps,
-    which does not have a high degree of accuracy, but should be good
-    enough to casually order the events.
+    See the :ref:`guide-extending` guide for more information.
 
 - All events now include a ``pid`` field, which is the process id of the
   process that sent the event.
 
-- Events now supports timezones.
-
-    A new ``utcoffset`` field is now sent with every event.  This is a
-    signed integer telling the difference from UTC time in hours,
-    so e.g. an even sent from the Europe/London timezone in daylight savings
-    time will have an offset of 1.
-
-    :class:`@events.Receiver` will automatically convert the timestamps
-    to the destination timezone.
-
 - Event heartbeats are now calculated based on the time when the event
   was received by the monitor, and not the time reported by the worker.
 
@@ -305,6 +484,11 @@ In Other News
     time and the internal time is greater than 15 seconds, suggesting
     that the clocks are out of sync.
 
+- Many parts of the Celery codebase now uses a montonic clock.
+
+    The montonic clock function is built-in starting from Python 3.4,
+    but we also have fallback implementaions for Linux and OS X.
+
 - :program:`celery worker` now supports a ``--detach`` argument to start
   the worker as a daemon in the background.
 
@@ -327,6 +511,11 @@ In Other News
     will only send worker related events and silently drop any attempts
     to send events related to any other group.
 
+- ``Result.revoke`` will no longer wait for replies.
+
+    You can add the ``reply=True`` argument if you really want to wait for
+    responses from the workers.
+
 - Better support for link and link_error tasks for chords.
 
     Contributed by Steeve Morin.
@@ -352,45 +541,6 @@ In Other News
 
     Contributed by Mher Movisyan.
 
-
-- Now supports Setuptools extra requirements.
-
-    +-------------+-------------------------+---------------------------+
-    | Extension   | Requirement entry       | Type                      |
-    +=============+=========================+===========================+
-    | Redis       | ``celery[redis]``       | transport, result backend |
-    +-------------+-------------------------+---------------------------+
-    | MongoDB``   | ``celery[mongodb]``     | transport, result backend |
-    +-------------+-------------------------+---------------------------+
-    | CouchDB     | ``celery[couchdb]``     | transport                 |
-    +-------------+-------------------------+---------------------------+
-    | Beanstalk   | ``celery[beanstalk]``   | transport                 |
-    +-------------+-------------------------+---------------------------+
-    | ZeroMQ      | ``celery[zeromq]``      | transport                 |
-    +-------------+-------------------------+---------------------------+
-    | Zookeeper   | ``celery[zookeeper]``   | transport                 |
-    +-------------+-------------------------+---------------------------+
-    | SQLAlchemy  | ``celery[sqlalchemy]``  | transport, result backend |
-    +-------------+-------------------------+---------------------------+
-    | librabbitmq | ``celery[librabbitmq]`` | transport (C amqp client) |
-    +-------------+-------------------------+---------------------------+
-
-    Examples using :program:`pip install`:
-
-    .. code-block:: bash
-
-        pip install celery[redis]
-        pip install celery[librabbitmq]
-
-        pip install celery[redis,librabbitmq]
-
-        pip install celery[mongodb]
-        pip install celery[couchdb]
-        pip install celery[beanstalk]
-        pip install celery[zeromq]
-        pip install celery[zookeeper]
-        pip install celery[sqlalchemy]
-
 - New settings :setting:`CELERY_EVENT_QUEUE_TTL` and
   :setting:`CELERY_EVENT_QUEUE_EXPIRES`.
 
@@ -410,7 +560,6 @@ In Other News
 
     .. _`Couchbase`: http://www.couchbase.com
 
-
 - CentOS init script now supports starting multiple worker instances.
 
     See the script header for details.
@@ -421,95 +570,6 @@ In Other News
 
     Fix contributed by Idan Kamara
 
-- Worker node names now consists of a name and a hostname separated by '@'.
-
-    This change is to more easily identify multiple instances running
-    on the same machine.
-
-    If a custom name is not specified then the
-    worker will use the name 'celery' in default, resulting in a
-    fully qualified node name of 'celery@hostname':
-
-    .. code-block:: bash
-
-        $ celery worker -n example.com
-        celery@example.com
-
-    To set the name you must include the @:
-
-    .. code-block:: bash
-
-        $ celery worker -n worker1@example.com
-        worker1@example.com
-
-    This also means that the worker will identify itself using the full
-    nodename in events and broadcast messages, so where before
-    a worker would identify as 'worker1.example.com', it will now
-    use 'celery@worker1.example.com'.
-
-    Remember that the ``-n`` argument also supports simple variable
-    substitutions, so if the current hostname is *jerry.example.com*
-    then ``%h`` will expand into that:
-
-    .. code-block:: bash
-
-        $ celery worker -n worker1@%h
-        worker1@jerry.example.com
-
-    The table of substitutions is as follows:
-
-    +---------------+---------------------------------------+
-    | Variable      | Substitution                          |
-    +===============+=======================================+
-    | ``%h``        | Full hostname (including domain name) |
-    +---------------+---------------------------------------+
-    | ``%d``        | Domain name only                      |
-    +---------------+---------------------------------------+
-    | ``%n``        | Hostname only (without domain name)   |
-    +---------------+---------------------------------------+
-    | ``%%``        | The character ``%``                   |
-    +---------------+---------------------------------------+
-
-- Task decorator can now create "bound tasks"
-
-    This means that the function will be a method in the resulting
-    task class and so will have a ``self`` argument that can be used
-    to refer to the current task:
-
-    .. code-block:: python
-
-        @app.task(bind=True)
-        def send_twitter_status(self, oauth, tweet):
-            try:
-                twitter = Twitter(oauth)
-                twitter.update_status(tweet)
-            except (Twitter.FailWhaleError, Twitter.LoginError) as exc:
-                raise self.retry(exc=exc)
-
-    Using *bound tasks* is now the recommended approach whenever
-    you need access to the current task or request context.
-    Previously one would have to refer to the name of the task
-    instead (``send_twitter_status.retry``), but this could lead to problems
-    in some instances where the registered task was no longer the same
-    object.
-
-- Workers now synchronizes revoked tasks with its neighbors.
-
-    This happens at startup and causes a one second startup delay
-    to collect broadcast responses from other workers.
-
-- Workers logical clock value is now persisted so that the clock
-  is not reset when a worker restarts.
-
-    The logical clock is also synchronized with other nodes
-    in the same cluster (neighbors), so this means that the logical
-    epoch will start at the point when the first worker in the cluster
-    starts.
-
-    You may notice that the logical clock is an integer value and increases
-    very rapidly. It will take several millennia before the clock overflows 64 bits,
-    so this is not a concern.
-
 - New setting :setting:`BROKER_LOGIN_METHOD`
 
     This setting can be used to specify an alternate login method
@@ -520,28 +580,6 @@ In Other News
 - The ``dump_conf`` remote control command will now give the string
   representation for types that are not JSON compatible.
 
-- Calling a subtask will now execute the task directly as documented.
-
-    A misunderstanding led to ``Signature.__call__`` being an alias of
-    ``.delay`` but this does not conform to the calling API of ``Task`` which
-    should call the underlying task method.
-
-    This means that:
-
-    .. code-block:: python
-
-        @app.task
-        def add(x, y):
-            return x + y
-
-        add.s(2, 2)()
-
-    does the same as calling the task directly:
-
-    .. code-block:: python
-
-        add(2, 2)
-
 - Function `celery.security.setup_security` is now :func:`celery.setup_security`.
 
 - Message expires value is now forwarded at retry (Issue #980).
@@ -609,7 +647,6 @@ In Other News
         # Create graph from the current cluster
         $ celery graph workers | dot -T png -o workers.png
 
-
         # Create graph from a specified list of workers
         $ celery graph workers nodes:w1,w2,w3 | dot -T png workers.png
 
@@ -634,7 +671,6 @@ In Other News
 
         import celery
 
-
         class Celery(celery.Celery):
 
             def __init__(self, *args, **kwargs):
@@ -663,21 +699,69 @@ In Other News
 
         $ C_IMPDEBUG=1 celery shell
 
+- Message headers now available as part of the task request.
 
-- :class:`celery.apps.worker.Worker` has been refactored as a subclass of
-  :class:`celery.worker.WorkController`.
+    Example setting and retreiving a header value::
 
-    This removes a lot of duplicate functionality.
+        @app.task(bind=True)
+        def t(self):
+            return self.request.headers.get('sender')
 
+        >>> t.apply_async(headers={'sender': 'George Costanza'})
 
 - :class:`@events.Receiver` is now a :class:`kombu.mixins.ConsumerMixin`
   subclass.
 
+- New :signal:`task_send`` signal dispatched before a task message
+  is sent and can be used to modify the final message fields (Issue #1281).
+
 - ``celery.platforms.PIDFile`` renamed to :class:`celery.platforms.Pidfile`.
 
 - ``celery.results.BaseDictBackend`` has been removed, replaced by
   :class:``celery.results.BaseBackend``.
 
+- MongoDB Backend: Can now be configured using an URL
+
+    See :ref:`example-mongodb-result-config`.
+
+- MongoDB Backend: No longer using deprecated ``pymongo.Connection``.
+
+- MongoDB Backend: Now disables ``auto_start_request``.
+
+- MongoDB Backend: Now enables ``use_greenlets`` when eventlet/gevent is used.
+
+- ``subtask()`` / ``maybe_subtask()`` renamed to
+  ``signature()``/``maybe_signature()``.
+
+    Aliases still available for backwards compatibility.
+
+- The ``correlation_id`` message property is now automatically set to the
+  id of the task.
+
+- The task message ``eta`` and ``expires`` fields now includes timezone
+  information.
+
+- All result backends ``store_result``/``mark_as_*`` methods must now accept
+  a ``request`` keyword argument.
+
+- Events now emit warning if the broken ``yajl`` library is used.
+
+- The :signal:`celeryd_init` signal now takes an extra keyword argument:
+  ``option``.
+
+    This is the mapping of parsed command line arguments, and can be used to
+    prepare new preload arguments (``app.user_options['preload']``).
+
+- New callback: ``Celery.on_configure``.
+
+    This callback is called when an app is about to be configured (a
+    configuration key is required).
+
+- Eventlet/gevent/solo/threads pools now properly handles BaseException errors
+  raised by tasks.
+
+- Multi: Now properly handles both ``-f`` and ``--logfile`` options
+  (Issue #1541).
 
 .. _v310-experimental:
 
@@ -725,6 +809,13 @@ Scheduled Removals
                 }
             }
 
+- Worker: No longer forks on :sig:`HUP`
+
+    This means that the worker will reuse the same pid, which makes it
+    easier for process supervisors.
+
+    Contributed by Jameel Al-Aziz.
+
 - The ``CELERY_AMQP_TASK_RESULT_EXPIRES`` setting is no longer supported.
 
     Use :setting:`CELERY_TASK_RESULT_EXPIRES` instead.
@@ -739,6 +830,40 @@ Scheduled Removals
 - The ``Celery.with_default_connection`` method has been removed in favor
   of ``with app.connection_or_acquire``.
 
+- The :signal:`task_revoked` signal now accepts new ``request`` argument
+  (Issue #1555).
+
+    The revoked signal is dispatched after the task request is removed from
+    the stack, so it must instead use the :class:`~celery.worker.job.Request`
+    object to get information about the task.
+
+- Worker: New :option:`-X` command line argument to exclude queues
+  (Issue #1399).
+
+    The :option:`-X` argument is the inverse of the :option:`-Q` argument
+    and accepts a list of queues to exclude (not consume from):
+
+    .. code-block:: bash
+
+        # Consume from all queues in CELERY_QUEUES, but not the 'foo' queue.
+        $ celery worker -A proj -l info -X foo
+
+- New public API to push and pop from the current task stack:
+
+    :func:`celery.app.push_current_task` and
+    :func:`celery.app.pop_current_task``.
+
+- ``RetryTaskError`` has been renamed to :exc:`~celery.exceptions.Retry`.
+
+    The old name is still available for backwards compatibility.
+
+- New semipredicate exception :exc:`~celery.exceptions.Reject`
+
+    This exception can be raised to reject/requeue the task message,
+    see :ref:`task-semipred-reject` for examples.
+
+- Semipredicates documented: :ref:`task-semipredicates` (Retry/Ignore/Reject).
+
 .. _v310-deprecations:
 
 Deprecations
@@ -756,7 +881,50 @@ See the :ref:`deprecation-timeline`.
 Fixes
 =====
 
-- XXX
+- AMQP Backend: join did not convert exceptions when using the json
+  serializer.
+
+- Worker: Workaround for unicode errors in logs (Issue #427)
+
+- Task methods: ``.apply_async`` now works properly if args list is None
+  (Issue #1459).
+
+- Autoscale and ``pool_grow``/``pool_shrink`` remote control commands
+  will now also automatically increase and decrease the consumer prefetch count.
+
+    Fix contributed by Daniel M. Taub.
+
+- Optimization: Improved performance of ``ResultSet.join_native()``.
+
+    Contributed by Stas Rudakou.
+
+- ``celery control pool_`` commands did not coerce string arguments to int.
+
+- Redis/Cache chords: Callback result is now set to failure if the group
+  disappeared from the database (Issue #1094).
+
+- Worker: Now makes sure that the shutdown process is not initiated multiple
+  times.
+
+- Adds :envvar:`C_FAKEFORK` envvar for simple init script/multi debugging
+
+    This means that you can now do:
+
+    .. code-block:: bash
+
+            $ C_FAKEFORK=1 celery multi start 10
+
+    or:
+
+    .. code-block:: bash
+
+        $ C_FAKEFORK=1 /etc/init.d/celeryd start
+
+    to avoid the daemonization step to see errors that are not visible
+    due to missing stdout/stderr.
+
+    A ``dryrun`` command has been added to the generic init script that
+    enables this option.
 
 .. _v310-internal:
 
@@ -780,3 +948,8 @@ Internal changes
 - Removed unused and never documented global instance
   ``celery.events.state.state``.
 
+- :class:`celery.apps.worker.Worker` has been refactored as a subclass of
+  :class:`celery.worker.WorkController`.
+
+    This removes a lot of duplicate functionality.
+