Results are now disabled by default.

The AMQP backend was not a good default because often the users were
not consuming the results, resulting in thousands of queues.

While the queues can be configured to expire if left unused, it was not
possible to enable this by default because this was only available in
recent RabbitMQ versions (2.1.1+)

With this change enabling a result backend will be a conscious choice,
which will hopefully lead the user to read the documentation and be aware
of any common pitfalls with the particular backend.

The default backend is now a dummy backend
(:class:`celery.backends.base.DisabledBackend`).  Saving state is simply an
noop operation, and AsyncResult.wait(), .result, .state, etc will raise
a :exc:`NotImplementedError` telling the user to configure the result backend.

Changelog

@@ -5,6 +5,39 @@
 .. contents::
     :local:
 
+.. _version-2.3.0:
+
+2.3.0
+=====
+:release-date: TBA
+:status: in development
+:branch: master
+
+.. _v230-important:
+
+Important Notes
+---------------
+
+* Results are now disabled by default.
+
+    The AMQP backend was not a good default because often the users were
+    not consuming the results, resulting in thousands of queues.
+
+    While the queues can be configured to expire if left unused, it was not
+    possible to enable this by default because this was only available in
+    recent RabbitMQ versions (2.1.1+)
+
+    With this change enabling a result backend will be a conscious choice,
+    which will hopefully lead the user to read the documentation and be aware
+    of any common pitfalls with the particular backend.
+
+    The default backend is now a dummy backend
+    (:class:`celery.backends.base.DisabledBackend`).  Saving state is simply an
+    noop operation, and AsyncResult.wait(), .result, .state, etc will raise
+    a :exc:`NotImplementedError` telling the user to configure the result backend.
+
+    For help choosing a backend please see :ref:`task-result-backends`.
+
 .. _version-2.2.5:
 
 2.2.5

celery/app/defaults.py

@@ -78,7 +78,7 @@ NAMESPACES = {
         "IGNORE_RESULT": Option(False, type="bool"),
         "MAX_CACHED_RESULTS": Option(5000, type="int"),
         "MESSAGE_COMPRESSION": Option(None, type="string"),
-        "RESULT_BACKEND": Option("amqp"),
+        "RESULT_BACKEND": Option(None, type="string"),
         "RESULT_DBURI": Option(),
         "RESULT_ENGINE_OPTIONS": Option(None, type="dict"),
         "RESULT_EXCHANGE": Option("celeryresults"),

celery/backends/__init__.py

@@ -10,13 +10,15 @@ BACKEND_ALIASES = {
     "tyrant": "celery.backends.tyrant.TyrantBackend",
     "database": "celery.backends.database.DatabaseBackend",
     "cassandra": "celery.backends.cassandra.CassandraBackend",
+    "disabled": "celery.backends.base.DisabledBackend",
 }
 
 _backend_cache = {}
 
 
-def get_backend_cls(backend, loader=None):
+def get_backend_cls(backend=None, loader=None):
     """Get backend class by name/alias"""
+    backend = backend or "disabled"
     loader = loader or current_app.loader
     if backend not in _backend_cache:
         aliases = dict(BACKEND_ALIASES, **loader.override_backends)

celery/backends/base.py

@@ -261,3 +261,18 @@ class KeyValueStoreBackend(BaseDictBackend):
         if meta:
             meta = pickle.loads(str(meta))
             return meta
+
+
+class DisabledBackend(BaseBackend):
+
+    def store_result(self, *args, **kwargs):
+        pass
+
+    def _is_disabled(self, *args, **kwargs):
+        raise NotImplementedError("No result backend configured.  "
+                "Please see the documentation for more information.")
+
+    wait_for = _is_disabled
+    get_status = _is_disabled
+    get_result = _is_disabled
+    get_traceback = _is_disabled

docs/configuration.rst

@@ -89,9 +89,10 @@ CELERY_RESULT_BACKEND
 ~~~~~~~~~~~~~~~~~~~~~
 
 The backend used to store task results (tombstones).
+Disabled by default.
 Can be one of the following:
 
-* database (default)
+* database
     Use a relational database supported by `SQLAlchemy`_.
     See :ref:`conf-database-result-backend`.
 

docs/getting-started/first-steps-with-celery.rst

@@ -140,18 +140,41 @@ broker will hold on to the task until a worker server has consumed and
 executed it.
 
 Right now we have to check the worker log files to know what happened
-with the task.  This is because we didn't keep the
-:class:`~celery.result.AsyncResult` object returned.
+with the task.  Applying a task returns an
+:class:`~celery.result.AsyncResult`, if you have configured a result store
+the :class:`~celery.result.AsyncResult` enables you to check the state of
+the task, wait for the task to finish, get its return value
+or exception/traceback if the task failed, and more.
 
-The :class:`~celery.result.AsyncResult` lets us check the state of the task,
-wait for the task to finish, get its return value or exception/traceback
-if the task failed, and more.
+Keeping Results
+---------------
 
-Let's execute the task again -- but this time we'll keep track of the task
-by holding on to the :class:`~celery.result.AsyncResult`::
+If you want to keep track of the tasks state, Celery needs to store or send
+the states somewhere.  There are several
+built-in backends to choose from: SQLAlchemy/Django ORM, Memcached, Redis,
+AMQP, MongoDB, Tokyo Tyrant and Redis -- or you can define your own.
+
+For this example we will use the `amqp` result backend, which sends states
+as messages.  The backend is configured via the ``CELERY_RESULT_BACKEND``
+option, in addition individual result backends may have additional settings
+you can configure::
+
+    CELERY_RESULT_BACKEND = "amqp"
+
+    #: We want the results to expire in 5 minutes, note that this requires
+    #: RabbitMQ version 2.1.1 or higher, so please comment out if you have
+    #: an earlier version.
+    CELERY_AMQP_TASK_RESULT_EXPIRES = 300
+
+To read more about result backends please see :ref:`task-result-backends`.
+
+Now with the result backend configured, let's execute the task again.
+This time we'll hold on to the :class:`~celery.result.AsyncResult`::
 
     >>> result = add.delay(4, 4)
 
+Here's some examples of what you can do when you have results::
+
     >>> result.ready() # returns True if the task has finished processing.
     False
 

docs/userguide/tasks.rst

@@ -475,10 +475,9 @@ Celery needs to store or send the states somewhere.  There are several
 built-in backends to choose from: SQLAlchemy/Django ORM, Memcached, Redis,
 AMQP, MongoDB, Tokyo Tyrant and Redis -- or you can define your own.
 
-There is no backend that works well for every single use case, but for
-historical reasons the default backend is the AMQP backend.  You should read
-about the strenghts and weaknesses of each backend, and choose the most
-appropriate for your own needs.
+No backend works well for every use case.
+You should read about the strenghts and weaknesses of each backend, and choose
+the most appropriate for your needs.
 
 
 .. seealso::