Merge remote-tracking branch 'upstream/master'

docs/django/first-steps-with-django.rst

@@ -2,201 +2,180 @@
  First steps with Django
 =========================
 
-Configuring your Django project to use Celery
-=============================================
+Using Celery with Django
+========================
 
-You need four simple steps to use celery with your Django project.
+To use Celery with your Django project you must first define
+an instance of the Celery library.
 
-    1. Install the ``django-celery`` library:
-
-        .. code-block:: bash
+If you have a modern Django project layout like
 
-            $ pip install django-celery
+    - proj/
+      - proj/__init__.py
+      - proj/settings.py
+      - proj/urls.py
+    - manage.py
 
-    2. Add the following lines to ``settings.py``:
+then the recommended way is to create a new `proj/proj/celery.py` module
+that defines the Celery instance:
 
-        .. code-block:: python
+:file: `proj/proj/celery.py`
 
-            import djcelery
-            djcelery.setup_loader()
+.. code-block:: python
 
-    3. Add ``djcelery`` to ``INSTALLED_APPS``.
+    from celery import Celery
+    from django.conf import settings
 
-    4. Create the celery database tables.
+    celery = Celery('proj.celery')
+    celery.config_from_object(settings)
+    celery.autodiscover_tasks(settings.INSTALLED_APPS, related_name='tasks')
 
-        If you are using south_ for schema migrations, you'll want to:
+    @celery.task(bind=True)
+    def debug_task(self):
+        print('Request: {0!r}'.format(self.request))
 
-        .. code-block:: bash
+Let's explain what happens here.
+First we create the Celery app instance:
 
-            $ python manage.py migrate djcelery
+.. code-block:: python
 
-        For those who are not using south, a normal ``syncdb`` will work:
+    celery = Celery('proj')
 
-        .. code-block:: bash
+Then we add the Django settings module as a configuration source
+for Celery.  This means that you don't have to use multiple
+configuration files, and instead configure Celery directly
+from the Django settings.
 
-            $ python manage.py syncdb
+.. code-block:: python
 
-.. _south: http://pypi.python.org/pypi/South/
+    celery.config_from_object(settings)
 
-By default Celery uses `RabbitMQ`_ as the broker, but there are several
-alternatives to choose from, see :ref:`celerytut-broker`.
+Next, a common practice for reusable apps is to define all tasks
+in a separate ``tasks.py`` module, and Celery does have a way to
+autodiscover these modules:
 
-All settings mentioned in the Celery documentation should be added
-to your Django project's ``settings.py`` module. For example
-you can configure the :setting:`BROKER_URL` setting to specify
-what broker to use::
+.. code-block:: python
 
-    BROKER_URL = 'amqp://guest:guest@localhost:5672/'
+    celery.autodiscover_tasks(settings.INSTALLED_APPS, related_name='tasks')
 
-That's it.
+With the line above Celery will automatically discover tasks in reusable
+apps if you follow the ``tasks.py`` convention::
 
-.. _`RabbitMQ`: http://www.rabbitmq.com/
+    - app1/
+        - app1/tasks.py
+        - app2/models.py
+    - app2/
+        - app2/tasks.py
+        - app2/models.py
 
-Special note for mod_wsgi users
--------------------------------
+This way you do not have to manually add the individual modules
+to the :setting:`CELERY_IMPORTS` setting.
 
-If you're using ``mod_wsgi`` to deploy your Django application you need to
-include the following in your ``.wsgi`` module::
 
-    import djcelery
-    djcelery.setup_loader()
+Finally, the ``debug_task`` example is a task that dumps
+its own request information.  This is using the new ``bind=True`` task option
+introduced in Celery 3.1 to easily refer to the current task instance.
 
-Defining and calling tasks
-==========================
 
-Tasks are defined by wrapping functions in the ``@task`` decorator.
-It is a common practice to put these in their own module named ``tasks.py``,
-and the worker will automatically go through the apps in ``INSTALLED_APPS``
-to import these modules.
+The `celery` command
+--------------------
 
-For a simple demonstration create a new Django app called
-``celerytest``.  To create this app you need to be in the directory
-of your Django project where ``manage.py`` is located and execute:
+To use the :program:`celery` command with Django you need to
+set up the :envvar:`DJANGO_SETTINGS_MODULE` environment variable:
 
 .. code-block:: bash
 
-    $ python manage.py startapp celerytest
+    $ DJANGO_SETTINGS_MODULE='proj.settings' celery -A proj worker -l info
+
+    $ DJANGO_SETTINGS_MODULE='proj.settings' celery -A proj status
 
-Next you have to add the new app to ``INSTALLED_APPS`` so that your
-Django project recognizes it.  This setting is a tuple/list so just
-append ``celerytest`` as a new element at the end
+If you find this inconvienient you can create a small wrapper script
+alongside ``manage.py`` that automatically binds to your app, e.g. ``proj/celery.py`
+
+:file:`proj/celery.py`
 
 .. code-block:: python
 
-    INSTALLED_APPS = (
-        ...,
-        'djcelery',
-        'celerytest',
-    )
+    #!/usr/bin/env python
+    import os
 
-After the new app has been created and added to ``INSTALLED_APPS``,
-you can define your tasks by creating a new file called ``celerytest/tasks.py``:
+    from proj.celery import celery
 
-.. code-block:: python
 
-    from celery import task
+    if __name__ == '__main__':
+        os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.celery')
+        celery.start()
 
-    @task()
-    def add(x, y):
-        return x + y
+Then you can use this command directly:
 
-Our example task is pretty pointless, it just returns the sum of two
-arguments, but it will do for demonstration, and it is referred to in many
-parts of the Celery documentation.
+.. code-block:: bash
 
-.. admonition:: Relative Imports
+    $ ./celery.py status
 
-    You have to be consistent in how you import the task module, e.g. if
-    you have ``project.app`` in ``INSTALLED_APPS`` then you also
-    need to import the tasks ``from project.app`` or else the names
-    of the tasks will be different.
 
-    See :ref:`task-naming-relative-imports`
+Using the Django ORM/Cache as a result backend.
+-----------------------------------------------
 
-Starting the worker process
-===========================
+The ``django-celery`` library defines result backends that
+uses the Django ORM and Django Cache frameworks.
 
-In a production environment you will want to run the worker in the background
-as a daemon - see :ref:`daemonizing` - but for testing and
-development it is useful to be able to start a worker instance by using the
-``celery worker`` manage command, much as you would use Django's runserver:
+To use this with your project you need to follow these three steps:
 
-.. code-block:: bash
+    1. Install the ``django-celery`` library:
 
-    $ python manage.py celery worker --loglevel=info
+        .. code-block:: bash
 
-For a complete listing of the command-line options available,
-use the help command:
+            $ pip install django-celery
 
-.. code-block:: bash
+    2. Add ``djcelery`` to ``INSTALLED_APPS``.
 
-    $ python manage.py celery help
+    3. Create the celery database tables.
 
-Calling our task
-================
+        If you are using south_ for schema migrations, you'll want to:
 
-Now that the worker is running, open up a new python interactive shell
-with ``python manage.py shell`` to actually call the task you defined::
+        .. code-block:: bash
 
-    >>> from celerytest.tasks import add
+            $ python manage.py migrate djcelery
 
-    >>> add.delay(2, 2)
+        For those who are not using south, a normal ``syncdb`` will work:
 
+        .. code-block:: bash
 
-Note that if you open a regular python shell by simply running ``python``
-you will need to import your Django application's settings by running::
+            $ python manage.py syncdb
 
-    # Replace 'myproject' with your project's name
-    >>> from myproject import settings
+.. _south: http://pypi.python.org/pypi/South/
 
+.. admonition:: Relative Imports
 
-The ``delay`` method used above is a handy shortcut to the ``apply_async`` 
-method which enables you to have greater control of the task execution.
-To read more about calling tasks, including specifying the time at which
-the task should be processed see :ref:`guide-calling`.
+    You have to be consistent in how you import the task module, e.g. if
+    you have ``project.app`` in ``INSTALLED_APPS`` then you also
+    need to import the tasks ``from project.app`` or else the names
+    of the tasks will be different.
 
-.. note::
+    See :ref:`task-naming-relative-imports`
 
-    Tasks need to be stored in a real module, they can't
-    be defined in the python shell or IPython/bpython. This is because the
-    worker server must be able to import the task function.
+Starting the worker process
+===========================
 
-The task should now be processed by the worker you started earlier,
-and you can verify that by looking at the worker's console output.
+In a production environment you will want to run the worker in the background
+as a daemon - see :ref:`daemonizing` - but for testing and
+development it is useful to be able to start a worker instance by using the
+``celery worker`` manage command, much as you would use Django's runserver:
 
-Calling a task returns an :class:`~celery.result.AsyncResult` instance,
-which can be used to check the state of the task, wait for the task to finish
-or get its return value (or if the task failed, the exception and traceback).
+.. code-block:: bash
 
-By default django-celery stores this state in the Django database.
-You may consider choosing an alternate result backend or disabling
-states alltogether (see :ref:`task-result-backends`).
+    $ DJANGO_SETTINGS_MODULE='proj.settings' celery -A proj worker -l info
 
-To demonstrate how the results work call the task again, but this time
-keep the result instance returned::
+For a complete listing of the command-line options available,
+use the help command:
 
-    >>> result = add.delay(4, 4)
-    >>> result.ready() # returns True if the task has finished processing.
-    False
-    >>> result.result # task is not ready, so no return value yet.
-    None
-    >>> result.get()   # Waits until the task is done and returns the retval.
-    8
-    >>> result.result # direct access to result, doesn't re-raise errors.
-    8
-    >>> result.successful() # returns True if the task didn't end in failure.
-    True
+.. code-block:: bash
 
-If the task raises an exception, the return value of ``result.successful()``
-will be ``False``, and ``result.result`` will contain the exception instance
-raised by the task.
+    $ celery help
 
 Where to go from here
 =====================
 
-To learn more you should read the `Celery User Guide`_, and the
-`Celery Documentation`_ in general.
-
-
-.. _`Celery User Guide`: http://docs.celeryproject.org/en/latest/userguide/
-.. _`Celery Documentation`: http://docs.celeryproject.org/
+If you want to learn more you should continue to the
+:ref:`Next Steps <next-steps>` tutorial, and after that you
+can study the :ref:`User Guide <guide>`.

examples/django/README.rst

@@ -10,13 +10,13 @@ Contents
 
 This is the project iself, created using
 ``django-admin.py startproject proj``, and then the settings module
-(``proj/settings.py``) was modified to add ``tasks`` and ``demoapp`` to
+(``proj/settings.py``) was modified to add ``demoapp`` to
 ``INSTALLED_APPS``
 
-``tasks/``
+``proj/celery.py``
 ----------
 
-This app contains the Celery application instance for this project,
+This module contains the Celery application instance for this project,
 we take configuration from Django settings and use ``autodiscover_tasks`` to
 find task modules inside all packages listed in ``INSTALLED_APPS``.
 
@@ -24,8 +24,8 @@ find task modules inside all packages listed in ``INSTALLED_APPS``.
 ------------
 
 Example generic app.  This is decoupled from the rest of the project by using
-the ``@shared_task`` decorator.  Shared tasks are shared between all Celery
-instances.
+the ``@shared_task`` decorator.  This decorator returns a proxy that always
+points to the currently active Celery instance.
 
 
 Starting the worker
@@ -36,4 +36,4 @@ worker:
 
 .. code-block:: bash
 
-    $ DJANGO_SETTINGS_MODULE='proj.settings' celery -A tasks worker -l info
+    $ DJANGO_SETTINGS_MODULE='proj.settings' celery -A proj worker -l info

examples/django/tasks/__init__.py → examples/django/proj/celery.py

@@ -4,7 +4,7 @@ from celery import Celery
 from django.conf import settings
 
 
-celery = Celery('tasks', broker='amqp://localhost')
+celery = Celery('proj.celery')
 celery.config_from_object(settings)
 celery.autodiscover_tasks(settings.INSTALLED_APPS)
 

examples/django/tasks/models.py

@@ -1,3 +0,0 @@
-from django.db import models
-
-# Create your models here.

examples/django/tasks/tests.py

@@ -1,16 +0,0 @@
-"""
-This file demonstrates writing tests using the unittest module. These will pass
-when you run "manage.py test".
-
-Replace this with more appropriate tests for your application.
-"""
-
-from django.test import TestCase
-
-
-class SimpleTest(TestCase):
-    def test_basic_addition(self):
-        """
-        Tests that 1 + 1 always equals 2.
-        """
-        self.assertEqual(1 + 1, 2)

examples/django/tasks/views.py

@@ -1 +0,0 @@
-# Create your views here.