Browse Source

New Q/A: Can I send some tasks to only some servers?

Ask Solem 16 years ago
parent
commit
a81199fa21
1 changed files with 69 additions and 0 deletions
  1. 69 0
      docs/faq.rst

+ 69 - 0
docs/faq.rst

@@ -27,3 +27,72 @@ celeryd is not doing anything, just hanging
 
 **Answer:** See `MySQL is throwing deadlock errors, what can I do?`_.
 
+
+Can I send some tasks to only some servers?
+--------------------------------------------
+
+As of now there is only one use-case that works like this, and that is
+tasks of type ``A`` can be sent to servers ``x`` and ``y``, while tasks
+of type ``B`` can be sent to server ``z``. One server can't handle more than
+one routing_key, but this is coming in a later release.
+
+Say you have two servers, ``x``, and ``y`` that handles regular tasks,
+and one server ``z``, that only handles feed related tasks, you can use this
+configuration:
+
+    * Servers ``x`` and ``y``: settings.py:
+
+    .. code-block:: python
+
+        AMQP_SERVER = "rabbit"
+        AMQP_PORT = 5678
+        AMQP_USER = "myapp"
+        AMQP_PASSWORD = "secret"
+        AMQP_VHOST = "myapp"
+
+        CELERY_AMQP_CONSUMER_QUEUE = "regular_tasks"
+        CELERY_AMQP_EXCHANGE = "tasks"
+        CELERY_AMQP_PUBLISHER_ROUTING_KEY = "task.regular"
+        CELERY_AMQP_CONSUMER_ROUTING_KEY = "task.#"
+        CELERY_AMQP_EXCHANGE_TYPE = "topic"
+
+    * Server ``z``: settings.py:
+
+    .. code-block:: python
+
+        AMQP_SERVER = "rabbit"
+        AMQP_PORT = 5678
+        AMQP_USER = "myapp"
+        AMQP_PASSWORD = "secret"
+        AMQP_VHOST = "myapp"
+        
+        CELERY_AMQP_CONSUMER_QUEUE = "feed_tasks"
+        CELERY_AMQP_EXCHANGE = "tasks"
+        CELERY_AMQP_PUBLISHER_ROUTING_KEY = "task.regular"
+        CELERY_AMQP_CONSUMER_ROUTING_KEY = "task.feed.#"
+        CELERY_AMQP_EXCHANGE_TYPE = "topic"
+
+Now to make a Task run on the ``z`` server you need to set its
+``routing_key`` attribute so it starts with the words ``"task.feed."``:
+
+.. code-block:: python
+
+    from feedaggregator.models import Feed
+    from celery.task import Task
+
+    class FeedImportTask(Task):
+        name = "import_feed"
+        routing_key = "task.feed.importer"
+
+        def run(self, feed_url):
+            # something importing the feed
+            Feed.objects.import_feed(feed_url)
+
+
+You can also override this using the ``routing_key`` argument to
+:func:`celery.task.apply_async`:
+
+    >>> from celery.task import apply_async
+    >>> from myapp.tasks import RefreshFeedTask
+    >>> apply_async(RefreshFeedTask, args=["http://cnn.com/rss"],
+    ...             routing_key="task.feed.importer")