PyPANTS PYPANTS!!

AUTHORS

@@ -1,6 +1,2 @@
 Ask Solem <askh@opera.com>
 Grégoire Cachet <gregoire@audacy.fr>
-
-Thanks to Rune Halvorsen <runeh@opera.com> for the name.
-Thanks to Anton Tsigularov <antont@opera.com> for the previous name (crunchy)
-which we had to abandon because of an existing project with that name.

Changelog → CHANGELOG

@@ -53,6 +53,10 @@ arguments, so be sure to flush your task queue before you upgrade.
 	* Task errors are now logged using loglevel ``ERROR`` instead of ``INFO``,
 		and backtraces are dumped. Thanks to Grégoire Cachet.
 
+	* Make every new worker process re-establish it's Django DB connection,
+		this solving the "MySQL connection died?" exceptions.
+		Thanks to Vitaly Babiy and Jirka Vejrazka.
+
 	* **IMOPORTANT** Now using pickle to encode task arguments. This means you
 		now can pass complex python objects to tasks as arguments.
 

FAQ

@@ -0,0 +1,124 @@
+============================
+ Frequently Asked Questions
+============================
+
+Questions
+=========
+
+MySQL is throwing deadlock errors, what can I do?
+-------------------------------------------------
+
+**Answer:** MySQL has default isolation level set to ``REPEATABLE-READ``,
+if you don't really need that, set it to ``READ-COMMITTED``.
+You can do that by adding the following to your ``my.cnf``::
+
+    [mysqld]
+    transaction-isolation = READ-COMMITTED
+
+For more information about InnoDBs transaction model see `MySQL - The InnoDB
+Transaction Model and Locking`_ in the MySQL user manual.
+
+(Thanks to Honza Kral and Anton Tsigularov for this solution)
+
+.. _`MySQL - The InnoDB Transaction Model and Locking`: http://dev.mysql.com/doc/refman/5.1/en/innodb-transaction-model.html
+
+celeryd is not doing anything, just hanging
+--------------------------------------------
+
+**Answer:** See `MySQL is throwing deadlock errors, what can I do?`_.
+
+I'm having ``IntegrityError: Duplicate Key`` errors. Why?
+----------------------------------------------------------
+
+**Answer:** See `MySQL is throwing deadlock errors, what can I do?`_.
+Thanks to howsthedotcom.
+
+Why won't my Task run?
+----------------------
+
+**Answer:** Did you register the task in the applications ``tasks.py`` module?
+(or in some other module Django loads by default, like ``models.py``?).
+Also there might be syntax errors preventing the tasks module being imported.
+
+You can find out if the celery daemon is able to run the task by executing the
+task manually:
+
+    >>> from myapp.tasks import MyPeriodicTask
+    >>> MyPeriodicTask.delay()
+
+Watch celery daemons logfile (or output if not running as a daemon), to see
+if it's able to find the task, or if some other error is happening.
+
+Why won't my Periodic Task run?
+-------------------------------
+
+See `Why won't my Task run?`_.
+
+Can I send some tasks to only some servers?
+--------------------------------------------
+
+As of now there is only one use-case that works like this, and that is
+tasks of type ``A`` can be sent to servers ``x`` and ``y``, while tasks
+of type ``B`` can be sent to server ``z``. One server can't handle more than
+one routing_key, but this is coming in a later release.
+
+Say you have two servers, ``x``, and ``y`` that handles regular tasks,
+and one server ``z``, that only handles feed related tasks, you can use this
+configuration:
+
+    * Servers ``x`` and ``y``: settings.py:
+
+    .. code-block:: python
+
+        AMQP_SERVER = "rabbit"
+        AMQP_PORT = 5678
+        AMQP_USER = "myapp"
+        AMQP_PASSWORD = "secret"
+        AMQP_VHOST = "myapp"
+
+        CELERY_AMQP_CONSUMER_QUEUE = "regular_tasks"
+        CELERY_AMQP_EXCHANGE = "tasks"
+        CELERY_AMQP_PUBLISHER_ROUTING_KEY = "task.regular"
+        CELERY_AMQP_CONSUMER_ROUTING_KEY = "task.#"
+        CELERY_AMQP_EXCHANGE_TYPE = "topic"
+
+    * Server ``z``: settings.py:
+
+    .. code-block:: python
+
+        AMQP_SERVER = "rabbit"
+        AMQP_PORT = 5678
+        AMQP_USER = "myapp"
+        AMQP_PASSWORD = "secret"
+        AMQP_VHOST = "myapp"
+        
+        CELERY_AMQP_CONSUMER_QUEUE = "feed_tasks"
+        CELERY_AMQP_EXCHANGE = "tasks"
+        CELERY_AMQP_PUBLISHER_ROUTING_KEY = "task.regular"
+        CELERY_AMQP_CONSUMER_ROUTING_KEY = "task.feed.#"
+        CELERY_AMQP_EXCHANGE_TYPE = "topic"
+
+Now to make a Task run on the ``z`` server you need to set its
+``routing_key`` attribute so it starts with the words ``"task.feed."``:
+
+.. code-block:: python
+
+    from feedaggregator.models import Feed
+    from celery.task import Task
+
+    class FeedImportTask(Task):
+        name = "import_feed"
+        routing_key = "task.feed.importer"
+
+        def run(self, feed_url):
+            # something importing the feed
+            Feed.objects.import_feed(feed_url)
+
+
+You can also override this using the ``routing_key`` argument to
+:func:`celery.task.apply_async`:
+
+    >>> from celery.task import apply_async
+    >>> from myapp.tasks import RefreshFeedTask
+    >>> apply_async(RefreshFeedTask, args=["http://cnn.com/rss"],
+    ...             routing_key="task.feed.importer")

MANIFEST.in

@@ -1,9 +1,10 @@
 include AUTHORS
-include README.rst
+include CHANGELOG
+include README
 include MANIFEST.in
-include LICENSE
+include LICENCE
 include TODO
-include Changelog
+include THANKS
 recursive-include celery *
 recursive-include docs *
 recursive-include testproj *

THANKS

@@ -0,0 +1,4 @@
+Thanks to Rune Halvorsen <runeh@opera.com> for the name.
+Thanks to Anton Tsigularov <antont@opera.com> for the previous name (crunchy)
+    which we had to abandon because of an existing project with that name.
+Thanks to Vitaly Babiy for bug reports.

TODO

@@ -1,7 +1,2 @@
-* Store result/errors in the TaskMeta store.
-* Catch exceptions from pure function tasks.
-* Need a tutorial
-* Need more documentation
-* Need a way to run without a AMQP server (see the ``backends`` branch of
-  carrot, it should make it able to run just using the python ``Queue``
-  module.)
+Please see our Issue Tracker at GitHub:
+    http://github.com/ask/celery/issues

celery/managers.py

@@ -66,7 +66,8 @@ class PeriodicTaskManager(models.Manager):
         for task_name, task in periodic_tasks.items():
             task_meta, created = self.get_or_create(name=task_name)
             # task_run.every must be a timedelta object.
-            run_every_drifted = task.run_every + SERVER_DRIFT
+            run_every_drifted = task.run_every + \
+                                    timedelta(seconds=SERVER_DRIFT)
             run_at = task_meta.last_run_at + task.run_every
             if datetime.now() > run_at:
                 waiting.append(task_meta)

docs/changelog.rst

@@ -1 +1 @@
-../Changelog
+../CHANGELOG

docs/faq.rst

@@ -1,124 +0,0 @@
-============================
- Frequently Asked Questions
-============================
-
-Questions
-=========
-
-MySQL is throwing deadlock errors, what can I do?
--------------------------------------------------
-
-**Answer:** MySQL has default isolation level set to ``REPEATABLE-READ``,
-if you don't really need that, set it to ``READ-COMMITTED``.
-You can do that by adding the following to your ``my.cnf``::
-
-    [mysqld]
-    transaction-isolation = READ-COMMITTED
-
-For more information about InnoDBs transaction model see `MySQL - The InnoDB
-Transaction Model and Locking`_ in the MySQL user manual.
-
-(Thanks to Honza Kral and Anton Tsigularov for this solution)
-
-.. _`MySQL - The InnoDB Transaction Model and Locking`: http://dev.mysql.com/doc/refman/5.1/en/innodb-transaction-model.html
-
-celeryd is not doing anything, just hanging
---------------------------------------------
-
-**Answer:** See `MySQL is throwing deadlock errors, what can I do?`_.
-
-I'm having ``IntegrityError: Duplicate Key`` errors. Why?
-----------------------------------------------------------
-
-**Answer:** See `MySQL is throwing deadlock errors, what can I do?`_.
-Thanks to howsthedotcom.
-
-Why won't my Task run?
-----------------------
-
-**Answer:** Did you register the task in the applications ``tasks.py`` module?
-(or in some other module Django loads by default, like ``models.py``?).
-Also there might be syntax errors preventing the tasks module being imported.
-
-You can find out if the celery daemon is able to run the task by executing the
-task manually:
-
-    >>> from myapp.tasks import MyPeriodicTask
-    >>> MyPeriodicTask.delay()
-
-Watch celery daemons logfile (or output if not running as a daemon), to see
-if it's able to find the task, or if some other error is happening.
-
-Why won't my Periodic Task run?
--------------------------------
-
-See `Why won't my Task run?`_.
-
-Can I send some tasks to only some servers?
---------------------------------------------
-
-As of now there is only one use-case that works like this, and that is
-tasks of type ``A`` can be sent to servers ``x`` and ``y``, while tasks
-of type ``B`` can be sent to server ``z``. One server can't handle more than
-one routing_key, but this is coming in a later release.
-
-Say you have two servers, ``x``, and ``y`` that handles regular tasks,
-and one server ``z``, that only handles feed related tasks, you can use this
-configuration:
-
-    * Servers ``x`` and ``y``: settings.py:
-
-    .. code-block:: python
-
-        AMQP_SERVER = "rabbit"
-        AMQP_PORT = 5678
-        AMQP_USER = "myapp"
-        AMQP_PASSWORD = "secret"
-        AMQP_VHOST = "myapp"
-
-        CELERY_AMQP_CONSUMER_QUEUE = "regular_tasks"
-        CELERY_AMQP_EXCHANGE = "tasks"
-        CELERY_AMQP_PUBLISHER_ROUTING_KEY = "task.regular"
-        CELERY_AMQP_CONSUMER_ROUTING_KEY = "task.#"
-        CELERY_AMQP_EXCHANGE_TYPE = "topic"
-
-    * Server ``z``: settings.py:
-
-    .. code-block:: python
-
-        AMQP_SERVER = "rabbit"
-        AMQP_PORT = 5678
-        AMQP_USER = "myapp"
-        AMQP_PASSWORD = "secret"
-        AMQP_VHOST = "myapp"
-        
-        CELERY_AMQP_CONSUMER_QUEUE = "feed_tasks"
-        CELERY_AMQP_EXCHANGE = "tasks"
-        CELERY_AMQP_PUBLISHER_ROUTING_KEY = "task.regular"
-        CELERY_AMQP_CONSUMER_ROUTING_KEY = "task.feed.#"
-        CELERY_AMQP_EXCHANGE_TYPE = "topic"
-
-Now to make a Task run on the ``z`` server you need to set its
-``routing_key`` attribute so it starts with the words ``"task.feed."``:
-
-.. code-block:: python
-
-    from feedaggregator.models import Feed
-    from celery.task import Task
-
-    class FeedImportTask(Task):
-        name = "import_feed"
-        routing_key = "task.feed.importer"
-
-        def run(self, feed_url):
-            # something importing the feed
-            Feed.objects.import_feed(feed_url)
-
-
-You can also override this using the ``routing_key`` argument to
-:func:`celery.task.apply_async`:
-
-    >>> from celery.task import apply_async
-    >>> from myapp.tasks import RefreshFeedTask
-    >>> apply_async(RefreshFeedTask, args=["http://cnn.com/rss"],
-    ...             routing_key="task.feed.importer")

docs/faq.rst

@@ -0,0 +1 @@
+../FAQ

docs/introduction.rst

@@ -1 +1 @@
-../README.rst
+../README

setup.py

@@ -13,6 +13,7 @@ except ImportError:
 
 import celery
 
+
 class RunTests(Command):
     description = "Run the django test suite from the testproj dir."
 
@@ -45,10 +46,10 @@ py_major_version = py_version_info[0]
 py_minor_version = py_version_info[1]
 
 if (py_major_version == 2 and py_minor_version <=5) or py_major_version < 2:
-    install_requires.append("multiprocessing")    
+    install_requires.append("multiprocessing")
+
+long_description = codecs.open("README", "r", "utf-8").read()
 
-long_description = codecs.open("README.rst", "r", "utf-8").read()
-                   
 
 setup(
     name='celery',