Parcourir la source

Added example project using database/redis as the message queue.

Ask Solem il y a 15 ans
Parent
commit
b117046ad6

+ 110 - 0
examples/ghetto-queue/README.rst

@@ -0,0 +1,110 @@
+=======================================================
+ Example Celery project using a database message queue
+=======================================================
+
+Short instructions
+==================
+
+Quick rundown of the tutorial::
+
+1. Install the `ghettoq`_ plugin.
+
+    $ pip install ghettoq
+
+    $ celeryinit
+
+2. Open up two terminals, in the first you run:
+
+    $ celeryd --loglevel=INFO
+
+  In the other you run the test program:
+
+    $ python ./test.py
+
+Voila, you've executed some tasks!
+
+Instructions
+============
+
+This example uses the database as a message queue (commonly called a "ghetto
+queue"). Excellent for testing, but not very useful for production
+installations.
+
+To try it out you have to install the `GhettoQ`_ package first::
+
+    $ pip install ghettoq
+
+This package is an add on to `Carrot`_; the messaging abstraction celery
+uses, that enables the use of databases as message queues. Currently it
+supports `Redis`_ and relational databases via the Django ORM.
+
+.. _`ghettoq`: http://pypi.python.org/pypi/ghettoq
+.. _`Carrot`: http://pypi.python.org/pypi/carrot
+.. _`Redis`: http://code.google.com/p/redis/
+
+
+The provided ``celeryconfig.py`` configures the settings used to drive celery.
+
+Next we have to create the database tables by issuing the ``celeryinit``
+command::
+
+    $ celeryinit
+
+We're using SQLite3, so this creates a database file (``celery.db`` as
+specified in the config file). SQLite is great, but when used in combination
+with Django it doesn't handle concurrency well, to protect your program from
+lock problems, celeryd will only spawn one worker process. However -- with
+other database drivers you can specify as many worker processes as you want.
+
+
+With the setup done, let's run the worker::
+
+    $ celeryd --loglevel=INFO
+
+
+You should see the worker starting up. As it will continue running in
+the foreground, we have to open up another terminal to run our test program::
+
+    $ python test.py
+
+
+The test program simply runs the ``add`` task, which is a simple task adding
+numbers. You can also run the task manually if you want::
+
+    >>> from tasks import add
+    >>> result = add.delay(4, 4)
+    >>> result.wait()
+    8
+
+Using Redis instead
+===================
+
+To use redis instead you have to configure the following directives in 
+``celeryconfig.py``::
+
+    CARROT_BACKEND = "ghettoq.taproot.Redis"
+    BROKER_HOST = "localhost"
+    BROKER_PORT = 6379
+
+Modules
+=======
+
+    * celeryconfig.py
+
+        The celery configuration module.
+
+    * tasks.py
+
+        Tasks are defined in this module. This module is automatically
+        imported by the worker because it's listed in
+        celeryconfig's ``CELERY_IMPORTS`` directive.
+
+    * test.py
+
+        Simple test program running tasks.
+
+
+More information
+================
+
+http://celeryproject.org

+ 0 - 0
examples/ghetto-queue/__init__.py


+ 21 - 0
examples/ghetto-queue/celeryconfig.py

@@ -0,0 +1,21 @@
+import os
+import sys
+sys.path.insert(0, os.getcwd())
+
+DATABASE_ENGINE = "sqlite3"
+DATABASE_NAME = "celery.db"
+
+# or for Redis use ghettoq.taproot.Redis
+CARROT_BACKEND = "ghettoq.taproot.Database"
+
+# not needed for database.
+# BROKER_HOST = "localhost"
+# BROKER_USER = "guest"
+# BROKER_PASSWORD = "guest"
+# BROKER_VHOST = "/"
+
+# Need to add ghettoq when using the database backend, so
+# the database tables are created at syncdb.
+INSTALLED_APPS = ("celery", "ghettoq")
+
+CELERY_IMPORTS = ("tasks", )

+ 11 - 0
examples/ghetto-queue/tasks.py

@@ -0,0 +1,11 @@
+from celery.decorators import task
+
+
+@task()
+def add(x, y):
+    return x + y
+
+
+@task()
+def mul(x, y):
+    return x * y

+ 15 - 0
examples/ghetto-queue/test.py

@@ -0,0 +1,15 @@
+from tasks import add
+
+
+print(">>> from tasks import add")
+print(">>> add(4, 4)")
+res = add(4, 4)
+print(repr(res))
+
+print(">>> add.delay(4, 4)")
+res = add.delay(4, 4)
+print(repr(res))
+
+print(">>> add.delay(4, 4).wait()")
+res = add.delay(4, 4).wait()
+print(repr(res))