introduction.html 31 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471
  1. <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  2. "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
  3. <html xmlns="http://www.w3.org/1999/xhtml">
  4. <head>
  5. <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  6. <title>celery - Distributed Task Queue for Django/Python &mdash; Celery v0.7.0 (unstable) documentation</title>
  7. <link rel="stylesheet" href="static/nature.css" type="text/css" />
  8. <link rel="stylesheet" href="static/pygments.css" type="text/css" />
  9. <script type="text/javascript">
  10. var DOCUMENTATION_OPTIONS = {
  11. URL_ROOT: '',
  12. VERSION: '0.7.0 (unstable)',
  13. COLLAPSE_MODINDEX: false,
  14. FILE_SUFFIX: '.html',
  15. HAS_SOURCE: true
  16. };
  17. </script>
  18. <script type="text/javascript" src="static/jquery.js"></script>
  19. <script type="text/javascript" src="static/doctools.js"></script>
  20. <link rel="top" title="Celery v0.7.0 (unstable) documentation" href="index.html" />
  21. <link rel="next" title="Cookbook" href="cookbook/index.html" />
  22. <link rel="prev" title="Celery - Distributed Task Queue" href="index.html" />
  23. </head>
  24. <body>
  25. <div class="related">
  26. <h3>Navigation</h3>
  27. <ul>
  28. <li class="right" style="margin-right: 10px">
  29. <a href="genindex.html" title="General Index"
  30. accesskey="I">index</a></li>
  31. <li class="right" >
  32. <a href="modindex.html" title="Global Module Index"
  33. accesskey="M">modules</a> |</li>
  34. <li class="right" >
  35. <a href="cookbook/index.html" title="Cookbook"
  36. accesskey="N">next</a> |</li>
  37. <li class="right" >
  38. <a href="index.html" title="Celery - Distributed Task Queue"
  39. accesskey="P">previous</a> |</li>
  40. <li><a href="index.html">Celery v0.7.0 (unstable) documentation</a> &raquo;</li>
  41. </ul>
  42. </div>
  43. <div class="document">
  44. <div class="documentwrapper">
  45. <div class="bodywrapper">
  46. <div class="body">
  47. <div class="section" id="celery-distributed-task-queue-for-django-python">
  48. <h1>celery - Distributed Task Queue for Django/Python<a class="headerlink" href="#celery-distributed-task-queue-for-django-python" title="Permalink to this headline">¶</a></h1>
  49. <table class="docutils field-list" frame="void" rules="none">
  50. <col class="field-name" />
  51. <col class="field-body" />
  52. <tbody valign="top">
  53. <tr class="field"><th class="field-name">Version:</th><td class="field-body">0.7.0</td>
  54. </tr>
  55. </tbody>
  56. </table>
  57. <div class="section" id="introduction">
  58. <h2>Introduction<a class="headerlink" href="#introduction" title="Permalink to this headline">¶</a></h2>
  59. <p><strong>NOTE:</strong> See the FAQ for information about using celery outside of Django.</p>
  60. <p><tt class="docutils literal"><span class="pre">celery</span></tt> is a distributed task queue framework for Django/Python.</p>
  61. <p>It is used for executing tasks <em>asynchronously</em>, routed to one or more
  62. worker servers, running concurrently using multiprocessing.</p>
  63. <p>It is designed to solve certain problems related to running websites
  64. demanding high-availability and performance.</p>
  65. <p>It is perfect for filling caches, posting updates to twitter, mass
  66. downloading data like syndication feeds or web scraping. Use-cases are
  67. plentiful. Implementing these features asynchronously using <tt class="docutils literal"><span class="pre">celery</span></tt> is
  68. easy and fun, and the performance improvements can make it more than
  69. worthwhile.</p>
  70. </div>
  71. <div class="section" id="overview">
  72. <h2>Overview<a class="headerlink" href="#overview" title="Permalink to this headline">¶</a></h2>
  73. <p>This is a high level overview of the architecture.</p>
  74. <img alt="http://cloud.github.com/downloads/ask/celery/Celery-Overview-v4.jpg" src="http://cloud.github.com/downloads/ask/celery/Celery-Overview-v4.jpg" />
  75. <p>The broker is an AMQP server pushing tasks to the worker servers.
  76. A worker server is a networked machine running <tt class="docutils literal"><span class="pre">celeryd</span></tt>. This can be one or
  77. more machines, depending on the workload. See <a class="reference internal" href="#a-look-inside-the-worker">A look inside the worker</a> to
  78. see how the worker server works.</p>
  79. <p>The result of the task can be stored for later retrieval (called its
  80. &#8220;tombstone&#8221;).</p>
  81. </div>
  82. <div class="section" id="features">
  83. <h2>Features<a class="headerlink" href="#features" title="Permalink to this headline">¶</a></h2>
  84. <ul>
  85. <li><p class="first">Uses AMQP messaging (RabbitMQ, ZeroMQ, Qpid) to route tasks to the
  86. worker servers. Experimental support for STOMP (ActiveMQ) is also
  87. available.</p>
  88. </li>
  89. <li><p class="first">You can run as many worker servers as you want, and still
  90. be <em>guaranteed that the task is only executed once.</em></p>
  91. </li>
  92. <li><p class="first">Tasks are executed <em>concurrently</em> using the Python 2.6
  93. <tt class="docutils literal"><span class="pre">multiprocessing</span></tt> module (also available as a back-port
  94. to older python versions)</p>
  95. </li>
  96. <li><p class="first">Supports <em>periodic tasks</em>, which makes it a (better) replacement
  97. for cronjobs.</p>
  98. </li>
  99. <li><p class="first">When a task has been executed, the return value can be stored using
  100. either a MySQL/Oracle/PostgreSQL/SQLite database, Memcached,
  101. or Tokyo Tyrant back-end. For high-performance you can also use
  102. AMQP to publish results.</p>
  103. </li>
  104. <li><p class="first">If the task raises an exception, the exception instance is stored,
  105. instead of the return value.</p>
  106. </li>
  107. <li><p class="first">All tasks has a Universally Unique Identifier (UUID), which is the
  108. task id, used for querying task status and return values.</p>
  109. </li>
  110. <li><p class="first">Tasks can be retried if they fail, with a configurable maximum number
  111. of retries.</p>
  112. </li>
  113. <li><p class="first">Tasks can be configured to run at a specific time and date in the
  114. future (ETA) or you can set a countdown in seconds for when the
  115. task should be executed.</p>
  116. </li>
  117. <li><p class="first">Supports <em>task-sets</em>, which is a task consisting of several sub-tasks.
  118. You can find out how many, or if all of the sub-tasks has been executed.
  119. Excellent for progress-bar like functionality.</p>
  120. </li>
  121. <li><p class="first">Has a <tt class="docutils literal"><span class="pre">map</span></tt> like function that uses tasks, called <tt class="docutils literal"><span class="pre">dmap</span></tt>.</p>
  122. </li>
  123. <li><p class="first">However, you rarely want to wait for these results in a web-environment.
  124. You&#8217;d rather want to use Ajax to poll the task status, which is
  125. available from a URL like <tt class="docutils literal"><span class="pre">celery/&lt;task_id&gt;/status/</span></tt>. This view
  126. returns a JSON-serialized data structure containing the task status,
  127. and the return value if completed, or exception on failure.</p>
  128. </li>
  129. <li><p class="first">The worker can collect statistics, like, how many tasks has been
  130. executed by type, and the time it took to process them. Very useful
  131. for monitoring and profiling.</p>
  132. </li>
  133. <li><dl class="first docutils">
  134. <dt>Pool workers are supervised, so if for some reason a worker crashes</dt>
  135. <dd><p class="first last">it is automatically replaced by a new worker.</p>
  136. </dd>
  137. </dl>
  138. </li>
  139. <li><p class="first">Can be configured to send e-mails to the administrators when a task
  140. fails.</p>
  141. </li>
  142. </ul>
  143. </div>
  144. <div class="section" id="api-reference-documentation">
  145. <h2>API Reference Documentation<a class="headerlink" href="#api-reference-documentation" title="Permalink to this headline">¶</a></h2>
  146. <p>The <a class="reference external" href="http://ask.github.com/celery/">API Reference</a> is hosted at Github
  147. (<a class="reference external" href="http://ask.github.com/celery">http://ask.github.com/celery</a>)</p>
  148. </div>
  149. <div class="section" id="installation">
  150. <h2>Installation<a class="headerlink" href="#installation" title="Permalink to this headline">¶</a></h2>
  151. <p>You can install <tt class="docutils literal"><span class="pre">celery</span></tt> either via the Python Package Index (PyPI)
  152. or from source.</p>
  153. <p>To install using <tt class="docutils literal"><span class="pre">pip</span></tt>,:</p>
  154. <div class="highlight-python"><pre>$ pip install celery</pre>
  155. </div>
  156. <p>To install using <tt class="docutils literal"><span class="pre">easy_install</span></tt>,:</p>
  157. <div class="highlight-python"><pre>$ easy_install celery</pre>
  158. </div>
  159. <div class="section" id="downloading-and-installing-from-source">
  160. <h3>Downloading and installing from source<a class="headerlink" href="#downloading-and-installing-from-source" title="Permalink to this headline">¶</a></h3>
  161. <p>Download the latest version of <tt class="docutils literal"><span class="pre">celery</span></tt> from
  162. <a class="reference external" href="http://pypi.python.org/pypi/celery/">http://pypi.python.org/pypi/celery/</a></p>
  163. <p>You can install it by doing the following,:</p>
  164. <div class="highlight-python"><pre>$ tar xvfz celery-0.0.0.tar.gz
  165. $ cd celery-0.0.0
  166. $ python setup.py build
  167. # python setup.py install # as root</pre>
  168. </div>
  169. </div>
  170. <div class="section" id="using-the-development-version">
  171. <h3>Using the development version<a class="headerlink" href="#using-the-development-version" title="Permalink to this headline">¶</a></h3>
  172. <p>You can clone the repository by doing the following:</p>
  173. <div class="highlight-python"><pre>$ git clone git://github.com/ask/celery.git</pre>
  174. </div>
  175. </div>
  176. </div>
  177. <div class="section" id="usage">
  178. <h2>Usage<a class="headerlink" href="#usage" title="Permalink to this headline">¶</a></h2>
  179. <div class="section" id="installing-rabbitmq">
  180. <h3>Installing RabbitMQ<a class="headerlink" href="#installing-rabbitmq" title="Permalink to this headline">¶</a></h3>
  181. <p>See <a class="reference external" href="http://www.rabbitmq.com/install.html">Installing RabbitMQ</a> over at RabbitMQ&#8217;s website. For Mac OS X
  182. see <a class="reference external" href="http://playtype.net/past/2008/10/9/installing_rabbitmq_on_osx/">Installing RabbitMQ on OS X</a>.</p>
  183. </div>
  184. <div class="section" id="setting-up-rabbitmq">
  185. <h3>Setting up RabbitMQ<a class="headerlink" href="#setting-up-rabbitmq" title="Permalink to this headline">¶</a></h3>
  186. <p>To use celery we need to create a RabbitMQ user, a virtual host and
  187. allow that user access to that virtual host:</p>
  188. <div class="highlight-python"><pre>$ rabbitmqctl add_user myuser mypassword
  189. $ rabbitmqctl add_vhost myvhost</pre>
  190. </div>
  191. <p>From RabbitMQ version 1.6.0 and onward you have to use the new ACL features
  192. to allow access:</p>
  193. <div class="highlight-python"><pre>$ rabbitmqctl set_permissions -p myvhost myuser "" ".*" ".*"</pre>
  194. </div>
  195. <p>See the RabbitMQ <a class="reference external" href="http://www.rabbitmq.com/admin-guide.html">Admin Guide</a> for more information about <a class="reference external" href="http://www.rabbitmq.com/admin-guide.html#access-control">access control</a>.</p>
  196. <p>If you are still using version 1.5.0 or below, please use <tt class="docutils literal"><span class="pre">map_user_vhost</span></tt>:</p>
  197. <div class="highlight-python"><pre>$ rabbitmqctl map_user_vhost myuser myvhost</pre>
  198. </div>
  199. </div>
  200. <div class="section" id="configuring-your-django-project-to-use-celery">
  201. <h3>Configuring your Django project to use Celery<a class="headerlink" href="#configuring-your-django-project-to-use-celery" title="Permalink to this headline">¶</a></h3>
  202. <p>You only need three simple steps to use celery with your Django project.</p>
  203. <ol class="arabic">
  204. <li><p class="first">Add <tt class="docutils literal"><span class="pre">celery</span></tt> to <tt class="docutils literal"><span class="pre">INSTALLED_APPS</span></tt>.</p>
  205. </li>
  206. <li><p class="first">Create the celery database tables:</p>
  207. <div class="highlight-python"><pre>$ python manage.py syncdb</pre>
  208. </div>
  209. </li>
  210. <li><dl class="first docutils">
  211. <dt>Configure celery to use the AMQP user and virtual host we created</dt>
  212. <dd><p class="first">before, by adding the following to your <tt class="docutils literal"><span class="pre">settings.py</span></tt>:</p>
  213. <div class="last highlight-python"><div class="highlight"><pre><span class="n">AMQP_SERVER</span> <span class="o">=</span> <span class="s">&quot;localhost&quot;</span>
  214. <span class="n">AMQP_PORT</span> <span class="o">=</span> <span class="mf">5672</span>
  215. <span class="n">AMQP_USER</span> <span class="o">=</span> <span class="s">&quot;myuser&quot;</span>
  216. <span class="n">AMQP_PASSWORD</span> <span class="o">=</span> <span class="s">&quot;mypassword&quot;</span>
  217. <span class="n">AMQP_VHOST</span> <span class="o">=</span> <span class="s">&quot;myvhost&quot;</span>
  218. </pre></div>
  219. </div>
  220. </dd>
  221. </dl>
  222. </li>
  223. </ol>
  224. <p>That&#8217;s it.</p>
  225. <p>There are more options available, like how many processes you want to process
  226. work in parallel (the <tt class="docutils literal"><span class="pre">CELERY_CONCURRENCY</span></tt> setting), and the backend used
  227. for storing task statuses. But for now, this should do. For all of the options
  228. available, please consult the <a class="reference external" href="http://ask.github.com/celery/">API Reference</a></p>
  229. <p><strong>Note</strong>: If you&#8217;re using SQLite as the Django database back-end,
  230. <tt class="docutils literal"><span class="pre">celeryd</span></tt> will only be able to process one task at a time, this is
  231. because SQLite doesn&#8217;t allow concurrent writes.</p>
  232. </div>
  233. <div class="section" id="running-the-celery-worker-server">
  234. <h3>Running the celery worker server<a class="headerlink" href="#running-the-celery-worker-server" title="Permalink to this headline">¶</a></h3>
  235. <p>To test this we&#8217;ll be running the worker server in the foreground, so we can
  236. see what&#8217;s going on without consulting the logfile:</p>
  237. <div class="highlight-python"><pre>$ python manage.py celeryd</pre>
  238. </div>
  239. <p>However, in production you probably want to run the worker in the
  240. background, as a daemon:</p>
  241. <div class="highlight-python"><pre>$ python manage.py celeryd --detach</pre>
  242. </div>
  243. <p>For a complete listing of the command line arguments available, with a short
  244. description, you can use the help command:</p>
  245. <div class="highlight-python"><pre>$ python manage.py help celeryd</pre>
  246. </div>
  247. </div>
  248. <div class="section" id="defining-and-executing-tasks">
  249. <h3>Defining and executing tasks<a class="headerlink" href="#defining-and-executing-tasks" title="Permalink to this headline">¶</a></h3>
  250. <p><strong>Please note</strong> All of these tasks has to be stored in a real module, they can&#8217;t
  251. be defined in the python shell or ipython/bpython. This is because the celery
  252. worker server needs access to the task function to be able to run it.
  253. So while it looks like we use the python shell to define the tasks in these
  254. examples, you can&#8217;t do it this way. Put them in the <tt class="docutils literal"><span class="pre">tasks</span></tt> module of your
  255. Django application. The worker server will automatically load any <tt class="docutils literal"><span class="pre">tasks.py</span></tt>
  256. file for all of the applications listed in <tt class="docutils literal"><span class="pre">settings.INSTALLED_APPS</span></tt>.
  257. Executing tasks using <tt class="docutils literal"><span class="pre">delay</span></tt> and <tt class="docutils literal"><span class="pre">apply_async</span></tt> can be done from the
  258. python shell, but keep in mind that since arguments are pickled, you can&#8217;t
  259. use custom classes defined in the shell session.</p>
  260. <p>While you can use regular functions, the recommended way is to define
  261. a task class. This way you can cleanly upgrade the task to use the more
  262. advanced features of celery later.</p>
  263. <p>This is a task that basically does nothing but take some arguments,
  264. and return a value:</p>
  265. <div class="highlight-python"><div class="highlight"><pre><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">celery.task</span> <span class="kn">import</span> <span class="n">Task</span>
  266. <span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">celery.registry</span> <span class="kn">import</span> <span class="n">tasks</span>
  267. <span class="gp">&gt;&gt;&gt; </span><span class="k">class</span> <span class="nc">MyTask</span><span class="p">(</span><span class="n">Task</span><span class="p">):</span>
  268. <span class="gp">... </span> <span class="k">def</span> <span class="nf">run</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">some_arg</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
  269. <span class="gp">... </span> <span class="n">logger</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_logger</span><span class="p">(</span><span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
  270. <span class="gp">... </span> <span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s">&quot;Did something: </span><span class="si">%s</span><span class="s">&quot;</span> <span class="o">%</span> <span class="n">some_arg</span><span class="p">)</span>
  271. <span class="gp">... </span> <span class="k">return</span> <span class="mf">42</span>
  272. <span class="gp">&gt;&gt;&gt; </span><span class="n">tasks</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">MyTask</span><span class="p">)</span>
  273. </pre></div>
  274. </div>
  275. <p>Now if we want to execute this task, we can use the <tt class="docutils literal"><span class="pre">delay</span></tt> method of the
  276. task class (this is a handy shortcut to the <tt class="docutils literal"><span class="pre">apply_async</span></tt> method which gives
  277. you greater control of the task execution).</p>
  278. <div class="highlight-python"><div class="highlight"><pre><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">myapp.tasks</span> <span class="kn">import</span> <span class="n">MyTask</span>
  279. <span class="gp">&gt;&gt;&gt; </span><span class="n">MyTask</span><span class="o">.</span><span class="n">delay</span><span class="p">(</span><span class="n">some_arg</span><span class="o">=</span><span class="s">&quot;foo&quot;</span><span class="p">)</span>
  280. </pre></div>
  281. </div>
  282. <p>At this point, the task has been sent to the message broker. The message
  283. broker will hold on to the task until a celery worker server has successfully
  284. picked it up.</p>
  285. <p><em>Note</em> If everything is just hanging when you execute <tt class="docutils literal"><span class="pre">delay</span></tt>, please check
  286. that RabbitMQ is running, and that the user/password has access to the virtual
  287. host you configured earlier.</p>
  288. <p>Right now we have to check the celery worker logfiles to know what happened with
  289. the task. This is because we didn&#8217;t keep the <tt class="docutils literal"><span class="pre">AsyncResult</span></tt> object returned
  290. by <tt class="docutils literal"><span class="pre">delay</span></tt>.</p>
  291. <p>The <tt class="docutils literal"><span class="pre">AsyncResult</span></tt> lets us find the state of the task, wait for the task to
  292. finish and get its return value (or exception if the task failed).</p>
  293. <p>So, let&#8217;s execute the task again, but this time we&#8217;ll keep track of the task:</p>
  294. <div class="highlight-python"><div class="highlight"><pre><span class="gp">&gt;&gt;&gt; </span><span class="n">result</span> <span class="o">=</span> <span class="n">MyTask</span><span class="o">.</span><span class="n">delay</span><span class="p">(</span><span class="s">&quot;do_something&quot;</span><span class="p">,</span> <span class="n">some_arg</span><span class="o">=</span><span class="s">&quot;foo bar baz&quot;</span><span class="p">)</span>
  295. <span class="gp">&gt;&gt;&gt; </span><span class="n">result</span><span class="o">.</span><span class="n">ready</span><span class="p">()</span> <span class="c"># returns True if the task has finished processing.</span>
  296. <span class="go">False</span>
  297. <span class="gp">&gt;&gt;&gt; </span><span class="n">result</span><span class="o">.</span><span class="n">result</span> <span class="c"># task is not ready, so no return value yet.</span>
  298. <span class="go">None</span>
  299. <span class="gp">&gt;&gt;&gt; </span><span class="n">result</span><span class="o">.</span><span class="n">get</span><span class="p">()</span> <span class="c"># Waits until the task is done and return the retval.</span>
  300. <span class="go">42</span>
  301. <span class="gp">&gt;&gt;&gt; </span><span class="n">result</span><span class="o">.</span><span class="n">result</span>
  302. <span class="go">42</span>
  303. <span class="gp">&gt;&gt;&gt; </span><span class="n">result</span><span class="o">.</span><span class="n">successful</span><span class="p">()</span> <span class="c"># returns True if the task didn&#39;t end in failure.</span>
  304. <span class="go">True</span>
  305. </pre></div>
  306. </div>
  307. <p>If the task raises an exception, the <tt class="docutils literal"><span class="pre">result.success()</span></tt> will be <tt class="xref docutils literal"><span class="pre">False</span></tt>,
  308. and <tt class="docutils literal"><span class="pre">result.result</span></tt> will contain the exception instance raised.</p>
  309. </div>
  310. <div class="section" id="auto-discovery-of-tasks">
  311. <h3>Auto-discovery of tasks<a class="headerlink" href="#auto-discovery-of-tasks" title="Permalink to this headline">¶</a></h3>
  312. <p><tt class="docutils literal"><span class="pre">celery</span></tt> has an auto-discovery feature like the Django Admin, that
  313. automatically loads any <tt class="docutils literal"><span class="pre">tasks.py</span></tt> module in the applications listed
  314. in <tt class="docutils literal"><span class="pre">settings.INSTALLED_APPS</span></tt>. This autodiscovery is used by the celery
  315. worker to find registered tasks for your Django project.</p>
  316. </div>
  317. <div class="section" id="periodic-tasks">
  318. <h3>Periodic Tasks<a class="headerlink" href="#periodic-tasks" title="Permalink to this headline">¶</a></h3>
  319. <p>Periodic tasks are tasks that are run every <tt class="docutils literal"><span class="pre">n</span></tt> seconds.
  320. Here&#8217;s an example of a periodic task:</p>
  321. <div class="highlight-python"><div class="highlight"><pre><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">celery.task</span> <span class="kn">import</span> <span class="n">PeriodicTask</span>
  322. <span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">celery.registry</span> <span class="kn">import</span> <span class="n">tasks</span>
  323. <span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">timedelta</span>
  324. <span class="gp">&gt;&gt;&gt; </span><span class="k">class</span> <span class="nc">MyPeriodicTask</span><span class="p">(</span><span class="n">PeriodicTask</span><span class="p">):</span>
  325. <span class="gp">... </span> <span class="n">run_every</span> <span class="o">=</span> <span class="n">timedelta</span><span class="p">(</span><span class="n">seconds</span><span class="o">=</span><span class="mf">30</span><span class="p">)</span>
  326. <span class="gp">...</span>
  327. <span class="gp">... </span> <span class="k">def</span> <span class="nf">run</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
  328. <span class="gp">... </span> <span class="n">logger</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_logger</span><span class="p">(</span><span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
  329. <span class="gp">... </span> <span class="n">logger</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s">&quot;Running periodic task!&quot;</span><span class="p">)</span>
  330. <span class="gp">...</span>
  331. <span class="gp">&gt;&gt;&gt; </span><span class="n">tasks</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">MyPeriodicTask</span><span class="p">)</span>
  332. </pre></div>
  333. </div>
  334. <p><strong>Note:</strong> Periodic tasks does not support arguments, as this doesn&#8217;t
  335. really make sense.</p>
  336. </div>
  337. </div>
  338. <div class="section" id="a-look-inside-the-worker">
  339. <h2>A look inside the worker<a class="headerlink" href="#a-look-inside-the-worker" title="Permalink to this headline">¶</a></h2>
  340. <img alt="http://cloud.github.com/downloads/ask/celery/InsideTheWorker-v2.jpg" src="http://cloud.github.com/downloads/ask/celery/InsideTheWorker-v2.jpg" />
  341. </div>
  342. <div class="section" id="getting-help">
  343. <h2>Getting Help<a class="headerlink" href="#getting-help" title="Permalink to this headline">¶</a></h2>
  344. <div class="section" id="mailing-list">
  345. <h3>Mailing list<a class="headerlink" href="#mailing-list" title="Permalink to this headline">¶</a></h3>
  346. <p>For discussions about the usage, development, and future of celery,
  347. please join the <a class="reference external" href="http://groups.google.com/group/celery-users/">celery-users</a> mailing list.</p>
  348. </div>
  349. <div class="section" id="irc">
  350. <h3>IRC<a class="headerlink" href="#irc" title="Permalink to this headline">¶</a></h3>
  351. <p>Come chat with us on IRC. The <a class="reference external" href="irc://irc.freenode.net/celery">#celery</a> channel is located at the <a class="reference external" href="http://freenode.net">Freenode</a>
  352. network.</p>
  353. </div>
  354. </div>
  355. <div class="section" id="bug-tracker">
  356. <h2>Bug tracker<a class="headerlink" href="#bug-tracker" title="Permalink to this headline">¶</a></h2>
  357. <p>If you have any suggestions, bug reports or annoyances please report them
  358. to our issue tracker at <a class="reference external" href="http://github.com/ask/celery/issues/">http://github.com/ask/celery/issues/</a></p>
  359. </div>
  360. <div class="section" id="contributing">
  361. <h2>Contributing<a class="headerlink" href="#contributing" title="Permalink to this headline">¶</a></h2>
  362. <p>Development of <tt class="docutils literal"><span class="pre">celery</span></tt> happens at Github: <a class="reference external" href="http://github.com/ask/celery">http://github.com/ask/celery</a></p>
  363. <p>You are highly encouraged to participate in the development
  364. of <tt class="docutils literal"><span class="pre">celery</span></tt>. If you don&#8217;t like Github (for some reason) you&#8217;re welcome
  365. to send regular patches.</p>
  366. </div>
  367. <div class="section" id="license">
  368. <h2>License<a class="headerlink" href="#license" title="Permalink to this headline">¶</a></h2>
  369. <p>This software is licensed under the <tt class="docutils literal"><span class="pre">New</span> <span class="pre">BSD</span> <span class="pre">License</span></tt>. See the <tt class="docutils literal"><span class="pre">LICENSE</span></tt>
  370. file in the top distribution directory for the full license text.</p>
  371. </div>
  372. </div>
  373. </div>
  374. </div>
  375. </div>
  376. <div class="sphinxsidebar">
  377. <div class="sphinxsidebarwrapper">
  378. <h3><a href="index.html">Table Of Contents</a></h3>
  379. <ul>
  380. <li><a class="reference external" href="">celery - Distributed Task Queue for Django/Python</a><ul>
  381. <li><a class="reference external" href="#introduction">Introduction</a></li>
  382. <li><a class="reference external" href="#overview">Overview</a></li>
  383. <li><a class="reference external" href="#features">Features</a></li>
  384. <li><a class="reference external" href="#api-reference-documentation">API Reference Documentation</a></li>
  385. <li><a class="reference external" href="#installation">Installation</a><ul>
  386. <li><a class="reference external" href="#downloading-and-installing-from-source">Downloading and installing from source</a></li>
  387. <li><a class="reference external" href="#using-the-development-version">Using the development version</a></li>
  388. </ul>
  389. </li>
  390. <li><a class="reference external" href="#usage">Usage</a><ul>
  391. <li><a class="reference external" href="#installing-rabbitmq">Installing RabbitMQ</a></li>
  392. <li><a class="reference external" href="#setting-up-rabbitmq">Setting up RabbitMQ</a></li>
  393. <li><a class="reference external" href="#configuring-your-django-project-to-use-celery">Configuring your Django project to use Celery</a></li>
  394. <li><a class="reference external" href="#running-the-celery-worker-server">Running the celery worker server</a></li>
  395. <li><a class="reference external" href="#defining-and-executing-tasks">Defining and executing tasks</a></li>
  396. <li><a class="reference external" href="#auto-discovery-of-tasks">Auto-discovery of tasks</a></li>
  397. <li><a class="reference external" href="#periodic-tasks">Periodic Tasks</a></li>
  398. </ul>
  399. </li>
  400. <li><a class="reference external" href="#a-look-inside-the-worker">A look inside the worker</a></li>
  401. <li><a class="reference external" href="#getting-help">Getting Help</a><ul>
  402. <li><a class="reference external" href="#mailing-list">Mailing list</a></li>
  403. <li><a class="reference external" href="#irc">IRC</a></li>
  404. </ul>
  405. </li>
  406. <li><a class="reference external" href="#bug-tracker">Bug tracker</a></li>
  407. <li><a class="reference external" href="#contributing">Contributing</a></li>
  408. <li><a class="reference external" href="#license">License</a></li>
  409. </ul>
  410. </li>
  411. </ul>
  412. <h4>Previous topic</h4>
  413. <p class="topless"><a href="index.html"
  414. title="previous chapter">Celery - Distributed Task Queue</a></p>
  415. <h4>Next topic</h4>
  416. <p class="topless"><a href="cookbook/index.html"
  417. title="next chapter">Cookbook</a></p>
  418. <h3>This Page</h3>
  419. <ul class="this-page-menu">
  420. <li><a href="sources/introduction.txt"
  421. rel="nofollow">Show Source</a></li>
  422. </ul>
  423. <div id="searchbox" style="display: none">
  424. <h3>Quick search</h3>
  425. <form class="search" action="search.html" method="get">
  426. <input type="text" name="q" size="18" />
  427. <input type="submit" value="Go" />
  428. <input type="hidden" name="check_keywords" value="yes" />
  429. <input type="hidden" name="area" value="default" />
  430. </form>
  431. <p class="searchtip" style="font-size: 90%">
  432. Enter search terms or a module, class or function name.
  433. </p>
  434. </div>
  435. <script type="text/javascript">$('#searchbox').show(0);</script>
  436. </div>
  437. </div>
  438. <div class="clearer"></div>
  439. </div>
  440. <div class="related">
  441. <h3>Navigation</h3>
  442. <ul>
  443. <li class="right" style="margin-right: 10px">
  444. <a href="genindex.html" title="General Index"
  445. >index</a></li>
  446. <li class="right" >
  447. <a href="modindex.html" title="Global Module Index"
  448. >modules</a> |</li>
  449. <li class="right" >
  450. <a href="cookbook/index.html" title="Cookbook"
  451. >next</a> |</li>
  452. <li class="right" >
  453. <a href="index.html" title="Celery - Distributed Task Queue"
  454. >previous</a> |</li>
  455. <li><a href="index.html">Celery v0.7.0 (unstable) documentation</a> &raquo;</li>
  456. </ul>
  457. </div>
  458. <div class="footer">
  459. &copy; Copyright 2009, Ask Solem.
  460. Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 0.6.2.
  461. </div>
  462. </body>
  463. </html>