faq.html 18 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265
  1. <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  2. "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
  3. <html xmlns="http://www.w3.org/1999/xhtml">
  4. <head>
  5. <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  6. <title>Frequently Asked Questions &mdash; Celery v0.4.2 (stable) documentation</title>
  7. <link rel="stylesheet" href="static/nature.css" type="text/css" />
  8. <link rel="stylesheet" href="static/pygments.css" type="text/css" />
  9. <script type="text/javascript">
  10. var DOCUMENTATION_OPTIONS = {
  11. URL_ROOT: '',
  12. VERSION: '0.4.2 (stable)',
  13. COLLAPSE_MODINDEX: false,
  14. FILE_SUFFIX: '.html',
  15. HAS_SOURCE: true
  16. };
  17. </script>
  18. <script type="text/javascript" src="static/jquery.js"></script>
  19. <script type="text/javascript" src="static/doctools.js"></script>
  20. <link rel="top" title="Celery v0.4.2 (stable) documentation" href="index.html" />
  21. <link rel="next" title="Module API Reference" href="reference/index.html" />
  22. <link rel="prev" title="Tutorial: Creating a click counter using carrot and celery" href="tutorials/clickcounter.html" />
  23. </head>
  24. <body>
  25. <div class="related">
  26. <h3>Navigation</h3>
  27. <ul>
  28. <li class="right" style="margin-right: 10px">
  29. <a href="genindex.html" title="General Index"
  30. accesskey="I">index</a></li>
  31. <li class="right" >
  32. <a href="modindex.html" title="Global Module Index"
  33. accesskey="M">modules</a> |</li>
  34. <li class="right" >
  35. <a href="reference/index.html" title="Module API Reference"
  36. accesskey="N">next</a> |</li>
  37. <li class="right" >
  38. <a href="tutorials/clickcounter.html" title="Tutorial: Creating a click counter using carrot and celery"
  39. accesskey="P">previous</a> |</li>
  40. <li><a href="index.html">Celery v0.4.2 (stable) documentation</a> &raquo;</li>
  41. </ul>
  42. </div>
  43. <div class="document">
  44. <div class="documentwrapper">
  45. <div class="bodywrapper">
  46. <div class="body">
  47. <div class="section" id="frequently-asked-questions">
  48. <h1>Frequently Asked Questions<a class="headerlink" href="#frequently-asked-questions" title="Permalink to this headline">¶</a></h1>
  49. <div class="section" id="questions">
  50. <h2>Questions<a class="headerlink" href="#questions" title="Permalink to this headline">¶</a></h2>
  51. <div class="section" id="mysql-is-throwing-deadlock-errors-what-can-i-do">
  52. <h3>MySQL is throwing deadlock errors, what can I do?<a class="headerlink" href="#mysql-is-throwing-deadlock-errors-what-can-i-do" title="Permalink to this headline">¶</a></h3>
  53. <p><strong>Answer:</strong> MySQL has default isolation level set to <tt class="docutils literal"><span class="pre">REPEATABLE-READ</span></tt>,
  54. if you don&#8217;t really need that, set it to <tt class="docutils literal"><span class="pre">READ-COMMITTED</span></tt>.
  55. You can do that by adding the following to your <tt class="docutils literal"><span class="pre">my.cnf</span></tt>:</p>
  56. <div class="highlight-python"><div class="highlight"><pre><span class="p">[</span><span class="n">mysqld</span><span class="p">]</span>
  57. <span class="n">transaction</span><span class="o">-</span><span class="n">isolation</span> <span class="o">=</span> <span class="n">READ</span><span class="o">-</span><span class="n">COMMITTED</span>
  58. </pre></div>
  59. </div>
  60. <p>For more information about InnoDBs transaction model see <a class="reference external" href="http://dev.mysql.com/doc/refman/5.1/en/innodb-transaction-model.html">MySQL - The InnoDB
  61. Transaction Model and Locking</a> in the MySQL user manual.</p>
  62. <p>(Thanks to Honza Kral and Anton Tsigularov for this solution)</p>
  63. </div>
  64. <div class="section" id="celeryd-is-not-doing-anything-just-hanging">
  65. <h3>celeryd is not doing anything, just hanging<a class="headerlink" href="#celeryd-is-not-doing-anything-just-hanging" title="Permalink to this headline">¶</a></h3>
  66. <p><strong>Answer:</strong> See <a class="reference internal" href="#mysql-is-throwing-deadlock-errors-what-can-i-do">MySQL is throwing deadlock errors, what can I do?</a>.</p>
  67. </div>
  68. <div class="section" id="i-m-having-integrityerror-duplicate-key-errors-why">
  69. <h3>I&#8217;m having <tt class="docutils literal"><span class="pre">IntegrityError:</span> <span class="pre">Duplicate</span> <span class="pre">Key</span></tt> errors. Why?<a class="headerlink" href="#i-m-having-integrityerror-duplicate-key-errors-why" title="Permalink to this headline">¶</a></h3>
  70. <p><strong>Answer:</strong> See <a class="reference internal" href="#mysql-is-throwing-deadlock-errors-what-can-i-do">MySQL is throwing deadlock errors, what can I do?</a>.
  71. Thanks to howsthedotcom.</p>
  72. </div>
  73. <div class="section" id="why-won-t-my-task-run">
  74. <h3>Why won&#8217;t my Task run?<a class="headerlink" href="#why-won-t-my-task-run" title="Permalink to this headline">¶</a></h3>
  75. <p><strong>Answer:</strong> Did you register the task in the applications <tt class="docutils literal"><span class="pre">tasks.py</span></tt> module?
  76. (or in some other module Django loads by default, like <tt class="docutils literal"><span class="pre">models.py</span></tt>?).
  77. Also there might be syntax errors preventing the tasks module being imported.</p>
  78. <p>You can find out if the celery daemon is able to run the task by executing the
  79. task manually:</p>
  80. <div class="highlight-python"><div class="highlight"><pre><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">myapp.tasks</span> <span class="kn">import</span> <span class="n">MyPeriodicTask</span>
  81. <span class="gp">&gt;&gt;&gt; </span><span class="n">MyPeriodicTask</span><span class="o">.</span><span class="n">delay</span><span class="p">()</span>
  82. </pre></div>
  83. </div>
  84. <p>Watch celery daemons logfile (or output if not running as a daemon), to see
  85. if it&#8217;s able to find the task, or if some other error is happening.</p>
  86. </div>
  87. <div class="section" id="why-won-t-my-periodic-task-run">
  88. <h3>Why won&#8217;t my Periodic Task run?<a class="headerlink" href="#why-won-t-my-periodic-task-run" title="Permalink to this headline">¶</a></h3>
  89. <p><strong>Answer:</strong> See <a class="reference internal" href="#why-won-t-my-task-run">Why won&#8217;t my Task run?</a>.</p>
  90. </div>
  91. <div class="section" id="how-do-i-discard-all-waiting-tasks">
  92. <h3>How do I discard all waiting tasks?<a class="headerlink" href="#how-do-i-discard-all-waiting-tasks" title="Permalink to this headline">¶</a></h3>
  93. <p><strong>Answer:</strong> Use <tt class="docutils literal"><span class="pre">celery.task.discard_all()</span></tt>, like this:</p>
  94. <div class="highlight-python"><div class="highlight"><pre><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">celery.task</span> <span class="kn">import</span> <span class="n">discard_all</span>
  95. <span class="gp">&gt;&gt;&gt; </span><span class="n">discard_all</span><span class="p">()</span>
  96. <span class="go">1753</span>
  97. </pre></div>
  98. </div>
  99. <p>The number <tt class="docutils literal"><span class="pre">1753</span></tt> is the number of messages deleted.</p>
  100. <p>You can also start celeryd with the <tt class="docutils literal"><span class="pre">--discard</span></tt> argument which will
  101. accomplish the same thing.</p>
  102. </div>
  103. <div class="section" id="i-ve-discarded-messages-but-there-are-still-messages-left-in-the-queue">
  104. <h3>I&#8217;ve discarded messages, but there are still messages left in the queue?<a class="headerlink" href="#i-ve-discarded-messages-but-there-are-still-messages-left-in-the-queue" title="Permalink to this headline">¶</a></h3>
  105. <p><strong>Answer:</strong> Tasks are acknowledged (removed from the queue) as soon
  106. as they are actually executed. After the worker has received a task, it will
  107. take some time until it is actually executed, especially if there are a lot
  108. of tasks already waiting for execution. Messages that are not acknowledged are
  109. hold on to by the worker until it closes the connection to the broker (AMQP
  110. server). When that connection is closed (e.g because the worker was stopped)
  111. the tasks will be re-sent by the broker to the next available worker (or the
  112. same worker when it has been restarted), so to properly purge the queue of
  113. waiting tasks you have to stop all the workers, and then discard the tasks
  114. using <tt class="docutils literal"><span class="pre">discard_all</span></tt>.</p>
  115. </div>
  116. <div class="section" id="can-i-send-some-tasks-to-only-some-servers">
  117. <h3>Can I send some tasks to only some servers?<a class="headerlink" href="#can-i-send-some-tasks-to-only-some-servers" title="Permalink to this headline">¶</a></h3>
  118. <p><strong>Answer:</strong> As of now there is only one use-case that works like this,
  119. and that is tasks of type <tt class="docutils literal"><span class="pre">A</span></tt> can be sent to servers <tt class="docutils literal"><span class="pre">x</span></tt> and <tt class="docutils literal"><span class="pre">y</span></tt>,
  120. while tasks of type <tt class="docutils literal"><span class="pre">B</span></tt> can be sent to server <tt class="docutils literal"><span class="pre">z</span></tt>. One server can&#8217;t
  121. handle more than one routing_key, but this is coming in a later release.</p>
  122. <p>Say you have two servers, <tt class="docutils literal"><span class="pre">x</span></tt>, and <tt class="docutils literal"><span class="pre">y</span></tt> that handles regular tasks,
  123. and one server <tt class="docutils literal"><span class="pre">z</span></tt>, that only handles feed related tasks, you can use this
  124. configuration:</p>
  125. <blockquote>
  126. <ul class="simple">
  127. <li>Servers <tt class="docutils literal"><span class="pre">x</span></tt> and <tt class="docutils literal"><span class="pre">y</span></tt>: settings.py:</li>
  128. </ul>
  129. <div class="highlight-python"><div class="highlight"><pre><span class="n">AMQP_SERVER</span> <span class="o">=</span> <span class="s">&quot;rabbit&quot;</span>
  130. <span class="n">AMQP_PORT</span> <span class="o">=</span> <span class="mf">5678</span>
  131. <span class="n">AMQP_USER</span> <span class="o">=</span> <span class="s">&quot;myapp&quot;</span>
  132. <span class="n">AMQP_PASSWORD</span> <span class="o">=</span> <span class="s">&quot;secret&quot;</span>
  133. <span class="n">AMQP_VHOST</span> <span class="o">=</span> <span class="s">&quot;myapp&quot;</span>
  134. <span class="n">CELERY_AMQP_CONSUMER_QUEUE</span> <span class="o">=</span> <span class="s">&quot;regular_tasks&quot;</span>
  135. <span class="n">CELERY_AMQP_EXCHANGE</span> <span class="o">=</span> <span class="s">&quot;tasks&quot;</span>
  136. <span class="n">CELERY_AMQP_PUBLISHER_ROUTING_KEY</span> <span class="o">=</span> <span class="s">&quot;task.regular&quot;</span>
  137. <span class="n">CELERY_AMQP_CONSUMER_ROUTING_KEY</span> <span class="o">=</span> <span class="s">&quot;task.#&quot;</span>
  138. <span class="n">CELERY_AMQP_EXCHANGE_TYPE</span> <span class="o">=</span> <span class="s">&quot;topic&quot;</span>
  139. </pre></div>
  140. </div>
  141. <ul class="simple">
  142. <li>Server <tt class="docutils literal"><span class="pre">z</span></tt>: settings.py:</li>
  143. </ul>
  144. <div class="highlight-python"><div class="highlight"><pre><span class="n">AMQP_SERVER</span> <span class="o">=</span> <span class="s">&quot;rabbit&quot;</span>
  145. <span class="n">AMQP_PORT</span> <span class="o">=</span> <span class="mf">5678</span>
  146. <span class="n">AMQP_USER</span> <span class="o">=</span> <span class="s">&quot;myapp&quot;</span>
  147. <span class="n">AMQP_PASSWORD</span> <span class="o">=</span> <span class="s">&quot;secret&quot;</span>
  148. <span class="n">AMQP_VHOST</span> <span class="o">=</span> <span class="s">&quot;myapp&quot;</span>
  149. <span class="n">CELERY_AMQP_EXCHANGE</span> <span class="o">=</span> <span class="s">&quot;tasks&quot;</span>
  150. <span class="n">CELERY_AMQP_PUBLISHER_ROUTING_KEY</span> <span class="o">=</span> <span class="s">&quot;task.regular&quot;</span>
  151. <span class="n">CELERY_AMQP_EXCHANGE_TYPE</span> <span class="o">=</span> <span class="s">&quot;topic&quot;</span>
  152. <span class="c"># This is the settings different for this server:</span>
  153. <span class="n">CELERY_AMQP_CONSUMER_QUEUE</span> <span class="o">=</span> <span class="s">&quot;feed_tasks&quot;</span>
  154. <span class="n">CELERY_AMQP_CONSUMER_ROUTING_KEY</span> <span class="o">=</span> <span class="s">&quot;feed.#&quot;</span>
  155. </pre></div>
  156. </div>
  157. </blockquote>
  158. <p>Now to make a Task run on the <tt class="docutils literal"><span class="pre">z</span></tt> server you need to set its
  159. <tt class="docutils literal"><span class="pre">routing_key</span></tt> attribute so it starts with the words <tt class="docutils literal"><span class="pre">&quot;task.feed.&quot;</span></tt>:</p>
  160. <div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">feedaggregator.models</span> <span class="kn">import</span> <span class="n">Feed</span>
  161. <span class="kn">from</span> <span class="nn">celery.task</span> <span class="kn">import</span> <span class="n">Task</span>
  162. <span class="k">class</span> <span class="nc">FeedImportTask</span><span class="p">(</span><span class="n">Task</span><span class="p">):</span>
  163. <span class="n">name</span> <span class="o">=</span> <span class="s">&quot;import_feed&quot;</span>
  164. <span class="n">routing_key</span> <span class="o">=</span> <span class="s">&quot;feed.importer&quot;</span>
  165. <span class="k">def</span> <span class="nf">run</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">feed_url</span><span class="p">):</span>
  166. <span class="c"># something importing the feed</span>
  167. <span class="n">Feed</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">import_feed</span><span class="p">(</span><span class="n">feed_url</span><span class="p">)</span>
  168. </pre></div>
  169. </div>
  170. <p>You can also override this using the <tt class="docutils literal"><span class="pre">routing_key</span></tt> argument to
  171. <tt class="xref docutils literal"><span class="pre">celery.task.apply_async()</span></tt>:</p>
  172. <div class="highlight-python"><div class="highlight"><pre><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">celery.task</span> <span class="kn">import</span> <span class="n">apply_async</span>
  173. <span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">myapp.tasks</span> <span class="kn">import</span> <span class="n">RefreshFeedTask</span>
  174. <span class="gp">&gt;&gt;&gt; </span><span class="n">apply_async</span><span class="p">(</span><span class="n">RefreshFeedTask</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">[</span><span class="s">&quot;http://cnn.com/rss&quot;</span><span class="p">],</span>
  175. <span class="gp">... </span> <span class="n">routing_key</span><span class="o">=</span><span class="s">&quot;feed.importer&quot;</span><span class="p">)</span>
  176. </pre></div>
  177. </div>
  178. </div>
  179. </div>
  180. </div>
  181. </div>
  182. </div>
  183. </div>
  184. <div class="sphinxsidebar">
  185. <div class="sphinxsidebarwrapper">
  186. <h3><a href="index.html">Table Of Contents</a></h3>
  187. <ul>
  188. <li><a class="reference external" href="">Frequently Asked Questions</a><ul>
  189. <li><a class="reference external" href="#questions">Questions</a><ul>
  190. <li><a class="reference external" href="#mysql-is-throwing-deadlock-errors-what-can-i-do">MySQL is throwing deadlock errors, what can I do?</a></li>
  191. <li><a class="reference external" href="#celeryd-is-not-doing-anything-just-hanging">celeryd is not doing anything, just hanging</a></li>
  192. <li><a class="reference external" href="#i-m-having-integrityerror-duplicate-key-errors-why">I&#8217;m having <tt class="docutils literal"><span class="pre">IntegrityError:</span> <span class="pre">Duplicate</span> <span class="pre">Key</span></tt> errors. Why?</a></li>
  193. <li><a class="reference external" href="#why-won-t-my-task-run">Why won&#8217;t my Task run?</a></li>
  194. <li><a class="reference external" href="#why-won-t-my-periodic-task-run">Why won&#8217;t my Periodic Task run?</a></li>
  195. <li><a class="reference external" href="#how-do-i-discard-all-waiting-tasks">How do I discard all waiting tasks?</a></li>
  196. <li><a class="reference external" href="#i-ve-discarded-messages-but-there-are-still-messages-left-in-the-queue">I&#8217;ve discarded messages, but there are still messages left in the queue?</a></li>
  197. <li><a class="reference external" href="#can-i-send-some-tasks-to-only-some-servers">Can I send some tasks to only some servers?</a></li>
  198. </ul>
  199. </li>
  200. </ul>
  201. </li>
  202. </ul>
  203. <h4>Previous topic</h4>
  204. <p class="topless"><a href="tutorials/clickcounter.html"
  205. title="previous chapter">Tutorial: Creating a click counter using carrot and celery</a></p>
  206. <h4>Next topic</h4>
  207. <p class="topless"><a href="reference/index.html"
  208. title="next chapter">Module API Reference</a></p>
  209. <h3>This Page</h3>
  210. <ul class="this-page-menu">
  211. <li><a href="sources/faq.txt"
  212. rel="nofollow">Show Source</a></li>
  213. </ul>
  214. <div id="searchbox" style="display: none">
  215. <h3>Quick search</h3>
  216. <form class="search" action="search.html" method="get">
  217. <input type="text" name="q" size="18" />
  218. <input type="submit" value="Go" />
  219. <input type="hidden" name="check_keywords" value="yes" />
  220. <input type="hidden" name="area" value="default" />
  221. </form>
  222. <p class="searchtip" style="font-size: 90%">
  223. Enter search terms or a module, class or function name.
  224. </p>
  225. </div>
  226. <script type="text/javascript">$('#searchbox').show(0);</script>
  227. </div>
  228. </div>
  229. <div class="clearer"></div>
  230. </div>
  231. <div class="related">
  232. <h3>Navigation</h3>
  233. <ul>
  234. <li class="right" style="margin-right: 10px">
  235. <a href="genindex.html" title="General Index"
  236. >index</a></li>
  237. <li class="right" >
  238. <a href="modindex.html" title="Global Module Index"
  239. >modules</a> |</li>
  240. <li class="right" >
  241. <a href="reference/index.html" title="Module API Reference"
  242. >next</a> |</li>
  243. <li class="right" >
  244. <a href="tutorials/clickcounter.html" title="Tutorial: Creating a click counter using carrot and celery"
  245. >previous</a> |</li>
  246. <li><a href="index.html">Celery v0.4.2 (stable) documentation</a> &raquo;</li>
  247. </ul>
  248. </div>
  249. <div class="footer">
  250. &copy; Copyright 2009, Ask Solem.
  251. Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 0.6.1.
  252. </div>
  253. </body>
  254. </html>