yangck
/
celery


			
							123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282
							<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    
    <title>Frequently Asked Questions &mdash; Celery v0.4.12 (stable) documentation</title>
    <link rel="stylesheet" href="static/nature.css" type="text/css" />
    <link rel="stylesheet" href="static/pygments.css" type="text/css" />
    <script type="text/javascript">
      var DOCUMENTATION_OPTIONS = {
        URL_ROOT:    '',
        VERSION:     '0.4.12 (stable)',
        COLLAPSE_MODINDEX: false,
        FILE_SUFFIX: '.html',
        HAS_SOURCE:  true
      };
    </script>
    <script type="text/javascript" src="static/jquery.js"></script>
    <script type="text/javascript" src="static/doctools.js"></script>
    <link rel="top" title="Celery v0.4.12 (stable) documentation" href="index.html" />
    <link rel="next" title="Module API Reference" href="reference/index.html" />
    <link rel="prev" title="Tutorial: Creating a click counter using carrot and celery" href="tutorials/clickcounter.html" /> 
  </head>
  <body>
    <div class="related">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="genindex.html" title="General Index"
             accesskey="I">index</a></li>
        <li class="right" >
          <a href="modindex.html" title="Global Module Index"
             accesskey="M">modules</a> |</li>
        <li class="right" >
          <a href="reference/index.html" title="Module API Reference"
             accesskey="N">next</a> |</li>
        <li class="right" >
          <a href="tutorials/clickcounter.html" title="Tutorial: Creating a click counter using carrot and celery"
             accesskey="P">previous</a> |</li>
        <li><a href="index.html">Celery v0.4.12 (stable) documentation</a> &raquo;</li> 
      </ul>
    </div>  

    <div class="document">
      <div class="documentwrapper">
        <div class="bodywrapper">
          <div class="body">
            
  <div class="section" id="frequently-asked-questions">
<h1>Frequently Asked Questions<a class="headerlink" href="#frequently-asked-questions" title="Permalink to this headline">¶</a></h1>
<div class="section" id="questions">
<h2>Questions<a class="headerlink" href="#questions" title="Permalink to this headline">¶</a></h2>
<div class="section" id="mysql-is-throwing-deadlock-errors-what-can-i-do">
<h3>MySQL is throwing deadlock errors, what can I do?<a class="headerlink" href="#mysql-is-throwing-deadlock-errors-what-can-i-do" title="Permalink to this headline">¶</a></h3>
<p><strong>Answer:</strong> MySQL has default isolation level set to <tt class="docutils literal"><span class="pre">REPEATABLE-READ</span></tt>,
if you don&#8217;t really need that, set it to <tt class="docutils literal"><span class="pre">READ-COMMITTED</span></tt>.
You can do that by adding the following to your <tt class="docutils literal"><span class="pre">my.cnf</span></tt>:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="p">[</span><span class="n">mysqld</span><span class="p">]</span>
<span class="n">transaction</span><span class="o">-</span><span class="n">isolation</span> <span class="o">=</span> <span class="n">READ</span><span class="o">-</span><span class="n">COMMITTED</span>
</pre></div>
</div>
<p>For more information about InnoDBs transaction model see <a class="reference external" href="http://dev.mysql.com/doc/refman/5.1/en/innodb-transaction-model.html">MySQL - The InnoDB
Transaction Model and Locking</a> in the MySQL user manual.</p>
<p>(Thanks to Honza Kral and Anton Tsigularov for this solution)</p>
</div>
<div class="section" id="celeryd-is-not-doing-anything-just-hanging">
<h3>celeryd is not doing anything, just hanging<a class="headerlink" href="#celeryd-is-not-doing-anything-just-hanging" title="Permalink to this headline">¶</a></h3>
<dl class="docutils">
<dt><strong>Answer:</strong> See <a class="reference internal" href="#mysql-is-throwing-deadlock-errors-what-can-i-do">MySQL is throwing deadlock errors, what can I do?</a>.</dt>
<dd>or <cite>Why is Task.delay/apply* just hanging?</cite>.</dd>
</dl>
</div>
<div class="section" id="why-is-task-delay-apply-just-hanging">
<h3>Why is Task.delay/apply* just hanging?`<a class="headerlink" href="#why-is-task-delay-apply-just-hanging" title="Permalink to this headline">¶</a></h3>
<p><strong>Answer:</strong> <tt class="xref docutils literal"><span class="pre">amqplib</span></tt> hangs if it isn&#8217;t able to authenticate to the
AMQP server, so make sure you are able to access the configured vhost using
the user and password.</p>
</div>
<div class="section" id="why-won-t-celeryd-run-on-freebsd">
<h3>Why won&#8217;t celeryd run on FreeBSD?<a class="headerlink" href="#why-won-t-celeryd-run-on-freebsd" title="Permalink to this headline">¶</a></h3>
<p><strong>Answer:</strong> multiprocessing.Pool requires a working POSIX semaphore
implementation which isn&#8217;t enabled in FreeBSD by default. You have to enable
POSIX semaphores in the kernel and manually recompile multiprocessing.</p>
</div>
<div class="section" id="i-m-having-integrityerror-duplicate-key-errors-why">
<h3>I&#8217;m having <tt class="docutils literal"><span class="pre">IntegrityError:</span> <span class="pre">Duplicate</span> <span class="pre">Key</span></tt> errors. Why?<a class="headerlink" href="#i-m-having-integrityerror-duplicate-key-errors-why" title="Permalink to this headline">¶</a></h3>
<p><strong>Answer:</strong> See <a class="reference internal" href="#mysql-is-throwing-deadlock-errors-what-can-i-do">MySQL is throwing deadlock errors, what can I do?</a>.
Thanks to howsthedotcom.</p>
</div>
<div class="section" id="why-won-t-my-task-run">
<h3>Why won&#8217;t my Task run?<a class="headerlink" href="#why-won-t-my-task-run" title="Permalink to this headline">¶</a></h3>
<p><strong>Answer:</strong> Did you register the task in the applications <tt class="docutils literal"><span class="pre">tasks.py</span></tt> module?
(or in some other module Django loads by default, like <tt class="docutils literal"><span class="pre">models.py</span></tt>?).
Also there might be syntax errors preventing the tasks module being imported.</p>
<p>You can find out if the celery daemon is able to run the task by executing the
task manually:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">myapp.tasks</span> <span class="kn">import</span> <span class="n">MyPeriodicTask</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">MyPeriodicTask</span><span class="o">.</span><span class="n">delay</span><span class="p">()</span>
</pre></div>
</div>
<p>Watch celery daemons logfile (or output if not running as a daemon), to see
if it&#8217;s able to find the task, or if some other error is happening.</p>
</div>
<div class="section" id="why-won-t-my-periodic-task-run">
<h3>Why won&#8217;t my Periodic Task run?<a class="headerlink" href="#why-won-t-my-periodic-task-run" title="Permalink to this headline">¶</a></h3>
<p><strong>Answer:</strong> See <a class="reference internal" href="#why-won-t-my-task-run">Why won&#8217;t my Task run?</a>.</p>
</div>
<div class="section" id="how-do-i-discard-all-waiting-tasks">
<h3>How do I discard all waiting tasks?<a class="headerlink" href="#how-do-i-discard-all-waiting-tasks" title="Permalink to this headline">¶</a></h3>
<p><strong>Answer:</strong> Use <tt class="docutils literal"><span class="pre">celery.task.discard_all()</span></tt>, like this:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">celery.task</span> <span class="kn">import</span> <span class="n">discard_all</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">discard_all</span><span class="p">()</span>
<span class="go">1753</span>
</pre></div>
</div>
<p>The number <tt class="docutils literal"><span class="pre">1753</span></tt> is the number of messages deleted.</p>
<p>You can also start celeryd with the <tt class="docutils literal"><span class="pre">--discard</span></tt> argument which will
accomplish the same thing.</p>
</div>
<div class="section" id="i-ve-discarded-messages-but-there-are-still-messages-left-in-the-queue">
<h3>I&#8217;ve discarded messages, but there are still messages left in the queue?<a class="headerlink" href="#i-ve-discarded-messages-but-there-are-still-messages-left-in-the-queue" title="Permalink to this headline">¶</a></h3>
<p><strong>Answer:</strong> Tasks are acknowledged (removed from the queue) as soon
as they are actually executed. After the worker has received a task, it will
take some time until it is actually executed, especially if there are a lot
of tasks already waiting for execution. Messages that are not acknowledged are
hold on to by the worker until it closes the connection to the broker (AMQP
server). When that connection is closed (e.g because the worker was stopped)
the tasks will be re-sent by the broker to the next available worker (or the
same worker when it has been restarted), so to properly purge the queue of
waiting tasks you have to stop all the workers, and then discard the tasks
using <tt class="docutils literal"><span class="pre">discard_all</span></tt>.</p>
</div>
<div class="section" id="can-i-send-some-tasks-to-only-some-servers">
<h3>Can I send some tasks to only some servers?<a class="headerlink" href="#can-i-send-some-tasks-to-only-some-servers" title="Permalink to this headline">¶</a></h3>
<p><strong>Answer:</strong> As of now there is only one use-case that works like this,
and that is tasks of type <tt class="docutils literal"><span class="pre">A</span></tt> can be sent to servers <tt class="docutils literal"><span class="pre">x</span></tt> and <tt class="docutils literal"><span class="pre">y</span></tt>,
while tasks of type <tt class="docutils literal"><span class="pre">B</span></tt> can be sent to server <tt class="docutils literal"><span class="pre">z</span></tt>. One server can&#8217;t
handle more than one routing_key, but this is coming in a later release.</p>
<p>Say you have two servers, <tt class="docutils literal"><span class="pre">x</span></tt>, and <tt class="docutils literal"><span class="pre">y</span></tt> that handles regular tasks,
and one server <tt class="docutils literal"><span class="pre">z</span></tt>, that only handles feed related tasks, you can use this
configuration:</p>
<blockquote>
<ul class="simple">
<li>Servers <tt class="docutils literal"><span class="pre">x</span></tt> and <tt class="docutils literal"><span class="pre">y</span></tt>: settings.py:</li>
</ul>
<div class="highlight-python"><div class="highlight"><pre><span class="n">AMQP_SERVER</span> <span class="o">=</span> <span class="s">&quot;rabbit&quot;</span>
<span class="n">AMQP_PORT</span> <span class="o">=</span> <span class="mf">5678</span>
<span class="n">AMQP_USER</span> <span class="o">=</span> <span class="s">&quot;myapp&quot;</span>
<span class="n">AMQP_PASSWORD</span> <span class="o">=</span> <span class="s">&quot;secret&quot;</span>
<span class="n">AMQP_VHOST</span> <span class="o">=</span> <span class="s">&quot;myapp&quot;</span>

<span class="n">CELERY_AMQP_CONSUMER_QUEUE</span> <span class="o">=</span> <span class="s">&quot;regular_tasks&quot;</span>
<span class="n">CELERY_AMQP_EXCHANGE</span> <span class="o">=</span> <span class="s">&quot;tasks&quot;</span>
<span class="n">CELERY_AMQP_PUBLISHER_ROUTING_KEY</span> <span class="o">=</span> <span class="s">&quot;task.regular&quot;</span>
<span class="n">CELERY_AMQP_CONSUMER_ROUTING_KEY</span> <span class="o">=</span> <span class="s">&quot;task.#&quot;</span>
<span class="n">CELERY_AMQP_EXCHANGE_TYPE</span> <span class="o">=</span> <span class="s">&quot;topic&quot;</span>
</pre></div>
</div>
<ul class="simple">
<li>Server <tt class="docutils literal"><span class="pre">z</span></tt>: settings.py:</li>
</ul>
<div class="highlight-python"><div class="highlight"><pre><span class="n">AMQP_SERVER</span> <span class="o">=</span> <span class="s">&quot;rabbit&quot;</span>
<span class="n">AMQP_PORT</span> <span class="o">=</span> <span class="mf">5678</span>
<span class="n">AMQP_USER</span> <span class="o">=</span> <span class="s">&quot;myapp&quot;</span>
<span class="n">AMQP_PASSWORD</span> <span class="o">=</span> <span class="s">&quot;secret&quot;</span>
<span class="n">AMQP_VHOST</span> <span class="o">=</span> <span class="s">&quot;myapp&quot;</span>

<span class="n">CELERY_AMQP_EXCHANGE</span> <span class="o">=</span> <span class="s">&quot;tasks&quot;</span>
<span class="n">CELERY_AMQP_PUBLISHER_ROUTING_KEY</span> <span class="o">=</span> <span class="s">&quot;task.regular&quot;</span>
<span class="n">CELERY_AMQP_EXCHANGE_TYPE</span> <span class="o">=</span> <span class="s">&quot;topic&quot;</span>
<span class="c"># This is the settings different for this server:</span>
<span class="n">CELERY_AMQP_CONSUMER_QUEUE</span> <span class="o">=</span> <span class="s">&quot;feed_tasks&quot;</span>
<span class="n">CELERY_AMQP_CONSUMER_ROUTING_KEY</span> <span class="o">=</span> <span class="s">&quot;feed.#&quot;</span>
</pre></div>
</div>
</blockquote>
<p>Now to make a Task run on the <tt class="docutils literal"><span class="pre">z</span></tt> server you need to set its
<tt class="docutils literal"><span class="pre">routing_key</span></tt> attribute so it starts with the words <tt class="docutils literal"><span class="pre">&quot;task.feed.&quot;</span></tt>:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">feedaggregator.models</span> <span class="kn">import</span> <span class="n">Feed</span>
<span class="kn">from</span> <span class="nn">celery.task</span> <span class="kn">import</span> <span class="n">Task</span>

<span class="k">class</span> <span class="nc">FeedImportTask</span><span class="p">(</span><span class="n">Task</span><span class="p">):</span>
    <span class="n">name</span> <span class="o">=</span> <span class="s">&quot;import_feed&quot;</span>
    <span class="n">routing_key</span> <span class="o">=</span> <span class="s">&quot;feed.importer&quot;</span>

    <span class="k">def</span> <span class="nf">run</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">feed_url</span><span class="p">):</span>
        <span class="c"># something importing the feed</span>
        <span class="n">Feed</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">import_feed</span><span class="p">(</span><span class="n">feed_url</span><span class="p">)</span>
</pre></div>
</div>
<p>You can also override this using the <tt class="docutils literal"><span class="pre">routing_key</span></tt> argument to
<tt class="xref docutils literal"><span class="pre">celery.task.apply_async()</span></tt>:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">celery.task</span> <span class="kn">import</span> <span class="n">apply_async</span>
<span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">myapp.tasks</span> <span class="kn">import</span> <span class="n">RefreshFeedTask</span>
<span class="gp">&gt;&gt;&gt; </span><span class="n">apply_async</span><span class="p">(</span><span class="n">RefreshFeedTask</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">[</span><span class="s">&quot;http://cnn.com/rss&quot;</span><span class="p">],</span>
<span class="gp">... </span>            <span class="n">routing_key</span><span class="o">=</span><span class="s">&quot;feed.importer&quot;</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
</div>


          </div>
        </div>
      </div>
      <div class="sphinxsidebar">
        <div class="sphinxsidebarwrapper">
            <h3><a href="index.html">Table Of Contents</a></h3>
            <ul>
<li><a class="reference external" href="">Frequently Asked Questions</a><ul>
<li><a class="reference external" href="#questions">Questions</a><ul>
<li><a class="reference external" href="#mysql-is-throwing-deadlock-errors-what-can-i-do">MySQL is throwing deadlock errors, what can I do?</a></li>
<li><a class="reference external" href="#celeryd-is-not-doing-anything-just-hanging">celeryd is not doing anything, just hanging</a></li>
<li><a class="reference external" href="#why-is-task-delay-apply-just-hanging">Why is Task.delay/apply* just hanging?`</a></li>
<li><a class="reference external" href="#why-won-t-celeryd-run-on-freebsd">Why won&#8217;t celeryd run on FreeBSD?</a></li>
<li><a class="reference external" href="#i-m-having-integrityerror-duplicate-key-errors-why">I&#8217;m having <tt class="docutils literal"><span class="pre">IntegrityError:</span> <span class="pre">Duplicate</span> <span class="pre">Key</span></tt> errors. Why?</a></li>
<li><a class="reference external" href="#why-won-t-my-task-run">Why won&#8217;t my Task run?</a></li>
<li><a class="reference external" href="#why-won-t-my-periodic-task-run">Why won&#8217;t my Periodic Task run?</a></li>
<li><a class="reference external" href="#how-do-i-discard-all-waiting-tasks">How do I discard all waiting tasks?</a></li>
<li><a class="reference external" href="#i-ve-discarded-messages-but-there-are-still-messages-left-in-the-queue">I&#8217;ve discarded messages, but there are still messages left in the queue?</a></li>
<li><a class="reference external" href="#can-i-send-some-tasks-to-only-some-servers">Can I send some tasks to only some servers?</a></li>
</ul>
</li>
</ul>
</li>
</ul>

            <h4>Previous topic</h4>
            <p class="topless"><a href="tutorials/clickcounter.html"
                                  title="previous chapter">Tutorial: Creating a click counter using carrot and celery</a></p>
            <h4>Next topic</h4>
            <p class="topless"><a href="reference/index.html"
                                  title="next chapter">Module API Reference</a></p>
            <h3>This Page</h3>
            <ul class="this-page-menu">
              <li><a href="sources/faq.txt"
                     rel="nofollow">Show Source</a></li>
            </ul>
          <div id="searchbox" style="display: none">
            <h3>Quick search</h3>
              <form class="search" action="search.html" method="get">
                <input type="text" name="q" size="18" />
                <input type="submit" value="Go" />
                <input type="hidden" name="check_keywords" value="yes" />
                <input type="hidden" name="area" value="default" />
              </form>
              <p class="searchtip" style="font-size: 90%">
              Enter search terms or a module, class or function name.
              </p>
          </div>
          <script type="text/javascript">$('#searchbox').show(0);</script>
        </div>
      </div>
      <div class="clearer"></div>
    </div>
    <div class="related">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="genindex.html" title="General Index"
             >index</a></li>
        <li class="right" >
          <a href="modindex.html" title="Global Module Index"
             >modules</a> |</li>
        <li class="right" >
          <a href="reference/index.html" title="Module API Reference"
             >next</a> |</li>
        <li class="right" >
          <a href="tutorials/clickcounter.html" title="Tutorial: Creating a click counter using carrot and celery"
             >previous</a> |</li>
        <li><a href="index.html">Celery v0.4.12 (stable) documentation</a> &raquo;</li> 
      </ul>
    </div>
    <div class="footer">
      &copy; Copyright 2009, Ask Solem.
      Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 0.6.1.
    </div>
  </body>
</html>