celery.datastructures.html 15 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277
  1. <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
  2. "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
  3. <html xmlns="http://www.w3.org/1999/xhtml">
  4. <head>
  5. <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  6. <title>Datastructures - celery.datastructures &mdash; Celery v0.2.14 documentation</title>
  7. <link rel="stylesheet" href="../static/agogo.css" type="text/css" />
  8. <link rel="stylesheet" href="../static/pygments.css" type="text/css" />
  9. <script type="text/javascript">
  10. var DOCUMENTATION_OPTIONS = {
  11. URL_ROOT: '../',
  12. VERSION: '0.2.14',
  13. COLLAPSE_MODINDEX: false,
  14. FILE_SUFFIX: '.html',
  15. HAS_SOURCE: true
  16. };
  17. </script>
  18. <script type="text/javascript" src="../static/jquery.js"></script>
  19. <script type="text/javascript" src="../static/doctools.js"></script>
  20. <link rel="top" title="Celery v0.2.14 documentation" href="../index.html" />
  21. <link rel="up" title="Module API Reference" href="index.html" />
  22. <link rel="next" title="Logging - celery.log" href="celery.log.html" />
  23. <link rel="prev" title="Configuration - celery.conf" href="celery.conf.html" />
  24. </head>
  25. <body>
  26. <div class="header-wrapper">
  27. <div class="header">
  28. <h1><a href="../index.html">Celery v0.2.14 documentation</a></h1>
  29. <div class="rel">
  30. <a href="../genindex.html" title="General Index"
  31. accesskey="I">index</a> |
  32. <a href="../modindex.html" title="Global Module Index"
  33. accesskey="M">modules</a> |
  34. <a href="celery.log.html" title="Logging - celery.log"
  35. accesskey="N">next</a> |
  36. <a href="celery.conf.html" title="Configuration - celery.conf"
  37. accesskey="P">previous</a>
  38. </div>
  39. </div>
  40. </div>
  41. <div class="content-wrapper">
  42. <div class="content">
  43. <div class="document">
  44. <div class="documentwrapper">
  45. <div class="bodywrapper">
  46. <div class="body">
  47. <div class="section" id="module-celery.datastructures">
  48. <h1>Datastructures - celery.datastructures<a class="headerlink" href="#module-celery.datastructures" title="Permalink to this headline">¶</a></h1>
  49. <p>Custom Datastructures</p>
  50. <dl class="class">
  51. <dt id="celery.datastructures.PositionQueue">
  52. <em class="property">
  53. class </em><tt class="descclassname">celery.datastructures.</tt><tt class="descname">PositionQueue</tt><big>(</big><em>length</em><big>)</big><a class="headerlink" href="#celery.datastructures.PositionQueue" title="Permalink to this definition">¶</a></dt>
  54. <dd><p>A positional queue of a specific length, with slots that are either
  55. filled or unfilled. When all of the positions are filled, the queue
  56. is considered <a title="celery.datastructures.PositionQueue.full" class="reference internal" href="#celery.datastructures.PositionQueue.full"><tt class="xref docutils literal"><span class="pre">full()</span></tt></a>.</p>
  57. <table class="docutils field-list" frame="void" rules="none">
  58. <col class="field-name" />
  59. <col class="field-body" />
  60. <tbody valign="top">
  61. <tr class="field"><th class="field-name">Parameter:</th><td class="field-body"><em>length</em> &#8211; see <a title="celery.datastructures.PositionQueue.length" class="reference internal" href="#celery.datastructures.PositionQueue.length"><tt class="xref docutils literal"><span class="pre">length</span></tt></a>.</td>
  62. </tr>
  63. </tbody>
  64. </table>
  65. <dl class="attribute">
  66. <dt id="celery.datastructures.PositionQueue.length">
  67. <tt class="descname">length</tt><a class="headerlink" href="#celery.datastructures.PositionQueue.length" title="Permalink to this definition">¶</a></dt>
  68. <dd>The number of items required for the queue to be considered full.</dd></dl>
  69. <dl class="class">
  70. <dt id="celery.datastructures.PositionQueue.UnfilledPosition">
  71. <em class="property">
  72. class </em><tt class="descname">UnfilledPosition</tt><big>(</big><em>position</em><big>)</big><a class="headerlink" href="#celery.datastructures.PositionQueue.UnfilledPosition" title="Permalink to this definition">¶</a></dt>
  73. <dd>Describes an unfilled slot.</dd></dl>
  74. <dl class="attribute">
  75. <dt id="celery.datastructures.PositionQueue.filled">
  76. <tt class="descclassname">PositionQueue.</tt><tt class="descname">filled</tt><a class="headerlink" href="#celery.datastructures.PositionQueue.filled" title="Permalink to this definition">¶</a></dt>
  77. <dd>Returns the filled slots as a list.</dd></dl>
  78. <dl class="method">
  79. <dt id="celery.datastructures.PositionQueue.full">
  80. <tt class="descclassname">PositionQueue.</tt><tt class="descname">full</tt><big>(</big><big>)</big><a class="headerlink" href="#celery.datastructures.PositionQueue.full" title="Permalink to this definition">¶</a></dt>
  81. <dd>Returns <tt class="xref docutils literal"><span class="pre">True</span></tt> if all of the slots has been filled.</dd></dl>
  82. </dd></dl>
  83. <dl class="class">
  84. <dt id="celery.datastructures.TaskProcessQueue">
  85. <em class="property">
  86. class </em><tt class="descclassname">celery.datastructures.</tt><tt class="descname">TaskProcessQueue</tt><big>(</big><em>limit</em>, <em>reap_timeout=None</em>, <em>logger=None</em>, <em>done_msg=None</em><big>)</big><a class="headerlink" href="#celery.datastructures.TaskProcessQueue" title="Permalink to this definition">¶</a></dt>
  87. <dd><p>Queue of running child processes, which starts waiting for the
  88. processes to finish when the queue limit has been reached.</p>
  89. <table class="docutils field-list" frame="void" rules="none">
  90. <col class="field-name" />
  91. <col class="field-body" />
  92. <tbody valign="top">
  93. <tr class="field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
  94. <li><em>limit</em> &#8211; see <a title="celery.datastructures.TaskProcessQueue.limit" class="reference internal" href="#celery.datastructures.TaskProcessQueue.limit"><tt class="xref docutils literal"><span class="pre">limit</span></tt></a> attribute.</li>
  95. <li><em>logger</em> &#8211; see <a title="celery.datastructures.TaskProcessQueue.logger" class="reference internal" href="#celery.datastructures.TaskProcessQueue.logger"><tt class="xref docutils literal"><span class="pre">logger</span></tt></a> attribute.</li>
  96. <li><em>done_msg</em> &#8211; see <a title="celery.datastructures.TaskProcessQueue.done_msg" class="reference internal" href="#celery.datastructures.TaskProcessQueue.done_msg"><tt class="xref docutils literal"><span class="pre">done_msg</span></tt></a> attribute.</li>
  97. </ul>
  98. </td>
  99. </tr>
  100. </tbody>
  101. </table>
  102. <dl class="attribute">
  103. <dt id="celery.datastructures.TaskProcessQueue.limit">
  104. <tt class="descname">limit</tt><a class="headerlink" href="#celery.datastructures.TaskProcessQueue.limit" title="Permalink to this definition">¶</a></dt>
  105. <dd>The number of processes that can run simultaneously until
  106. we start collecting results.</dd></dl>
  107. <dl class="attribute">
  108. <dt id="celery.datastructures.TaskProcessQueue.logger">
  109. <tt class="descname">logger</tt><a class="headerlink" href="#celery.datastructures.TaskProcessQueue.logger" title="Permalink to this definition">¶</a></dt>
  110. <dd>The logger used to print the <a title="celery.datastructures.TaskProcessQueue.done_msg" class="reference internal" href="#celery.datastructures.TaskProcessQueue.done_msg"><tt class="xref docutils literal"><span class="pre">done_msg</span></tt></a>.</dd></dl>
  111. <dl class="attribute">
  112. <dt id="celery.datastructures.TaskProcessQueue.done_msg">
  113. <tt class="descname">done_msg</tt><a class="headerlink" href="#celery.datastructures.TaskProcessQueue.done_msg" title="Permalink to this definition">¶</a></dt>
  114. <dd>Message logged when a tasks result has been collected.
  115. The message is logged with loglevel <tt class="xref docutils literal"><span class="pre">logging.INFO</span></tt>.</dd></dl>
  116. <dl class="method">
  117. <dt id="celery.datastructures.TaskProcessQueue.add">
  118. <tt class="descname">add</tt><big>(</big><em>result</em>, <em>task_name</em>, <em>task_id</em><big>)</big><a class="headerlink" href="#celery.datastructures.TaskProcessQueue.add" title="Permalink to this definition">¶</a></dt>
  119. <dd><p>Add a process to the queue.</p>
  120. <p>If the queue is full, it will wait for the first task to finish,
  121. collects its result and remove it from the queue, so it&#8217;s ready
  122. to accept new processes.</p>
  123. <table class="docutils field-list" frame="void" rules="none">
  124. <col class="field-name" />
  125. <col class="field-body" />
  126. <tbody valign="top">
  127. <tr class="field"><th class="field-name">Parameters:</th><td class="field-body"><ul class="first last simple">
  128. <li><em>result</em> &#8211; A <tt class="xref docutils literal"><span class="pre">multiprocessing.AsyncResult</span></tt> instance, as
  129. returned by <tt class="xref docutils literal"><span class="pre">multiprocessing.Pool.apply_async()</span></tt>.</li>
  130. <li><em>task_name</em> &#8211; Name of the task executed.</li>
  131. <li><em>task_id</em> &#8211; Id of the task executed.</li>
  132. </ul>
  133. </td>
  134. </tr>
  135. </tbody>
  136. </table>
  137. </dd></dl>
  138. <dl class="method">
  139. <dt id="celery.datastructures.TaskProcessQueue.get_worker_pids">
  140. <tt class="descname">get_worker_pids</tt><big>(</big><big>)</big><a class="headerlink" href="#celery.datastructures.TaskProcessQueue.get_worker_pids" title="Permalink to this definition">¶</a></dt>
  141. <dd><p>Returns the process id&#8217;s of all the pool workers.</p>
  142. <table class="docutils field-list" frame="void" rules="none">
  143. <col class="field-name" />
  144. <col class="field-body" />
  145. <tbody valign="top">
  146. <tr class="field"><th class="field-name">Return type:</th><td class="field-body">list</td>
  147. </tr>
  148. </tbody>
  149. </table>
  150. </dd></dl>
  151. <dl class="method">
  152. <dt id="celery.datastructures.TaskProcessQueue.on_ready">
  153. <tt class="descname">on_ready</tt><big>(</big><em>ret_value</em>, <em>task_name</em>, <em>task_id</em><big>)</big><a class="headerlink" href="#celery.datastructures.TaskProcessQueue.on_ready" title="Permalink to this definition">¶</a></dt>
  154. <dd><p>What to do when a worker returns with a result.</p>
  155. <p>If <a title="celery.datastructures.TaskProcessQueue.done_msg" class="reference internal" href="#celery.datastructures.TaskProcessQueue.done_msg"><tt class="xref docutils literal"><span class="pre">done_msg</span></tt></a> is defined, it will log this
  156. format string, with level <tt class="docutils literal"><span class="pre">logging.INFO</span></tt>,
  157. using these format variables:</p>
  158. <ul>
  159. <li><p class="first">%(name)</p>
  160. <blockquote>
  161. <p>The name of the task completed</p>
  162. </blockquote>
  163. </li>
  164. <li><p class="first">%(id)</p>
  165. <blockquote>
  166. <p>The UUID of the task completed.</p>
  167. </blockquote>
  168. </li>
  169. <li><p class="first">%(return_value)</p>
  170. <blockquote>
  171. <p>Return value of the task function.</p>
  172. </blockquote>
  173. </li>
  174. </ul>
  175. </dd></dl>
  176. <dl class="method">
  177. <dt id="celery.datastructures.TaskProcessQueue.wait_for_result">
  178. <tt class="descname">wait_for_result</tt><big>(</big><big>)</big><a class="headerlink" href="#celery.datastructures.TaskProcessQueue.wait_for_result" title="Permalink to this definition">¶</a></dt>
  179. <dd><p>Waits for the first process in the pool to finish.</p>
  180. <p>This operation is blocking.</p>
  181. </dd></dl>
  182. </dd></dl>
  183. </div>
  184. </div>
  185. </div>
  186. </div>
  187. </div>
  188. <div class="sidebar">
  189. <h3>Contents</h3>
  190. <ul class="current">
  191. <li class="toctree-l1"><a class="reference external" href="../introduction.html">celery - Distributed Task Queue for Django.</a></li>
  192. <li class="toctree-l1"><a class="reference external" href="../faq.html">Frequently Asked Questions</a></li>
  193. <li class="toctree-l1 current"><a class="reference external" href="index.html">Module API Reference</a><ul class="current">
  194. <li class="toctree-l2"><a class="reference external" href="celery.task.html">Tasks - celery.task</a></li>
  195. <li class="toctree-l2"><a class="reference external" href="celery.result.html">Task Result - celery.result</a></li>
  196. <li class="toctree-l2"><a class="reference external" href="celery.registry.html">Task Registry - celery.registry</a></li>
  197. <li class="toctree-l2"><a class="reference external" href="celery.discovery.html">Task Discovery - celery.discovery</a></li>
  198. <li class="toctree-l2"><a class="reference external" href="celery.worker.html">Multiprocessing Worker - celery.worker</a></li>
  199. <li class="toctree-l2"><a class="reference external" href="celery.backends.html">Backends - celery.backends</a></li>
  200. <li class="toctree-l2"><a class="reference external" href="celery.backends.base.html">Backend: Base - celery.backends.base</a></li>
  201. <li class="toctree-l2"><a class="reference external" href="celery.backends.database.html">Backend: Database - celery.backends.database</a></li>
  202. <li class="toctree-l2"><a class="reference external" href="celery.backends.cache.html">Backend: Cache - celery.backends.cache</a></li>
  203. <li class="toctree-l2"><a class="reference external" href="celery.backends.tyrant.html">Backend: Tokyo Tyrant - celery.backends.tyrant</a></li>
  204. <li class="toctree-l2"><a class="reference external" href="celery.conf.html">Configuration - celery.conf</a></li>
  205. <li class="toctree-l2 current"><a class="current reference external" href="">Datastructures - celery.datastructures</a></li>
  206. <li class="toctree-l2"><a class="reference external" href="celery.log.html">Logging - celery.log</a></li>
  207. <li class="toctree-l2"><a class="reference external" href="celery.managers.html">Django Model Managers - celery.managers</a></li>
  208. <li class="toctree-l2"><a class="reference external" href="celery.models.html">Django Models - celery.models</a></li>
  209. <li class="toctree-l2"><a class="reference external" href="celery.messaging.html">Messaging - celery.messaging</a></li>
  210. <li class="toctree-l2"><a class="reference external" href="celery.platform.html">Platform Specific - celery.platform</a></li>
  211. <li class="toctree-l2"><a class="reference external" href="celery.timer.html">Timers - celery.timer</a></li>
  212. <li class="toctree-l2"><a class="reference external" href="celery.bin.celeryd.html">Celery Worker Daemon - celery.bin.celeryd</a></li>
  213. </ul>
  214. </li>
  215. </ul>
  216. <h3 style="margin-top: 1.5em;">Search</h3>
  217. <form class="search" action="../search.html" method="get">
  218. <input type="text" name="q" size="18" />
  219. <input type="submit" value="Go" />
  220. <input type="hidden" name="check_keywords" value="yes" />
  221. <input type="hidden" name="area" value="default" />
  222. </form>
  223. <p class="searchtip" style="font-size: 90%">
  224. Enter search terms or a module, class or function name.
  225. </p>
  226. </div>
  227. <div class="clearer"></div>
  228. </div>
  229. </div>
  230. <div class="footer-wrapper">
  231. <div class="footer">
  232. <div class="left">
  233. <a href="../genindex.html" title="General Index"
  234. >index</a> |
  235. <a href="../modindex.html" title="Global Module Index"
  236. >modules</a> |
  237. <a href="celery.log.html" title="Logging - celery.log"
  238. >next</a> |
  239. <a href="celery.conf.html" title="Configuration - celery.conf"
  240. >previous</a>
  241. <br/>
  242. <a href="../sources/reference/celery.datastructures.txt"
  243. rel="nofollow">Show Source</a>
  244. </div>
  245. <div class="right">
  246. &copy; Copyright 2009, Ask Solem.<br/>
  247. Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 0.6.1.
  248. </div>
  249. <div class="clearer"></div>
  250. </div>
  251. </div>
  252. </body>
  253. </html>