contributing.rst 27 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014
  1. .. _contributing:
  2. ==============
  3. Contributing
  4. ==============
  5. .. contents::
  6. :local:
  7. .. _community-code-of-conduct:
  8. Community Code of Conduct
  9. =========================
  10. The goal is to maintain a diverse community that is pleasant for everyone.
  11. That is why we would greatly appreciate it if everyone contributing to and
  12. interacting with the community also followed this Code of Conduct.
  13. The Code of Conduct covers our behavior as members of the community,
  14. in any forum, mailing list, wiki, website, Internet relay chat (IRC), public
  15. meeting or private correspondence.
  16. The Code of Conduct is heavily based on the `Ubuntu Code of Conduct`_, and
  17. the `Pylons Code of Conduct`_.
  18. .. _`Ubuntu Code of Conduct`: http://www.ubuntu.com/community/conduct
  19. .. _`Pylons Code of Conduct`: http://docs.pylonshq.com/community/conduct.html
  20. Be considerate.
  21. ---------------
  22. Your work will be used by other people, and you in turn will depend on the
  23. work of others. Any decision you take will affect users and colleagues, and
  24. we expect you to take those consequences into account when making decisions.
  25. Even if it's not obvious at the time, our contributions to Ubuntu will impact
  26. the work of others. For example, changes to code, infrastructure, policy,
  27. documentation and translations during a release may negatively impact
  28. others work.
  29. Be respectful.
  30. --------------
  31. The Celery community and its members treat one another with respect. Everyone
  32. can make a valuable contribution to Celery. We may not always agree, but
  33. disagreement is no excuse for poor behavior and poor manners. We might all
  34. experience some frustration now and then, but we cannot allow that frustration
  35. to turn into a personal attack. It's important to remember that a community
  36. where people feel uncomfortable or threatened is not a productive one. We
  37. expect members of the Celery community to be respectful when dealing with
  38. other contributors as well as with people outside the Celery project and with
  39. users of Celery.
  40. Be collaborative.
  41. -----------------
  42. Collaboration is central to Celery and to the larger free software community.
  43. We should always be open to collaboration. Your work should be done
  44. transparently and patches from Celery should be given back to the community
  45. when they are made, not just when the distribution releases. If you wish
  46. to work on new code for existing upstream projects, at least keep those
  47. projects informed of your ideas and progress. It many not be possible to
  48. get consensus from upstream, or even from your colleagues about the correct
  49. implementation for an idea, so don't feel obliged to have that agreement
  50. before you begin, but at least keep the outside world informed of your work,
  51. and publish your work in a way that allows outsiders to test, discuss and
  52. contribute to your efforts.
  53. When you disagree, consult others.
  54. ----------------------------------
  55. Disagreements, both political and technical, happen all the time and
  56. the Celery community is no exception. It is important that we resolve
  57. disagreements and differing views constructively and with the help of the
  58. community and community process. If you really want to go a different
  59. way, then we encourage you to make a derivative distribution or alternate
  60. set of packages that still build on the work we've done to utilize as common
  61. of a core as possible.
  62. When you are unsure, ask for help.
  63. ----------------------------------
  64. Nobody knows everything, and nobody is expected to be perfect. Asking
  65. questions avoids many problems down the road, and so questions are
  66. encouraged. Those who are asked questions should be responsive and helpful.
  67. However, when asking a question, care must be taken to do so in an appropriate
  68. forum.
  69. Step down considerately.
  70. ------------------------
  71. Developers on every project come and go and Celery is no different. When you
  72. leave or disengage from the project, in whole or in part, we ask that you do
  73. so in a way that minimizes disruption to the project. This means you should
  74. tell people you are leaving and take the proper steps to ensure that others
  75. can pick up where you leave off.
  76. .. _reporting-bugs:
  77. Reporting Bugs
  78. ==============
  79. .. _vulnsec:
  80. Security
  81. --------
  82. You must never report security related issues, vulnerabilities or bugs
  83. including senstive information to the bug tracker, or elsewhere in public.
  84. Instead sensitive bugs must be sent by email to security@celeryproject.org.
  85. If you'd like to submit the information encrypted our PGP key is::
  86. -----BEGIN PGP PUBLIC KEY BLOCK-----
  87. Version: GnuPG v1.4.15 (Darwin)
  88. mQENBFJpWDkBCADFIc9/Fpgse4owLNvsTC7GYfnJL19XO0hnL99sPx+DPbfr+cSE
  89. 9wiU+Wp2TfUX7pCLEGrODiEP6ZCZbgtiPgId+JYvMxpP6GXbjiIlHRw1EQNH8RlX
  90. cVxy3rQfVv8PGGiJuyBBjxzvETHW25htVAZ5TI1+CkxmuyyEYqgZN2fNd0wEU19D
  91. +c10G1gSECbCQTCbacLSzdpngAt1Gkrc96r7wGHBBSvDaGDD2pFSkVuTLMbIRrVp
  92. lnKOPMsUijiip2EMr2DvfuXiUIUvaqInTPNWkDynLoh69ib5xC19CSVLONjkKBsr
  93. Pe+qAY29liBatatpXsydY7GIUzyBT3MzgMJlABEBAAG0MUNlbGVyeSBTZWN1cml0
  94. eSBUZWFtIDxzZWN1cml0eUBjZWxlcnlwcm9qZWN0Lm9yZz6JATgEEwECACIFAlJp
  95. WDkCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEOArFOUDCicIw1IH/26f
  96. CViDC7/P13jr+srRdjAsWvQztia9HmTlY8cUnbmkR9w6b6j3F2ayw8VhkyFWgYEJ
  97. wtPBv8mHKADiVSFARS+0yGsfCkia5wDSQuIv6XqRlIrXUyqJbmF4NUFTyCZYoh+C
  98. ZiQpN9xGhFPr5QDlMx2izWg1rvWlG1jY2Es1v/xED3AeCOB1eUGvRe/uJHKjGv7J
  99. rj0pFcptZX+WDF22AN235WYwgJM6TrNfSu8sv8vNAQOVnsKcgsqhuwomSGsOfMQj
  100. LFzIn95MKBBU1G5wOs7JtwiV9jefGqJGBO2FAvOVbvPdK/saSnB+7K36dQcIHqms
  101. 5hU4Xj0RIJiod5idlRC5AQ0EUmlYOQEIAJs8OwHMkrdcvy9kk2HBVbdqhgAREMKy
  102. gmphDp7prRL9FqSY/dKpCbG0u82zyJypdb7QiaQ5pfPzPpQcd2dIcohkkh7G3E+e
  103. hS2L9AXHpwR26/PzMBXyr2iNnNc4vTksHvGVDxzFnRpka6vbI/hrrZmYNYh9EAiv
  104. uhE54b3/XhXwFgHjZXb9i8hgJ3nsO0pRwvUAM1bRGMbvf8e9F+kqgV0yWYNnh6QL
  105. 4Vpl1+epqp2RKPHyNQftbQyrAHXT9kQF9pPlx013MKYaFTADscuAp4T3dy7xmiwS
  106. crqMbZLzfrxfFOsNxTUGE5vmJCcm+mybAtRo4aV6ACohAO9NevMx8pUAEQEAAYkB
  107. HwQYAQIACQUCUmlYOQIbDAAKCRDgKxTlAwonCNFbB/9esir/f7TufE+isNqErzR/
  108. aZKZo2WzZR9c75kbqo6J6DYuUHe6xI0OZ2qZ60iABDEZAiNXGulysFLCiPdatQ8x
  109. 8zt3DF9BMkEck54ZvAjpNSern6zfZb1jPYWZq3TKxlTs/GuCgBAuV4i5vDTZ7xK/
  110. aF+OFY5zN7ciZHkqLgMiTZ+RhqRcK6FhVBP/Y7d9NlBOcDBTxxE1ZO1ute6n7guJ
  111. ciw4hfoRk8qNN19szZuq3UU64zpkM2sBsIFM9tGF2FADRxiOaOWZHmIyVZriPFqW
  112. RUwjSjs7jBVNq0Vy4fCu/5+e+XLOUBOoqtM5W7ELt0t1w9tXebtPEetV86in8fU2
  113. =0chn
  114. -----END PGP PUBLIC KEY BLOCK-----
  115. Other bugs
  116. ----------
  117. Bugs can always be described to the :ref:`mailing-list`, but the best
  118. way to report an issue and to ensure a timely response is to use the
  119. issue tracker.
  120. 1) Create a GitHub account.
  121. You need to `create a GitHub account`_ to be able to create new issues
  122. and participate in the discussion.
  123. .. _`create a GitHub account`: https://github.com/signup/free
  124. 2) Determine if your bug is really a bug.
  125. You should not file a bug if you are requesting support. For that you can use
  126. the :ref:`mailing-list`, or :ref:`irc-channel`.
  127. 3) Make sure your bug hasn't already been reported.
  128. Search through the appropriate Issue tracker. If a bug like yours was found,
  129. check if you have new information that could be reported to help
  130. the developers fix the bug.
  131. 4) Collect information about the bug.
  132. To have the best chance of having a bug fixed, we need to be able to easily
  133. reproduce the conditions that caused it. Most of the time this information
  134. will be from a Python traceback message, though some bugs might be in design,
  135. spelling or other errors on the website/docs/code.
  136. If the error is from a Python traceback, include it in the bug report.
  137. We also need to know what platform you're running (Windows, OSX, Linux, etc),
  138. the version of your Python interpreter, and the version of Celery, and related
  139. packages that you were running when the bug occurred.
  140. 5) Submit the bug.
  141. By default `GitHub`_ will email you to let you know when new comments have
  142. been made on your bug. In the event you've turned this feature off, you
  143. should check back on occasion to ensure you don't miss any questions a
  144. developer trying to fix the bug might ask.
  145. .. _`GitHub`: http://github.com
  146. .. _issue-trackers:
  147. Issue Trackers
  148. --------------
  149. Bugs for a package in the Celery ecosystem should be reported to the relevant
  150. issue tracker.
  151. * Celery: http://github.com/celery/celery/issues/
  152. * Django-Celery: http://github.com/celery/django-celery/issues
  153. * Celery-Pylons: http://bitbucket.org/ianschenck/celery-pylons/issues
  154. * Kombu: http://github.com/celery/kombu/issues
  155. If you are unsure of the origin of the bug you can ask the
  156. :ref:`mailing-list`, or just use the Celery issue tracker.
  157. Contributors guide to the codebase
  158. ==================================
  159. There's a seperate section for internal details,
  160. including details about the codebase and a style guide.
  161. Read :ref:`internals-guide` for more!
  162. .. _versions:
  163. Versions
  164. ========
  165. Version numbers consists of a major version, minor version and a release number.
  166. Since version 2.1.0 we use the versioning semantics described by
  167. semver: http://semver.org.
  168. Stable releases are published at PyPI
  169. while development releases are only available in the GitHub git repository as tags.
  170. All version tags starts with “v”, so version 0.8.0 is the tag v0.8.0.
  171. .. _git-branches:
  172. Branches
  173. ========
  174. Current active version branches:
  175. * master (http://github.com/celery/celery/tree/master)
  176. * 3.1 (http://github.com/celery/celery/tree/3.1)
  177. * 3.0 (http://github.com/celery/celery/tree/3.0)
  178. You can see the state of any branch by looking at the Changelog:
  179. https://github.com/celery/celery/blob/master/Changelog
  180. If the branch is in active development the topmost version info should
  181. contain metadata like::
  182. 2.4.0
  183. ======
  184. :release-date: TBA
  185. :status: DEVELOPMENT
  186. :branch: master
  187. The ``status`` field can be one of:
  188. * ``PLANNING``
  189. The branch is currently experimental and in the planning stage.
  190. * ``DEVELOPMENT``
  191. The branch is in active development, but the test suite should
  192. be passing and the product should be working and possible for users to test.
  193. * ``FROZEN``
  194. The branch is frozen, and no more features will be accepted.
  195. When a branch is frozen the focus is on testing the version as much
  196. as possible before it is released.
  197. ``master`` branch
  198. -----------------
  199. The master branch is where development of the next version happens.
  200. Maintenance branches
  201. --------------------
  202. Maintenance branches are named after the version, e.g. the maintenance branch
  203. for the 2.2.x series is named ``2.2``. Previously these were named
  204. ``releaseXX-maint``.
  205. The versions we currently maintain is:
  206. * 2.3
  207. This is the current series.
  208. * 2.2
  209. This is the previous series, and the last version to support Python 2.4.
  210. * 2.1
  211. This is the last version to use the ``carrot`` AMQP library.
  212. Recent versions use ``kombu``.
  213. Archived branches
  214. -----------------
  215. Archived branches are kept for preserving history only,
  216. and theoretically someone could provide patches for these if they depend
  217. on a series that is no longer officially supported.
  218. An archived version is named ``X.Y-archived``.
  219. Our currently archived branches are:
  220. * 2.1-archived
  221. * 2.0-archived
  222. * 1.0-archived
  223. Feature branches
  224. ----------------
  225. Major new features are worked on in dedicated branches.
  226. There is no strict naming requirement for these branches.
  227. Feature branches are removed once they have been merged into a release branch.
  228. Tags
  229. ====
  230. Tags are used exclusively for tagging releases. A release tag is
  231. named with the format ``vX.Y.Z``, e.g. ``v2.3.1``.
  232. Experimental releases contain an additional identifier ``vX.Y.Z-id``, e.g.
  233. ``v3.0.0-rc1``. Experimental tags may be removed after the official release.
  234. .. _contributing-changes:
  235. Working on Features & Patches
  236. =============================
  237. .. note::
  238. Contributing to Celery should be as simple as possible,
  239. so none of these steps should be considered mandatory.
  240. You can even send in patches by email if that is your preferred
  241. work method. We won't like you any less, any contribution you make
  242. is always appreciated!
  243. However following these steps may make maintainers life easier,
  244. and may mean that your changes will be accepted sooner.
  245. Forking and setting up the repository
  246. -------------------------------------
  247. First you need to fork the Celery repository, a good introduction to this
  248. is in the Github Guide: `Fork a Repo`_.
  249. After you have cloned the repository you should checkout your copy
  250. to a directory on your machine:
  251. .. code-block:: bash
  252. $ git clone git@github.com:username/celery.git
  253. When the repository is cloned enter the directory to set up easy access
  254. to upstream changes:
  255. .. code-block:: bash
  256. $ cd celery
  257. .. code-block:: bash
  258. $ git remote add upstream git://github.com/celery/celery.git
  259. .. code-block:: bash
  260. $ git fetch upstream
  261. If you need to pull in new changes from upstream you should
  262. always use the :option:`--rebase` option to ``git pull``:
  263. .. code-block:: bash
  264. git pull --rebase upstream master
  265. With this option you don't clutter the history with merging
  266. commit notes. See `Rebasing merge commits in git`_.
  267. If you want to learn more about rebasing see the `Rebase`_
  268. section in the Github guides.
  269. If you need to work on a different branch than ``master`` you can
  270. fetch and checkout a remote branch like this::
  271. git checkout --track -b 3.0-devel origin/3.0-devel
  272. For a list of branches see :ref:`git-branches`.
  273. .. _`Fork a Repo`: http://help.github.com/fork-a-repo/
  274. .. _`Rebasing merge commits in git`:
  275. http://notes.envato.com/developers/rebasing-merge-commits-in-git/
  276. .. _`Rebase`: http://help.github.com/rebase/
  277. .. _contributing-testing:
  278. Running the unit test suite
  279. ---------------------------
  280. To run the Celery test suite you need to install a few dependencies.
  281. A complete list of the dependencies needed are located in
  282. :file:`requirements/test.txt`.
  283. Installing the test requirements:
  284. .. code-block:: bash
  285. $ pip install -U -r requirements/test.txt
  286. When installation of dependencies is complete you can execute
  287. the test suite by calling ``nosetests``:
  288. .. code-block:: bash
  289. $ nosetests
  290. Some useful options to :program:`nosetests` are:
  291. * :option:`-x`
  292. Stop running the tests at the first test that fails.
  293. * :option:`-s`
  294. Don't capture output
  295. * :option:`--nologcapture`
  296. Don't capture log output.
  297. * :option:`-v`
  298. Run with verbose output.
  299. If you want to run the tests for a single test file only
  300. you can do so like this:
  301. .. code-block:: bash
  302. $ nosetests celery.tests.test_worker.test_worker_job
  303. .. _contributing-pull-requests:
  304. Creating pull requests
  305. ----------------------
  306. When your feature/bugfix is complete you may want to submit
  307. a pull requests so that it can be reviewed by the maintainers.
  308. Creating pull requests is easy, and also let you track the progress
  309. of your contribution. Read the `Pull Requests`_ section in the Github
  310. Guide to learn how this is done.
  311. You can also attach pull requests to existing issues by following
  312. the steps outlined here: http://bit.ly/koJoso
  313. .. _`Pull Requests`: http://help.github.com/send-pull-requests/
  314. .. _contributing-coverage:
  315. Calculating test coverage
  316. ~~~~~~~~~~~~~~~~~~~~~~~~~
  317. Code coverage in HTML:
  318. .. code-block:: bash
  319. $ nosetests --with-coverage3 --cover3-html
  320. The coverage output will then be located at
  321. :file:`celery/tests/cover/index.html`.
  322. Code coverage in XML (Cobertura-style):
  323. .. code-block:: bash
  324. $ nosetests --with-coverage3 --cover3-xml --cover3-xml-file=coverage.xml
  325. The coverage XML output will then be located at :file:`coverage.xml`
  326. .. _contributing-tox:
  327. Running the tests on all supported Python versions
  328. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  329. There is a ``tox`` configuration file in the top directory of the
  330. distribution.
  331. To run the tests for all supported Python versions simply execute:
  332. .. code-block:: bash
  333. $ tox
  334. If you only want to test specific Python versions use the :option:`-e`
  335. option:
  336. .. code-block:: bash
  337. $ tox -e py26
  338. Building the documentation
  339. --------------------------
  340. To build the documentation you need to install the dependencies
  341. listed in :file:`requirements/docs.txt`:
  342. .. code-block:: bash
  343. $ pip install -U -r requirements/docs.txt
  344. After these dependencies are installed you should be able to
  345. build the docs by running:
  346. .. code-block:: bash
  347. $ cd docs
  348. $ rm -rf .build
  349. $ make html
  350. Make sure there are no errors or warnings in the build output.
  351. After building succeeds the documentation is available at :file:`.build/html`.
  352. .. _contributing-verify:
  353. Verifying your contribution
  354. ---------------------------
  355. To use these tools you need to install a few dependencies. These dependencies
  356. can be found in :file:`requirements/pkgutils.txt`.
  357. Installing the dependencies:
  358. .. code-block:: bash
  359. $ pip install -U -r requirements/pkgutils.txt
  360. pyflakes & PEP8
  361. ~~~~~~~~~~~~~~~
  362. To ensure that your changes conform to PEP8 and to run pyflakes
  363. execute:
  364. .. code-block:: bash
  365. $ paver flake8
  366. To not return a negative exit code when this command fails use the
  367. :option:`-E` option, this can be convenient while developing:
  368. .. code-block:: bash
  369. $ paver flake8 -E
  370. API reference
  371. ~~~~~~~~~~~~~
  372. To make sure that all modules have a corresponding section in the API
  373. reference please execute:
  374. .. code-block:: bash
  375. $ paver autodoc
  376. $ paver verifyindex
  377. If files are missing you can add them by copying an existing reference file.
  378. If the module is internal it should be part of the internal reference
  379. located in :file:`docs/internals/reference/`. If the module is public
  380. it should be located in :file:`docs/reference/`.
  381. For example if reference is missing for the module ``celery.worker.awesome``
  382. and this module is considered part of the public API, use the following steps:
  383. .. code-block:: bash
  384. $ cd docs/reference/
  385. $ cp celery.schedules.rst celery.worker.awesome.rst
  386. .. code-block:: bash
  387. $ vim celery.worker.awesome.rst
  388. # change every occurance of ``celery.schedules`` to
  389. # ``celery.worker.awesome``
  390. .. code-block:: bash
  391. $ vim index.rst
  392. # Add ``celery.worker.awesome`` to the index.
  393. .. code-block:: bash
  394. # Add the file to git
  395. $ git add celery.worker.awesome.rst
  396. $ git add index.rst
  397. $ git commit celery.worker.awesome.rst index.rst \
  398. -m "Adds reference for celery.worker.awesome"
  399. .. _coding-style:
  400. Coding Style
  401. ============
  402. You should probably be able to pick up the coding style
  403. from surrounding code, but it is a good idea to be aware of the
  404. following conventions.
  405. * All Python code must follow the `PEP-8`_ guidelines.
  406. `pep8.py`_ is an utility you can use to verify that your code
  407. is following the conventions.
  408. .. _`PEP-8`: http://www.python.org/dev/peps/pep-0008/
  409. .. _`pep8.py`: http://pypi.python.org/pypi/pep8
  410. * Docstrings must follow the `PEP-257`_ conventions, and use the following
  411. style.
  412. Do this:
  413. .. code-block:: python
  414. def method(self, arg):
  415. """Short description.
  416. More details.
  417. """
  418. or:
  419. .. code-block:: python
  420. def method(self, arg):
  421. """Short description."""
  422. but not this:
  423. .. code-block:: python
  424. def method(self, arg):
  425. """
  426. Short description.
  427. """
  428. .. _`PEP-257`: http://www.python.org/dev/peps/pep-0257/
  429. * Lines should not exceed 78 columns.
  430. You can enforce this in :program:`vim` by setting the ``textwidth`` option:
  431. .. code-block:: vim
  432. set textwidth=78
  433. If adhering to this limit makes the code less readable, you have one more
  434. character to go on, which means 78 is a soft limit, and 79 is the hard
  435. limit :)
  436. * Import order
  437. * Python standard library (`import xxx`)
  438. * Python standard library ('from xxx import`)
  439. * Third party packages.
  440. * Other modules from the current package.
  441. or in case of code using Django:
  442. * Python standard library (`import xxx`)
  443. * Python standard library ('from xxx import`)
  444. * Third party packages.
  445. * Django packages.
  446. * Other modules from the current package.
  447. Within these sections the imports should be sorted by module name.
  448. Example:
  449. .. code-block:: python
  450. import threading
  451. import time
  452. from collections import deque
  453. from Queue import Queue, Empty
  454. from .datastructures import TokenBucket
  455. from .utils import timeutils
  456. from .utils.compat import all, izip_longest, chain_from_iterable
  457. * Wildcard imports must not be used (`from xxx import *`).
  458. * For distributions where Python 2.5 is the oldest support version
  459. additional rules apply:
  460. * Absolute imports must be enabled at the top of every module::
  461. from __future__ import absolute_import
  462. * If the module uses the with statement and must be compatible
  463. with Python 2.5 (celery is not) then it must also enable that::
  464. from __future__ import with_statement
  465. * Every future import must be on its own line, as older Python 2.5
  466. releases did not support importing multiple features on the
  467. same future import line::
  468. # Good
  469. from __future__ import absolute_import
  470. from __future__ import with_statement
  471. # Bad
  472. from __future__ import absolute_import, with_statement
  473. (Note that this rule does not apply if the package does not include
  474. support for Python 2.5)
  475. * Note that we use "new-style` relative imports when the distribution
  476. does not support Python versions below 2.5
  477. .. code-block:: python
  478. from . import submodule
  479. .. _feature-with-extras:
  480. Contributing features requiring additional libraries
  481. ====================================================
  482. Some features like a new result backend may require additional libraries
  483. that the user must install.
  484. We use setuptools `extra_requires` for this, and all new optional features
  485. that require 3rd party libraries must be added.
  486. 1) Add a new requirements file in `requirements/extras`
  487. E.g. for the Cassandra backend this is
  488. :file:`requirements/extras/cassandra.txt`, and the file looks like this::
  489. pycassa
  490. These are pip requirement files so you can have version specifiers and
  491. multiple packages are separated by newline. A more complex example could
  492. be:
  493. # pycassa 2.0 breaks Foo
  494. pycassa>=1.0,<2.0
  495. thrift
  496. 2) Modify ``setup.py``
  497. After the requirements file is added you need to add it as an option
  498. to ``setup.py`` in the ``extras_require`` section::
  499. extra['extras_require'] = {
  500. # ...
  501. 'cassandra': extras('cassandra.txt'),
  502. }
  503. 3) Document the new feature in ``docs/includes/installation.txt``
  504. You must add your feature to the list in the :ref:`bundles` section
  505. of :file:`docs/includes/installation.txt`.
  506. After you've made changes to this file you need to render
  507. the distro :file:`README` file:
  508. .. code-block:: bash
  509. $ pip install -U requirements/pkgutils.txt
  510. $ paver readme
  511. That's all that needs to be done, but remember that if your feature
  512. adds additional configuration options then these needs to be documented
  513. in ``docs/configuration.rst``. Also all settings need to be added to the
  514. ``celery/app/defaults.py`` module.
  515. Result backends require a separate section in the ``docs/configuration.rst``
  516. file.
  517. .. _contact_information:
  518. Contacts
  519. ========
  520. This is a list of people that can be contacted for questions
  521. regarding the official git repositories, PyPI packages
  522. Read the Docs pages.
  523. If the issue is not an emergency then it is better
  524. to :ref:`report an issue <reporting-bugs>`.
  525. Committers
  526. ----------
  527. Ask Solem
  528. ~~~~~~~~~
  529. :github: https://github.com/ask
  530. :twitter: http://twitter.com/#!/asksol
  531. Mher Movsisyan
  532. ~~~~~~~~~~~~~~
  533. :github: https://github.com/mher
  534. :twitter: http://twitter.com/#!/movsm
  535. Steeve Morin
  536. ~~~~~~~~~~~~
  537. :github: https://github.com/steeve
  538. :twitter: http://twitter.com/#!/steeve
  539. Website
  540. -------
  541. The Celery Project website is run and maintained by
  542. Mauro Rocco
  543. ~~~~~~~~~~~
  544. :github: https://github.com/fireantology
  545. :twitter: https://twitter.com/#!/fireantology
  546. with design by:
  547. Jan Henrik Helmers
  548. ~~~~~~~~~~~~~~~~~~
  549. :web: http://www.helmersworks.com
  550. :twitter: http://twitter.com/#!/helmers
  551. .. _packages:
  552. Packages
  553. ========
  554. celery
  555. ------
  556. :git: https://github.com/celery/celery
  557. :CI: http://travis-ci.org/#!/celery/celery
  558. :PyPI: http://pypi.python.org/pypi/celery
  559. :docs: http://docs.celeryproject.org
  560. kombu
  561. -----
  562. Messaging library.
  563. :git: https://github.com/celery/kombu
  564. :CI: http://travis-ci.org/#!/celery/kombu
  565. :PyPI: http://pypi.python.org/pypi/kombu
  566. :docs: http://kombu.readthedocs.org
  567. billiard
  568. --------
  569. Fork of multiprocessing containing improvements
  570. that will eventually be merged into the Python stdlib.
  571. :git: https://github.com/celery/billiard
  572. :PyPI: http://pypi.python.org/pypi/billiard
  573. librabbitmq
  574. -----------
  575. Very fast Python AMQP client written in C.
  576. :git: https://github.com/celery/librabbitmq
  577. :PyPI: http://pypi.python.org/pypi/librabbitmq
  578. celerymon
  579. ---------
  580. Celery monitor web-service.
  581. :git: https://github.com/celery/celerymon
  582. :PyPI: http://pypi.python.org/pypi/celerymon
  583. django-celery
  584. -------------
  585. Django <-> Celery Integration.
  586. :git: https://github.com/celery/django-celery
  587. :PyPI: http://pypi.python.org/pypi/django-celery
  588. :docs: http://docs.celeryproject.org/en/latest/django
  589. cl
  590. --
  591. Actor library.
  592. :git: https://github.com/celery/cl
  593. :PyPI: http://pypi.python.org/pypi/cl
  594. cyme
  595. ----
  596. Distributed Celery Instance manager.
  597. :git: https://github.com/celery/cyme
  598. :PyPI: http://pypi.python.org/pypi/cyme
  599. :docs: http://cyme.readthedocs.org/
  600. Deprecated
  601. ----------
  602. - Flask-Celery
  603. :git: https://github.com/ask/Flask-Celery
  604. :PyPI: http://pypi.python.org/pypi/Flask-Celery
  605. - carrot
  606. :git: https://github.com/ask/carrot
  607. :PyPI: http://pypi.python.org/pypi/carrot
  608. - ghettoq
  609. :git: https://github.com/ask/ghettoq
  610. :PyPI: http://pypi.python.org/pypi/ghettoq
  611. - kombu-sqlalchemy
  612. :git: https://github.com/ask/kombu-sqlalchemy
  613. :PyPI: http://pypi.python.org/pypi/kombu-sqlalchemy
  614. - django-kombu
  615. :git: https://github.com/ask/django-kombu
  616. :PyPI: http://pypi.python.org/pypi/django-kombu
  617. - pylibrabbitmq
  618. Old name for :mod:`librabbitmq`.
  619. :git: :const:`None`
  620. :PyPI: http://pypi.python.org/pypi/pylibrabbitmq
  621. .. _release-procedure:
  622. Release Procedure
  623. =================
  624. Updating the version number
  625. ---------------------------
  626. The version number must be updated two places:
  627. * :file:`celery/__init__.py`
  628. * :file:`docs/include/introduction.txt`
  629. After you have changed these files you must render
  630. the :file:`README` files. There is a script to convert sphinx syntax
  631. to generic reStructured Text syntax, and the paver task `readme`
  632. does this for you:
  633. .. code-block:: bash
  634. $ paver readme
  635. Now commit the changes:
  636. .. code-block:: bash
  637. $ git commit -a -m "Bumps version to X.Y.Z"
  638. and make a new version tag:
  639. .. code-block:: bash
  640. $ git tag vX.Y.Z
  641. $ git push --tags
  642. Releasing
  643. ---------
  644. Commands to make a new public stable release::
  645. $ paver releaseok # checks pep8, autodoc index, runs tests and more
  646. $ paver removepyc # Remove .pyc files
  647. $ git clean -xdn # Check that there's no left-over files in the repo
  648. $ python setup.py sdist upload # Upload package to PyPI
  649. If this is a new release series then you also need to do the
  650. following:
  651. * Go to the Read The Docs management interface at:
  652. http://readthedocs.org/projects/celery/?fromdocs=celery
  653. * Enter "Edit project"
  654. Change default branch to the branch of this series, e.g. ``2.4``
  655. for series 2.4.
  656. * Also add the previous version under the "versions" tab.