.. _contributing: ============== Contributing ============== Welcome! This document is fairly extensive and you aren't really expected to study this in detail for small contributions; The most important rule is that contributing must be easy and that the community is friendly and not nitpicking on details, such as coding style. If you're reporting a bug you should read the Reporting bugs section below to ensure that your bug report contains enough information to successfully diagnose the issue, and if you're contributing code you should try to mimic the conventions you see surrounding the code you're working on, but in the end all patches will be cleaned up by the person merging the changes so don't worry too much. .. contents:: :local: .. _community-code-of-conduct: Community Code of Conduct ========================= The goal is to maintain a diverse community that's pleasant for everyone. That's why we would greatly appreciate it if everyone contributing to and interacting with the community also followed this Code of Conduct. The Code of Conduct covers our behavior as members of the community, in any forum, mailing list, wiki, website, Internet relay chat (IRC), public meeting or private correspondence. The Code of Conduct is heavily based on the `Ubuntu Code of Conduct`_, and the `Pylons Code of Conduct`_. .. _`Ubuntu Code of Conduct`: http://www.ubuntu.com/community/conduct .. _`Pylons Code of Conduct`: http://docs.pylonshq.com/community/conduct.html Be considerate -------------- Your work will be used by other people, and you in turn will depend on the work of others. Any decision you take will affect users and colleagues, and we expect you to take those consequences into account when making decisions. Even if it's not obvious at the time, our contributions to Celery will impact the work of others. For example, changes to code, infrastructure, policy, documentation and translations during a release may negatively impact others work. Be respectful ------------- The Celery community and its members treat one another with respect. Everyone can make a valuable contribution to Celery. We may not always agree, but disagreement is no excuse for poor behavior and poor manners. We might all experience some frustration now and then, but we cannot allow that frustration to turn into a personal attack. It's important to remember that a community where people feel uncomfortable or threatened isn't a productive one. We expect members of the Celery community to be respectful when dealing with other contributors as well as with people outside the Celery project and with users of Celery. Be collaborative ---------------- Collaboration is central to Celery and to the larger free software community. We should always be open to collaboration. Your work should be done transparently and patches from Celery should be given back to the community when they're made, not just when the distribution releases. If you wish to work on new code for existing upstream projects, at least keep those projects informed of your ideas and progress. It many not be possible to get consensus from upstream, or even from your colleagues about the correct implementation for an idea, so don't feel obliged to have that agreement before you begin, but at least keep the outside world informed of your work, and publish your work in a way that allows outsiders to test, discuss, and contribute to your efforts. When you disagree, consult others --------------------------------- Disagreements, both political and technical, happen all the time and the Celery community is no exception. It's important that we resolve disagreements and differing views constructively and with the help of the community and community process. If you really want to go a different way, then we encourage you to make a derivative distribution or alternate set of packages that still build on the work we've done to utilize as common of a core as possible. When you're unsure, ask for help -------------------------------- Nobody knows everything, and nobody is expected to be perfect. Asking questions avoids many problems down the road, and so questions are encouraged. Those who are asked questions should be responsive and helpful. However, when asking a question, care must be taken to do so in an appropriate forum. Step down considerately ----------------------- Developers on every project come and go and Celery is no different. When you leave or disengage from the project, in whole or in part, we ask that you do so in a way that minimizes disruption to the project. This means you should tell people you're leaving and take the proper steps to ensure that others can pick up where you leave off. .. _reporting-bugs: Reporting Bugs ============== .. _vulnsec: Security -------- You must never report security related issues, vulnerabilities or bugs including sensitive information to the bug tracker, or elsewhere in public. Instead sensitive bugs must be sent by email to ``security@celeryproject.org``. If you'd like to submit the information encrypted our PGP key is:: -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v1.4.15 (Darwin) mQENBFJpWDkBCADFIc9/Fpgse4owLNvsTC7GYfnJL19XO0hnL99sPx+DPbfr+cSE 9wiU+Wp2TfUX7pCLEGrODiEP6ZCZbgtiPgId+JYvMxpP6GXbjiIlHRw1EQNH8RlX cVxy3rQfVv8PGGiJuyBBjxzvETHW25htVAZ5TI1+CkxmuyyEYqgZN2fNd0wEU19D +c10G1gSECbCQTCbacLSzdpngAt1Gkrc96r7wGHBBSvDaGDD2pFSkVuTLMbIRrVp lnKOPMsUijiip2EMr2DvfuXiUIUvaqInTPNWkDynLoh69ib5xC19CSVLONjkKBsr Pe+qAY29liBatatpXsydY7GIUzyBT3MzgMJlABEBAAG0MUNlbGVyeSBTZWN1cml0 eSBUZWFtIDxzZWN1cml0eUBjZWxlcnlwcm9qZWN0Lm9yZz6JATgEEwECACIFAlJp WDkCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEOArFOUDCicIw1IH/26f CViDC7/P13jr+srRdjAsWvQztia9HmTlY8cUnbmkR9w6b6j3F2ayw8VhkyFWgYEJ wtPBv8mHKADiVSFARS+0yGsfCkia5wDSQuIv6XqRlIrXUyqJbmF4NUFTyCZYoh+C ZiQpN9xGhFPr5QDlMx2izWg1rvWlG1jY2Es1v/xED3AeCOB1eUGvRe/uJHKjGv7J rj0pFcptZX+WDF22AN235WYwgJM6TrNfSu8sv8vNAQOVnsKcgsqhuwomSGsOfMQj LFzIn95MKBBU1G5wOs7JtwiV9jefGqJGBO2FAvOVbvPdK/saSnB+7K36dQcIHqms 5hU4Xj0RIJiod5idlRC5AQ0EUmlYOQEIAJs8OwHMkrdcvy9kk2HBVbdqhgAREMKy gmphDp7prRL9FqSY/dKpCbG0u82zyJypdb7QiaQ5pfPzPpQcd2dIcohkkh7G3E+e hS2L9AXHpwR26/PzMBXyr2iNnNc4vTksHvGVDxzFnRpka6vbI/hrrZmYNYh9EAiv uhE54b3/XhXwFgHjZXb9i8hgJ3nsO0pRwvUAM1bRGMbvf8e9F+kqgV0yWYNnh6QL 4Vpl1+epqp2RKPHyNQftbQyrAHXT9kQF9pPlx013MKYaFTADscuAp4T3dy7xmiwS crqMbZLzfrxfFOsNxTUGE5vmJCcm+mybAtRo4aV6ACohAO9NevMx8pUAEQEAAYkB HwQYAQIACQUCUmlYOQIbDAAKCRDgKxTlAwonCNFbB/9esir/f7TufE+isNqErzR/ aZKZo2WzZR9c75kbqo6J6DYuUHe6xI0OZ2qZ60iABDEZAiNXGulysFLCiPdatQ8x 8zt3DF9BMkEck54ZvAjpNSern6zfZb1jPYWZq3TKxlTs/GuCgBAuV4i5vDTZ7xK/ aF+OFY5zN7ciZHkqLgMiTZ+RhqRcK6FhVBP/Y7d9NlBOcDBTxxE1ZO1ute6n7guJ ciw4hfoRk8qNN19szZuq3UU64zpkM2sBsIFM9tGF2FADRxiOaOWZHmIyVZriPFqW RUwjSjs7jBVNq0Vy4fCu/5+e+XLOUBOoqtM5W7ELt0t1w9tXebtPEetV86in8fU2 =0chn -----END PGP PUBLIC KEY BLOCK----- Other bugs ---------- Bugs can always be described to the `mailing-list`_, but the best way to report an issue and to ensure a timely response is to use the issue tracker. 1) **Create a GitHub account**. You need to `create a GitHub account`_ to be able to create new issues and participate in the discussion. .. _`create a GitHub account`: https://github.com/signup/free 2) **Determine if your bug is really a bug**. You shouldn't file a bug if you're requesting support. For that you can use the `mailing-list`_, or `irc-channel`_. 3) **Make sure your bug hasn't already been reported**. Search through the appropriate Issue tracker. If a bug like yours was found, check if you have new information that could be reported to help the developers fix the bug. 4) **Check if you're using the latest version**. A bug could be fixed by some other improvements and fixes - it might not have an existing report in the bug tracker. Make sure you're using the latest releases of celery, billiard, kombu, amqp, and vine. 5) **Collect information about the bug**. To have the best chance of having a bug fixed, we need to be able to easily reproduce the conditions that caused it. Most of the time this information will be from a Python traceback message, though some bugs might be in design, spelling or other errors on the website/docs/code. A) If the error is from a Python traceback, include it in the bug report. B) We also need to know what platform you're running (Windows, macOS, Linux, etc.), the version of your Python interpreter, and the version of Celery, and related packages that you were running when the bug occurred. C) If you're reporting a race condition or a deadlock, tracebacks can be hard to get or might not be that useful. Try to inspect the process to get more diagnostic data. Some ideas: * Enable Celery's ``breakpoint_signal`` and use it to inspect the process's state. This will allow you to open a ``pdb`` session. * Collect tracing data using `strace`_(Linux), ``dtruss`` (macOS), and ``ktrace`` (BSD), `ltrace`_, and `lsof`_. D) Include the output from the ``celery report`` command: :: $ celery -A proj report This will also include your configuration settings and it try to remove values for keys known to be sensitive, but make sure you also verify the information before submitting so that it doesn't contain confidential information like API tokens and authentication credentials. 6) **Submit the bug**. By default `GitHub`_ will email you to let you know when new comments have been made on your bug. In the event you've turned this feature off, you should check back on occasion to ensure you don't miss any questions a developer trying to fix the bug might ask. .. _`GitHub`: https://github.com .. _`strace`: https://en.wikipedia.org/wiki/Strace .. _`ltrace`: https://en.wikipedia.org/wiki/Ltrace .. _`lsof`: https://en.wikipedia.org/wiki/Lsof .. _issue-trackers: Issue Trackers -------------- Bugs for a package in the Celery ecosystem should be reported to the relevant issue tracker. * ``celery``: https://github.com/celery/celery/issues/ * ``kombu``: https://github.com/celery/kombu/issues * ``amqp``: https://github.com/celery/py-amqp/issues * ``vine``: https://github.com/celery/vine/issues * ``librabbitmq``: https://github.com/celery/librabbitmq/issues * ``django-celery-beat``: https://github.com/celery/django-celery-beat/issues * ``django-celery-results``: https://github.com/celery/django-celery-results/issues If you're unsure of the origin of the bug you can ask the `mailing-list`_, or just use the Celery issue tracker. Contributors guide to the code base =================================== There's a separate section for internal details, including details about the code base and a style guide. Read `internals-guide`_ for more! .. _versions: Versions ======== Version numbers consists of a major version, minor version and a release number. Since version 2.1.0 we use the versioning semantics described by SemVer: http://semver.org. Stable releases are published at PyPI while development releases are only available in the GitHub git repository as tags. All version tags starts with “v”, so version 0.8.0 is the tag v0.8.0. .. _git-branches: Branches ======== Current active version branches: * dev (which git calls "master") (https://github.com/celery/celery/tree/master) * 3.1 (https://github.com/celery/celery/tree/3.1) * 3.0 (https://github.com/celery/celery/tree/3.0) You can see the state of any branch by looking at the Changelog: https://github.com/celery/celery/blob/master/Changelog If the branch is in active development the topmost version info should contain meta-data like: :: 2.4.0 ====== :release-date: TBA :status: DEVELOPMENT :branch: dev (git calls this master) The ``status`` field can be one of: * ``PLANNING`` The branch is currently experimental and in the planning stage. * ``DEVELOPMENT`` The branch is in active development, but the test suite should be passing and the product should be working and possible for users to test. * ``FROZEN`` The branch is frozen, and no more features will be accepted. When a branch is frozen the focus is on testing the version as much as possible before it is released. dev branch ---------- The dev branch (called "master" by git), is where development of the next version happens. Maintenance branches -------------------- Maintenance branches are named after the version -- for example, the maintenance branch for the 2.2.x series is named ``2.2``. Previously these were named ``releaseXX-maint``. The versions we currently maintain is: * 3.1 This is the current series. * 3.0 This is the previous series, and the last version to support Python 2.5. Archived branches ----------------- Archived branches are kept for preserving history only, and theoretically someone could provide patches for these if they depend on a series that's no longer officially supported. An archived version is named ``X.Y-archived``. Our currently archived branches are: * ``2.5-archived`` * ``2.4-archived`` * ``2.3-archived`` * ``2.1-archived`` * ``2.0-archived`` * ``1.0-archived`` Feature branches ---------------- Major new features are worked on in dedicated branches. There's no strict naming requirement for these branches. Feature branches are removed once they've been merged into a release branch. Tags ==== - Tags are used exclusively for tagging releases. A release tag is named with the format ``vX.Y.Z`` -- for example ``v2.3.1``. - Experimental releases contain an additional identifier ``vX.Y.Z-id`` -- for example ``v3.0.0-rc1``. - Experimental tags may be removed after the official release. .. _contributing-changes: Working on Features & Patches ============================= .. note:: Contributing to Celery should be as simple as possible, so none of these steps should be considered mandatory. You can even send in patches by email if that's your preferred work method. We won't like you any less, any contribution you make is always appreciated! However following these steps may make maintainers life easier, and may mean that your changes will be accepted sooner. Forking and setting up the repository ------------------------------------- First you need to fork the Celery repository, a good introduction to this is in the GitHub Guide: `Fork a Repo`_. After you have cloned the repository you should checkout your copy to a directory on your machine: :: $ git clone git@github.com:username/celery.git When the repository is cloned enter the directory to set up easy access to upstream changes: :: $ cd celery $ git remote add upstream git://github.com/celery/celery.git $ git fetch upstream If you need to pull in new changes from upstream you should always use the ``--rebase`` option to ``git pull``: :: git pull --rebase upstream master With this option you don't clutter the history with merging commit notes. See `Rebasing merge commits in git`_. If you want to learn more about rebasing see the `Rebase`_ section in the GitHub guides. If you need to work on a different branch than the one git calls ``master``, you can fetch and checkout a remote branch like this:: git checkout --track -b 3.0-devel origin/3.0-devel .. _`Fork a Repo`: http://help.github.com/fork-a-repo/ .. _`Rebasing merge commits in git`: http://notes.envato.com/developers/rebasing-merge-commits-in-git/ .. _`Rebase`: http://help.github.com/rebase/ .. _contributing-testing: Running the unit test suite --------------------------- To run the Celery test suite you need to install a few dependencies. A complete list of the dependencies needed are located in ``requirements/test.txt``. If you're working on the development version, then you need to install the development requirements first: :: $ pip install -U -r requirements/dev.txt THIS REQUIREMENT FILE MAY NOT BE PRESENT, SKIP IF NOT FOUND. Both the stable and the development version have testing related dependencies, so install these next: :: $ pip install -U -r requirements/test.txt $ pip install -U -r requirements/default.txt After installing the dependencies required, you can now execute the test suite by calling ``py.test =1.0,<2.0 thrift 2) Modify ``setup.py`` After the requirements file is added you need to add it as an option to ``setup.py`` in the ``extras_require`` section:: extra['extras_require'] = { # ... 'cassandra': extras('cassandra.txt'), } 3) Document the new feature in ``docs/includes/installation.txt`` You must add your feature to the list in the `bundles`_ section of ``docs/includes/installation.txt``. After you've made changes to this file you need to render the distro ``README`` file: :: $ pip install -U requirements/pkgutils.txt $ make readme That's all that needs to be done, but remember that if your feature adds additional configuration options then these needs to be documented in ``docs/configuration.rst``. Also all settings need to be added to the ``celery/app/defaults.py`` module. Result backends require a separate section in the ``docs/configuration.rst`` file. .. _contact_information: Contacts ======== This is a list of people that can be contacted for questions regarding the official git repositories, PyPI packages Read the Docs pages. If the issue isn't an emergency then it's better to `report an issue`_. Committers ---------- Ask Solem ~~~~~~~~~ :github: https://github.com/ask :twitter: http://twitter.com/#!/asksol Asif Saif Uddin ~~~~~~~~~~~~~~~ :github: https://github.com/auvipy :twitter: https://twitter.com/#!/auvipy Dmitry Malinovsky ~~~~~~~~~~~~~~~~~ :github: https://github.com/malinoff :twitter: https://twitter.com/__malinoff__ Ionel Cristian Mărieș ~~~~~~~~~~~~~~~~~~~~~ :github: https://github.com/ionelmc :twitter: https://twitter.com/ionelmc Mher Movsisyan ~~~~~~~~~~~~~~ :github: https://github.com/mher :twitter: http://twitter.com/#!/movsm Omer Katz ~~~~~~~~~ :github: https://github.com/thedrow :twitter: https://twitter.com/the_drow Steeve Morin ~~~~~~~~~~~~ :github: https://github.com/steeve :twitter: http://twitter.com/#!/steeve Website ------- The Celery Project website is run and maintained by Mauro Rocco ~~~~~~~~~~~ :github: https://github.com/fireantology :twitter: https://twitter.com/#!/fireantology with design by: Jan Henrik Helmers ~~~~~~~~~~~~~~~~~~ :web: http://www.helmersworks.com :twitter: http://twitter.com/#!/helmers .. _packages: Packages ======== ``celery`` ---------- :git: https://github.com/celery/celery :CI: http://travis-ci.org/#!/celery/celery :Windows-CI: https://ci.appveyor.com/project/ask/celery :PyPI: ``celery`` :docs: http://docs.celeryproject.org ``kombu`` --------- Messaging library. :git: https://github.com/celery/kombu :CI: http://travis-ci.org/#!/celery/kombu :Windows-CI: https://ci.appveyor.com/project/ask/kombu :PyPI: ``kombu`` :docs: https://kombu.readthedocs.io ``amqp`` -------- Python AMQP 0.9.1 client. :git: https://github.com/celery/py-amqp :CI: http://travis-ci.org/#!/celery/py-amqp :Windows-CI: https://ci.appveyor.com/project/ask/py-amqp :PyPI: ``amqp`` :docs: https://amqp.readthedocs.io ``vine`` -------- Promise/deferred implementation. :git: https://github.com/celery/vine/ :CI: http://travis-ci.org/#!/celery/vine/ :Windows-CI: https://ci.appveyor.com/project/ask/vine :PyPI: ``vine`` :docs: https://vine.readthedocs.io ``billiard`` ------------ Fork of multiprocessing containing improvements that'll eventually be merged into the Python stdlib. :git: https://github.com/celery/billiard :CI: http://travis-ci.org/#!/celery/billiard/ :Windows-CI: https://ci.appveyor.com/project/ask/billiard :PyPI: ``billiard`` ``django-celery-beat`` ---------------------- Database-backed Periodic Tasks with admin interface using the Django ORM. :git: https://github.com/celery/django-celery-beat :CI: http://travis-ci.org/#!/celery/django-celery-beat :Windows-CI: https://ci.appveyor.com/project/ask/django-celery-beat :PyPI: ``django-celery-beat`` ``django-celery-results`` ------------------------- Store task results in the Django ORM, or using the Django Cache Framework. :git: https://github.com/celery/django-celery-results :CI: http://travis-ci.org/#!/celery/django-celery-results :Windows-CI: https://ci.appveyor.com/project/ask/django-celery-results :PyPI: ``django-celery-results`` ``librabbitmq`` --------------- Very fast Python AMQP client written in C. :git: https://github.com/celery/librabbitmq :PyPI: ``librabbitmq`` ``cell`` -------- Actor library. :git: https://github.com/celery/cell :PyPI: ``cell`` ``cyme`` -------- Distributed Celery Instance manager. :git: https://github.com/celery/cyme :PyPI: ``cyme`` :docs: https://cyme.readthedocs.io/ Deprecated ---------- - ``django-celery`` :git: https://github.com/celery/django-celery :PyPI: ``django-celery`` :docs: http://docs.celeryproject.org/en/latest/django - ``Flask-Celery`` :git: https://github.com/ask/Flask-Celery :PyPI: ``Flask-Celery`` - ``celerymon`` :git: https://github.com/celery/celerymon :PyPI: ``celerymon`` - ``carrot`` :git: https://github.com/ask/carrot :PyPI: ``carrot`` - ``ghettoq`` :git: https://github.com/ask/ghettoq :PyPI: ``ghettoq`` - ``kombu-sqlalchemy`` :git: https://github.com/ask/kombu-sqlalchemy :PyPI: ``kombu-sqlalchemy`` - ``django-kombu`` :git: https://github.com/ask/django-kombu :PyPI: ``django-kombu`` - ``pylibrabbitmq`` Old name for ``librabbitmq``. :git: ``None`` :PyPI: ``pylibrabbitmq`` .. _release-procedure: Release Procedure ================= Updating the version number --------------------------- The version number must be updated two places: * ``celery/__init__.py`` * ``docs/include/introduction.txt`` After you have changed these files you must render the ``README`` files. There's a script to convert sphinx syntax to generic reStructured Text syntax, and the make target `readme` does this for you: :: $ make readme Now commit the changes: :: $ git commit -a -m "Bumps version to X.Y.Z" and make a new version tag: :: $ git tag vX.Y.Z $ git push --tags Releasing --------- Commands to make a new public stable release: :: $ make distcheck # checks pep8, autodoc index, runs tests and more $ make dist # NOTE: Runs git clean -xdf and removes files not in the repo. $ python setup.py sdist upload --sign --identity='Celery Security Team' $ python setup.py bdist_wheel upload --sign --identity='Celery Security Team' If this is a new release series then you also need to do the following: * Go to the Read The Docs management interface at: http://readthedocs.org/projects/celery/?fromdocs=celery * Enter "Edit project" Change default branch to the branch of this series, for example, use the ``2.4`` branch for the 2.4 series. * Also add the previous version under the "versions" tab. .. _`mailing-list`: http://groups.google.com/group/celery-users .. _`irc-channel`: http://docs.celeryproject.org/en/latest/getting-started/resources.html#irc .. _`internals-guide`: http://docs.celeryproject.org/en/latest/internals/guide.html .. _`bundles`: http://docs.celeryproject.org/en/latest/getting-started/introduction.html#bundles .. _`report an issue`: http://docs.celeryproject.org/en/latest/contributing.html#reporting-bugs