Common Task Strategies - celery.task.strategy¶

celery.task.strategy.even_time_distribution(task, size, time_window, iterable, **apply_kwargs)¶

With an iterator yielding task args, kwargs tuples, evenly distribute the processing of its tasks throughout the time window available.

Parameters:	task – The kind of task (a celery.task.base.Task.) size – Total number of elements the iterator gives. time_window – Total time available, in minutes. iterable – Iterable yielding task args, kwargs tuples. **apply_kwargs – Additional keyword arguments to be passed on to celery.execute.apply_async().

Example

>>> class RefreshAllFeeds(Task):
...
...     def run(self, **kwargs):
...         feeds = Feed.objects.all()
...         total = feeds.count()
...
...         time_window = REFRESH_FEEDS_EVERY_INTERVAL_MINUTES
...
...         def iter_feed_task_args(iterable):
...             for feed in iterable:
...                 yield ([feed.feed_url], {}) # args, kwargs tuple
...
...         it = iter_feed_task_args(feeds.iterator())
...
...         even_time_distribution(RefreshFeedTask, total,
...                                time_window, it)