Task Pool - celery.pool

Process Pools.

class celery.pool.TaskPool(limit, logger=None)

Process Pool for processing tasks in parallel.

Parameters:
  • limit – see limit attribute.
  • logger – see logger attribute.
limit
The number of processes that can run simultaneously.
logger
The logger used for debugging.
apply_async(target, args=None, kwargs=None, callbacks=None, errbacks=None, meta=None)

Equivalent of the :func:apply built-in function.

All callbacks and errbacks should complete immediately since otherwise the thread which handles the result will get blocked.

full()

Is the pool full?

Returns:True if the maximum number of concurrent processes has been reached.
get_worker_pids()
Returns the process id’s of all the pool workers.
on_ready(callbacks, errbacks, meta, ret_value)
What to do when a worker task is ready and its return value has been collected.
on_return(tid, callbacks, errbacks, meta, ret_value)
What to do when the process returns.
start()

Run the task pool.

Will pre-fork all workers so they’re ready to accept tasks.

stop()
Terminate the pool.

Previous topic

Worker Controller Threads - celery.worker.controllers

Next topic

Process Supervisor - celery.supervisor

This Page