Datastructures - celery.datastructures¶

Custom Datastructures

class celery.datastructures.PositionQueue(length)¶

A positional queue of a specific length, with slots that are either filled or unfilled. When all of the positions are filled, the queue is considered full().

Parameter:	length – see length.

length¶

The number of items required for the queue to be considered full.

class UnfilledPosition(position)¶

Describes an unfilled slot.

PositionQueue.filled¶

Returns the filled slots as a list.

PositionQueue.full()¶

Returns True if all of the slots has been filled.

class celery.datastructures.TaskProcessQueue(limit, logger=None, done_msg=None)¶

Queue of running child processes, which starts waiting for the processes to finish when the queue limit has been reached.

Parameters:	limit – see limit attribute. logger – see logger attribute. done_msg – see done_msg attribute.

limit¶

The number of processes that can run simultaneously until we start collecting results.

logger¶

The logger used to print the done_msg.

done_msg¶

Message logged when a tasks result has been collected. The message is logged with loglevel logging.INFO.

add(result, task_name, task_id)¶

Add a process to the queue.

If the queue is full, it will start to collect return values from the tasks executed. When all return values has been collected, it deletes the current queue and is ready to accept new processes.

Parameters:	result – A multiprocessing.AsyncResult instance, as returned by multiprocessing.Pool.apply_async(). task_name – Name of the task executed. task_id – Id of the task executed.