Multiprocessing Worker - celery.worker¶

celery.worker

exception celery.worker.EmptyQueue¶

The message queue is currently empty.

class celery.worker.TaskDaemon(concurrency=None, logfile=None, loglevel=None, queue_wakeup_after=None)¶

Executes tasks waiting in the task queue.

concurrency¶

The number of simultaneous processes doing work (default: celery.conf.DAEMON_CONCURRENCY)

loglevel¶

The loglevel used (default: logging.INFO)

logfile¶

The logfile used, if no logfile is specified it uses stderr (default: celery.conf.DAEMON_LOG_FILE).

queue_wakeup_after¶

The time it takes for the daemon to wake up after the queue is empty, so it can check for more work (default: celery.conf.QUEUE_WAKEUP_AFTER).

empty_msg_emit_every¶

How often the daemon emits the Waiting for queue message. If this is None, the message will never be logged. (default: celery.conf.EMPTY_MSG_EMIT_EVERY)

logger¶

The logging.Logger instance used for logging.

pool¶

The multiprocessing.Pool instance used.

task_consumer¶

The celery.messaging.TaskConsumer instance used.

connection_diagnostics()¶

Diagnose the AMQP connection, and reset if necessary.

execute_next_task()¶

Execute the next task on the queue using the multiprocessing pool.

Catches all exceptions and logs them with level logging.CRITICAL.

fetch_next_task()¶

Fetch the next task from the AMQP broker.

Raises :class`EmptyQueue` exception if there is no messages waiting on the queue.

Return type:	TaskWrapper instance.

receive_message()¶

Receive the next message from the Task consumer queue.

Tries to reset the AMQP connection if not available. Returns None if no message is waiting on the queue.

Return type:	carrot.messaging.Message instance.

receive_message_cc()¶

UNUSED.

reset_connection()¶

Reset the AMQP connection, and reinitialize the celery.messaging.TaskConsumer instance.

run()¶

The worker server’s main loop.

run_periodic_tasks()¶

Schedule all waiting periodic tasks for execution.

Return type:	list of celery.models.PeriodicTaskMeta objects.

schedule_retry_tasks()¶

Reschedule all requeued tasks waiting for retry.

class celery.worker.TaskWrapper(task_name, task_id, task_func, args, kwargs)¶

Class defining a task to be run.

execute(loglevel, logfile)¶

Execute the task in a jail() and store its result and status in the task meta backend.

execute_using_pool(pool, loglevel, logfile)¶

Like execute, but using the multiprocessing pool.

Return type:	multiprocessing.AsyncResult instance.

extend_kwargs_with_logging(loglevel, logfile)¶

Extend the tasks keyword arguments with standard task arguments.

These are logfile, loglevel, task_id and task_name.

classmethod from_message(message)¶

Create a TaskWrapper from a message returned by celery.messaging.TaskConsumer.

If the message is not a proper task it raises UnknownTask exception.

Return type:	TaskWrapper instance.

exception celery.worker.UnknownTask¶

Got an unknown task in the queue. The message is requeued and ignored.

celery.worker.jail(task_id, callable_, args, kwargs)¶

Wraps the task in a jail which saves the status and result of the task execution to the task meta backend.

If the call results in an exception, it saves the exception as the task result, and sets the task status to FAILURE.

If the call was successful, it saves the result to the task result backend, and sets the task status to DONE.