Celery task schedule (Ensuring a task is only executed one at a time)

celery, django, python

Solution

It is invalid to access local variables since you can have several celery workers running tasks. And those workers might even be on different hosts. So, basically, there is as many `is_locked` variable instances as many Celery workers are running your `async_work` task. Thus, even though your code won't raise any errors you wouldn't get desired effect with it.

To achieve you goal you need to configure Celery to run only one worker. Since any worker can process a single task at any given time you get what you need.

EDIT:

According to Workers Guide > Concurrency:

By default multiprocessing is used to perform concurrent execution of tasks, but you can also use Eventlet. The number of worker processes/threads can be changed using the `--concurrency` argument and defaults to the number of CPUs available on the machine.

Thus you need to run the worker like this:

$ celery worker --concurrency=1

EDIT 2:

Surprisingly there's another solution, moreover it is even in the official docs, see the Ensuring a task is only executed one at a time article.

Problem

I have a task, somewhat like this: ``` @task() def async_work(info): ... ``` At any moment, I may call async_work with some info. For some reason, I need to make sure that only one async_work is running at a time, other calling request must wait for. So I come up with the following code: ``` is_locked = False @task() def async_work(info): while is_locked: pass is_locked = True ... is_locked = False ``` But it says it's invalid to access local variables... How to solve it?

Original source