Can celery celerybeat dynamically add/remove tasks

Question

I have a project that does not include Django, so i can't use djcelery.

But i found the modification of django-celery DatabaseSchedule that using sqlalchemy.

It works fine as like djceley's DatabaseScheule did. But the only problem is that it doesn't seem to send tasks that were added in runtime, then i restart the celery-beat, the tasks that were added before will be sent successfully.

So, is it possible to dynamically add/remove tasks without restarting celery-beat?

Thanks for any advice. And sorry for my bad english.

Sorry, it was my fault. The tasks that were added in runtime will not be picked up instantly, you can change the DatabaseScheduler.sync_every value to a faster one. So it works after waiting for several minutes.

Thanks for tuomur's help.

Answer 1

Solution 1:

I too had the same problem, The problem is, Celery periodic scheduler will look for [djcelery_periodictasks] which will have [last_update] value. If there is no change in the [last_update] then the celery will not check the [djcelery_periodictask] table.

What we have to do is, When ever you add/remove a task update the [last_update] time in [djcelery_periodictasks].

In my senario, I have the periodic scheduelr in python. However my Web UI is written in C#. my web application will add/remove the entry in [djcelery_periodictask] and update the [last_update] time in [djcelery_periodictasks].

By this way, i have solved this issue.

When we use the standard flower UI, If we add/remove the task it will update the [last_update] time in [djcelery_periodictasks]. IF you have developed your own custom UI / application you have to include the functionality to update [last_update] time in [djcelery_periodictasks] to dynamically add/remove the task.

Solution2:

use flower : https://github.com/mher/flower