Back
Long running tasks, how to?
Hey guys, for long running tasks, like processing a large database dump or using some browser automation, is locking a worker on it the only way to do so? Given the limited amount of workers we can spin.
TL;DR
Question: Is locking a worker the only way to handle long running tasks, considering the limited number of available workers?
Solution: One possible solution to handle long running tasks without locking a worker is to use task queuing and scheduling systems like Celery. This allows you to distribute the tasks across multiple workers and prioritize them based on their importance and resource requirements. This way, you can handle long running tasks efficiently without blocking available workers.