How to handle asyncronous task queue in a server application?

Question

I am working on an api with flask which uses web scraping to supply data. suppose i have a function named update for the web scraping task which scapes the required data and caches the data in a database. How can I force the function to execute sequentially?

Answer 1

For handling asynchronous task queue in Flask, you can use Celery.

http://www.celeryproject.org/

Here is an example :
https://stackoverflow.com/questions/31866796/making-an-asynchronous-task-in-flask

Answer 2

sleep is not appropriate, because you cannot be sure how much time is needed for the response be fetched.

Answer 3

Gordon Oh that seems like a great solution. 😄

But, I am saying can I use a simpler time.sleep() in this case?

Answer 4

Thanks for the response. 😊