How can I write code to request lots of data from an API with a limit of 1 query per second? Should I use a loop and setTimeout?

Question

The API I want to use has a limit of 1 query per second and I want to make quite a lot of requests (to analyse some data).

I'm not sure how best to do this as it's the first time I'm attempting to do something like this.

Ideally I'd like to query the data and send it to a database (I'm familiar with Mongo and SQL) but I don't mind any way to store the data I get back as I can add it to a database later.

Any and all suggestions are welcome!

Answer 1

first make sure that there are no other options. sometimes theres an endpoint where you can fetch a multiple data in single request.

loop or setTimeout. i'll say setTimeout. it may have a fraction of inaccuracy but its simpler and enough get the job done.

Answer 2

Rhea Assuming you are using Nodejs, check this out

https://code.sololearn.com/cfgu1kiaTO0Z/?ref=app

Answer 3

Thanks Taste for your advice. I think I might have to use a loop and setTimeout to make requests for each item in a dataset, hopefully that is not a bad idea/mix

Answer 4

try setInterval

Answer 5

I'll give writing that code a go, thank you

Answer 6

perhaps check of limit is server side job and you simply send request after receive answer and wait

Answer 7

I have a dataset and would need to send a request for each individual item.

The API I want to use has a limit of 1 query per second