hosnipogiOriginal post
Rank 1: Thread
Hello,
I keep getting a timeout error after sometime when I do multiple concurrent consecutive api calls to update or insert to my collection. It sometimes happen in after about ~1000 updates not really sure.
Im trying to do high frequency updates in parallel. Here is an example of how code works:
JavaScript
const jobs = lines.map((line) => async () => { await exampleAppwriteApiInsert(line); });
/** * Slice the jobs into chunks of 3 */ const chunks = sliceIntoChunks({ array: jobs, chunkSize: 3 });
/** * Heres an example of how it would look like after `sliceIntoChunks` * * [ * [promise1, promise2, promise3], * [promise4, promise5, promise6], * [promise7, promise8, promise9], * ] */
/** * My attempt to execute the example 3 chunked jobs in parallel, * the jobs inside the chunk will execute sequentially, meaning will await until the previous job finishes before moving to next job * https://stackoverflow.com/questions/30823653/is-node-js-native-promise-all-processing-in-parallel-or-sequentially */
const syncJobs = chunks.map((ch) => ch.reduce((p, fn) => p.then(fn), Promise.resolve()) );
await Promise.all(syncJobs);The first api calls in the batch works but eventually I get an AppwriteException [Error]: connect ETIMEDOUT error. Also seems like im not getting relevant error logs when I do docker logs -f appwrite
Summary
The user is experiencing a timeout error when making multiple concurrent API calls to update or insert data into a collection. They are using a NextJS app to make the calls to the Appwrite Docker installed on the same server. They have tried chunking the requests and executing them in parallel using `Promise.all`, but they still encounter the error. They suspect that the server may not be able to handle the high number of concurrent connections. A possible solution suggested by another user is to scale the Appwrite application horizontally by increasing the number of instances. The user is also advised to check the Traefik container and the server's specifications. No further