AppwriteException [Error]: connect ETIMEDOUT When trying to do concurrent api calls
Hello,
I keep getting a timeout error after sometime when I do multiple concurrent consecutive api calls to update or insert to my collection. It sometimes happen in after about ~1000 updates not really sure.
Im trying to do high frequency updates in parallel. Here is an example of how code works:
const jobs = lines.map((line) => async () => {
await exampleAppwriteApiInsert(line);
});
/**
* Slice the jobs into chunks of 3
*/
const chunks = sliceIntoChunks({ array: jobs, chunkSize: 3 });
/**
* Heres an example of how it would look like after `sliceIntoChunks`
*
* [
* [promise1, promise2, promise3],
* [promise4, promise5, promise6],
* [promise7, promise8, promise9],
* ]
*/
/**
* My attempt to execute the example 3 chunked jobs in parallel,
* the jobs inside the chunk will execute sequentially, meaning will await until the previous job finishes before moving to next job
* https://stackoverflow.com/questions/30823653/is-node-js-native-promise-all-processing-in-parallel-or-sequentially
*/
const syncJobs = chunks.map((ch) =>
ch.reduce((p, fn) => p.then(fn), Promise.resolve())
);
await Promise.all(syncJobs);
The first api calls in the batch works but eventually I get an AppwriteException [Error]: connect ETIMEDOUT error. Also seems like im not getting relevant error logs when I do docker logs -f appwrite
The server probably isn't able to handle that many concurrent connections. The easiest thing to do is to limit the number of concurrent requests.
hello, basically i'm doing the call from the same server the Appwrite Docker is installed, so in my case I have a NextJS app doing the calls via getStaticProps. So Im calling the appwrite update apis to an appwrite localhost endpoint, the error I'm getting now after some time looks like this:
err: {
"type": "AppwriteException",
"message": "connect EADDRNOTAVAIL 127.0.0.1:31074 - Local (127.0.0.1:0)",
"stack":
Error: connect EADDRNOTAVAIL 127.0.0.1:31074 - Local (127.0.0.1:0)
at Client.call (/root/app/testProject-dev/node_modules/node-appwrite/lib/client.js:177:23)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Databases.listDocuments (/root/app/testProject-dev/node_modules/node-appwrite/lib/services/databases.js:1002:16)
at async Appwrite.listDocument (/root/app/testProject-dev/.next/server/chunks/8996.js:72:24)
at async AppwriteMedia.upsertMedia (/root/app/testProject-dev/.next/server/chunks/8996.js:333:21)
at async AppwriteMedia.storeImageIfNotExist (/root/app/testProject-dev/.next/server/chunks/8996.js:343:23)
at async Promise.all (index 0)
at async getStaticProps$1 (/root/app/testProject-dev/.next/server/pages/[username]/gallery/[page].js:82:17)
at async /root/app/testProject-dev/node_modules/@sentry/nextjs/cjs/server/utils/wrapperUtils.js:38:14
}
My guess is traefic container itself but im not really sure where to start checking. Here's my server specs if it helps:
CPU(s): 16
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 1
with about 125gbs of memory
i've been looking at https://dev.to/appwrite/30daysofappwrite-docker-swarm-integration-2io9 as a possible fix, but not really sure if this can fix it
Scaling horizontally would help because you'd have more instances to handle the load. I don't know the details of how to scale though
For now you have a single server And you're calling Appwrite within a function?
Is this your code flow?
I think it's not the case
And you have a pretty strong server
Can you try to set the _APP_WORKER_PER_CORE to let's say 18 then you'll have 288 workers.
Then reload your docker
docker compose up -d
Then if it failed then it will be best to horizontally scale your Appwrite application
Check here to see more about variables https://appwrite.io/docs/environment-variables#general
