Performance degradation over time (Executing functions/Creating documents)

Did a function stresstest and sometimes I do receive this error. I was just using the dart starter function so nothing special. And also it always drops after around 15sec and the server can only handle around 150 functions per second consistently.

Selfhosted, Appwrite 1.5.7
Stresstest: oha with 100 concurrent Connections

how are you running the function? it's sync or async?

It's sync.

Logs from openruntime executor.

Only difference in the docker-compose file is that I removed mariadb because I am hosting it on my other server

@Meldiron do you know what could be happening here?

How it looks like over time:

It is almost consistent at around 1800 per 10 sec. Maybe a bottleneck somewhere

What Open Runtimes Executor version do you use? You can find it in docker-compose.yml

0.5.5

Both warnings were fixed in 0.5.6. You can safely upgrade.

oh lol, will try it out. thanks

Regarding "Address not available", sorry, first time seing this 😬 cc @user

If you look into docker-compose.yml, do you have Open Runtimes Proxy there?

eh, no.

I guess you don't mean " openruntimes-executor:
container_name: openruntimes-executor"

Yeah, I meant openruntimes-proxy. It's good that you dont have it there, I was just double checking

Hmm, I can think about scenario when runtime is crashing, causing this. If you start your function, and run benchmark.. and then do docker ps. What does it say in the status? It should say something like "Up for X minutes". I need to ensure it says real time since first execution. If it's anything smaller, it could mean that a runtime crashed in the meantime - possibly causing this

This is how it looks now:

Overall it is better now but it still drops to half instead of like one third after some time

Like this?

Another run and it looks almost identical to the previous one. First 20sec are fine then it drops

do you have the same Address not available error?

Nope. No error in appwrite or openruntime-executor container at all

It's gone since I upgraded to executor 0.5.6