Adding Support for More Executors

There is a proxy in front of the executor

*runtime container

Oh yes I'm a bit confused about the naming 😄

Okay so

1. Increase the number of Function Workers
2. Increase the number of instances of the executor and figure out a good load balancing solution

As I saw you can't increase the number of function workers

I would increase the number of function workers first since that's easier

If you're using swarm, you can increase the number of replicas.

If you just have a docker-compose.yml file, you can try duplicating the service in the compose file

Anyway I'm working on the Kubernetes adapter to can use open-runtime in Kubernetes environment and tried to find a better solution how can scale it more efficiently but until now without any success

hm interesting

What have you tried if you don't mind me asking?

As I mentioned the main problem is the design concept. The executor manages their own runtime containers. Start them up and keep running and send them the necessary commands. The only solution that I found is to increase the number of executors, so every executor can manage their own containers, but we have to pay attention to the proxy too because the proxy always try to redirect the function invocation request to that executor which has the right container but doesn't deal with it if that container is still working on the previous task or not

The best concept would be near the same as Knative built up and working

Executor can handle multiple requests. Runtime containers can handle multiple requests. Where is the problem? 🧐

Yeah you're right executor can start the same script multiple times, so can run it concurrently in the same container but that's another topic how efficient is this

Well I'm gonna be working it out

https://docs.docker.com/config/containers/multi-service_container It's ok to have multiple processes, but to get the most benefit out of Docker, avoid one container being responsible for multiple aspects of your overall application.

It's a very advanced topic because at this point the Linux PID also comes into the picture. What is the PID, how does it work, and why is it important, how does Docker containers running on the host system etc.

Why is it a bad idea to just have a round robin for any existing executors?

Because if the executor get a request to run a function and doesn't have a running container with that function, then first he must initialize it and just after that will be able to execute

Could you not just have one parent executor or parent instance that can then signal to the children that are actually running them to run?

This is the case if you run a single executor

As I mentioned the best and most efficient way would be to study how Knative working and to go in that direction

Google started Knative and it's working nearly the same as GCP executes serverless

So maybe they know better how to properly serve tons of requests in time

For me, it's also fine, that we're reinventing the wheel, but it will absolutely take a lot of time to get there, where others are already. But this is the same topic in which I can't fully agree with the stakeholders. Maybe I have more experience 🤣