Function could not reaolve host

Posting specifics here upon @Steven's request.

See attached image to see turning point from working to error and the appropriate timings.

Error seems to be: An internal curl error has occurred within the executor! Error Msg: Could not resolve host: 63a72192c36ed0fa715f-647a176f38c4b46c7887

@Steven

You're self hosting?

Function could not reaolve host

And that's the error for the first failed one too?

Yes

For all the fails, that seems to be it

So the reason why this might be happening is the container gets removed (usually due to inactivity) but the removal fails so it's in this inconsistent state. Make sure to 👍 this related issue https://github.com/appwrite/appwrite/issues/3776

There are some things you can do to mitigate this and they all involve preventing the runtime container from being removed.

One option is to increase the inactivity threshold (there's an environment variable for it) to a value where the runtime container won't be removed.

Another option is to have your function execute regularly (but don't do anything) so that it doesn't get considered as inactive.

You can do both of these too

Thanks for the help Steven.

Sorry, I've been quite busy so only just saw this.

A few questions:

1. One option is to increase the inactivity threshold (there's an environment variable for it) to a value where the runtime container won't be removed. Alright. What is the default inactivity threshold and how would I increase it? It seems weird to me that you have to stop things from being removed. In theory, people could have functions that are tiggered once a month but that doesn't mean that they want them to be unusable. Is there a way to make the container remove properly?

2. Another option is to have your function execute regularly (but don't do anything) so that it doesn't get considered as inactive. Could I achieve this with a cron job then? Would it matter if the function execution failed when it was executed via CRON (because I wouldn't be able to pass in valid data probably).

3. Is there a way to setup some kind of monitoring that will notify me when functions are failing a lot? Wether they be my fault or the AppWrite/the server's (like a curl error), it would be good to know when vital functions aren't working right so that we can get ahead of the issue.

Again- I appreciate the help. Sorry if I came off as unpleasant the other day, I was just very annoyed that AppWrite was playing up because I felt bad for my client.

Inactivity threshold

The default is 60 seconds but actually can be up to 61 minutes, check here: https://discord.com/channels/564160730845151244/564160731327758347/1096480144434151584 To change it update the _APP_FUNCTIONS_INACTIVE_THRESHOLD with the number of seconds you want in the .env file You can read more about it here: https://appwrite.io/docs/environment-variables#functions

Keep alive!

If you need your function to be "alive" check this https://discord.com/channels/564160730845151244/1102625724243906640/1102628184219979886 and https://discord.com/channels/564160730845151244/1102936099745177700/1115665275191312524

Function monitoring

The last part can be more difficult will have to wait for Steven's suggestions. You can try some 3rd party software for monitoring docker containers

In theory, people could have functions that are tiggered once a month but that doesn't mean that they want them to be unusable.

Of course, that's why we're working on improvements to functions.

Is there a way to make the container remove properly?

Hard to say because I don't know why your function wasn't removed properly.

Is there a way to setup some kind of monitoring that will notify me when functions are failing a lot?

Sure, I remember seeing some community member do something like this with the server sdk

Do you remember which member did this?

No sorry

@Steven This happened again today on another function.

Seems like it's prone to happen on all functions...

so I need to increase inactivity and then set up a CRON for like every hour on each function and execute it? 😫

@Steven

the same logic applies to all functions