Appwrite Cloud performance

Hi team,

I have a few remarks and questions regarding Appwrite Cloud performance since it is “much slower” than my current self hosted instance.

I fully understand that Appwrite Cloud is in beta and things can only get better 🤗 . Also on self-hosted server my database is directly under Appwrite so ofc everything is snappier.

Any call to database on Cloud is an average of 400ms, sometimes under 300ms, and frequently above 500ms, while on self-hosted I get on average 80ms-120ms consistently, depending on the amount of data. Interacting with database is not faster from server functions.

Requests to starter functions from templates, that immediately return a value, take on average 150-200ms to complete on Cloud, with frequent spikes to 400-500ms. Same functions on self-hosted instance on cheapest Hetzner server (shared ARM, 2C/4GB) take on average 60-70ms, barely ever going above 100ms. Same outcome with all runtimes.

There seems to be much more latency on Appwrite Cloud. I am in Europe, at equal distance from Frankfurt (Appwrite Cloud) and Nuremberg (Hetzner).

While this is generally not a problem and it is fast when looking at a single request, but I have a fairly complex app where I have to execute multiple request to fetch and process data. For something that take 1 second on self-hosted, I am looking at almost 10 seconds of processing on Appwrite Cloud, even with dedicated functions.

I already optimised my queries in bunch of ways, with and without relationships, but there is no way around it when I have to fetch and clear data from 10+ different collections, or validate multiple cases before creating documents.

Will it be possible to buy more processing power at some point, or is it possible to somehow boost performance now? I am already on Pro plan if that makes any difference (it does mention dedicated database on pricing page). I really want to switch to Appwrite Cloud but I can’t seem to wrap my head around performance compared to cheapo 4€ server 😅

Thanks for raising this and thanks for your steps taken to improve things on your end.

What's your project id?

I'll also bring this up with the team

I have 2 duplicated projects testing this paralino-dev & 66522414001cbb3070d6.

Also just did more testing with GraphQL and that does improve fetching data. It is still not as fast as self hosted but there are less network requests which I notice are causing the most of delays. For example, functions executions in console are displayed in a range of few milliseconds for both Appwrite Cloud and self hosted, but actual request take much more time on Cloud.

Also I believe 90% of those bottlenecks could be solved with database transactions and/or join queries, whenever that is added to Appwrite. Do you maybe have some vague idea when could we expect this?

Thanks!

Not sure on the timeline...maybe some bulk operations in 1.7

Good to know we will have that. I'm waiting for transactions as well, because sometimes my script execution times out, and databases are created without some attributes.

Any update on this and on dedicated instances (is this something you plan on adding)? Or anything we can do now to stabilise and improve performance?

I also tried one of the templates from recent blog post from Appwrite -> https://github.com/appwrite/templates/tree/main/node/query-neon-postgres

And using this is way faster than built in Appwrite database 😅. How come? I even tried using Supabase as a database in Appwrite functions, also much faster and more importantly, both of those options are really consistent in response time. There is no huge discrepancy between multiple request, always in the range of 60-110ms. Appwrite often just jumps to 1 second or above for single request with 400ms average of just reading single specific document. Same thing if fetching multiple documents from collections.

Btw , I am testing this response times both in functions and manually firing bunch of request with curl and in Insomnia. So more than 10000 requests over the last few days on both self hosted and cloud instance for functions, auth and database.

What I am worried about now is my account being blocked for slow queries like few others reported here last few days. I decided against using relationship and GraphQL queries because of that, so I am joining data manually. But I still have to perform bulk operations which would result in slow queries and longer function execution.

How are response times on self hosted instance?

I've tried 3 different setups and here are some averages:

1. Appwrite and database on same server (default)

- Fetching one document -> 50-80ms
- Fetching collection with 3 documents -> 60-90ms
- Fetching collection with 25 documents -> 80-120ms
- Create/Update/Delete document -> 60-90ms
- Fetch user (account/me) -> 50-100ms
- Starter function execution -> 50-70ms

2. Appwrite and database on different servers at different geo locations (Helsinki and Nuremberg)

- Fetching one document -> 70-100ms
- Fetching collection with 3 documents -> 80-150ms
- Fetching collection with 25 documents -> 90-190ms
- Create/Update/Delete document -> 100-160ms
- Fetch user (account/me) -> 70-130ms
- Starter function execution -> 50-70ms

3. Appwrite and database on different server at same geo location

- Surprisingly same averages as in default self hosted server where database is under Appwrite.
But maybe it is to be expected since to reach database we are still firing actual network request that leaves Appwrite instance and then goes back to it.
So even if we have different servers here it is still same round trip.

I have already discussed this roundtrip case long time ago, issue is still open btw -> GitHub

There were some occasional peaks of up to 400ms if fetching data after some time idling, but after that consecutive requests are pretty consistent and fast. 
Also functions execution are not for cold start, in that case we are looking at 3+ seconds in each case. You can notice that in case when database is not on the same location as Appwrite, response time worsens, which is expected due to longer distance.

So on average Appwrite Cloud is 10x slower than self hosted instance, and in worst case scenario up to 20x slower. There is already similar issue reported on GitHub

This is from a function?

can you try cloud again and compare?

I am encountering a similar issue. When attempting to fetch data, particularly images, from Appwrite Cloud in a React Native application, the process is significantly slow and requires a strong internet connection for the images to display properly.

Do you mean using this: https://marketplace.digitalocean.com/apps/appwrite That is not really managed, you still have to take care of everything as if you would install it manually, it only installs it for you automatically. You still have to manage the server (secure it, monitoring, do backups). That is identical to self-hosting, you just start faster. Using managed server/VPS instances takes some of the server maintenance from you, but you still have to take care of Appwrite instance (upgrades, migrations, backups, monitoring...)

Hmm, did about 500 functions executions and ~4000 database requests and it is much faster now, still around 2x slower than self-hosted instance, but not 10x slower 👏. Was there some issue with my project or you did some magic 😅

This is what I got (average ranges where 90% of requests fall in)

- Fetching one document -> 80-180ms
- Fetching collection with 3 documents -> 90-170ms
- Fetching collection with 25 documents -> 90-210ms
- Create/Update/Delete document -> 150-340ms
- Fetch user (account/me) -> 190-380ms
- Starter function execution -> 200-350ms

It is more consistent now, but there is still bigger difference between consecutive request compared to self-hosted. Peaks where not that extreme as before, maybe up to 30% slower from above upper range (not counting function cold start).

I'll have do more testing through out the day to see how it behaves, since before there was quite a difference depending on time of day.

Getting closer to performance of second use case I had, where Appwrite and database instance were on different servers at different geo locations, some 800km apart. Moving it closer together does improve response time quite a bit.

This is probably a different issue. You should resize and optimize the images and use the preview endpoint for a faster loading

This solution involves more requests but I will try it and come back as soon as possible

Nope

Instead of getting the image as usual you get it with the preview endpoint. The amount of requests is the same plus bandwidth usage will be reduced if you optimise it properly with the mentioned endpoint

@chrissiku It is this one -> https://appwrite.io/docs/references/cloud/client-web/storage#getFilePreview But it won't work if your images are larger than 10MB to begin with

Thank you

On it, I'll give you feedback

the max file size I've configured is 2MB,

so it should work, I've been reading that documentation, once done, I'll let you know the result.

Thank you for your help guys

Create a separated post if you need more help or have doubts of this :appwritepeepo:

wildo

We made a change to cloud 😁

Those numbers are awesome! 200ms is acceptable to us... Except account.get.... That should be much faster 🧐