Collection total count does not show > 5000
hello, i have a collection containing 8k docs, but the console only shows total of 5k. Also, when i create new document manually, the count does not increase. When i query the total count, it is also showing just 5000
count is an expensive operation. So it's limited till 5000. while if you view more in the console by going till the end, you can see more documents. I would recommend if you wanted to keep a count operation then maybe make a seperate collection to store that for better performance
^ What Biswa said 😉 Count is a O(N) operation and pretty evil in general. We discourage it. Numbers exceeding 5000 will just be 5000+
Exact counts can be tracked many different ways. With metadata tables that are incremented by an Appwrite function triggered by database document events, or run periodically.
We also recommend cursor based pagination like what's used by GitHub. ID + Offset instead of offsetting by number.
How ID + Offset works?
You basically find the ID of the last item from the previous page and fetch the next N items
This way, if there were inserts or deletes, you will avoid duplicate items in your pages
For example
Given items [a, b, c, d, e, f] The first page is [a, b, c] The next page would be
Query.limit(3),
Query.cursorAfter(c),
If you use offset pagination, and someone inserted a1, the new data is: [a, a1, b, c, d, e, f]
If you used offset pagination
Query.limit(3),
Query.offset(3),
Might actually return c, d, e instead of d, e, f
So we recommend cursor pagination because it performs better and avoids the above described pitfall
Thats pretty interesting, but how that cursor can work? it orders data based on the creation time?
I mean, data should be sorted somehow to use an offset cursor, right?
@VincentGe Thanks for the info
Data is always sorted
It's got a natural sort order, never gonna fetch data at total random
You can also specify order in Appwrite during queries, but there should be a natural order never the less
@VincentGe I understand now, thank you so much
i see, thanks @Maniac_Fighter and @VincentGe
