How do I manage the appwrite collection database having more than 5000 documents
How do I manage the appwrite collection database having more than 5000 documents
What do you mean by manage?
@Suii 🚀
I can only retreive 5000 docs on the console. the rest is not retrevied
my colletion contain like 30K docs
Do you need all 30k at one time?
You can get more with filter and pagination, althought it shows 5k in the console count
expecting paged result.
i try with filter - no result
it should be more pages to go to.
Huh yea that seems like a bug imo
seem appwrite limits to last 5000 docs only
+1
Is there a GitHub issue for this?
Nah I asked about this it’s a performance limitation and their implementation of pagination. To give the page numbers like they do they have to use offset pagination, and since the total is limited to 5000 they don’t know what’s beyond or something like that
console might be not useful for collection containing more than 5000 documents as of now
You can filter down to what you need or search for it, the only limitation is pagination over 5000 afaik
that's expected behavoir, but what I see is. it is not fetched. let say I have 30K , I only fetch the first 5K for filteration/pagination.
as here, i expect it should be more paginated, 96 x 53 =5088 only. what else , it should be more pagined number
Sure, the limit is because they are using offset pagination to calculate those pages and the Appwrite endpoint for collections returns at most 5000 for the total. But the filtering and searching should not be effected by that, the only issue you have is if you want to go through every single page for a collection over 5000 records?
That shouldn’t be how filtering works, it doesn’t fetch the first 5000 it fetches them as you go to the page, it’s showing 5000+ and stopping the pagination there because that is the max number returned in the total when doing list documents
the problem i am facing is , let say I have 30K doucments, it returns the first 5K documents. then, I try to filter to get some againts those 30K doucments, but it returns NOTHING. even there are in the database itselft, I already verified in the mariadb
