How do I manage the appwrite collection database having more than 5000 documents

How do I manage the appwrite collection database having more than 5000 documents

What do you mean by manage?

@Suii 🚀

I can only retreive 5000 docs on the console. the rest is not retrevied

my colletion contain like 30K docs

Do you need all 30k at one time?

You can get more with filter and pagination, althought it shows 5k in the console count

expecting paged result.

i try with filter - no result

it should be more pages to go to.

Huh yea that seems like a bug imo

seem appwrite limits to last 5000 docs only

+1

Is there a GitHub issue for this?

Nah I asked about this it’s a performance limitation and their implementation of pagination. To give the page numbers like they do they have to use offset pagination, and since the total is limited to 5000 they don’t know what’s beyond or something like that

console might be not useful for collection containing more than 5000 documents as of now

You can filter down to what you need or search for it, the only limitation is pagination over 5000 afaik

that's expected behavoir, but what I see is. it is not fetched. let say I have 30K , I only fetch the first 5K for filteration/pagination.

as here, i expect it should be more paginated, 96 x 53 =5088 only. what else , it should be more pagined number

Sure, the limit is because they are using offset pagination to calculate those pages and the Appwrite endpoint for collections returns at most 5000 for the total. But the filtering and searching should not be effected by that, the only issue you have is if you want to go through every single page for a collection over 5000 records?

That shouldn’t be how filtering works, it doesn’t fetch the first 5000 it fetches them as you go to the page, it’s showing 5000+ and stopping the pagination there because that is the max number returned in the total when doing list documents

the problem i am facing is , let say I have 30K doucments, it returns the first 5K documents. then, I try to filter to get some againts those 30K doucments, but it returns NOTHING. even there are in the database itselft, I already verified in the mariadb