Help with Appwrite offset pagination to fetch more than 25 docs

Hi, I’m trying to fetch all documents using offset-based pagination in Appwrite, but I ran into some issues. Below is my code and the approach I followed. The problem is that it either doesn’t fetch all documents correctly or returns fewer than expected.

'''python import requests import json from datetime import datetime

API_ENDPOINT = "" PROJECT_ID = "" API_KEY = "" DATABASE_ID = "" COLLECTION_ID = ""

HEADERS = { "X-Appwrite-Project": PROJECT_ID, "X-Appwrite-Key": API_KEY }

def export_collection_to_file(database_id, collection_id): timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") output_file = f"data_{timestamp}.json"

url = f"{API_ENDPOINT}/databases/{database_id}/collections/{collection_id}/documents"
all_documents = []
limit = 5000

try:
    print(f"start to request collection {collection_id} documents...")
    response = requests.get(url, headers=HEADERS, params={"limit": limit}, timeout=10)
    response.raise_for_status()
    documents = response.json().get("documents", [])

    if not documents:
        print("no docs to export.")
    else:
        all_documents.extend(documents)
        print(f"successfully get {len(all_documents)} documents:")
        for doc in all_documents:
            print(f"document ID: {doc['$id']} has been successfully exported!")
        with open(output_file, "w", encoding="utf-8") as f:
            json.dump(all_documents, f, ensure_ascii=False, indent=4)

        print(f"Collection documents have been successfully exported and saved to {output_file}")

except requests.exceptions.RequestException as e:
    print("Request failed or timed out: {e}")

if name == "main": export_collection_to_file(DATABASE_ID, COLLECTION_ID)

@LiamBMXCould you take a look when you have time? Thanks! 😄

Try creating a loop to query the default amount of documents at a time until there are none left to query, I’m not on my pc right now so I can’t check or run code but I will try some stuff in the morning if you can’t get it working

Okay, I'll try it, thanks!