Hi guys, I need to import 20millions documents into the database, I am currently running some scripts that add the documents one by one. Its has been a week and I only managed to import 1million due to hitting rate limits very often.
What are the options to populate my collections a bit faster? I've seen some mentions that says the rate limit only applies for 1 connection and that I could have different connections for increasing it? in practice, how you guys implement that?
On the roadmap for 1.7 is bulk inserting, other than that I think this is all you can do. Depending on how you're inserting them you can look at doing concurrent insertions in your code.
Recommended threads
- Database Write Limits hit
Hello Appwrite Admins, I'm a GitHub Education user, and about a week ago, my database was really badly optimized, resulting in about 600k writes in a single day...
- How i can call increment with operators ...
- Attribute not found in schema on REST AP...
I'm querying a appwrite collection via the REST API on appwrite cloud 1.9.5 (no SDK) via a cloudflare worker and keep getting: ``` {"message":"Invalid query: A...