Hi guys, I need to import 20millions documents into the database, I am currently running some scripts that add the documents one by one. Its has been a week and I only managed to import 1million due to hitting rate limits very often.
What are the options to populate my collections a bit faster? I've seen some mentions that says the rate limit only applies for 1 connection and that I could have different connections for increasing it? in practice, how you guys implement that?
On the roadmap for 1.7 is bulk inserting, other than that I think this is all you can do. Depending on how you're inserting them you can look at doing concurrent insertions in your code.
Recommended threads
- I'm getting an error on the console "j?....
On my self hosted instance version 1.8.1 the console is giving me this error when trying to view the rows for a table I recently created. My application is read...
- Websites hosted on my appwrite sites hav...
Hello, all my websites hosted on appwrite sites are not running I am getting this message "This site can’t be reached drivehub.appwrite.network took too long t...
- Database Write Limits hit
Hello Appwrite Admins, I'm a GitHub Education user, and about a week ago, my database was really badly optimized, resulting in about 600k writes in a single day...