Back
Bulk Document Upload from CSV
Hello - I’m currently uploading data from a CSV to a collection in my database.
My current workflow is as follows:
1.) Convert CSV to JSON using Pandas 2.) Loop over each JSON object and upload using create_document()
I have over 50k rows of data which is taking way longer than I expected (two hours and only 15k records have been uploaded)
I’ve read that batch upload is a common feature request but all of these requests were made over a year ago.
For context: I’m using the Python Server SDK and API key.
Any help will go a really long way!
TL;DR
Developers are seeking a more efficient solution for bulk document upload from CSV using Appwrite. They are currently converting CSV to JSON and uploading each JSON object individually, which is slow for large datasets. Looking for suggestions to speed up the process, especially with batch upload capabilities.Found this. Still wanting to see if anyone has found/ can walk me through a more efficient solution
