Back
[DISCUSSION] Upload file chunks to /tmp instead?
If we upload a file bigger than the max. chunk size "5MB" then the file gets split up in chunks. Appwrite will create a temporary folder in "/storage/uploads/" with the prefix "tmp_<filename>".
Final path:
/storage/uploads/myfile.pdf
The temporary directory would be:
/storage/uploads/tmp_myfile.pdf/
Each chunk is stored in this temporary directory with a name like:
TypeScript
/storage/uploads/tmp_myfile.pdf/myfile.part.1
/storage/uploads/tmp_myfile.pdf/myfile.part.2
etc.```
I think a cleaner approach would be if we move the temporary folder to a designated place like **"/tmp"**.
/tmp/uploads/tmp_myfile.pdf/
TypeScript
I only want final files to be in **"/storage/uploads/"** and no chunk files because I do not care about them.
Why this is a dealbreaker for me?
I don't think the appwrite workers will clean the chunks up after a certain time.
They remain there until the end of time and take up space.
I do not want them when I do backups. I have written a cronjob that runs every 5 minutes that searches for **"/tmp_*"** folders in **"/storage/uploads/"** and deletes them when they are older than 60 minutes. But it gets more resource intensive the more files have been uploaded so far.
And for me personally this is not a clean approach. They are still displayed in Appwrite storage and when trying to delete them manually after we have deleted the tmp folder it will result in an error.
```Warning: scandir(): (errno 2): No such file or directory in /usr/src/code/vendor/utopia-php/storage/src/Storage/Device/Local.php on line 526
Warning: foreach() argument must be of type array|object, false given in /usr/src/code/vendor/utopia-php/storage/src/Storage/Device/Local.php on line 526
Warning: rmdir(/storage/uploads/app-test/x/tmp_6694dd45964b0958a472.mp4/): No such file or directory in /usr/src/code/vendor/utopia-php/storage/src/Storage/Device/Local.php on line 252
[Error] Timestamp: 2024-07-15T10:01:43+00:00
[Error] Method: DELETE
[Error] URL: /v1/storage/buckets/:bucketId/files/:fileId
[Error] Type: Appwrite\Extend\Exception
[Error] Message: Failed to delete file from device
[Error] File: /usr/src/code/app/controllers/api/storage.php
[Error] Line: 1632```
Another reason:
In my current appwrite setup I upload all files in **"/storage/uploads/"** to my S3 bucket after 5 seconds and cache a copy on the appwrite server. This way Appwrite does not need to redownload the file from the S3 bucket everytime when a user downloads that file. Currently it will upload all the file chunks too.
It will upload all file chunks to S3 and when the user has finished uploading, the file gets build and that file will be uploaded to S3 too and then it will delete the tmp folder from my S3. It's not really efficient.
Would moving the file chunks to a different folder like **"/tmp"** work without breaking things?
TL;DR
Developers are suggesting moving file chunks from "/storage/uploads/" to "/tmp" for a cleaner approach. This is important to save space and for backups. Cleaning chunks with a cronjob in "/storage/uploads/" folder is resource-intensive. Moving chunks to "/tmp" may be more efficient for uploading to S3.