Uploading file with multipart

I am trying to upload file using storage api and go. When file size is a multiple of chunk size, it completes successfully. But if there is a small remaining part, then the request fails with status code 500 Server Error. The docker logs show:

Warning: file_get_contents(/storage/uploads/app-testsdk/testBucket/tmp_testFile.txt/testFile.part.8): Failed to open stream: No such file or directory in /usr/src/code/vendor/utopia-php/storage/src/Storage/Device/Local.php on line 124
[Error] Timestamp: 2023-04-17T13:30:18+00:00
[Error] Method: POST
[Error] URL: /v1/storage/buckets/:bucketId/files
[Error] Type: Exception
[Error] Message: Failed to read chunk /storage/uploads/app-testsdk/testBucket/tmp_testFile.txt/testFile.part.8
[Error] File: /usr/src/code/vendor/utopia-php/storage/src/Storage/Device/Local.php
[Error] Line: 126

What might be the issue?

I am using chunk size of 1024 and size of 7168(7 chunks) completes successfully but file of size 7173 fails.

The last content range header is bytes 7168-7172/7173

Hi - let me look into this

Not sure how relevant this is but I tried tracking the directory where appwrite stores the files:

/storage/uploads/app-testsdk/testBucket/tmp_test12MB.txt # cat test12MB.txt_chunks.log 
1
2
6

The last line shouldn't be 6? I am using appwrite 1.3.1 and the same code works with appwrite cloud.

Have you used our official SDKs for file upload, or are you doing chunking yourself.

I am writing an SDK for go. The current one doesn't support chunking 😅

I dont think its a problem with chunking since it works with appwrite cloud.

I finally figured it out: For the last block, the end byte must be equal to total size. For a file size of 7173, the final header should be bytes 7168-7173/7173. However, this seems to be wrong because:

1. According to https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Range

   Content-Range: <unit> <range-start>-<range-end>/<size> <range-end> An integer in the given unit indicating the end position (zero-indexed & inclusive) of the requested range. Range end should be inclusive.

2. According to https://httpwg.org/specs/rfc9110.html#field.content-range

   A Content-Range field value is invalid if it contains a range-resp that has a last-pos value less than its first-pos value, or a complete-length value less than or equal to its last-pos value. The recipient of an invalid Content-Range MUST NOT attempt to recombine the received content with a stored representation. range-end must be less than complete-length

Opened a github issue cause this probably belong there

please paste the link here as well ❤️

https://github.com/appwrite/appwrite/issues/5392