S3 API: `403 AccessDenied: Signature does not match` with Rclone (`provider = Other`)

Hi team,

I configured Appwrite Cloud S3 as an S3 remote in rclone using provider = Other:

• Endpoint: https://sfo.cloud.appwrite.io/v1/s3
• Region: auto
• Access Key ID: <Project ID>
• Secret Access Key: <API Key>
  (See attached screenshot for exact config)

Whenever Rclone attempts to list (ListBuckets / operations/list), it immediately fails:

{
  "error": "error in ListJSON: operation error S3: ListBuckets, https response error StatusCode: 403, RequestID: 8f0fcfbdd36afa8e, HostID: , api error AccessDenied: Signature does not match.",
  "input": { "fs": "appwrite unrollit:", "remote": "" },
  "path": "operations/list",
  "status": 500
}

cc @Torsten Dittmann

checking 🙂

ok thanks 🙏

i might have found a bug investigating this, but not 100% sure on it yet

also, i believe there's a need for a dedicated origin and server for s3. file-sdk doesnt support origins with paths and s3.head() fails as well

yeah thats a known limitation, we dont support virtual-hosted-style URLs

thanks for the follow-up

i see. i guess that's why even in the dashboard object keys in the format of folder/file only show file as object name

yeah, folders are supported though - just not implemented as UI for now

looks like rclone does something funky, aws cli works just fine

found it: rclone signs the Accept-Encoding header, but the CDN/proxy modifies it before Appwrite verifies the SigV4 signature. Appwrite therefore correctly calculates a different signature and returns 403.

you can workaround it, rclone has a setting for it I believe, but let me check if its solvable on my end

gotcha. i was using rclown.com yesterday and the issue popped up so i thought it might be from rclown. then i decided to use rclone directly and that's where i confirmed the issue wasn't rclown related

nah, its something that only happens because our CDN normalizes the Accept-Encoding header

https://www.fastly.com/documentation/reference/http/http-headers/Accept-Encoding/#normalization

but it also gives me a header with the original values I can prioritize if exists - I have an agent working on a fix right now

unfortunately that wasn't reproduced in CI tests 😛

i want to ask, what's the maximum file size limit when uploading through s3 clients?

fix should be live 🙂 same as the buckets setting

it will follow the same rules as if you would use appwrite sdk client

thanks. now showing up on rclown and rclone

my bucket limit is 5GB but it was failing on some object less than 20MB. you can check the screenshots for the error messages and the objects.
i also asked my agent to investigate and give me a summary

thanks for reporting, i will throw an agent on it 🙂

you're welcome. and thanks for the fix