[SOLVED] Understanding S3 setup with appwrite

Hey, i'm planning to change the storage from local to S3, tho i have some questions to see before starting the migration.

1. Does all the /storage/<storage_type> will be migrated to S3 ?
2. How does the S3 path looks like ? is it like <bucket>/<storage_type>/<files> or something else ? (since the getDevice root is APP_STORAGE_UPLOADS = '/storage/uploads' could it be <bucket>/storage/<storage_type>/<files> ?)
3. Do you use ACL requests ? since i'm planning to use garageHQ, they don't have ACL/Policies endpoints (see https://garagehq.deuxfleurs.fr/documentation/reference-manual/s3-compatibility/)
4. Do you use public facing object ? or for public / get objects you use a presign ? as garage don't allow public access for now, but allow presign requests

also, i don't use minio because the opensource part is now deprecated, and they want us to use AlStor instead

Also looking at the executor i guess it's the same, just the envs are :

- OPR_EXECUTOR_STORAGE_DEVICE=$_APP_STORAGE_DEVICE
- OPR_EXECUTOR_STORAGE_S3_ACCESS_KEY=$_APP_STORAGE_S3_ACCESS_KEY
- OPR_EXECUTOR_STORAGE_S3_SECRET=$_APP_STORAGE_S3_SECRET
- OPR_EXECUTOR_STORAGE_S3_REGION=$_APP_STORAGE_S3_REGION
- OPR_EXECUTOR_STORAGE_S3_BUCKET=$_APP_STORAGE_S3_BUCKET
- OPR_EXECUTOR_STORAGE_S3_ENDPOINT=$_APP_STORAGE_S3_ENDPOINT

and i also need to keep the /tmp volume for the runtimes to access the data from the executor if i'm not wrong

up

up ? it's basicly the last service to migrate on my cluster ...

1. If you mean that all the previous data will be moved from local to S3 automatically, nope
2. Not sure, but it should be similar to the local volume.
3. Afaik not, it just has full access.
4. No, everything goes and is served through the Appwrite instance

thanks for your reply, for the first one i meant, does all the folders under the /storage path (basicly all shared volumes) need to be migrated to S3 ?

If you have existing data yes. Make sure to have a backup before proceeding

so after a lot of searching and testing, it does work, tho regarding what's moved :

|     Path                | Garage S3? | Notes           |
|-------------------------|------------|------------------|
| `/storage/uploads`      |    Yes     | User files, export outputs |
| `/storage/functions`    |    Yes     | Function deployment tarballs |
| `/storage/sites`        |    Yes     | Site deployment tarballs |
| `/storage/builds`       |    Yes     | Build artifacts |
| `/storage/imports`      |    Yes     | CSV/JSON migration staging (if used) |
| `/storage/cache`        |    No      | HTTP response cache — RWX PVC only |
| `/storage/config`       |    No      | Traefik TLS snippets (Ingress setups often empty) |
| `/storage/certificates` |    No      | Let's Encrypt PEMs (Ingress setups often empty) |

regarding the bucket, everything is in a subdir storage so it's s3://<bucket>/storage/<type>

tho didn't fully tests like public buckets, but since your said appwrite proxy everything, it should work

other thing to consider while migrating apparently, the database includes a / at the start of the path, and so when using s3 it's not found because the path gets a double /. Removing the first / and cleaning the redis cache made the migration process works

S3 migration seams to have worked, just needed to flush redis to reload the modified paths from the db

and in 1 minute i will tell if the executor also works (but it should)

yup executor works 🙂

currently making a compose only migration guide that i will post here before putting it as "SOLVED" so others will be able to setup S3 properly

[SOLVED] Understanding S3 setup with appwrite