What does a "failed" index mean?

It means it didn't successfully created. You should check the docker logs for the appwrite-worker-databases container for more info

this happened a while ago and those logs are long lost. Can I delete the index and attempt to re-create it?

Maybe

The log looks new: it just says "appwrite_vm /var/lib/docker/containers/5333ffe519c45470a703b31bbe7ea2e4adccfad4fe9dffc170f480b3028a8691 # more 5333ffe519c45470a703b31bbe7ea2e4adccfad4fe9dffc170f480b3028a8691-json.log
{"log":"\u001b[32mAppwrite database worker v1 has started\n","stream":"stdout","time":"2023-05-02T02:30:33.965141813Z"}
{"log":"\u001b[0m\n","stream":"stdout","time":"2023-05-02T02:30:33.965189813Z"}
{"log":"[notice] Starting worker 5333ffe519c4:7:v1-database\n","stream":"stdout","time":"2023-05-02T02:30:33.974499813Z"}"

Why are you looking at the logs like this?

Am I doing something stupid?

Recreating also fails

With what?

Check the logs

You seem to be suggesting there's an easier way to view the log aside from reading on the terminal. Now the log is ginormous

You typically check docker logs using the docker logs command or docker compose logs. There are various options you can pass too

A-ha! It says "Index already exists"

But that's occurring upon recreating it after I deleted it, so it appears that 1) the deletion code isn't doing its job and 2) the recreation code should first call delete on its own. The deletion code doesn't appear to be reporting any errors.

The thing is you don't know what happened the first time it failed. It could be the index already exists but it failed to create some of the Appwrite metadata so Appwrite thinks the index doesn't exist but it does exist in mariadb.

To fix this, the easiest thing to do is recreate the collection. Otherwise, you'd need to manually delete rows in some of the tables and the index in mariadb

This is a dev server, but how would we deal with this on a real-world production server?

OK, by recreate the collection you mean just database fields and this won't delete the actual data or am I confused?

No, deleting the collection would delete the data.

You would have to do the 2nd option

which particular rows would I delete and how would that fix it? Also, if this happened on a production server, this failure seems somewhat catastrophic.

You'd have to dig around a bit...I can help you look tomorrow if you need.

It's just annoying to see a failed index. It's not that catastrophic.

this is just a dev server, but i can recreate the schema from scratch when we go into production to avoid any failed indices from getting copied over

I suggest using the Appwrite CLI to quickly deploy your collections.

https://appwrite.io/docs/command-line-deployment

If I understand correctly, I will produce an appwrite.json file from the dev server, and I copy it to the new server and upload it, but what isn't clear is how do I access the CLI? Is this happening on the shell of the appwrite containers themselves?

You don't need to copy it to the server. You can use the CLI from wherever

See https://appwrite.io/docs/command-line