[SOLVED] Collection version management

As my production app evolves, I make changes to the structure of the collections that are used. Some of the structure changes require a 'migration' of the existing documents so that they match the new structure. To manage this, I have written a tool that I run after each structure change (if required) that does the heavy lifting, but it's a fairly brute-force approach: as the number of structure changes grows, so does the number of 'migrations'.

In an ideal world I would like to know which iteration of the structure a collection is currently at, and only apply the necessary migrations applicable to that iteration. In other words, I would like to 'version' a collection and then say something like "If the collection is less than this version, then do this migration."

Has anyone solved a similar problem? Any thoughts about how I might go about doing this?

As of now there's no official way to achieve such a thing with Appwrite.

What usually can be done, is creating script file using Appwrite Database Server side SDK. https://appwrite.io/docs/server/databases

Then, you be able to manage - somehow - migration using git to control the current migration.

But, it still won't give you the experience you get in Laravel for example, and it will still require some extra effort.

Thanks for the reply @Binyamin. What you've described is largely what I'm doing now. The challenge is the "somehow" that you mention. Without having a property on a collection where this information could be stored, it's practically impossible when you have multiple databases in use (it's very challenging with just a development, testing, staging and production database - add to those a production database per customer and it becomes a nightmare).

I agree.

The only way to mitigate it now, is to have some local db (could be simple as json) that holds the migration.

Then before adding or changing attribute you can verifry if the attribute exist, and how using the getAttribute
https://appwrite.io/docs/server/databases?sdk=nodejs-default#databasesGetAttribute

Though for now it will be still painful to manage and track..

Maybe you can take inspiration from various migration systems and do the same. From my experience, there's typically:

1. a metadata table that stores which migrations have been executed
2. migration scripts with a up/forward and down/back
3. a coordinator that checks the metadata table, runs the necessary migration scripts, and updates the metadata table

That's actually a very good idea, thank you @Steven. I think I'll go the route of having a meta table. It's an extra collection but it will only be visible to the backend processes.

Thank you both for your input!

[SOLVED] Collection version management