Graphql-engine: Best practises for Migration version control?

[joshuarobs]

What are the best practises for Migration version control in Hasura?

Here are some I can think of, but I don't know which one's the best:

1. The "standard" way: Init once (hasura migrate create "init" --from-server), make changes to the db schema and VC all these states. Migrate to another server when you want to.
2. Store specific versions: same as above, but you run the hasura migrate create "init" --from-server command whenever you want to make a new "snapshot" of the latest version, put this folder outside of the migrations/ folder (I'm assuming there'd be conflicts if you kept it in there, even if you rename the folder, but I haven't tried), and whenever you want to upgrade the schema, you'd just refer to that folder.
3. Store next/current version: same as above, but you'd just have a specific folder called next and store the sql and yaml files in here, and just refer to that. If you want a history of all the previous "deployed" databases, you'd have to look in your git repo as there'd only be one of these folders at a time.

I've only tried the first approach, and seeing hasura migrate status on the server, made me think this is the only way to go, given that the versioning system was determined by the name of the folder, and that the folders are using some sort of incrementing id based on the current timestamp. If this is true, does that mean that my theoretical approaches (2 & 3) won't work?

(I'm not sure how versioning would work if you uploaded a folder called next, but I do think if you did some manual renaming of a "snapshot", like in approach 2, you could have a migration history on the server like v1.0.0, v1.0.1, v1.1.0, v2.0.0, etc, but this wouldn't be automated. Not sure if this is redundant, considering one can just go to the last stable merge/commit in version control and use that for rolling back, instead of going to the folder of the last used version.)

Just looking to get some clarification.

Also, if I try to have more than one "init" (e.g. 157804XXXXXX_init folder) in the migrations/ folder, migration to another server failed for me. This was resolved by deleting all folders like this, but only keeping one. Is this meant to happen?

[shahidhk]

@joshuarobs I assume that you're trying to use Hasura migrations to version control your Postgres schema as well as Hasura metadata.

Here are some points on Hasura migrations:

• They are incremental, versioned by an integer (timestamp)
• Names are just for human readability
• Each migration contains up and down actions for Postgres schema and Hasura metadata
• You can use Hasura console form CLI (hasura console command) to generate these migrations automatically when you do something on the console
• You can also write these migrations manually

Now, coming to how we intended Hasura migrations to be used:

• Install Hasura CLI
• Create a Hasura project: hasura init (creates migrations directory)
• (Optional) Snapshot your current database state and Hasura metadata as the "first" migration: hasura migrate create init --from-server and mark it as applied: hasura migrate apply --version "123" --skip-execution
• Use the Hasura console from CLI: hasura console
• As you do actions on the console (opens at http://localhost:9695), migrations are generated in the migrations directory
• Each migration version is also marked as applied on the server (see with hasura migarte status)
• To apply migrations on another server, use the apply command: hasura migrate apply --endpoint http://another-hasura.com
• Apply command only applies migrations that are not already applied on this server.

Let me know if you have other questions.

[dmi3y]

Thank you @shahidhk, what about a twist, we already using the migration tool (Sequelize) and want incremental metadata migrations, are there any recommendations for such use case?

[simpleshadow]

Ended up in a similar situation as @dmi3y. Was not clear that hasura console is required for Hasura migrations to work.

I followed the above steps but began operating in the console without hasura console with the (false) assumption I could later hasura migrate create to create a migration from the last state to the current state.

Part of my reasoning was that I did not want Hasura creating thousands of those migrations while I was setting up my initial database tinkering. 😅

Fortunately, I didn't do much between the two states that I could nuke the migration table and perform the changes again, but there might need to be a clear warning in the console that changes are not being tracked.

[shahidhk]

what about a twist, we already using the migration tool (Sequelize) and want incremental metadata migrations, are there any recommendations for such use case?

@dmi3y Yes! You can still use Hasura migrations for incremental metadata. You can use the console to autogenerate them or write them manually.

@simpleshadow For automatically creating migrations, the CLI is required.

[plmercereau]

@shahidhk How does incremental metadata work with config v2?
Now that there is a strict distinction between migrations and metadata folder, and that the metadata folder is designed as kind of a snapshot of the current metadata, I wouldn't know how to keep track of the metadata updates

[rikinsk]

@plmercereau it is now recommended that you version control the metadata files and add commits when you have reached a stable checkpoint. See managing migrations

_PS: sorry for the delayed response._