Graphql-engine: Let's make resetting(squash) migrations EASY!

[jjangga0214]

Migration files can easily be piled up. So "squash" feature would be required in some projects. According to https://blog.hasura.io/resetting-hasura-migrations/, in order to squash, we should do

1. Remove every migration files (rm migrations/*).
2. Remove whole data from the table hdb_catalog.schema_migrations (TRUNCATE hdb_catalog.schema_migrations;)
3. Pull the metadata (and optionally schema) from hasura.
4. Apply the pulled metadata (and optionally schema) just back to hasura (with --skip-execution option as hasura obviously already have the metadata, but only needs "version" to be applied.)

Problems

In my humble opinion, there are these problems.

Inconsistency (high-level vs low-level migration tasks)

Users handle migration through cli. The high-level approach should be consistent. But suddenly having to take care about schema_migrations table is not enough abstract. So it does not give 'smooth' and 'expected' experience. I guess many even don't know schema_migrations, as docs(https://docs.hasura.io/1.0/graphql/manual/how-it-works/metadata-schema.html) only explicitly explains hdb_table, hdb_relationship, and hdb_permission, not schema_migrations and many others.

Somewhat hacky? (at least to me)

Users should specify --skip-execution. After they just pulled migrations files from hasura, they have to "pretend" applying migration only for writing a new row on schema_migrations. It makes me feel I am doing a "workaround" rather than a straight official way.

Too manual

There are total 4 manual steps for one "squash". These are cumbersome and error-prone.

"squash"ing does happen as projects grow. People obviously just don't want hundreds of migration files to be stacked, because it'd be inconvenient to open migration directory and it just simply looks not "neat". So the "squash" can't but happen and it should be automatic for convenience.

Solution

I suggest a command like

hasura migrate squash

which would safely execute the 4 steps automatically.

[heyrict]

Currently, to avoid inconsistency of database, I'm writing the migrations myself instead of using hasura console.

How do I squash migrations when using hasura console

1. do several migrations in hasura console
2. revert some migrations with hasura cli
   hasura migrate apply --down x
3. concatenate up and down migration files
   shell cat `ls *.up.yaml | tail -n x` > `ls *.up.yaml | tail -n 1` cat `ls *.down.yaml | tail -n x | tac` > `ls *.down.yaml | tail -n 1`
4. do migrations again
   hasura migrate apply --up x

If you use a similar workflow as me, a simple bash function should help as a temporary workaround.

hasura_migrate_squash(){
    hasura migrate apply --down $1;
    cat `ls *.up.yaml | tail -n $1`  > `ls *.up.yaml | tail -n 1`;
    cat `ls *.down.yaml | tail -n $1 | tac` > `ls *.down.yaml | tail -n 1`;
    hasura migrate apply --up $1;
}

Another solution

Another solution I come up with is to add a squash option to hasura console.

1. Check squash option.
2. Do migration A. Console will generate a file [timestamp_A]_migration_A.*.yaml.
3. Do migration B. Console will move [timestamp_A]_migration_A.*.yaml to [timestamp_A]_multiple_migrations.*.yaml or something like so, and concatenating the A and B migrations to this file.
   So the content of []multiple_migrations.up.yaml should be
   ...A ...B
   and content of []multiple_migrations.down.yaml should be
   ...B ...A

This solution suppose the developer has only one open window of hasura console, as the squash option is stored in the client (say, redux store).