Graphql-engine: Squashing runs forever / fails on "drop table" as down migrations are not valid

[mklueh]

Hi there,

I´ve created my schema and I´m now having 100+ migration files in my directory I´d like to merge.

My first attempt was to just run

hasura migrate squash --name squashed

without --from, as I wanted to include all changes, which is failing because down.yaml files of my "drop table" changes are just containing an empty array

down.yaml content

[]

log

WARN This command is currently experimental and hence in preview, correctness of squashed migration is not guaranteed!
SquashingSquashing?[31mFATA?[0m[0000] unable to squash migrations: 1580928345277 down migration not found

In my second attempt I´ve deleted the failing down migrations. This leads to the following endless running log output that I had aborted after a few minutes:

$ hasura migrate squash --from 1580927467522 --name squashed
WARN This command is currently experimental and hence in preview, correctness of squashed migration is not guaranteed!
SquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashing migrations from 1580927467522 to latest...?[37m◒
SquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashing migrations from 1580927467522 to latest...?[37m◒
SquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashing migrations from 1580927467522 to latest...?[37m◒
SquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashing migrations from 1580927467522 to latest...?[37m◒
SquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashingSquashing migrations from 1580927467522 to latest...?[37m◓?[0m
.
.
.
.

Once I move to the first migration after the table drops and pass that number into the --from parameter, it starts working and produces clear outputs

1580937879341
1580938405332
1580938430947
1580938688130
1580938712395
1580938735018
1581166965385
1581166979430
1581167011276
1581167033904
1581167044336
1581167065096
1581167071383
1581167077340
1581167219480
1581167223764
1581167228023
1581167305054
1581167361078
1581167400498
1581167467421
1581167590952
1581167622532
1581167649304
1581167654605
1581167666343
1581167671974
1581167676404
1581167678159
1581167750352
1581452665830
1581453505386
1581761463818
1581761531194
1581761535352
1581761536995
1581761544990
1581761631031
1581761636589
1581761638143
1581761643090
1581761655464
1581809783351
1581809794054

INFO Do you want to delete these migration source files? (y/N)

To get it working I´ve deleted the create and drop migrations of the tables that were affected and squashed everything into one migration with success, however the migrations were producing SQL errors and could not be applied, while the unsquashed migrations are working perfectly without those tables.

I´d guess this is not the expected behaviour and a bug but it would be glad if someone could elaborate on this issue.

[scriptonist]

@mklueh Apologies for the delay!

can you let me know the CLI version that you were using, you can find that using the following command.

hasura version

[mklueh]

@scriptonist I don´t remember anymore what version I had back then, but it must have been the latest version around that time

[scriptonist]

Thanks @mklueh , no worries

[tirumaraiselvan]

@scriptonist Is there any update on this issue?

@mklueh Any chance you have more info about the cli version? Or a sample repro?

[scriptonist]

Yes @tirumaraiselvan a bunch of bugs were reported on the squashing end of things, we will be addressing them ASAP

[scriptonist]

@mklueh I was not able to reproduce this 🤔, It'll be helpful if you can give it a try, with the latest version of CLI and see if the issue persists.

[mklueh]

Hi @scriptonist ,

I´m not sure if I can reproduce it either, I can try to restore the version from the time of this post and retry it with the current Hasura version this evening, but I cannot promise anything. Maybe it´s already solved

[mklueh]

@scriptonist So I´ve managed to find and restore the commit that had the failing migrations

First of all, the command I´ve used back then is not working anymore

$ hasura migrate squash --name squashed
INFO hasura cli is up to date                      version=1.2.1
time="2020-05-29T23:35:41+02:00" level=fatal msg="required flag(s) \"from\" not set"

It now requires the from command, so I´ve specified that providing the id of the first migration and it still hangs with the latest CLI version. The root cause is still this migration

time="2020-05-29T23:50:00+02:00" level=fatal msg="unable to squash migrations: 1580928345277 down migration not found"

I´ve at least managed to reproduce that issue easily and I believe the hanging was caused by that under further circumstances. To reproduce simply create a user table and afterwards drop it, but I´ve attached the migrations never the less.

hasura-migrations-issue.zip

Hasura + CLI were both on 1.2.1 while creating and droping those tables.

Hope that helps

[scriptonist]

Thanks for taking the time to do it @mklueh, really appreciate it. This will be of great help.

[pfic]

Any updates on this issue? I believe I am facing a similar problem. When squashing, I get the following error:

FATA[0003] unable to squash migrations: 1598634335411 down migration not found

The down migration specified contains only:

[]

Currently using Hasura console v1.3.2.