Kong: column "path_handling" of relation "routes" does not exist when migrating APIS from 0.14.1 to 1.5.0

[bathizte]

Summary

When trying to upgrade from Kong v.0.14.1 to Kong 1.5.0 our existing APIs using the super-nice command you introduce (thanks btw :+1: ) :

kong migrations migrate-apis 

You see a bunch of errors related to the absence of the column path_handing in the routes table :

migrating [...] failed (ERROR: column "path_handling" of relation "routes" does not exist)

Steps To Reproduce

1. Take a 0.14.1 kong instance with some APIs defined
2. Launch a kong 1.5.0 as per https://github.com/Kong/kong/blob/master/UPGRADE.md#3-suggested-upgrade-path
3. kong migrations up won't run since the APIs relation exist
4. kong migrations migrate-apis fails as summarized above

Our Workaround

We needed to apply the following schema migration first (which adds the path_handling column) :

_/usr/local/share/lua/5.1/kong/db/migrations/core/007_140_to_150.lua_

Suggested fix

Since the path_handling has been introduced a few days ago :
https://github.com/Kong/kong/commit/5146230e84e7d039fc501fdd5e0417213aea38b8

We now also need to trigger this migration
_kong/db/migrations/core/007_140_to_150.lua_
from
_kong/cmd/utils/migrations.lua_ ~327 ?

I may have missed something, such as kong migrations up --force was needed. In that case, it could be useful to others to mention it it he UPGRADE.md section.

[bungle]

@bathizte, did I read it correctly that it worked with --force? That is a good workaround and I am glad it worked, but this should not be required AFAIK. So I labeled it as a bug.

[bungle]

@hishamhm,

I think the order should be:

1. take 0.14.1 (db is updated to this)
2. take 1.5.0 code and run migrations migrate-apis and after that migrations update

So I think that this commit should be reverted:
https://github.com/Kong/kong/commit/5146230e84e7d039fc501fdd5e0417213aea38b8

[bathizte]

@bathizte, did I read it correctly that it worked with --force?

Huh, no, I did not mean that, I was just saying that _maybe I could have missed a step_ while reading the documentation.

To be crystal clear what worked was the following :

• Shutdown my 0.14.1 kong instance that has some APIs defined
• Launch a kong 1.5.0
• _kong migrations migrate-apis_ fails as summarized above
• Apply the content of the schema migration _kong/db/migrations/core/007_140_to_150.lua_, that is, add the the _path_handling_ column to the _routes_ table
• Then run kong migrations migrate-apis again, which now works
• Finally, run the migrations update & migrations finish

[bathizte]

I'll let you decide whether

• revert the commit 5146230 (isn't it needed for something ?)
• trigger the 007_140_to_150.lua migration

is the right way to fix the issue

[bungle]

I think if you revert https://github.com/Kong/kong/commit/5146230e84e7d039fc501fdd5e0417213aea38b8 then the migrate-apis will not fail in a first place. Then you run up and it should get that new column. What you did, you manually fixed this so that you added column, and then the migrate-apis worked, but it would have worked in a first place without that commit. We could also come up with patch that tries to add the column directly with migrate api. @hishamhm, do you know if there was any reason for that commit? The migrate-api only works with db that is 0.14.1.

[hishamhm]

@bungle let's just create the column during migrate-apis and things should work :+1:

[rubenlohaus]

as @hishamhm says, as workaround I only created the column before migrate-apis:

ALTER TABLE routes ADD COLUMN path_handling TEXT;

then up and finish

[kikito]

Hi, I have written a fix for this in #5572. I was able to reproduce the problem locally, and the changes in that PR fixed the issue.

[hishamhm]

Closing as #5572 is merged in preparation for Kong 1.5.1, which is coming shortly!