Flyway: Environment variable substitution in flyway.conf yields weird experience

[tarciosaraiva]

Which version and edition of Flyway are you using?

6.1

If this is not the latest version, can you reproduce the issue with the latest one as well?

(Many bugs are fixed in newer releases and upgrading will often resolve the issue)

Yes

Which client are you using? (Command-line, Java API, Maven plugin, Gradle plugin)

Command-line, docker based

Which database are you using (type & version)?

PostgreSQL 10

Which operating system are you using?

Docker under Mac OS and Amazon Linux

What did you do?

(Please include the content causing the issue, any relevant configuration settings, the SQL statement that failed (if relevant) and the command you ran.)

I have a simple configuration file that looks basically like this:

flyway.url=${JDBC_URL}
flyway.password=${FLYWAY_PASSWORD}
flyway.user=flyway

The password is dynamically generated and rotates every day and we are not discounting special characters. While testing the password contained the following special characters:

(,$[

Assigning the password to the environment variable FLYWAY_PASSWORD and running the Docker image was resulting in the Java-based error illegal group reference. An easy search gave me the SO result.

I then thought to myself that I needed to escape the characters. I did that (by hand) but forgot to escape the dollar sign and still got the same error. Tried again, this time escaping the dollar sign, and whatever Flyway does with the replacement worked but I got wrong password.

What did you expect to see?

I expected the password, passed in to the env variable, would be unescaped somehow or accounted for special characters.

What did you see instead?

I got a wrong password message.

[tarciosaraiva]

I thought I would mention that this problem goes away if I use the environment variable FLYWAY_PASSWORD without assigning to flyway.password in the configuration file.

Only now I noticed that Flyway actually expects an environment variable of that name anyway and I guess use them two caused a substitution error.

In any case, this is the debug info

DEBUG: Loading config file: /flyway/conf/flyway.conf
ERROR: Unexpected error
java.lang.IllegalArgumentException: Illegal group reference
    at java.util.regex.Matcher.appendReplacement(Matcher.java:857)
    at java.util.regex.Matcher.replaceAll(Matcher.java:955)
    at java.lang.String.replaceAll(String.java:2223)
    at org.flywaydb.core.internal.configuration.ConfigUtils.expandEnvironmentVariables(ConfigUtils.java:335)
    at org.flywaydb.core.internal.configuration.ConfigUtils.loadConfigurationFile(ConfigUtils.java:316)
    at org.flywaydb.commandline.Main.loadConfigurationFromConfigFiles(Main.java:423)
    at org.flywaydb.commandline.Main.main(Main.java:98)

Feel free to close this ticket if not relevant. I did not test the scenario using flyway.password=${MY_ENVVAR_PWD} where MY_ENVVAR_PWD has special characters.

[alextercete]

@tarciosaraiva Thanks for reporting this! We have produced a fix which will be available in 6.1.1.

[tarciosaraiva]

That's great to hear @alextercete - happy to help!