Presto: configure prestosql catalog and config.properties files dynamically

[mostafadgwell]

i am trying to use prestosql environment variables to pass some variables to configure the catalogs and even the presto worker or coordinator dynamically, i saw the following links and tried, but didn't find a correct way to implement like

https://prestosql.io/docs/current/security/secrets.html

but i didn't be able to configure them although printenv command has the variables configured correctly,
so, i appreciate any help or link that may help in this

[electrum]

The server will print the actual config values (after replacement) on startup. What is shown here?

[mostafadgwell]

thanks for your answer, and I tried but the server is not replacing these variables, so, I will describe in details the configurations I did, may it will help you

1- I am using prestosql version 337
2- I am using 2 catalogs which are added under the etc/catalog folder
3- the database configurations for mongo for example is as follows:

connector.name=mongodb
mongodb.seeds=${ENV:mongoip}:${ENV:mongoport}
mongodb.credentials=${ENV:mongousername}:${ENV:mongopassword}@test

4- I am running presto in a docker container which I am sure that I added the needed environment variables, and I am printing these variables correctly inside the container using printenv command.
5- I added the discovery server ip also as an environment variable in config.properties.
6- when starting the presto server, these variables are not replaced at all with the real variables exist in the environment variables
7- also, I changed the syntax of ${ENV:mongoport} to be ${mongoport} and to be $mongoport, without any useful result,
8- also, I tried to connect using presto-CLI, but same result, can't read the real values

so, I am asking for the correct way to write these variables in the configuration files to be able to dynamically configure the presto configurations

thanks in advance

[mostafadgwell]

hi guys,
I think this is a bug right?

[electrum]

It looks like we only support replacing entire values, not within a value. It shouldn’t be hard to change this to support your use case.

[ssquan]

It's better to store these catalog properties in a config map and mount it into containers.

[pcastellazzi]

I have a similar issue with compound values: property=${ENV:VALUE} works as expected, but property=${ENV:PART1}${ENV:PART2} is taken as a literal instead of a doing multiple replacements.

I tracked the issue to the implementation of replaceEnvironmentVariables.

[choiwaiyiu]

@electrum @findepi Hello, I am a newcomer. May I work on this issue?

[findepi]

@choiwaiyiu sure! as @pcastellazzi pointed out the necessary change would actually need to go in https://github.com/airlift/airlift (a library Presto uses extensively)

[choiwaiyiu]

@findepi Thank you for the clarification.
I have created a PR in the airlift project. https://github.com/airlift/airlift/pull/872
I am not able to invite reviewers but this PR is ready for review. Thank you.

[findepi]

https://github.com/airlift/airlift/pull/872 is merged -- thanks @choiwaiyiu !
this will be available to Presto when we upgrade to Airlift 201.

[choiwaiyiu]

@findepi This is actually my first contribution to open-source projects. Thank you for your guidance and advice!

[findepi]

https://github.com/prestosql/presto/pull/5381