moleculer-runner support for loading config file asynchronously

[jodaks]

Is your feature request related to a problem? Please describe.

We have a policy to not write any passwords/tokens on any config files or environment variables. So we store any security related config values in a vault and some on AWS Parameter Store (encrypted). This is to avoid checking in any passwords/tokens to the repository (since we actually checkin our .env file too)

One of those secure values includes the RabbitMQ url (because it includes the username and password). So before we can pass it as a value to the { transporter: { type: 'AMQP', options: { url: '[url_with_password]' } } }, we need to load the value by calling an async function first.

But because the way the config file is being loaded by the service runner, any async calls are not being waited upon before the config is loaded.

Describe the solution you'd like

A simple hack we did was to add an async/await to the service runner code. Because the runner already loads the config in a promise chain .then(loadConfigFile) line in the service-runner.js, adding async function loadConfigFile() { and configFile = await require(filePath); does the trick for us.

For reference here's what our moleculer.config.js file look like:

"use strict";

require('dotenv').config();
const fs = require("fs");
const path = require("path");
const paramStore = require("@company/engagement-common/lib/aws/param-store").default;
const bunyan = require("bunyan");
const _get = require("lodash/get");

// setup logging
const logPath = "logs/services.log";
const logDir = path.dirname(logPath);
fs.mkdirSync(logDir, {recursive: true});
const logger = bunyan.createLogger({ 
    name: "engagement", 
    level: _get(process.env, "LOG_LEVEL", "info"),
    streams: [{
        type: "rotating-file",
        path: logPath,
        period: '1d',
        count: 3
    }]  
});

/**
 * load configuration
 */
const getConfig = async function () {
    // TODO: load the following multiple params using one call instead of getting individual values
    const host = await paramStore.getValue(`/db/host`);
    const pwd = await paramStore.getValue(`/db/pwd`);
    const user = await paramStore.getValue(`/db/user`);
    const name = await paramStore.getValue(`/db/database`);
    const rabbitmq = await paramStore.getValue('/rabbitmq/connection');
    const searchvol = await paramStore.getValue('/search-volume-api/key');

    // put some stuff on the global.config
    global.config = {
        env: _get(process.env, "NODE_ENV", "development"),
        db: {
            host: host.Value,
            pwd: pwd.Value,
            user: user.Value,
            name: name.Value
        },
        process: {
            cacheLength: parseInt(_get(process.env, "CACHE_TTL", 60))
        },
        api: {
            searchVolumeKey: searchvol
        },
        logger,
        rabbitmq: rabbitmq.Value
    };

    // return the moleculer specific config
    return {
        namespace: "engagement",
        nodeID: null,

        logger: bindings => logger.child(bindings),

        transporter: {
            type: "AMQP",
            options: {
                url: global.config.rabbitmq,
                autoDeleteQueues: true
            }
        },

        cacher: "Memory",
        // other misc moleculer config follows...
    }
}

module.exports = getConfig();

Describe alternatives you've considered

We have no alternatives in mind as of the moment. Hence why I am asking for this feature/change request. If there is a better solution, please point me to the right direction. Thanks!

Additional context

No additional context

[icebob]

I can make changes that after the runner required the config file it checks it is a function. If yes, then call it and add the result to the Promise chain, so it can be async. If it's an object it does the same as before. What do you think?
By the way, I can only add it into the next 0.14 version.

[jodaks]

Sounds good, I think that would work as long as if it is a function it will still await if the return is a promise/async. And if it is not a promise, it would be loaded same way as if it got a standard config object (as per current behavior).

And yes, adding it on 0.14 works for us, because we are still in active development and will probably not be production ready anytime soon , so we should be able to migrate to 0.14 (assuming it will come out anytime soon) before our production release.

Thank you @icebob (and all contributors) for such an awesome framework!

[icebob]

Ok, then I'll change it accordingly

[icebob]

It's done. You could try it with npm i moleculerjs/moleculer#next

[jodaks]

works great, thanks @icebob!