Graphql-engine: Managing multiple servers from cli

[shahidhk]

There could be scenarios where a user is dealing with staging and production environments. Development work happens in staging and and then it is pushed to production.

Right now, to switch servers and apply migrations on a different server, endpoint inside config.yaml has to be edited.

Instead if the CLI could take a flag --config-file=<file-name>, these switches can be made quickly, just by keeping two files and switching them in the command.

For example, there are two files, config-staging.yaml and config-prod.yaml.

# open console on staging server
hasura --config-file config-staging.yaml console

# apply migrations on production server
hasura --config-file config-production.yaml migrate apply

The default value for this flag will be config.yaml so that the change is backwards compatible.

Flag can also be set using an env var HASURA_GRAPHQL_CLI_CONFIG_FILE.

I also propose to move the name of directory in which migrations are stored into config.yaml, instead of hard coded migrations. This gives the user flexibility to use the CLI in any directory structure.

Config file becomes:

# config.yaml
endpoint: https://graphql-engine-endpoint
migrations_directory: migrations

Suggestions/Feedback welcome. Use :+1: :-1: reactions.

[0x777]

@shahidhk we do not want to have multiple environments in the same config file?

[shahidhk]

@0x777 Possible. But we cannot do this without breaking existing setup or introducing versioning for config file.

If we are to have different environments in the same file, it could look like this:

# config.yaml
version: v1.0
environments:
- name: staging
  endpoint: https://staging-endpoint.com
  migrations_dir: migrations
- name: production
  endpoint: https://production-endpoint.com
  migrations_dir: migrations

And commands will be:

hasura --environment staging console
hasura --environment production migrate apply

The current config.yaml will be version v0.0. CLI will be able handle both versions.

[coco98]

Do you think it'll be simpler just to have multiple conf files? By the way, for now, you can always override the conf file by passing the right values as flags right?

[shahidhk]

@coco98 Implementation is simpler for multiple config files.

Yes, endpoint can be over-ridden by passing it as a flag or env var as of now.

[pradel]

As a user I think I like more the possibility to have different environments in the same file and specify the env in the cli command.
For now how can we specify the endpoint with the cli? You said by passing it as a flag but I can't find which one 🤔?

[shahidhk]

@pradel You can provide endpoint/access key in 3 different ways

1. In the config.yaml file
2. HASURA_GRAPHQL_ENDPOINT env var
3. hasura --endpoint <hasura-endpoint> console

This precedence is flag > env var > config file

[pradel]

@shahidhk Thanks that what I was looking for!
Maybe it can be a good idea to add --endpoint in the Flags section of the hasura --help command?

[shahidhk]

@pradel It comes up for the relevant commands as its not a global flag.

hasura console --help

Run a web server to serve Hasura Console for GraphQL Engine to manage database and build queries

Usage:
  hasura console [flags]

Examples:
  # Start console:
  hasura console

  # Start console on a different address and ports:
  hasura console --address 0.0.0.0 --console-port 8080 --api-port 8081

Flags:
      --access-key string     access key for Hasura GraphQL Engine
      --address string        address to use (default "localhost")
      --api-port string       port for serving migrate api (default "9693")
      --console-port string   port for serving console (default "9695")
      --endpoint string       http(s) endpoint for Hasura GraphQL Engine
  -h, --help                  help for console
      --no-browser            do not automatically open console in browser
      --static-dir string     directory where static assets mentioned in the console html template can be served from

Global Flags:
      --project string   hasura project directory where the commands should be executed. (default: current directory)

[pradel]

@shahidhk Okay I see, I tried only the hasura metadata --help and not hasura metadata apply --help that's why, thanks!

[PierBover]

@shahidhk is this feature being implemented?

(multiple environments in either a single file or multiple files)

I guess hasura --endpoint <hasura-endpoint> console works but it's not ideal.

[KevinColemanInc]

I wrote this bash script to help me manage this so I don't accidentally push to the wrong server.

hasura-export

#!/bin/bash

cp stg.config.yaml config.yaml
hasura metadata export
rm config.yaml
echo "downloaded stg"

but yeah, it would be nice if this support multiple envs

[denis-ryzhkov]

@shahidhk

You can provide endpoint/access key in 3 different ways

1. In the config.yaml file
2. HASURA_GRAPHQL_ENDPOINT env var
3. hasura --endpoint <hasura-endpoint> console

This precedence is flag > env var > config file

When I use HASURA_GRAPHQL_ENDPOINT: http://localhost:8080/
in docker-compose.yaml:environment
for image: hasura/graphql-engine:v1.2.2.cli-migrations-v2
then docker-compose logs shows this error:

level=fatal msg="version check: failed to get version from server:
failed making version api call:
Get http://localhost:8080/v1/version:
dial tcp 127.0.0.1:8080: connect: connection refused"

But when I use endpoint: http://localhost:8080/ in config.yaml instead, then it works.
Why environment variable does not work?
Is it caused by https://github.com/hasura/graphql-engine/issues/1558 ?

[scriptonist]

Hey @denis-ryzhkov , correct me if I got it wrong, but this is what I understood.

You are running a hasura instance with a docker-compose file which is similar to the following,

Docker compose file

version: '3.6'
services:
  postgres:
    image: postgres:12
    restart: always
    volumes:
    - db_data:/var/lib/postgresql/data
    environment:
      POSTGRES_PASSWORD: postgrespassword
  graphql-engine:
    image: hasura/graphql-engine:v1.2.2.cli-migrations-v2
    ports:
    - "8080:8080"
    depends_on:
    - "postgres"
    restart: always
    environment:
      HASURA_GRAPHQL_DATABASE_URL: postgres://postgres:postgrespassword@postgres:5432/postgres
      HASURA_GRAPHQL_ENABLE_CONSOLE: "true" # set to "false" to disable console
      HASURA_GRAPHQL_ENABLED_LOG_TYPES: startup, http-log, webhook-log, websocket-log, query-log
      ## uncomment next line to set an admin secret
      # HASURA_GRAPHQL_ADMIN_SECRET: myadminsecretkey
volumes:
  db_data:

Now, you created a project

$ hasura init

and you are trying to run let's say the console command

$ hasura console

This is working since we have http://localhost:8080 in config.yaml

The you created another project with a similar docker-compose file but exposing the server at http://localhost:8181 and you want to connect to this server from the CLI using the environment variable right?

If that's the case you can try

$  HASURA_GRAPHQL_ENDPOINT="http://localhost:8181"  hasura console

[denis-ryzhkov]

Hey @scriptonist , thank you for explanation

I want to configure Hasura via environment variables only, in a DRY way,
passing them to docker-compose.yaml, to remote deployment, and to hasura console,
so I tried to avoid hardcoding endpoint in config.yaml

Now I see .cli-migrations-v2 image needs neither docker-compose.yaml:HASURA_GRAPHQL_ENDPOINT nor config.yaml:endpoint to connect its built-in Hasura CLI to Hasura Engine,
so I just pass HASURA_GRAPHQL_ENDPOINT to hasura console only

My question is solved, thank you!