Prisma1: Cluster Configuration `cluster.yml`

[sorenbs]

The goal of this issue is to suggest a format for the configuration of a Graphcool cluster. It specifies information like the connection to the database, the function runtime that is being used or secrets being used by the servers.

Once a cluster config has been defined it can be transformed into an executable cluster config, e.g. a Docker Compose file or an AWS CloudFormation template. This executable config can then be submitted to the cluster solution in use e.g. Docker Swarm or AWS CloudFormation. This way the configuration of a Graphcool Cluster is independent of the actual Cluster Solution in use.

                                                       
                ┌───────────────────────┐              
                │    Cluster Config     │              
                └───────────────────────┘              
                            │ transformed into         
                            ▼                          
                 ┌────────────────────┐                
                 │                    │                
                 │ Executable Cluster │                
                 │       Config       │                
                 │                    │                
                 └────────────────────┘                
                            │submitted to              
                            ▼                          
                   ┌─────────────────┐                 
                   │Cluster Solution │                 
                   └─────────────────┘                 

The proposed YAML format looks like the following. The keys mean the following:

• database:
  
  
  
  • meta: This config specifies the database the schema of a service is stored in.
  
  
  • data: This database contains the actual data of services.
  
  
• messageQueue: This config specifies the Message Queue being used, e.g. RabbitMQ, Amazon Kinesis on In-Memory.
• cluster: Cluster settings
  
  
  
  • gateway: This is DNS name under which the Graphcool API is available. It is needed so that the System API knows how to call the Simple API for example. It depends on what the system administrator has set up. In the case of the local version this would be simply localhost.
  
  
  • clusterSecret: This secret is used to authenticate the CLI with the System and Deploy APIs.
  
  
• serviceLimits:
  
  
  
  • requestsPerSecond: max requests per second allowed per service
  
  
  • maxNodes: max number of nodes returned for a single relation
  
  

database:
  meta:
    type: mysql
    config:
      url: myhost
      port: 1234
      user: graphcool
      password: graphcool
      database: database
  data:
    type: mysql
    config:
      url: myhost
      replicaUrl: myReplicaHost
      port: 1234
      user: graphcool
      password: graphcool

cluster:
  clusterSecret: "sdjflksdfj"
  gateway: "gateway.supercorp.com"

messageQueue:
  type: rabbitmq
  config:
    url: amqp://foo:[email protected]/quiz

limits:
  requestsPerSecond: 30
  maxNodes: 1000

Considerations

• [ ] Cloud provider configuration

[sorenbs]

This should also specify log forwarding https://github.com/graphcool/graphcool/issues/586

[marktani]

The current default timeout of 15 seconds per function invocation should be configurable as well: 0 - 300 seconds.

[marktani]

All limits should be configurable. Examples:

• size limit for Strings/JSON/... fields
• rate limiting

[aaronholla]

Any update on this?

I have an upload script that keeps hitting pagination limits and field size limits. It would be nice to be able to adjust these database settings.The only way I can get the script to run consistently at the moment is removing all concurrency as the mutations are pretty complex.

[sorenbs]

@aaronholla - are you using Graphcool Framework or Prisma?

[aaronholla]

Prisma self hosted.

[sorenbs]

This is fairly high on our roadmap. The way we will tackle this is to firs introduce environment variables that can be set individually for everything that should be configurable. Later we will introduce a single configuration file where all of this can be configured.

If you or anyone else is interested in taking a stab at making the pagination limit configurable, then we are happy to accept a pull request.

What is the limit you are hitting for field size?

[aaronholla]

Im not very familiar with the source code, I can look into it to see if I can add environment variable. But if someone else more familiar would like to make a pull request I won't mind.

The field limit was really interesting I was just getting an internal server error. I narrowed it do the description fields being too long. Is there a directive to set a string field in the datamodel to a MySQL data type? (Like setting that feild to TEXT or LONGTEXT)

[raymclee]

is there any doc for this? thanks

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 10 days if no further activity occurs. Thank you for your contributions.

[fred8617]

same question for the limit length