Prisma1: Templates

[timsuchanek]

Introduction

As mentioned in #719 and #605 the current concept of modules is not fulfilling what it's promising, also the responsibilities and boundaries of a module are not clear.
The vision is, that we definitely want to have a nice packages, modules or plugins system, however you might call it.

The requirement you have to such a reusable collection of code and definitions is:

• Versioning, which allows updates
• Reusability without touching the code

Both are currently not the case with modules. So the idea of this proposal is to have a more "honest", short-term approach, that doesn't promote something it doesn't hold but helps to put together code, that you want to use in your project, like authentication integrations.

Later, when we have a better understanding of all this, we want to implement a solution that fits the requirements.

Until then, the proposal is to disable modules

Templates

The idea of a template is, that it is basically a collection of code, that you still have to add to your project manually. In other words, copy-pasting code from a GitHub repo and manually adding it to the graphcool.yml would be the same as "adding a template".
This may sound like a step backward from modules, but it fosters the understanding of developers of what is actually going on in the code they are pulling into the project.

With having to manually add it to the project, you're forced to grapple with the code and automatically get a better understanding of how pieces are put together.

CLI Command

In order to make "copy-pasting" easier, the CLI would still expose a command to gather code from github.

The current modules add command would be renamed to add-template:

gc add-template graphcool/templates/authentication/email-password

graphcool.yml

A template still just has a graphcool.yml, but the content of this file will be added to the main project, as comments. Concretely it will look like this:

github.com/graphcool/templates/authentication/email-password/graphcool.yml

# GraphQL types
types: ./types.graphql

# functions
functions:

  signup:
    type: resolver
    schema: src/signup.graphql
    handler:
      code: src/signup.js

  authenticate:
    type: resolver
    schema: src/authenticate.graphql
    handler:
      code: src/authenticate.js

  loggedInUser:
    type: resolver
    schema: src/loggedInUser.graphql
    handler:
      code: src/loggedInUser.js

# Permanent Auth Token / Root Tokens
rootTokens:
  - signup
  - authenticate
  - loggedInUser

Will be added to the main graphcool.yml like this:

functions:
# uncomment to activate the `email-password` template:
#
#  signup:
#    type: resolver
#    schema: src/email-password/signup.graphql
#    handler:
#      code: src/email-password/signup.js
#
#  authenticate:
#    type: resolver
#    schema: src/email-password/authenticate.graphql
#    handler:
#      code: src/email-password/authenticate.js
#
#  loggedInUser:
#    type: resolver
#    schema: src/email-password/loggedInUser.graphql
#    handler:
#      code: src/email-password/loggedInUser.js

  current-function:
    type: resolver
    schema: src/hello.graphql
    handler:
      code: src/hello.js


rootTokens:
# uncomment to activate the `email-password` template
#
#  - signup
#  - authenticate
#  - loggedInUser

The code of a function will be added to src/TEMPLATE_NAME.
If there are naming conflicts (function name, root token name, file name) - the template installation will gracefully be stopped and an error will be printed.

The comments are all added inline. That means functions are added where current functions are defined etc. If the top level doesn't exist yet, it will be appended to the graphcool.yml.

If the top-level type like functions already exists, the out-commented new functions will be prepended to the list of functions, because it's hard to know where a definition stops, like in this example:

functions:
  hello:
    code:
      src: ./hello.js
#   hello2:
#    code:
#      src: ./hello.js
#permissions:
#  - operation: User.read

types.graphql

The types.graphql file will be adjusted like this:

1. If there us no user type yet:

# uncomment to enable the `email-password` template:
# type User implements Node {
#  id: ID! @isUnique
#  email: String @isUnique
#  password: String
# }

1. If there already is a User type:

type User implements Node {
  id: ID! @isUnique
  currentField: String!

# uncomment to enable the `email-password` template:
#  email: String @isUnique
#  password: String
}

This is basically flattening the concept former known as modules to a more transparent, less-magic concept called templates.

[nikolasburk]

I’d prefer add-template over install-template as a CLI command

[kbrandwijk]

I'd prefer template add over add-template as a CLI command

[nikolasburk]

@kbrandwijk I think this would make sense if we planned to add more subcommands like template update or templete remove. But this is not the case - templates really should only be a lightweight temporary solution until we have worked out a proper module/package system.

Another thing I just realized that could be a bit confusing is that the gc init command takes the --template option. Unless we somehow connect this to our idea of templates that we're discussing here, we should probably rename that option.

[marktani]

@timsuchanek, you did not spec out this section:

1. If there already is a User type:

[timsuchanek]

@marktani thanks, just added that section

[kbrandwijk]

@nikolasburk But if you decide an additional command is needed you might end up with yet another breaking change...

Question: you add a template, you uncomment the changes, then the template is updated. Would you be able to add the template again, and only get the changes commented? For example, if a field is added to a type, or a function is added.

Would templates support the rename/deprecated attribute, so you can release a new version with a renamed field in the template?

[marktani]

Would you be able to add the template again, and only get the changes commented? For example, if a field is added to a type, or a function is added.

I think in this installment of templates this is not needed. What you would do is the same template again, which would add a few new commented-out lines. You as a developer need to 'merge' the old version of the template with the new version.

I think this is a great trade-off for now. Templates are very much not designed to provide any kind of package management or "context aware installation".

[kbrandwijk]

@marktani That's exactly what I meant. When a new field is added to a template, and I add it again, I would just get that one field added as commented-out line to my types.graphql.

[marktani]

But you were talking of only the changes to be commented. However, I don't think this should happen, as it would require a "diff" between the added things and the current state.

I think this diff should be the responsibility of the developer for now.

[kbrandwijk]

The diff already happens when adding the template. Because only the items that are not in your project yet, are added if I understood correctly. For example, if the template adds 2 fields to User, and I already have 1 of those fields on User, it would just add the other field, right?

[marktani]

Good point, this is not obvious from the proposal and needs further clarification. What's your take, @timsuchanek?

I don't have a strong opinion either way.

[sorenbs]

The diff already happens when adding the template. Because only the items that are not in your project yet, are added if I understood correctly

I think this is too much magic. What happens if the field exists, but the type is different? What happens if the function exists, but the code is different?

@tim please specify what should happen when there are conflicts in

• types.graphql
• graphcool.yml
• file-name conflicts

[timsuchanek]

As said in the proposal:

If there are naming conflicts (function name, root token name, file name) - the template installation will gracefully be stopped and an error will be printed.

We may want to add a --force option later, that still adds the comments even if there are conflicts.

[sorenbs]

The code of a function will be added to src/TEMPLATE_NAME

is TEMPLATE_NAME a folder?

if so, is this a convention that all templates have to adhere to, or is it something the CLI enforces?

Concretely, if a template is called some-template and has a file at /someFunction.js will the CLI do the following?:

1. add the function at /src/some-template/someFunction.js
2. update the new commented-out function object in graphcool.yml to have the correct path

[marktani]

I think the template could define a default value for TEMPLATE_NAME, and additionally the command would offer an optional parameter to overwrite this.

If the folder ./TEMPLATE_NAME already exists, the process of adding the template is aborted.

[timsuchanek]

As with modules I suggest taking either the repo name, if it points to a repo, or the directory name when it's a directory in a repo for the template name.

[sorenbs]

Ok

Then my current understanding is that this is convention based. So in a template called email-password this is true:

1. Either the repository or the folder is called email-password. We don't support the suggestion from @marktani of overriding this.
2. All files must already be placed in ./src/email-password/ and functions in graphcool.yml must use the full path when referencing files (see example below).
   
   
   
   1. The CLI doesn't change the file paths in the graphcool.yml file
   
   

#  signup:
#    type: resolver
#    schema: ./src/email-password/signup.graphql
#    handler:
#      code: ./src/email-password/signup.js

Is this correct?

[kbrandwijk]

So when you create a template, you already have to put your sources in a folder with that name? Or can I just place code files in ./src/ when creating a template?

[sorenbs]

That's the question :-) My proposal is the former but I'm sure @timsuchanek will clarify.

[lastmjs]

I just want to throw in a couple thoughts here. I've seen something similar to this situation happen before with third-party TypeScript types. If I remember correctly, in the past if you wanted to add third-party TS types, you had to use some specialized tooling to do it. I remember that being slightly cumbersome to learn and remember. With TypeScript 2.0, typings were simplified and included through simple npm installs. That has made things a lot simpler. I think graph.cool should do something similar. Keep things simple and just let these modules/templates be installed from npm. I'm not sure how it would work, but as little extra tooling to get this to work as possible will pay dividends in simplicity, in my opinion.

For example, I imagine something like the following happening someday. Let's say I want to add the email-password authentication module:

npm install @graphcool/modules/email-password

Now I have all of the code in my project. All I need to do is point my configuration files to where the code is installed in my node_modules directory. We could even have the graph.cool CLI aware that all modules/templates will be installed into node_modules/@graphcool/modules (some known location), and just refer to the modules/templates by name in the graph.cool configuration files, no extra plumbing necessary by the developer. That is similar to how TypeScript works. You can add a types or typings property to package.json, and the TypeScript compiler just knows to look under node_modules/@types (if I'm not mistaken).

[kbrandwijk]

Good point @lastmjs. I created a similar proposal here: https://github.com/graphcool/graphcool/issues/637. This was for allowing to install modules from npm/github etc., like npm does. Actually installing them as npm module is a nice thought, but most modules cannot be used 1-on-1. They have to be imported, and integrated into your project. Because of adding to types, adding relations, etc. So I don't think being able to use them as isolated modules would work.

[lastmjs]

My last thought on it is a postinstall script in each module that does that work.

[kbrandwijk]

I would like to reference the Heroku cli in this context. They have a plugin concept, that relies on npm packages. It works like this: first, you npm install the package, then you execute a cli command to actually add the plugin.

[lastmjs]

Cordova has a similar concept

[schickling]

Thanks a lot for bringing this up @lastmjs! We're definitely looking into this going forward but at least for now we need a simpler solution.

[kbrandwijk]

@schickling A lot of open questions, and the status changed from needs-review to accepted? So what's the final version?

By @nikolasburk :

Another thing I just realized that could be a bit confusing is that the gc init command takes the --template option. Unless we somehow connect this to our idea of templates that we're discussing here, we should probably rename that option.

By @marktani (about https://github.com/graphcool/graphcool/issues/720#issuecomment-333550056):

Good point, this is not obvious from the proposal and needs further clarification. What's your take, @timsuchanek?

By @sorenbs (about https://github.com/graphcool/graphcool/issues/720#issuecomment-333568140):

That's the question :-) My proposal is the former but I'm sure @timsuchanek will clarify.

[schickling]

Thanks for following up on this @kbrandwijk! Regarding the open questions:

1. gc init template argument: I'd suggest to disable the gc init --template workflow for now and look for a better solution here in #756

2. Type field diffing: I like the suggestion to just add the remaining fields ✅

3. Folder name injection: I agree with the solution to put all src files into a subfolder which is named after the template. The file references in the graphcool.yml file will be automatically adjusted as pointed out in (2) by @sorenbs in his comment.

[kbrandwijk]

Regarding 3: This means that the code does _not_ have to be in a subfolder below src already when creating the template? And that it would be put there when adding the template?

[timsuchanek]

We now implemented the add-template command and released it in 0.7.0!
Try graphcool add-template graphcool/templates/auth/email-password to get started :)