Netlify-cms: Override-able config (mainly for dev config)

[Brondahl]

Is your feature request related to a problem? Please describe.
The committed value of branch in config.yml is naturally production. When I'm doing dev on a branch that involves using the CMS to change .md files then have to choose between either:

• commit dev CMS changes to prod
• maintain a floating uncommitted change modifying config.yml to point at my dev branch.
• commit the above change and risk forgetting to revert it before merging into master.
  None of these options are fully satisfactory.

Describe the solution you'd like
Ideally I would have a (.gitignored) config.dev.yml, which defined an override for the branch config property, which would target my dev branch. Naturally other properties could be overridden, and might well be useful.
Some other form of override, which can be maintained in any git-friendly way, could be suitable.

Describe alternatives you've considered
See above. Don't override it. Override it but don't commit it. Override and commit that override.

Additional context
Nothing comes to mind.

[Brondahl]

@erquhart You look to be the technical lead on netlify-cms :)
Is this something that you'd consider in Netlify-CMS?
If so, then I'd be open to developing the PR to support it.

Note that this is conceptually related to https://github.com/netlify/netlify-cms/issues/1737, (in that it would solve that particular problem) but I'm proposing a much broader support to cover a much wider range of needs.

[erquhart]

I like the idea of supporting a workflow, like staging/production. I'd rather see a simplistic approach to that than something involving an additional config file.

[Brondahl]

Cool - I'm glad you're in favour of the conceptual feature :) Do you have something in mind for an alternative approach?

The file.<environment>.ext pattern is a pretty standard approach across a lot of frameworks and languages, (JS: .env.<environment>, C# appSettings.<environment>.json) and is something that I generally consider pretty simple?
It's very little additional effort for developers - if they've already got the system working serving 1 file in a known place, then having another file(s) beside it should be very painless

Anything programmatic is going to start being (IMO) a lot messier, as then your config is going to live in multiple places in the repository codebase (or if you add something where they have to programmatically read a file sitting beside config.yml, then that's significant extra complexity.)

[Brondahl]

The only other altenative option that comes to mind is having different environment options defined within the _existing_ config.yml file?

Obviously that is much easier to use, as it's working within the existing file and config setup.

But that seems like a poor idea for a couple of reasons:

• Changing the format of the existing file is inherrently riskier as a code change (for introducing bugs)
• Changing the format of the existing file is slightly harder to do in a backwards compatible manner
• It means you've got a single file which is mixing different environments together, (as well as with cross-environment defaults) (YMMV on whether you think this is a pro or a con :) )
• It would work fine for prod and staging where you want the changes committed, but less well for dev where you ideally want the dev settings git-ignored, or at least in a easily trackable place that you can watch out for in git. (you get dev settings changes mixed in a single git commit of a file with 'real' settings changes)

[robsonsobral]

Some others CMSs use this approach, too. If @erquhart is against it, maybe we could have different branchs per domain. Let's say, adding to:

branch: master

The config branchs:

branchs:
  master: domain.com
  dev: develop.domain.com

Or something like that.

This branch restriction is really causing friction with my clients contact team.

Thank you all!

[robsonsobral]

Duplicate?

[Brondahl]

@robsonsobral It's really not a duplicate. Like I said in the second comment:

this is conceptually related to #1737, (in that it would solve that particular problem)
but I'm proposing a much broader support to cover a much wider range of needs.

[Brondahl]

@robsonsobral The point immediately above is why your suggestion (which would definitely fix #1737) isn't a solution for THIS issue.
What about name (which I've just had to implement a work-around for, because we have different needs for prod and dev login)
What about publish_mode?

Lots of settings could be environment dependant.

You would have to have (or allow for) those environment sub-configs defined within EVERY config value.

[Brondahl]

(I imagine it's clear that even if we decide not to support this general option, I'm still in favour of supporting per-environment git-branches. I would just prefer to fix the problem at its root, rather than one particular common symptom.)

[Undistraction]

FWIW, I think this is worth thinking about not (just) in terms of environments, but in terms of branches. netlify makes it a breeze to push arbitrary branches and have them deployed. This makes it easy to have multiple staging deploys (per issue, per feature, per fix etc). In each case it makes sense to scope any changes to that branch so they are sandboxed. So just having staging and production really limits the potential.

[Brondahl]

@Undistraction If it's per-environment then it shouldn't be dictating the names of those environments. So you'd have an environment for each branch you want to care about. (In my case, my "staging" is actually master.mySite.com)

I think what I'm saying is, "yes, I agree, and what I had in mind doesn't cause the restriction you're concerned about. I think."

[Brondahl]

I'm imagining it would read an Env.Variable (possibly NODE_ENV, possibly something custom) to determine the correct config file(s) to load.
It could potentially also do some automatic stuff about checking the domain to detect dev etc.

[Undistraction]

@Brondahl I totally get where you're coming from. It is the standard way of doing things in many situations, but it feels to me like it is a constraint born of complexity - of having to maintain a staging environment with all its moving pieces. The beauty of static sites is that those moving pieces are gone (in many cases), so there is no need for a single staging environment. The way I have things set up now, I automatically build and deploy any time I push a branch. If I want to work on a feature and share it, I just push my branch and share the URL. I'm not limited to a single 'staging' environment, and I don't want to have to create a new config file every time I want to deploy a branch. This freedom is a great thing.

I guess there is a happy medium where you can define a staging file which matches branches with names that satisfy a regex.

I think this all ties in with a larger problem which is that changes you make with the CMS while running locally are not then available to your local app until you pull from that branch which is a real pita.

[Brondahl]

@erquhart We're still waiting on a response from you.
Do you have some other approach in mind?
Would you accept a PR that solved this problem in the way I've suggested?

[erquhart]

Netlify CMS is Git-centric, so using different branches is the way this is generally handled. Is there anything about that approach that doesn't work for your use case?

[erquhart]

@Brondahl revisiting this one.

I don't think this issue is broader than #1737, just different. The approach championed here is for devs doing local work on the CMS, whereas #1737 would work for non-technical editors creating content(who wouldn't be creating their own local config.dev.yml).

I'd like to hear from other CMS implementors who feel this would be valuable. @talves, @tech4him1, @barthc?

[talves]

Facts:

• Backend backend: configurations are specific to the registered backend you are using
• Not all backends will have a branch value or context
• Any backend (core or custom) can have any configuration you want

Assumptions:

• Any configuration config.yml read into the CMS at the time of load or init should be final

Solution(s):

1. Override your config using manual init to change the backend based on the context of your environment
2. Write a custom backend to handle the configuration based on your requirements
3. Change the config.yml for the context prior to your build using a node cli script in your workflow

---

I am of the opinion that the CMS should not be opinionated on the dev context vs a production context for setting up a config. Only the backend should drive those decisions.

** I reserve the right to change my mind 😜

[renestalder]

Maybe, this helps someone. What I do is simply use a Netlify CMS config with placeholders, copying that file on every build to my admin folder and search/replace the variables. This allows me to automatically change the branch without the need to think about that. I always target the current branch I'm working on. I never work on the production branch that is deployed and where my customers edit the content.

I found this very convenient to use in my projects. Especially I can be sure that if I send the link to the customer to test a preview branch on Netlify, that they really only edit files on that branch.

I don't feel like it's the best option for maintenance, but it's simple and it works.

./netlify-cms-config.yml

backend:
  name: gitlab
  repo: website/repo
  branch: NETLIFY_CMS_BRANCH # Will be replaced during build with the current branch

A little bash script for copy and replacement that checks on which branch I'm currently locally:

prebuild.sh

#!/usr/bin/env bash

echo "Starting prebuild process"

USER_SET_BRANCH=$1

DETECTED_BRANCH=$(git branch | sed -n -e 's/^\* \(.*\)/\1/p')
BRANCH="${USER_SET_BRANCH:-$DETECTED_BRANCH}"

# Set branch on which the CMS edits files by creating
# the netlify config from the netlify-cms-config and overwrite
# the branch setting.
yes | cp -rf ./netlify-cms-config.yml ./static/admin/config.yml
sed -i -e "s|NETLIFY_CMS_BRANCH|$BRANCH|g" ./static/admin/config.yml

If you're using npm, in your package.json:

package.json

{
  "scripts": {
    "prebuild": "chmod +x ./prebuild.sh && ./prebuild.sh"
  }
}

For using with Netlify, your netlify.toml then can look something like this, which passes the branch Netlify receives to the prebuild script.

netlify.toml

[build]
  publish = "dist/"
  command = "npm run prebuild -- $HEAD && npm run whateverbuildsyourapp"

[talves]

@renestalder this is exactly the way I would recommend someone think about their workflow.

I don't feel like it's the best option for maintenance, but it's simple and it works.

The best option is the option that works for you. This is a very viable option and should be considered by those needing a solution if they want to change their config.yml based on branch

I have recommended something similar, but instead of the overrides, I just copied based on the context (branch) of the file version, so there could be a test branch. test.config.yaml with different layout that exists when I have a proposed config change. If the branch config does not exist, then fall back to copying the master.config.yaml.

[gambinish]

@renestalder

Thank you!! I was really running into a wall with this exact issue and your solution helped me finally break through it.

A humble suggestion as far as naming conventions go:

Bootstrapping a gatsby boilerplate, my npm scripts in my package.json were as follows:

"scripts": {
    "prebuild": "chmod +x ./prebuild.sh && ./prebuild.sh",
    "clean": "rimraf .cache public",
    "start:app": "npm run develop",
    "start:lambda": "netlify-lambda serve src/lambda",
    "start": "run-p start:**",
    "build:app": "npm run clean && gatsby build",
    "build:lambda": "netlify-lambda build src/lambda",
    "build": "run-p build:**",
    "develop": "npm run clean && gatsby develop",
    "serve": "gatsby serve",
    "format": "prettier --trailing-comma es5 --no-semi --single-quote --write \"{gatsby-*.js,src/**/*.js}\"",
    "test": "echo \"Error: no test specified\" && exit 1"
  }

this caused the prebuild script to get triggered twice, once during the prebuild process, and once during actual build, causing the .sh script to fetch incorrectly as the prebuild script contained the build keyword.

Changing it (and the netlify.toml) to the following solved the issue.

"scripts": {
    "pre": "chmod +x ./prebuild.sh && ./prebuild.sh",
    "clean": "rimraf .cache public",
    "start:app": "npm run develop",
    "start:lambda": "netlify-lambda serve src/lambda",
    "start": "run-p start:**",
    "build:app": "npm run clean && gatsby build",
    "build:lambda": "netlify-lambda build src/lambda",
    "build": "run-p build:**",
    "develop": "npm run clean && gatsby develop",
    "serve": "gatsby serve",
    "format": "prettier --trailing-comma es5 --no-semi --single-quote --write \"{gatsby-*.js,src/**/*.js}\"",
    "test": "echo \"Error: no test specified\" && exit 1"
  }

Just wanted to add this here for any poor souls like myself stuck on this.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[erquhart]

The CMS as a client app has no way of figuring out what branch was used in a deploy, but I'd like to discuss finding a way to automating this. Maybe a helper script you can add to your build task.

[kiliw]

I came across this issue and finally found a solution which works in my case. So In order to help maybe a few others I just want to present it here:
My goal is to build a static page with gatsby and netlify cms. As it turned out when you start the page on your local machine, go to /admin to enter the cms and make some changes, an automatic PR is made with those changes but against the master branch.
In order to prevent some problems with missing content types which might not be already in master I want the PR made to my current feature branch.
Here is the solution:

// /static/admin/config.yml

backend:
  name: git-gateway
  repo: yourRepo
...

Instead of setting window.CMS_MANUAL_INIT = true as described by netlify-cms, I used the way described by @zboman here, see the gatsby-config.js below.

// src/cms/cms.js
import CMS, { init } from 'netlify-cms-app'

const config = {
  backend: {
    name: 'git-gateway',
    repo: 'yourRepo',
    branch: process.env.GATSBY_CMS_BRANCH,
  },
}

init({ config })

// gatsby-config.js

const { setBranchEnvironment } = require('./envHelper')

setBranchEnvironment()

module.exports = {
   plugins: [
    {
      resolve: 'gatsby-plugin-netlify-cms',
      options: {
        manualInit: true,
        modulePath: `${__dirname}/src/cms/cms.js`,
      },
    },
 ]
}
...

setBranchEnvironment checks if it runs in the netlify environment, if so it sets the GATSBY_CMS_BRANCH to HEAD which is defined in netlify otherwise to your local git branch name.

// envHelper.js
const gitRepoInfo = require('git-repo-info')

const setBranchEnvironment = () => {
  const { branch } = gitRepoInfo()
  if (process.env.NETLIFY) {
    process.env.GATSBY_CMS_BRANCH = process.env.HEAD
  } else {
    process.env.GATSBY_CMS_BRANCH = branch
  }
}

module.exports = {
  setBranchEnvironment,
}

So what I got now is that whenever I go the cms (be it from localhost or from the deploy preview of netlify) changes on the content are always made to the the feature branch instead of master.

I hope this helps.

[erezrokah]

I think a simpler way of satisfying the workflow in the original issue is to use https://www.netlifycms.org/docs/beta-features/#working-with-a-local-git-repository

As scoping by branch is covered by https://github.com/netlify/netlify-cms/issues/1737 I think we can close this.

Please comment if you think this is still relevant.