Dpl: Support for Heroku Provider to Deploy Specific Branches

[loganknecht]

Hi there. I hope this is an ok place to request this.

I've been looking through the documentation and I don't see any support for this. If I'm misunderstanding the dpl tool and/or it's usage I would really appreciate any correction on how to achieve what I'm trying to do.

Goal

I want to be able to use the dpl tool to deploy specified branches in my ci/cd.

Current Steps

• Install heroku CLI
  
  
  
  • curl https://cli-assets.heroku.com/install-ubuntu.sh | sh
  
  
• Installs deploy tool for heroku
  
  
  
  • apt-get update -qy
  
  
  • apt-get install -y ruby-dev
  
  
  • gem install dpl
  
  
  • dpl --provider=heroku --app=$HEROKU_APP_NAME --api-key=$HEROKU_API_KEY
  
  

This works. However, if my understanding is correct this is only doing a deploy of my master branch.

Points of Confusion

I'm reading the docs here for deploying specific branches. I see that there is indeed an option to specify a branch. However this looks like it only works when using a .travis.yml file.

Request

Please make it so the dpl command supports the ability to target a specific branch for the Heroku provider.

Perhaps the command might look like this
dpl --provider=heroku --app=$HEROKU_APP_NAME --api-key=$HEROKU_API_KEY --on=$GIT_BRANCH_NAME

[svenfuchs]

@loganknecht thanks for the request, especially a this well written one :)

if you don't have a .travis.yml file, can i ask what CI/CD context you are using dpl on?

as for your request, the condition mentioned in our documentation is a Bash condition, i.e. our build script compiler would wrap it into a Bash if statement, so I think dpl is not the right place to accept that condition. i'm going to defer to @BanzaiMan's opinion though.

in any case, in the meantime you can achieve the same thing by doing something like:

[[ $(git rev-parse --abbrev-ref HEAD) = master ]] && dpl --provider=heroku --app=$HEROKU_APP_NAME --api-key=$HEROKU_API_KEY

[loganknecht]

@svenfuchs Thank you for such a quick reply!

The CI/CD solution I'm using is GitLab.

The reference I'm using is this page.

The issue is that I'm definitely going to want to push my master branch for full deployments, but if I want to test in a staging environment before I merge to a develop I can't.

Gitlab CI

# CI Variables from GitLab
# https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
api-deploy:
  stage: deploy
  variables:
    HEROKU_APP_NAME: "example-name"
    HEROKU_API_KEY: "65a7e475-3d8a-41a3-952c-c82518f4c872"
  image: python:3.7
  script:
    # Assumes you start in `/builds/project-0/`
    # Install heroku CLI
    - curl https://cli-assets.heroku.com/install-ubuntu.sh | sh
    # Installs deploy tool for heroku
    - apt-get update -qy
    - apt-get install -y ruby-dev
    - gem install dpl
    - dpl --provider=heroku --app=$HEROKU_APP_NAME --api-key=$HEROKU_API_KEY

If I try to use the Heroku cli tool, I use the environment variable $HEROKU_API_KEY for authentication, but instead it will prohibit me from doing so when I run the git push $DESIRED_BRANCH:$TARGET_BRANCH because the authentication between git and the heroku cli are two different mechanisms.

As a result the Gitlab suggested solution is to use the above link, which shows the usage of the dplcommand, but that locks me into just targeting master branch destinations for deploy.

I will try your suggestion later, but I'm not sure if that will achieve my goal.

[svenfuchs]

@loganknecht thanks for the extra details.

i am not super familiar with GitLab's build configuration format, but it is my understanding that you can use GitLab's config format features to include/exclude things to only run on specific branches. Maybe this helps: https://docs.gitlab.com/ee/ci/yaml/#onlyexcept-advanced

also, again, since your dpl command is just a Bash command, you can most certainly use a Bash condition like so:

[[ $(git rev-parse --abbrev-ref HEAD) = master ]] && dpl --provider=heroku --app=$HEROKU_APP_NAME --api-key=$HEROKU_API_KEY

if none of this works then maybe it might be best to also turn to GitLab's support. your usecase seems like a very standard usecase, and i am sure they can point you in the right direction.

[loganknecht]

@svenfuchs I apologize ahead of time if I'm misunderstanding some of this.

The Gitlab feature to include/exclude is useful for determining a job to run. In my case, I want the job to run on every merge request. This job would deploy that branch to a specific heroku app.

Please see my gitlab configuration:

# CI Variables from GitLab
# https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
api-deploy:
  stage: deploy
  variables:
    HEROKU_APP_NAME: "example-name"
    HEROKU_API_KEY: "65a7e475-3d8a-41a3-952c-c82518f4c872"
  image: python:3.7
  script:
    # Assumes you start in `/builds/project-0/`
    # Install heroku CLI
    - curl https://cli-assets.heroku.com/install-ubuntu.sh | sh
    # Installs deploy tool for heroku
    - apt-get update -qy
    - apt-get install -y ruby-dev
    - gem install dpl
    - dpl --provider=heroku --app=$HEROKU_APP_NAME --api-key=$HEROKU_API_KEY

I first tried using just the heroku cli and git, but the authentication bit was what I got snagged on.

Then I used the dpl command like the guide demonstrated, however the dpl tool appears to pull only from the master branch when you use it.

Because it is targeting the master branch, but in my ci/cd deployment I want to use the feature branch, I have no way to control what branch gets deployed when using this command:
dpl --provider=heroku --app=$HEROKU_APP_NAME --api-key=$HEROKU_API_KEY

Every time dpl targets the master branch for deployment. This means if I want to test the feature branch changes in my heroku instance before my changes are merged into master I would have to merge the code into master to deploy it to heroku. And that would defeat the purpose of that effort.

The bash you suggested, does the same operation. Which would be fine, except it also does not allow me the ability to select which branch is deployed on the specified heroku instance.

Am I misunderstanding how to choose what gets deployed to heroku using the dpl tool?

[BanzaiMan]

As @svenfuchs mentioned before, the branch condition for deployment belongs to whatever is invoking dpl to perform deployment. You have to figure out how to inject this logic into GitLab's dpl invocation, or use the Bash logic to decide whether or not to invoke dpl, as suggested in https://github.com/travis-ci/dpl/issues/1001#issuecomment-499596614.

If you need anything beyond, please contact GitLab support. They are the experts.

[loganknecht]

@BanzaiMan Thank you for the reply.

However, I really think I'm being misunderstood here.

I understand that you can do branch filtering within the ci/cd system to determine what job gets run. That makes sense and is not a problem.

The issue here is that I want to select which branch in my ci/cd system gets pushed when using the dpl tool.

To my knowledge, which no one has confirmed or denied yet is that: The dpl tool only targets the master branch for deployments to the heroku app specified. Because of that all of my ci/cd configurations will always deploy the master branch regardless of the branch I'm on.

IF I was to deploy without using the dpl command it would look like this

api-deploy:
  stage: deploy
  variables:
    HEROKU_APP_NAME: "api-production"
    HEROKU_API_KEY: "THIS-IS-AN-EXAMPLE"
    HEROKU_APP_GIT_REMOTE_NAME: "api-production"
    DESIRED_BRANCH_TO_DEPLOY: "$CI_COMMIT_REF_NAME"
  image: python:3.7
  script:
    # Assumes you start in `/builds/project-0/`
    # Install heroku CLI
    - curl https://cli-assets.heroku.com/install-ubuntu.sh | sh
    # Configure Heroku CLI
    - heroku git:remote --app $HEROKU_APP_NAME
    - git remote rename heroku $HEROKU_APP_GIT_REMOTE_NAME
    - heroku stack:set container --app $HEROKU_APP_NAME
    - git checkout $DESIRED_BRANCH_TO_DEPLOY
    # Sets authentication with git for Heroku
    - echo -e "machine git.heroku.com\n  login [email protected]\n  password $HEROKU_API_KEY" >> ~/.netrc
    - cat ~/.netrc
    # Deploy
    - git push --force $HEROKU_APP_GIT_REMOTE_NAME $DESIRED_BRANCH_TO_DEPLOY:master

THAT will let me push the feature branch to the specified heroku app.

Why can't I do this with the dpl tool.

[BanzaiMan]

Why can't I do this with the dpl tool.

dpl does not care about the git branch your code is on. It doesn't care that your code is on any branch at all; it doesn't care if your code is in detached mode. It doesn't care that your code is using git, or any source code versioning software.

In order to introduce the git-branch-level logic, it will have to done on a higher level than dpl. We do this on Travis CI. You will have to do the same with GitLab.

[svenfuchs]

@loganknecht Sorry for the misunderstandings on our part. Please understand that we are trying to support you in using dpl on GitLab, while that service is not our expertise.

My understanding now is that you would like to deploy whatever git sha the env var $CI_COMMIT_REF_NAME contains to the same production app (as given in HEROKU_APP_NAME) on Heroku. Is that correct?

dpl should actually do exactly that.

The heroku provider has two strategies: api and git, api being the default.

The api strategy posts a build request to Heroku's API https://github.com/travis-ci/dpl/blob/master/lib/dpl/provider/heroku/api.rb#L34, and includes as a version whatever is given on --version, or the env var TRAVIS_COMMIT (not present on GitLab), or defaults to git rev-parse HEAD, i.e. the current head commit. See https://github.com/travis-ci/dpl/blob/master/lib/dpl/provider/heroku/api.rb#L92

The git strategy (which you are not using) would basically do something similar to what you say you'd do without using dpl, i.e. pushing the current head commit to the Heroku app git remote.

The option --on=[...] (and the .travis.yml integration mentioned on https://docs.travis-ci.com/user/deployment/heroku/#deploying-specific-branches) does something different (it allows enabling/disabling deployments on the given branch names).

[loganknecht]

@svenfuchs Thank you very much for taking the time to elaborate on this. I really appreciate the links you've provided and the clarification.

The $CI_COMMIT_REF_NAME will provide the explicit branch name. In this case it is: feature/ci_and_cd_pipeline-development.

With respect to the strategy is this the documentation for it?

I have tested the usage of the git argument and I think that appears to be doing the trick. However I can't confirm quite yet as the full pipeline is still failing.

The command is now: dpl --provider=heroku --strategy=git --app=$HEROKU_APP_NAME --api-key=$HEROKU_API_KEY

I will check back once I have resolved everything, and will confirm that this is indeed the solution.

[loganknecht]

I am confirming that using the git strategy is the solution I was looking for.

The winning command was:
dpl --provider=heroku --strategy=git --app=$HEROKU_APP_NAME --api-key=$HEROKU_API_KEY

@svenfuchs Thank you for your time, energy, and patience :)

[svenfuchs]

Glad you figured it out @loganknecht!

🎉

[kakashysen]

Hi @loganknecht I see that you found a solution but I didn't see how to use a different branch on dpl? Could you give me an example please? I have the same problem now

[loganknecht]

@kakashysen It's been a long time since I touched this, but this is the command I consistently use

dpl --provider=heroku --strategy=git --app=$HEROKU_APP_NAME --api-key=$HEROKU_API_KEY

The trick is to checkout the branch and run it with the correct options set.

I can't really say much more than that.