Git-secret: automate the GitHub release

[simbo1905]

Creating a release programmatically looks straightforward enough https://developer.github.com/v3/repos/releases/#create-a-release

Changelogs can be generated using something like github-changelog-generator . Perhaps we can run a bot of some sort to run generate the changelog and commit it.

[simbo1905]

Hub has a wrapper command to create, update and delete GitHub releases documented at https://hub.github.com/hub-release.1.html

[simbo1905]

Here is an example of getting travis to use to do the release which requires a git tag:

https://stackoverflow.com/a/28230711/329496

[simbo1905]

@sobolevn the travis docs use a api_key: "GITHUB OAUTH TOKEN". Can you create a token and add it as an environment variable GITHUB_OAUTH_TOKEN into the travis build to be able to automate releases as per that doc?

[sobolevn]

@simbo1905 will do tomorrow!

[joshrabinowitz]

@sobolevn @simbo1905

Is adding the GITHUB_OAUTH_TOKEN environment variable into the travis build a blocker for getting out a new release?

[simbo1905]

No it isn't a blocker as @sobolevn has done the release so there is a binary at https://github.com/sobolevn/git-secret/archive/v0.2.4.tar.gz

So having read the manual it looks like I just need to raise a PR for the brew formula with the new version. I will try the hub.github.com advanced features for doing that tonight since it looks like it optimised things like forking and raising a PR with boilerplate text. tonight i will look to script it from my mac and "pull the trigger". I will then raise a ticket here to get it to run inside of travis for the next release.

if we want to go end to end at the next release then we will need the github token in travis to push up a tar.gz release to use in the brew formula PR.

[joshrabinowitz]

@simbo1905 yes please do raise the PR so we can release 0.2.5. The tagged 0.2.4 seems to have included #201 'manual installation fails'

[simbo1905]

I have a draft script https://github.com/simbo1905/git-secret/commit/9683f1b0868ace36db6f30e7ced8aea040b04585 that bumps the release number updates the sha265, and pushes to a repo, from which I can create a PR. The PR page told me to test it locally before I submit it which I will do tonight.

We will need a repo that is a clone of homebrew that we can commit onto in order to create a PR to the main homebrew and an SSH key to let our CI deploy. That not needed for tonight but to automate this now that I understand the workflow.

[joshrabinowitz]

@simbo1905 on the other hand if we get the rpm and deb files out for 0.2.5, it looks like @ilovezfs can get the brew version published (See https://github.com/Homebrew/homebrew-core/pull/28891 )

[ilovezfs]

You can use brew bump-formula-pr to automate opening the PR.

[simbo1905]

thanks yes. currently debugging why the tar.gz won't locally pass brew test.

[ilovezfs]

==> Summary
🍺  /usr/local/Cellar/git-secret/HEAD-b85cc04: 20 files, 68KB, built in 7 seconds
iMac-TMP:~ joe$ brew test -vd git-secret
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/git-secret.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/gawk.rb
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/gnupg.rb
Testing git-secret
/usr/bin/sandbox-exec -f /tmp/homebrew20180614-9022-13blu08.sb /usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.3.3_2/bin/ruby -W0 -I /usr/local/Homebrew/Library/Homebrew -- /usr/local/Homebrew/Library/Homebrew/test.rb /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/git-secret.rb -vd --HEAD
/usr/local/Homebrew/Library/Homebrew/test.rb (Formulary::FromPathLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/git-secret.rb
/usr/local/Homebrew/Library/Homebrew/test.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/gnupg.rb
==> /usr/local/opt/gnupg/bin/gpg --batch --gen-key batch.gpg
gpg: directory '/tmp/git-secret-test-20180614-9023-1s2j4g4/.gnupg' created
gpg: keybox '/tmp/git-secret-test-20180614-9023-1s2j4g4/.gnupg/pubring.kbx' created
gpg: /tmp/git-secret-test-20180614-9023-1s2j4g4/.gnupg/trustdb.gpg: trustdb created
gpg: key 596C5DC211F8657C marked as ultimately trusted
gpg: directory '/tmp/git-secret-test-20180614-9023-1s2j4g4/.gnupg/openpgp-revocs.d' created
gpg: revocation certificate stored as '/tmp/git-secret-test-20180614-9023-1s2j4g4/.gnupg/openpgp-revocs.d/3897EF1BF0C5A9AA3EC575FD596C5DC211F8657C.rev'
==> git init
Initialized empty Git repository in /private/tmp/git-secret-test-20180614-9023-1s2j4g4/.git/
==> git config user.email [email protected]
==> git secret init
'/private/tmp/git-secret-test-20180614-9023-1s2j4g4/.gitsecret/' created.
cleaning up...
==> git secret tell -m
gpg: keybox '/private/tmp/git-secret-test-20180614-9023-1s2j4g4/.gitsecret/keys/pubring.kbx' created
gpg: /private/tmp/git-secret-test-20180614-9023-1s2j4g4/.gitsecret/keys/trustdb.gpg: trustdb created
==> git secret add shh.txt
1 item(s) added.
==> git secret hide
done. all 1 files are hidden.
==> /usr/local/opt/gnupg/bin/gpgconf --kill gpg-agent
iMac-TMP:~ joe$

HEAD @ b85cc04 passes for me.

[simbo1905]

you need to extract the release at https://github.com/sobolevn/git-secret/archive/v0.2.4.tar.gz and on my macos that dies at cp -R "$BATS_ROOT"/bin/* "$PREFIX"/bin with "utils/bin/*: No such file or directory"

[ilovezfs]

That was fixed in HEAD.

[ilovezfs]

See https://github.com/sobolevn/git-secret/issues/201#issuecomment-396551178

[simbo1905]

then it is not in the release so the release cannot be released 😀

[simbo1905]

i cannot modify the release as @sobolevn has the keys.

[ilovezfs]

Shall we just skip 0.2.4 in brew then? Or I can backport the patch.

[simbo1905]

we are definitely wanting to ship this as we don't want to call this closed until we have a brew release.

great idea to patch! there was a patch in the formula for the last release.

[simbo1905]

this was in the last release:

  # Upstream PR from 13 Jan 2018 "Make checksum command operating system based"
  patch do
    url "https://github.com/sobolevn/git-secret/pull/127.patch?full_index=1"
    sha256 "0f711bb7f2cd91e0770a92371ea50541aa8ae606c0542c164af7fc280dd956db"
  end

So i will try adding a patch at that PR.

[simbo1905]

trying

  # Upstream PR Revert "migrate from bats to bats-core " #203
  patch do
    url "https://github.com/sobolevn/git-secret/pull/203.patch?full_index=1"
    sha256 "c80d63075906d5e7f9145fc5c96d5dfbef0b6ef209f2e879dedb0b9febb0421a"
  end

[simbo1905]

that worked!

[ilovezfs]

🎉

[simbo1905]

And here it is my first brew PR!

https://github.com/Homebrew/homebrew-core/pull/29051

[simbo1905]

@sobolevn can we get that GITHUB_OAUTH_TOKEN as an env var in travis to automate the github release?

[sobolevn]

@simbo1905 I have added GITHUB_OAUTH_TOKEN to travis settings.
It should be now be available to you.

It currently has only repo scope, that should be enough.

[joshrabinowitz]

@simbo1905 @sobolevn -- I assume github and our tooling supports release candidates or pre-production releases?

Can we document how to run the release process without actually creating a new release, assuming that's possible? That will help us to streamline and fix the Github release process.

[simbo1905]

hub is the GitHub command line tool to make a release. It does support pre-releases. Here is a script I wrote to make releases driven by a chatbot for another project https://github.com/ocd-scm/ocd-chatbot/blob/master/bin/ocd-create-release.sh That script can run in the container image at https://hub.docker.com/r/simonmassey/ocd-tools

I am thinking that we might be able to get Travis-CI to have an extra deploy step that runs a version of that release script. I am currently travelling on business so I might not find time to set it up in my fork to figure it all out for a week or so.

We need an oAuth token for the git repo with write access but I can figure that out on my fork and document it to then setup in the main Travis.

[joshrabinowitz]

@simbo1905 that sounds great, I look forward to seeing it.

[simbo1905]

i have created PR #411 for this