Linkerd2: Publish homebrew install for linkerd CLI

[grampelberg]

It would be great to brew install linkerd.

[wmorgan]

Does it make sense to do something similar for npm?

[benjdlambert]

@grampelberg it looks like there is already a formula for linkerd which is the 1.x branch.

https://github.com/Homebrew/homebrew-core/blob/master/Formula/linkerd.rb

Maybe it should be brew install linkerd2?

[grampelberg]

@benjdlambert yeah, that sounds like a great idea to me! Once it is there, I can go update the docs so folks know about it =)

[murarisumit]

@benjdlambert @grampelberg are you guys working on brew package, I can go and create brew package for linkerd2 or we plan to updated the same package ?

[grampelberg]

@murarisumit I'm definitely not, would love the assistance there!

[murarisumit]

Picking this issue up, will try to build locally and then brew package.

[grampelberg]

That's awesome, thank you @murarisumit ! It probably isn't worth actually doing a build locally (just fetch the binary and put it in the right place). That said, I'm unfamiliar with how most brew packages do their thing.

[murarisumit]

Sorry guys for late update, forgot about the issue, started working on it. @grampelberg brew guys prefer building packages by source and then putting it there.

I tried to build cli on my machine, got into this issue. If any pointer in this direction would be helpful, I'm following this link: https://github.com/linkerd/linkerd2/blob/master/BUILD.md#building-the-cli-for-development

linkerd2 $ (master)
 $ > bin/fast-build
gcr.io/linkerd-io/controller:git-a9511aba
gcr.io/linkerd-io/web:git-a9511aba
gcr.io/linkerd-io/proxy-init:git-a9511aba
gcr.io/linkerd-io/grafana:git-a9511aba
gcr.io/linkerd-io/proxy:git-a9511aba
Unknown option: p
Type shasum -h for help
Actual digest of /var/folders/0j/000cnk895pg5prgjkwsqzn300000gp/T/dep.WYK/depbin does not match expected digest.

[siggy]

@murarisumit it looks like you have a version of shasum we did not anticipate, here's mine:

shasum -v
5.96

this should also work to build the cli:

bin/docker-build-cli-bin

the difference between bin/docker-build-cli-bin and bin/fast-build is that the former builds everything in Docker, so it's less sensitive to your build machine, but slower.

[murarisumit]

Thanks @siggy, it actually helped bin/docker-build-cli-bin, this ran successfully for me, and generated binary for all 3 platform, and shasum still fails, re-cloned repo and tried all over again.

shasum -v
6.01

In-effective, but for now I'll use bin/docker-build-cli-bin method to create binary. Will update for next release, if it works.

[klingerf]

Ah, there's actually a fix for the shasum issue on darwin in #2071, here:

https://github.com/linkerd/linkerd2/pull/2071/files#diff-20af0845e3268b60b911d81420604f7a

That will hopefully hit master soon, but you could maybe patch bin/dep locally in the meantime?

[murarisumit]

@klingerf I can patch locally, but in "brew formula" we need to mention the package "source url", then steps to build it, below is install snippet, that will be used in brew formula.

  url "https://github.com/linkerd/linkerd2/archive/stable-2.1.0.tar.gz"

  depends_on "go" => :build
  depends_on "docker" => :build

  def install
    ENV["GOPATH"] = buildpath

    srcpath = buildpath/"src/linkerd/linkerd2"
    outpath = srcpath/"target/cli/darwin"
    srcpath.install buildpath.children

    cd srcpath do
      system "bin/docker-build-cli-bin"
      prefix.install_metafiles
      bin.install outpath/"linkerd"
    end
  end

Even If patch locally, build will happen on brew's jenkins server and then binaries will be available to download. So probably will have to wait till it hits a release to use bin/fast-build .

Meanwhile, not able to get version to work properly. I couldn't find any environment variable or flag to be passed for version to work.

 $ > linkerd version
Client version: dev--sumit
Server version: unavailable

Or if git is used, it gives

 linkerd version
Client version: dev-3ff971fd-sumit
Server version: unavailable

Sorry, I new to linkerd project so too many questions out here
Is there any environment variable needed to be passed.

[siggy]

@murarisumit Thanks for all the info, it's helpful.

That dev--sumit version is coming from the script, because it assumes it is being built from inside a git repo:
https://github.com/linkerd/linkerd2/blob/377391cda984f71624f1fa779168bbf48d0aea0f/bin/_tag.sh#L27

Is it possible to build from the git repo, rather than stable-2.1.0.tar.gz? I see docker doing something like this:
https://github.com/Homebrew/homebrew-core/blob/d1fe7f91faf627a661965a94e3588f7c3a1b4bd2/Formula/docker.rb#L4-L6

[siggy]

something i also noticed in the docker install, completion support:
https://github.com/Homebrew/homebrew-core/blob/d1fe7f91faf627a661965a94e3588f7c3a1b4bd2/Formula/docker.rb#L31-L33

not sure if this is possible, but the output of linkerd completion bash and linkerd completion zsh provide great completion support. it would be awesome if that were included in the brew installation.

[murarisumit]

@siggy, thanks for the feedback, yes bash completion is very important.

I earlier also tried from git repo, as I mentioned in above comment. Got this

$ > linkerd version
Client version: dev-3ff971fd-sumit  // was looking for 
Server version: unavailable

When I downloaded the generated binary i get:

 $ > ./linkerd2-cli-stable-2.1.0-darwin version
Client version: stable-2.1.0

A rudimentary recipe is:

class Linkerd2 < Formula
  desc "Linkerd configuration command-line utility"
  homepage "https://github.com/linkerd/Linkerd2"
  # url "https://github.com/linkerd/linkerd2/archive/stable-2.1.0.tar.gz",
  url "https://github.com/linkerd/linkerd2.git",
      :tag      => "stable-2.1.0",
      :revision => "3ff971fd59b5300f1a13fb5317f3f90fe0445c82"

  depends_on "go" => :build
  depends_on "docker" => :build

  def install
    ENV["GOPATH"] = buildpath
    ENV["LINKERD_VERSION"] = version.to_s

    srcpath = buildpath/"src/linkerd/linkerd2"
    outpath = srcpath/"target/cli/darwin"
    srcpath.install buildpath.children

    cd srcpath do
      system "bin/docker-build-cli-bin"
      prefix.install_metafiles
      bin.install outpath/"linkerd"
    end
  end

  # test do
  # end
end

Command to try it out: brew install --build-from-source ./Formula/linkerd2.rb

Output is:

==> Summary
🍺  /usr/local/Cellar/linkerd2/2.1.0: 6 files, 27MB, built in 17 seconds

[siggy]

@murarisumit

i modified the Linkerd repo to add a bin/build-cli-bin script, which removes the docker dependency, and should be useful for the brew install (#2137)

i also took a crack at modifying the brew formula with the following changes:

• the dev-3ff971fd-sumit versions were due to the git repo getting dirty, ensure .brew_home is not present, and set CI_FORCE_CLEAN=1 to fail the build if not
• handle stable/edge/devel releases
• bash/zsh completion.

let me know what you think...

brew install --build-from-source ./Formula/linkerd2.rb --verbose --debug --HEAD

class Linkerd2 < Formula
  desc "Linkerd configuration command-line utility"
  homepage "https://linkerd.io"

  # TODO: wait for stable-2.2
  stable do
    url "https://github.com/linkerd/linkerd2.git",
      :tag      => "stable-2.1.0",
      :revision => "3ff971fd59b5300f1a13fb5317f3f90fe0445c82"
  end

  # TODO: wait for edge-19.1.3
  devel do
    url "https://github.com/linkerd/linkerd2.git",
      :tag      => "edge-19.1.2",
      :revision => "50d181cfb846933aaffbe003600692cb83fdf0f0"
  end

  head do
    url "https://github.com/linkerd/linkerd2.git"
  end

  depends_on "go" => :build

  def install
    ENV["GOPATH"] = buildpath
    ENV["CI_FORCE_CLEAN"] = "1"

    srcpath = buildpath/"src/github.com/linkerd/linkerd2"
    srcpath.install buildpath.children - [buildpath/".brew_home"]

    cd srcpath do
      system "bin/build-cli-bin"
      bin.install "target/cli/darwin/linkerd"

      # Install bash completion
      output = Utils.popen_read("#{bin}/linkerd completion bash")
      (bash_completion/"linkerd").write output

      # Install zsh completion
      output = Utils.popen_read("#{bin}/linkerd completion zsh")
      (zsh_completion/"linkerd").write output

      prefix.install_metafiles
    end
  end

  test do
    run_output = shell_output("#{bin}/linkerd 2>&1")
    assert_match "linkerd manages the Linkerd service mesh.", run_output

    version_output = shell_output("#{bin}/linkerd version --client 2>&1")
    assert_match "Client version: ", version_output
    assert_match stable.instance_variable_get(:@tag), version_output
  end
end

[murarisumit]

@siggy those scripts changes were really helpful, you have almost completed it up. Will test and update back here.

[siggy]

@murarisumit glar to hear it. another thought: what do you think about just replacing the existing likerd 1.0 formula with this one? linkerd 1.0 isn’t really a cli and doesn’t make much sense as a brew package.

[murarisumit]

HEAD works fine for me. Initially was thinking to creating a new formula/file for linkerd2, but as you suggestion makes sense, will update the original linkerd 1.0 formula. All credit for this formula goes to you

Update:
New release edge version also works fine

 $ > linkerd version --client
Client version: edge-19.1.3

[murarisumit]

Hi @siggy made PR for stable version here: https://github.com/Homebrew/homebrew-core/pull/36957.

Now they don't accept devel in homebrew-core, so removed those section.

One change I took from my side instead for dropping the old version, thought of keeping both the version first, then deprecating the old version in favor of new one. Let me know how that sounds to you.

[siggy]

@murarisumit Thanks for submitting that PR! I recommend we replace the existing linkerd.rb formula, as there was never much use case for installing Linkerd 1.0 via brew.

[murarisumit]

@siggy okies 👍, have made those changes into existing formula and pushed them.
Thanks for your comment up here in PR, was thinking about how to test more without k8s cluster