Cli: Build fails because of transitive dependency: go-md2man

[lszucs]

Recently added dependency go-md2man (maybe in https://github.com/urfave/cli/commit/40d4a25a01296d2f51bf7bbf0b6e8eb99ba4a84c#diff-2a0cb88ddd9482c046c169b7b242d36c ) causes compile time errors.

For some reason, the vendor folder is missing from the dependency. If I clone go-md2man I can build it fine. If I copy the vendor folder under our projects vendor, everything works as expected.

This is only an issue if depending on master branch (rev: 693af58b4d51b8fcc7f9d89576da170765980581), v1.21.0 is cool.

Issue filed in go-md2man repo: https://github.com/cpuguy83/go-md2man/issues/48

[lynncyrin]

@lszucs you mention a build failing, can you provide an example of it failing on CI?

[saschagrunert]

Hm, I think it can cause issues with go versions older than 1.12, because I'm not sure if this change really works: https://github.com/urfave/cli/commit/3a41d6d7851b15f132131444865e82b17baf0be0

There is a hash calculation mismatch between go 1.11 and and 1.12 which might cause problems.

I tried to reproduce the issue within the latest golang:1.12.8 container, but it seems to work fine:

> docker run -it -e GO111MODULE=on golang:1.12.8 bash -c 'git clone https://github.com/urfave/cli && pushd cli && go build ./...'
Cloning into 'cli'...
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 4415 (delta 1), reused 1 (delta 0), pack-reused 4408
Receiving objects: 100% (4415/4415), 1.53 MiB | 3.41 MiB/s, done.
Resolving deltas: 100% (2682/2682), done.
/go/cli /go
go: finding github.com/shurcooL/vfsgen v0.0.0-20181202132449-6a9ea43bcacd
go: finding github.com/shurcooL/httpfs v0.0.0-20190707220628-8d4bc4ba7749
go: finding github.com/BurntSushi/toml v0.3.1
go: finding github.com/cpuguy83/go-md2man v1.0.10
go: finding gopkg.in/yaml.v2 v2.2.2
go: finding github.com/russross/blackfriday v1.5.2
go: finding gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405
go: downloading github.com/BurntSushi/toml v0.3.1
go: downloading github.com/cpuguy83/go-md2man v1.0.10
go: downloading gopkg.in/yaml.v2 v2.2.2
go: extracting github.com/cpuguy83/go-md2man v1.0.10
go: downloading github.com/russross/blackfriday v1.5.2
go: extracting github.com/BurntSushi/toml v0.3.1
go: extracting gopkg.in/yaml.v2 v2.2.2
go: extracting github.com/russross/blackfriday v1.5.2

We could still vendor the dependencies in, but I'm not sure if this is a nice solution for this kind of project.

[lszucs]

@lynncyrin

Sorry for not being explicit about it: I receive the same kind of error as in the referenced issue.

$ go build
# github.com/bitrise-io/bitrise/vendor/github.com/cpuguy83/go-md2man/md2man
vendor/github.com/cpuguy83/go-md2man/md2man/md2man.go:11:16: undefined: blackfriday.EXTENSION_NO_INTRA_EMPHASIS
vendor/github.com/cpuguy83/go-md2man/md2man/md2man.go:12:16: undefined: blackfriday.EXTENSION_TABLES
vendor/github.com/cpuguy83/go-md2man/md2man/md2man.go:13:16: undefined: blackfriday.EXTENSION_FENCED_CODE
vendor/github.com/cpuguy83/go-md2man/md2man/md2man.go:14:16: undefined: blackfriday.EXTENSION_AUTOLINK
vendor/github.com/cpuguy83/go-md2man/md2man/md2man.go:15:16: undefined: blackfriday.EXTENSION_SPACE_HEADERS
vendor/github.com/cpuguy83/go-md2man/md2man/md2man.go:16:16: undefined: blackfriday.EXTENSION_FOOTNOTES
vendor/github.com/cpuguy83/go-md2man/md2man/md2man.go:17:16: undefined: blackfriday.EXTENSION_TITLEBLOCK
vendor/github.com/cpuguy83/go-md2man/md2man/md2man.go:19:29: too many arguments to conversion to blackfriday.Markdown: blackfriday.Markdown(doc, renderer, extensions)
vendor/github.com/cpuguy83/go-md2man/md2man/roff.go:19:9: cannot use &roffRenderer literal (type *roffRenderer) as type blackfriday.Renderer in return argument:
        *roffRenderer does not implement blackfriday.Renderer (missing RenderFooter method)
vendor/github.com/cpuguy83/go-md2man/md2man/roff.go:102:11: undefined: blackfriday.LIST_TYPE_ORDERED
vendor/github.com/cpuguy83/go-md2man/md2man/roff.go:102:11: too many errors

Go version is go1.12.8 darwin/amd64

Note the paths! We're using dep and vendor folders, while the referenced issue seems to use a different scheme.

../go/pkg/mod/github.com/cpuguy83/[email protected]/md2man/md2man.go:11:16: undefined: blackfriday.EXTENSION_NO_INTRA_EMPHASIS
../go/pkg/mod/github.com/cpuguy83/[email protected]/md2man/md2man.go:12:16: undefined: blackfriday.EXTENSION_TABLES
../go/pkg/mod/github.com/cpuguy83/[email protected]/md2man/md2man.go:13:16: undefined: blackfriday.EXTENSION_FENCED_CODE
../go/pkg/mod/github.com/cpuguy83/[email protected]/md2man/md2man.go:14:16: undefined: blackfriday.EXTENSION_AUTOLINK
../go/pkg/mod/github.com/cpuguy83/[email protected]/md2man/md2man.go:15:16: undefined: blackfriday.EXTENSION_SPACE_HEADERS
../go/pkg/mod/github.com/cpuguy83/[email protected]/md2man/md2man.go:16:16: undefined: blackfriday.EXTENSION_FOOTNOTES
../go/pkg/mod/github.com/cpuguy83/[email protected]/md2man/md2man.go:17:16: undefined: blackfriday.EXTENSION_TITLEBLOCK
../go/pkg/mod/github.com/cpuguy83/[email protected]/md2man/md2man.go:19:29: too many arguments to conversion to blackfriday.Markdown: blackfriday.Markdown(doc, renderer, extensions)
../go/pkg/mod/github.com/cpuguy83/[email protected]/md2man/roff.go:19:9: cannot use &roffRenderer literal (type *roffRenderer) as type blackfriday.Renderer in return argument:
        *roffRenderer does not implement blackfriday.Renderer (missing RenderFooter method)
../go/pkg/mod/github.com/cpuguy83/[email protected]/md2man/roff.go:102:11: undefined: blackfriday.LIST_TYPE_ORDERED
../go/pkg/mod/github.com/cpuguy83/[email protected]/md2man/roff.go:102:11: too many errors

[lynncyrin]

Ah, this looks like it may be the same issue as https://github.com/urfave/cli/issues/866?

[lynncyrin]

@lszucs would be you able to create a sandbox github repo that reproduces this issue on CI?

[lszucs]

@lszucs would be you able to create a sandbox github repo that reproduces this issue on CI?

@lynncyrin sure

https://github.com/bitrise-io/bitrise

you must checkout revision 62978c55dd201ecc250c49cc6a591de6d20e1c7e

and simply run go build

You should see the output mentioned above.

[saschagrunert]

Yeah If we disable go modules, then it surely needs the dependency. We could vendor the dependency in, WDYT @lynncyrin @AudriusButkevicius ?

[AudriusButkevicius]

It seems this issue should be raised with md2man, as there is not really much we can do if that does not correctly specify the version of the dependency? Or perhaps we can fix it in our go mod?

[lynncyrin]

I would prefer we not vendor dependencies - and instead figure out how to fix this using go modules.

[saschagrunert]

I'm not sure if we can fix this without vendoring...

[AudriusButkevicius]

So it works if you enable go module support, that's what we use for our package management.

The fact that someone tries to build it without go module support is somewhat a Henny Youngman problem, as essentially go modules is the thing that solves the problem they are having,

The fact that they use dep or whatever other dependency manager is completely irrelevant to us.