Dependabot-core: go modules: tolerate replace directive

[hazcod]

Hello,

Currently your go.mod parser does not allow // comments in the go.mod file.
Error thrown is 'was unable to parse your go.mod file'.
Please allow this to be used.

[hazcod]

I do not know any Ruby, but this would look something like: https://github.com/dependabot/dependabot-core/commit/64931aa37bec9486bb94197a6527e9d860a1bdae

[greysteil]

Thanks for this @hazcod. Looking at the tests here (and particularly this fixture) it looks like Dependabot can handle comments, so I'm guessing it's something else going wrong here.

[hazcod]

My go.mod:

module frontend

go 1.12

require (
    cloud.google.com/go v0.39.0 // indirect
    github.com/boombuler/barcode v1.0.0 // indirect
    github.com/gin-gonic/gin v1.4.0
    github.com/go-pg/pg v8.0.4+incompatible
    github.com/google/go-cmp v0.3.0 // indirect
    github.com/gosimple/slug v1.5.0
    github.com/hashicorp/golang-lru v0.5.1 // indirect
    github.com/hazcod/gosecurity v0.0.2
    github.com/kelseyhightower/envconfig v1.4.0 // indirect
    github.com/kr/pretty v0.1.0 // indirect
    github.com/mailgun/mailgun-go/v3 v3.5.0 // indirect
    github.com/mailru/easyjson v0.0.0-20190403194419-1ea4449da983 // indirect
    github.com/mattn/go-isatty v0.0.8 // indirect
    github.com/oblq/i18n v0.0.0-20181031085821-98eec2978e00
    github.com/pariz/gountries v0.0.0-20171019111738-adb00f6513a3
    github.com/pkg/errors v0.8.1
    github.com/pquerna/otp v1.1.0
    github.com/rainycape/unidecode v0.0.0-20150907023854-cb7f23ec59be // indirect
    github.com/company/framework v1.1.2
    golang.org/x/crypto v0.0.0-20190513172903-22d7a77e9e5f // indirect
    golang.org/x/net v0.0.0-20190522155817-f3200d17e092 // indirect
    golang.org/x/oauth2 v0.0.0-20190523182746-aaccbc9213b0 // indirect
    golang.org/x/sys v0.0.0-20190526052359-791d8a0f4d09 // indirect
    golang.org/x/text v0.3.2
    google.golang.org/appengine v1.6.0 // indirect
    google.golang.org/genproto v0.0.0-20190522204451-c2c4e71fbf69 // indirect
    google.golang.org/grpc v1.21.0 // indirect
    gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 // indirect
)

// fix for AppEngine not using -mod=vendor (auto uncommented in cloudbuild.yaml)
// https://github.com/golang/go/issues/27227#issuecomment-495283061
// replace github.com/company/framework => ./vendor/github.com/company/framework

// fix for go-gin import ugorji/go
// https://github.com/gin-gonic/gin/issues/1673
replace github.com/ugorji/go v1.1.4 => github.com/ugorji/go/codec v0.0.0-20190204201341-e444a5086c43

[greysteil]

Great, thanks! @hmarr is this a Go 1.12 issue?

[vallieres]

It might be the replace that is causing the problem? We have the same issue, but we have no comments in our go.mod files.

[hazcod]

Any ETA on this issue?

[greysteil]

It's with @hmarr but I know he's super busy with Dependabot infrastructure. @hmarr what do you reckon?

[hmarr]

Hoping to have this fixed this week - sorry it's taking a while, it's been a slightly crazy couple of weeks!

[hazcod]

No problem, congratulations with the acquisition!

[hmarr]

@hazcod I've just been testing our the go.mod you provided, and it looks like the issue is that github.com/company/framework v1.1.2 doesn't exist (or is private and Dependabot doesn't have access).

If you can share the name of the repo with me, then I can have a deeper dig into what's gone wrong, as it sounds like the error message you got wasn't quite right.

@vallieres are you able to share a repo / problematic go.mod demonstrating the issue you're seeing?

(For both, I'm harry at our company name dot com if you'd rather share privately via email).

[hazcod]

@hmar I've emailed you the go.mod files.
I thought about the repo being private, but since the dependabot app has access to these repos I didn't think of it as a problem (and it worked before).

[hmarr]

In case it helps with any of the issues mentioned here, I figured I'd mention that I shipped support for Go 1.12 yesterday.

[hmarr]

@hazcod thanks for sending those across - I replied to the email directly so let's continue that conversation there.

So far I haven't been able to reproduce any errors relating to comments or the replace directive, so if anyone has any further examples, I'd love to see them!

[hazcod]

@hmarr did you by accident fix the issue? My dependabot issues got closed on the project.

[greysteil]

Maybe a result of @hmarr's work on Go 1.12?

[hmarr]

@hazcod glad to hear you're no longer getting the error - it could've been Go 1.12 related.

I'm going to leave this open for now as I think the issue some other people are seeing is related to the replace directive - specifically, we don't support using the replace directive if it points to a vendored / local module, as we currently only fetch the go.mod and go.sum rather than the whole repo.

[hmarr]

https://github.com/dependabot/dependabot-core/pull/1193 will be deployed within the next 15 mins or so, and I'm hopeful that it'll fix the issue with the replace directive. If it doesn't, let me know and I'll have another look.