Dep: Make it possible to have your go code outside of gopath

[mastoj]

This one of my biggest annoyances with go, and I really can't understand why it has to be like this if you have a proper package tool.

With a lock file and vendor folder in place, shouldn't it be possible to have your go code outside the go path? All you dependencies should be installed in the vendor path anyhow, right?

I tried to run dep init in a random folder, but dep complains about not being in a go path.

Is it because go install is ran on the packages installed and the output is added to bin.

I totally understand if this might not be doable, but at the same time I can't understand why this limitation exists.

[codeblooded]

I second this 100%. The GOPATH has always forced me outside of the way I normally organize projects. I know many Go-tools depend on its structure to operate, however.

With vendor-ed dependencies, it should only need to check the vendor folder. However, I don't believe the assumption is made that all dependencies are in the vendor-ed folder (even though they should be). I believe the vendor path is searched first and then the GOPATH is searched. If so, why can't the project be placed elsewhere? The tool can still revert to searching wherever the $GOPATH variable is set when the vendor-ed dependency isn't available.

[aldencolerain]

I agree. For me, GOPATH and vendoring is by far the most frustrating and annoying aspect of the go tool-chain. There are countless issues and posts about this in so many go repos, channels, and mailing lists. I understand why some go contributors have been resistant, but I think its an enormous drag on adoption. I would love to help free go from the GOPATH.

I found this post here by @davecheney insightful in understanding the genesis of GOPATH: https://dave.cheney.net/2016/12/20/thinking-about-gopath.

It looks like this is on the 2018 roadmap check out this post by @sdboyer https://gist.github.com/sdboyer/def9b138af448f39300cb078b0e94cc3.

Maybe we just need these long time gophers to nudge us in the right direction. I'm happy to contribute and start working on this problem.

[ddevault]

GOPATH is the absolute worst and each time I think about using Go my enthusiasm lasts about as long as it takes me to remember how bloody stupid GOPATH is.

[sdboyer]

looks like folks have already covered most of the sentiments here :)

@aldencolerain linked to the gist i wrote a while back. as the gist notes, it's really more of a skeletal outline than a roadmap, but it does seem likely that the path into the toolchain will end up including a solution obviating GOPATH as a hard requirement.

and, to be clear, that really is the rub here. dep _could_ try to fix this problem on its own. but, because the options for doing so that are available to us outside the toolchain are quite different than the options available when toolchain changes are possible, it's pretty much a guarantee that an attempt to address it in dep would be at odds with the approach we ultimately adopt. and dep's modus operandi is to remain as orthogonal to the toolchain as possible; the transition is already going to be highly complex, even when we're dealing solely with the bare minimum set of requirements.

(closing as out of scope)

[cyberwillis]

Would't be nice if anyone could work in its own preferred structure?

Let's say HOME/myworkspace/myproject (forked from a specific company let's say github) and after made my changes and comited it "locally" I could just execute make, and let the toolchain grab my "local repository" and to clone it over ( GOROOT/src/company/myproject ) ovewriting it then building it, testing it and installing it without the need to commit to the official company source,
without the needs to change to the folder of go structure project, wihtout to executing pull manually and selecting the specific branch manually, to compile, test and install?

Would't be nice ?

[damien-roche]

It is utterly baffling we are still talking about the woes of GOPATH. Why has this not been solved? When will it be solved?

I just ran into the exact same issue. And now I'm off to find a better package manager which doesn't make such a crucial assumption about the organisation of my code. I have projects with different micro-services in different languages. Why is it Go is the only language which insists to be different?

Sort it out please.

[ddevault]

You should look into vgo, which fixes the GOPATH problem and is being merged into go upstream.

[damien-roche]

Interesting, thanks. Is that easy enough to integrate into development and not impede deployment to production?

[ddevault]

I think so, yes.

[damien-roche]

I've been leaning towards Glide, but I'll check it out. Cheers.

[francisdb]

Go 1.11 adds preliminary support for a new concept called “modules,” an alternative to GOPATH with integrated support for versioning and package distribution

https://golang.org/doc/go1.11#modules

[sammy007]

Surprising and disappointing to see this obviously excellent feature request is still an issue and closed.

[ddevault]

Surprising and disappointing to keep seeing this thread in my inbox after modules landed in go 1.11.

[sammy007]

Modules are new and immature, requires special version of golang, takes a second to call bullshit on it also. Looks like dependency management will continue to be an issue for decades in this toxic ecosystem.

[damien-roche]

I started symlinking to projects in GOPATH and started using Glide and all of my problems went away.

Unless this issue is locked and/or an adequate solution is provided by the language (modules is not adequate), expect to see many other developers turn up here and in your inbox.

[sammy007]

@damien-roche yeah always was a good idea.

Since I don't really need go get for main package and of course GOPATH is a total BS, I adopted a simple shell script from other project (from ethereum LOL) I found few years ago:

build/env.sh:

#!/bin/sh

set -e

if [ ! -f "build/env.sh" ]; then
    echo "$0 must be run from the root of the repository."
    exit 2
fi

project="my-project"
repositoryRoot="github.com/USERNAME"

# Create fake Go workspace if it doesn't exist yet.
workspace="$PWD/build/_workspace"
root="$PWD"
dir="$workspace/src/$repositoryRoot"
if [ ! -L "$dir/$project" ]; then
    mkdir -p "$dir"
    cd "$dir"
    ln -s ../../../../../. $project
    cd "$root"
fi

# Set up the environment to use the workspace.
# Also add Godeps workspace so we build using canned dependencies.
GOPATH="$workspace"
GOBIN="$PWD/build/bin"
export GOPATH GOBIN

# Run the command inside the workspace.
cd "$dir/$project"
PWD="$dir/$project"

# Launch the arguments with the configured environment.
exec "$@"

The Makefile:

.PHONY: all test clean

GOBIN = build/bin

all:
    build/env.sh go get github.com/golang/dep/cmd/dep
    build/env.sh $(GOBIN)/dep ensure
    build/env.sh go build -v -o $(GOBIN)/my-project

debug:
    build/env.sh go build -race -v -o $(GOBIN)/my-project

test: all
    build/env.sh go test -v ./...

vet: all
    build/env.sh go vet -shadow=false ./...

clean:
    rm -fr $(GOBIN)/my-project

[xhliu]

GOPATH is the most frustrating part of GO.

[bjm88]

With Go 1.11 and vgo out, can someone explain how its possible to support a monolithic repo that has many projects using many different languages/tech stacks. We have node, ui, and several go micro services. We want to establish a go_common project that isn't meant to have a main.go but just be imported like a local/private lib/module to our other go services. This go_common has common/shared code for logging, database, utility functions for formatting and crypto, as well as our database schema models represented as structs. We have GOPATH for 3rd part libs to be stored locally to me, but how do we reference our own shared module with other projects/modules in the new go 1.11 world?

Here is a stackoverflow asking it in detailed specific way...
https://stackoverflow.com/questions/52453335/how-to-share-a-local-module-in-go-1-11-for-golang-using-monolithic-repo-archite

[cyberwillis]

@bjm88 to me is easy ... I just GO AWAY !
I will not support a language to my projects that doesn't care about how I want to do things but just try to bend me to it's "best practices".. I don't care... I left!

[bjm88]

Hi, so go 1.11 and vgo Integration into main toolchain is great actually. It's a real module mgmt system, flexible and support versioning very cleanly. They also broke away from GOPATH and support monolithic mixed stack repo well now finally. See go.mod override location of required packages to point to shared local modules. See my comments on stack track linked above, it's nice and simple actually.

[publicvirtualvoid]

I don't normally write Go, but I keep hearing people are excited about it. I just found myself in a situation where I had to build some Go code, and this $GOPATH restriction really is very off-putting.

[damien-roche]

@publicvirtualvoid wait until you try to namespace your code :P

I found it surprisingly restrictive and even go as far as to say amateur (though smarter people than me contributed to the language). The best approach I've found is to symlink Go projects from $GOPATH and just do as your told. I'd also suggest Glide for package management as it seems impossible for the core team to come up with an adequate package manager themselves. Can't remember why but vgo was not satisfactory compared to other tools available.

[alper]

For anybody new to go this is a radical departure from how pretty much every other language functions. Same as with the blog article about why go's declaration syntax is so different, maybe have one about why this is the way it is and how to work with it.

[exotfboy]

Surprising and disappointing to see this obviously excellent feature request is still an issue and closed.

[bitfield]

Surprising and disappointing to see this obviously excellent feature request is still an issue and closed.

It's closed because it's been fixed.

[fuyi]

@bitfield Can you point out how this issue is fixed and what's the new way to manage dependency?

[damien-roche]

@fuyi more info here: https://github.com/golang/go/wiki/Modules

[bockyanggoh]

You know what, I thought Go looks really amazing, wanted to give it a go(pun intended), guess I will just go back to Spring Boot. Even though maven is ass slow, at least I can still build libraries with it.

[roshnet]

I use hard links to do the job. So, I work in my preferred directory, while the actual code runs in $GOPATH. Version control works flawless.