GOPATH is not a versioning system

Thu, 08 May 2014 17:55:00 +0000

I reproduced a build failure only after deleting my GOPATH. That sentence contains most of the problem.

My normal workspace had this shape:

$GOPATH/
 src/example.com/service
 src/github.com/some/dependency
 pkg/darwin_amd64
 bin

go get fetched the dependency’s current source into the expected import path. My machine had an older checkout that still provided a function the service used. A fresh machine fetched a newer revision where the function had changed. The import path named a repository location, not the exact source revision I had tested.

I briefly considered copying dependencies under a random directory in the service. In 2014 the go command does not define vendoring semantics that make such a copy transparently override the canonical import. Rewriting imports to local copies creates new package identities and a permanent maintenance chore.

For deployable applications, I record the repository revisions and make the build checkout those revisions into an isolated GOPATH. Some teams use scripts; others use emerging third-party dependency tools. None is yet the universal answer. Libraries have an even harder choice because pinning their transitive world can conflict with the application assembling the final program.

I also keep import paths canonical. Relative imports make a package depend on where it happens to sit and behave poorly outside a narrow local experiment.

The decisive experiment was a new empty workspace:

$ mkdir /tmp/service-build
$ GOPATH=/tmp/service-build go get example.com/service/cmd/server

That made every source checkout visible under one disposable root. I recorded git rev-parse HEAD for each dependency and reproduced the working set by checking out those revisions. The script fails if a revision cannot be fetched; silently falling back to a branch tip would restore the original uncertainty.

Import-path compatibility remains social rather than solver-enforced. If upstream makes an incompatible change at the same path, the go command cannot select “the old API” from the import statement. Forking under a new path changes package identity, so values from original and forked packages are distinct even when declarations match.

For every recorded revision I also keep its canonical repository URL. Import paths can redirect discovery, and repeatability requires knowing both what commit was selected and where the build expects to obtain it.

GOPATH solves workspace layout and import discovery well. It does not solve repeatable dependency selection. Saying that plainly avoids a lot of mystical advice about cleaning $GOPATH/pkg. My build was not stale; its input was unspecified.

GOPATH is a workspace, not a package

Wed, 18 Apr 2012 20:40:00 +0000

I set GOPATH to the directory containing one package and then wondered why go install produced a peculiar path. The variable was innocent. I had mistaken the whole workspace for one shelf in it.

A minimal workspace has a recognizable shape:

$GOPATH/
 src/example.org/hello/hello.go
 pkg/
 bin/

Source lives below src in directories corresponding to import paths. Installed package objects go below pkg, arranged for the target system. Installed commands go in bin. GOPATH names the root containing those areas, not src itself and not the package directory.

The surprise was that import paths and disk paths line up without making imports relative. Code can import example.org/hello; the go command searches the workspace’s src/example.org/hello. The source does not need to know how many .. components separate two packages on my machine.

This is a workspace convention, not a global registry. I can keep more than one workspace and select one through the environment. A GOPATH list can also provide multiple roots, although source and installation behaviour deserve attention when doing that. For a small project, one root is easier to reason about.

I prefer an import path that reflects where a package is expected to live, even before publishing it. It avoids renaming every import when the code leaves a scratch directory. I also keep third-party packages in the workspace layout rather than copying their files into my package and erasing their identity.

The caveat is that the workspace does not pin dependency versions. If I update a dependency in place, every package using that workspace sees the update on its next build. Reproducible releases therefore still require discipline about the exact source revisions checked out. The layout organizes code; it does not make dependency history disappear.

There is also no need to set GOROOT to the workspace. GOROOT identifies the Go installation and standard library. GOPATH identifies my code and other non-standard packages. Giving both variables the same value is less configuration than performance art.

After moving my package under src and pointing GOPATH at the parent workspace, go build, go test, and go install behaved as advertised. The directory tree is opinionated, but at least it has the courtesy to be visible.

Gopath on Roberto Selbach

GOPATH is not a versioning system

GOPATH is a workspace, not a package