Copyright © 2013 Nathan Youngman

Earlier this week, the Go Package Committee announced the roadmap for dep , a tool that aims to become the standard solution to vendor libraries and apps dependencies within the Go ecosystem. The goal of this roadmap is to introduce dep into the Go toolchain for the beginning of version 1.10 development phase. However, this goal is not guaranteed, and it needs to be finally approved and embraced by the Go community — including the Go team, of course.

In this text, dep authors also ask for help to the community in all kind of areas:

Right now, we NEED PEOPLE to help shoulder the workload of writing docs, writing code, clarifying some design issues, and kicking the tires.

One of the clearest ways to provide help is by simply testing the tool and giving some feedback. piladb is a perfect example to become an early adopter and introduce dep :

External dependencies are minimal, difficult to mess up with.

Releases are pre-built, it won’t break current functionality.

It’s a young project, good timing for vendoring changes.

The idea is to replace the excellent and simple gvt , used by piladb until now, with dep , and in the meantime get to know better its early design, features and limitations.

For giving a bit of context, these are the only third party dependencies that piladb is relying on:

github.com/gorilla/mux : implements HTTP router. It uses github.com/gorilla/context as unique external dependency.

: implements HTTP router. It uses as unique external dependency. github.com/mitchellh/go-homedir: used to get home directory in cross-compilation environments in a test file.

That’s all, really.

So the first thing we’ll do is to remove the current vendor directory from piladb:

⇒ cd $GOPATH/src/github.com/fern4lvarez/piladb

⇒ tree vendor

vendor

├── github.com

│ ├── gorilla

│ │ ├── context

│ │ │ ├── context.go

│ │ │ ├── doc.go

│ │ │ └── LICENSE

│ │ └── mux

│ │ ├── context_gorilla.go

│ │ ├── context_native.go

│ │ ├── doc.go

│ │ ├── LICENSE

│ │ ├── mux.go

│ │ ├── regexp.go

│ │ └── route.go

│ └── mitchellh

│ └── go-homedir

│ ├── homedir.go

│ └── LICENSE

└── manifest

⇒ rm -rf vendor

⇒ git commit -am "Remove gvt's vendor directory"

At this point, if we push these changes and someone go get 's this repo, it would download the latest version of these dependencies and store them in their $GOPATH . Let’s then install dep and inspect it:

⇒ go get -u github.com/golang/dep/...

⇒ dep -h

Usage: dep <command> Commands: init Initialize a new project with manifest and lock files

status Report the status of the project's dependencies

ensure Ensure a dependency is safely vendored in the project

remove Remove a dependency from the project

Following short dep documentation, we want now to initialize a new project using dep init , and this is what we get:

⇒ dep init

Cached github.com/gorilla/mux

⇒ dep status

PROJECT CONSTRAINT VERSION REVISION LATEST PKGS USED

github.com/gorilla/context * v1.1 a85d2e5 v1.1 1

github.com/gorilla/mux * v1.3.0 392c28f v1.3.0 1

github.com/mitchellh/go-homedir * branch master b8bc1bf b8bc1bf 1

⇒ cat lock.json

{

"memo": "56e9e2eb8dff4ae328332bf64aba30efaed5e2351b1dd942d089b814a3eac77c",

"projects": [

{

"name": "github.com/gorilla/context",

"version": "v1.1",

"revision": "a85d2e53ba63bdea074dbbbb5983f0516974e87b",

"packages": [

"."

]

},

{

"name": "github.com/gorilla/mux",

"version": "v1.3.0",

"revision": "392c28fe23e1c45ddba891b0320b3b5df220beea",

"packages": [

"."

]

},

{

"name": "github.com/mitchellh/go-homedir",

"branch": "master",

"revision": "b8bc1bf767474819792c23f32d8286a45736f1c6",

"packages": [

"."

]

}

]

}

⇒ cat manifest.json

⇒ tree vendor

vendor

└── github.com

├── gorilla

│ ├── context

│ │ ├── context.go

│ │ ├── context_test.go

│ │ ├── doc.go

│ │ ├── LICENSE

│ │ └── README.md

│ └── mux

│ ├── bench_test.go

│ ├── context_gorilla.go

│ ├── context_gorilla_test.go

│ ├── context_native.go

│ ├── context_native_test.go

│ ├── doc.go

│ ├── LICENSE

│ ├── mux.go

│ ├── mux_test.go

│ ├── old_test.go

│ ├── README.md

│ ├── regexp.go

│ └── route.go

└── mitchellh

└── go-homedir

├── homedir.go

├── homedir_test.go

├── LICENSE

└── README.md 6 directories, 22 files

So we’ve got few things going on here:

dep created a lock.json and manifest.json files in the root directory. Former contains meta-information about the vendored dependencies, latter is an empty JSON file. Note that gvt only had one manifest file inside vendor/ directory, not in root.

created a and files in the root directory. Former contains meta-information about the vendored dependencies, latter is an empty JSON file. Note that only had one file inside directory, not in root. Unlike gvt , dep copied the dependencies code in vendor/ including tests and markdown files . I’m not sure about the motivation for this decision, but I expect it might change during dep 's development process.

, copied the dependencies code in . I’m not sure about the motivation for this decision, but I expect it might change during 's development process. From these dependencies, github.com/gorilla/mux is the only package with proper labeled, versioned releases. Thus, dep installed by default the latest stable release, v1.3.0, instead of checking out latest master. In turn, v1.1 of github.com/gorilla/context is used too.

These are no big deal for piladb, but I’d like to keep using the very same version that I was using with gvt , so apparently I need to dig into dep ensure now:

⇒ dep ensure -h

Usage: dep ensure [spec...] Ensure is used to fetch project dependencies into the vendor folder, as well as

to set version constraints for specific dependencies. It takes user input,

solves the updated dependency graph of the project, writes any changes to the

manifest and lock file, and places dependencies in the vendor folder. Package spec: <path>[:alt location][@<version specifier>] Examples: dep ensure Populate vendor from existing manifest and lock

dep ensure github.com/pkg/foo@^1.0.1 Update a specific dependency to a specific version For more detailed usage examples, see dep ensure -examples. Flags: -examples print detailed usage examples (default: false)

-n dry run, don't actually ensure anything (default: false)

-override specify an override constraint spec (repeatable) (default: <none>)

-update ensure dependencies are at the latest version allowed by the manifest (default: false)

-v enable verbose logging (default: false)

I find dep ensure -examples very interesting, as it shows extended usage examples, making emphasis in SemVer notations and version constraints using tilde or caret, which are widely used in the npm ecosystem, but are not that common in Go package managers.

⇒ dep ensure github.com/gorilla/mux@master

⇒ cat lock.json

{

"memo": "f1b9d5d8b37b8fe02b788694611b16a4fd8e7097edf980fdb1af70d086efc022",

"projects": [

{

"name": "github.com/gorilla/context",

"version": "v1.1",

"revision": "a85d2e53ba63bdea074dbbbb5983f0516974e87b",

"packages": [

"."

]

},

{

"name": "github.com/gorilla/mux",

"branch": "master",

"revision": "599cba5e7b6137d46ddf58fb1765f5d928e69604",

"packages": [

"."

]

},

{

"name": "github.com/mitchellh/go-homedir",

"branch": "master",

"revision": "b8bc1bf767474819792c23f32d8286a45736f1c6",

"packages": [

"."

]

}

]

}

⇒ cat manifest.json

{

"dependencies": {

"github.com/gorilla/mux": {

"branch": "master"

}

}

}

⇒ git commit -am "dep: ensure master for github.com/gorilla/mux"

We can see that in lock.json the metadata for github.com/gorilla/mux was updated, pointing to master branch, and creating an entry in manifest.json file, specifying the new dependency. So now I assume whenever master branch is updated on the upstream, a dep ensure -update github.com/gorilla/mux will handle that.

Since github.com/gorilla/mux and github.com/gorilla/context are not connected as explicit dependencies, the latter wasn’t updated in the previous action, so it’s something I also need to do manually:

⇒ dep ensure github.com/gorilla/context@master

⇒ cat manifest.json

{

"dependencies": {

"github.com/gorilla/context": {

"branch": "master"

},

"github.com/gorilla/mux": {

"branch": "master"

}

}

} ⇒ dep status

PROJECT CONSTRAINT VERSION REVISION LATEST PKGS USED

github.com/gorilla/context branch master v1.1 a85d2e5 08b5f42 1

github.com/gorilla/mux branch master branch master 599cba5 599cba5 1

github.com/mitchellh/go-homedir * branch master b8bc1bf b8bc1bf 1

⇒ git commit -am "dep: ensure master for github.com/gorilla/context"

With this, I know that both dependencies are up-to-date with master , and I’m one command away from updating them in case of changes in upstream.

So far dep looks good, but it has a long way ahead and I’m sure things will change in its functionality, and design over this process. Probably you are a reader from the future and see how dep evolved over the time, but remember that as of this writing the status is as follows:

Alpha. Functionality is known to be broken, missing or incomplete. Command and file format changes are still planned. (…).

Regarding piladb, the dependencies management here is so trivial that playing around with dep is a no-brainer. I created a PR with these changes, where we’ll consider if adopting it now, or giving it a bit more time, until authors consider it is usable.

Happy Friday!