Lesser-known features of go test
Most gophers know and love go test, the testing tool that comes with Go’s official gc toolchain. It is quite possibly the simplest thing that works, and that is beautiful.
They know that running go test runs the tests for the package in the current directory, looking for *_test.go files, and within those files, looking for functions following the TestXxx(*testing.T) {} naming and signature (that is, a function that receives a *testing.T as sole argument, and that is named TestXxx where Xxx is any name that doesn’t start with a lowercase). This testing code does not pollute standard builds as it is compiled only when go test is used.
But there is much more hiding in there.
The black box test package
Usually, in Go, tests are in the same package as the code it tests (the system under test), giving access to internal implementation details. For black-box testing, go test supports using a package name with the “_test” suffix that gets compiled into its own separate package.
For example:
// in example.go
package example
var start int
func Add(n int) int {
start += n
return start
}
// in example_test.go
package example_test
import (
"testing"
. "bitbucket.org/splice/blog/example"
)
func TestAdd(t *testing.T) {
got := Add(1)
if got != 1 {
t.Errorf("got %d, want 1", got)
}
}
You can see the infamous dot-import in action. This is the one use case where it makes sense, when black-box testing a package and importing its exported symbols in the current package’s scope. It should almost always be avoided in other circumstances.
As explained in the linked style guide section on dot imports, the black-box test pattern can also be used to break import cycles (when the package under test “a” is imported by package “b”, and the test of “a” needs to import “b” – the test can be moved to the “a_test” package and can then import both “a” and “b” without cycle).
Skipping tests
Some tests may require a particular context to be executed. For example, some tests may require the presence of an external command, a specific file, or an environment variable to be set. Instead of letting those tests fail when that condition is not met, it is easy to simply skip those tests:
func TestSomeProtectedResource(t *testing.T) {
if os.Getenv("SOME_ACCESS_TOKEN") == "" {
t.Skip("skipping test; $SOME_ACCESS_TOKEN not set")
}
// ... the actual test
}
If go test -v is called (notice the verbose flag), the output will mention the skipped test:
=== RUN TestSomeProtectedResource
--- SKIP: TestSomeProtectedResource (0.00 seconds)
example_test.go:17: skipping test; $SOME_ACCESS_TOKEN not set
Often used with the skip feature is the -short command-line flag, that is made available to the code via testing.Short() that simply returns true if the flag is set (just like the -v flag is made available via testing.Verbose() so you can print additional debugging information when this condition is met).
When a test is known to take a while to run and you’re in a hurry, you can call go test -short and, provided the package developer was kind enough to implement this, long-running tests will be skipped. That’s how Go’s tests are run when installing from source, and here is an example of a long-running test skipped in short mode, from the stdlib:
func TestCountMallocs(t *testing.T) {
if testing.Short() {
t.Skip("skipping malloc count in short mode")
}
// rest of test...
}
Skipping is just one option, the -short flag is just an indication and the developer can choose to run the test but avoid some slower assertions instead.
There’s also the -timeout flag that can be used to force a test to panic if it doesn’t finish within this duration. For example, running go test -timeout 1s with this test:
func TestWillTimeout(t *testing.T) {
time.Sleep(2 * time.Second)
// pass if timeout > 2s
}
produces this output (truncated):
=== RUN TestWillTimeout
panic: test timed out after 1s
And running specific tests instead of the whole test suite is easy with go test -run TestNameRegexp.
Parallelizing tests
By default, tests for a specific package are executed sequentially, but it is possible to mark some tests as safe for parallel execution using t.Parallel() within the test function (assuming the argument is named t, as is the convention). Only those tests marked as parallel will be executed in parallel, so having just one is kind of useless. It should be called first in the test’s function body (after any skipping conditions) as it will reset the test’s duration:
func TestParallel(t *testing.T) {
t.Parallel()
// actual test...
}
The number of tests run simultaneously in parallel is the current value of GOMAXPROCS by default. It can be set to a specific value via the -parallel n flag (go test -parallel 4).
Another source of parallelization, albeit not at the granular test function level, but at the package level, is when go test p1 p2 p3 is called (that is, when it is called with multiple packages to test). In this case, the test binaries of all packages are built then run at the same time. This can be great for the total running time, but it can also lead to randomly failing tests if some shared resources are used by many packages (e.g. if some tests access the database and delete some rows used by tests in other packages).
To keep this under control, the -p flag can be used to specify the number of builds and tests to run in parallel. In a repository with many packages in subfolders, one can write go test ./... to test all packages, including the one in the current directory and all subdirectories. Running without the -p flag, the total running time should be close to the longest package to test (plus build times). Running go test -p 1 ./..., constraining the tool to build and test one package at a time, the total running time should be close to the sum of all individual package tests plus build times. You can play with it and run go test -p 3 ./... to see the effect on running time.
Yet another place where parallelization can occur (and should be tested) is in the package’s code. Thanks to Go’s awesome concurrency primitives, it isn’t unusual for a package to use goroutines, channels and other synchronization mechanisms. By default, unless GOMAXPROCS is set via an environment variable or explicity in the code, the tests are run with GOMAXPROCS=1. One can type GOMAXPROCS=2 go test to test with 2 CPUs, and then type GOMAXPROCS=4 go test to test with 4, but there is a better way: go test -cpu=1,2,4 will run the tests three times, with 1, 2 and 4 as GOMAXPROCS values.
The -cpu flag, combined with the data race detector flag -race is a match made in heaven (or hell, depending on how it goes). The race detector is an amazing tool that must be used for any serious concurrent development, but its overview is outside the scope of this blog post. The introduction post on the Go blog is a good place to start for any interested gopher.
And much more
The go test tool supports running benchmarks and assertable examples (!) in a similar manner as the test functions. The godoc tool even understands the example syntax and includes them in the generated documentation.
There is also much to be said on code coverage and profiling, two features supported by the testing tool. For those interested to dive further, you can start with “The cover story” and “Profiling Go programs”, both on the Go blog.
And before you roll out your own test utilities, you may want to look at the testing/iotest, testing/quick and net/http/httptest packages in the standard library!
Explore royalty-free sounds from leading artists, producers, and sound designers:
September 3, 2014
