Go1.3 on Roberto Selbach

Parallel benchmarks and wandering maps

Thu, 09 Oct 2014 13:10:00 +0000

I tested a locked lookup table with Go 1.3’s parallel benchmark support:

func BenchmarkLookupParallel(b *testing.B) {
	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			_ = table.Lookup("troy")
		}
	})
}

RunParallel creates goroutines and distributes benchmark iterations among them. This exposed lock contention that a serial benchmark politely concealed. It is still synthetic: all workers hammering one key may be less realistic than production, so I use representative key distributions too.

The same upgrade broke a test that compared map traversal output to a fixed string. Go 1.3 randomizes iteration order for small maps, making an assumption that was always invalid fail more reliably.

for k := range m {
	keys = append(keys, k)
}
sort.Strings(keys)

Sorting is correct when output order is part of the result. For equality, I compare contents instead of rendering iteration order.

Random order is not cryptographic randomness, nor should a program depend on getting a different order every pass. It is an implementation defense against accidental ordering assumptions, not a shuffling API.

I approve of the randomization. A runtime that accidentally rewards a bad assumption lets the assumption become an API. My test was not flaky; it was finally honest.

sync.Pool is not a cache

Thu, 18 Sep 2014 16:25:00 +0000

I used Go 1.3’s new sync.Pool to reuse temporary buffers in an HTTP formatter. Then I almost used it to cache parsed templates. The first idea reduced garbage; the second would have randomly forgotten application data.

A pool holds temporary values that may be removed at any time. Garbage collection is explicitly allowed to clear it. That makes it suitable for amortizing allocation of scratch objects, not for storing anything whose presence affects correctness.

My buffer pool looks like this:

var buffers = sync.Pool{
	New: func() interface{} {
		return new(bytes.Buffer)
	},
}

func render(v value) []byte {
	b := buffers.Get().(*bytes.Buffer)
	b.Reset()

	writeValue(b, v)
	out := append([]byte(nil), b.Bytes()...)
	if b.Len() < 64<<10 {
		b.Reset()
		buffers.Put(b)
	}
	return out
}

The copy before Put is essential. b.Bytes() aliases the buffer’s storage. Returning that slice and putting the buffer back would allow another goroutine to overwrite the caller’s result. Pool safety does not transfer ownership safety to pooled objects.

The pool synchronizes Get and Put, but an object obtained from it belongs to the caller until returned. Two goroutines must not use the same buffer concurrently. Go 1.3’s bytes.Buffer does not expose its capacity, so I use the rendered length as a simple retention policy: a response that grows a buffer past 64 KiB does not return that buffer to the pool. It is only a proxy for capacity, but it keeps a freshly oversized allocation out of the pool.

In a benchmark of the formatter, pooling reduced allocation volume:

BenchmarkRender-4 500000 4100 ns/op 2304 B/op 12 allocs/op
BenchmarkRenderPool-4 500000 3000 ns/op 768 B/op 5 allocs/op

The output copy accounts for necessary ownership. If the function wrote directly to an io.Writer, even that copy could disappear, probably a better API than ever more pooling.

New should return the expected type consistently. A pool may omit New, in which case Get can return nil and every caller must handle that. I define it when all callers need the same scratch type; this keeps allocation policy together without pretending values are permanent.

Copying a sync.Pool after first use is invalid because its synchronization state and local storage are not value-like application data. I keep pools as package variables or pointer-owned fields initialized once.

I would not add sync.Pool without allocation profiles and a benchmark. It obscures object lifetime and may provide little benefit when allocations are already short and cheap. A cache promises retrieval; a pool merely offers to rummage in a box the collector is free to empty.

The Go 1.3 linker got less clever

Thu, 28 Aug 2014 14:05:00 +0000

One of our larger binaries spent enough time linking that I could switch windows, forget why, and begin an unrelated task. Go 1.3’s linker refactor made that interruption noticeably shorter.

The old linker represented symbols and their data using structures that encouraged many small allocations and expensive traversals. The 1.3 work reorganized that machinery around more compact symbol data and a cleaner division of the linking stages. The user-facing command did not change:

$ go build ./cmd/server

The wait did. On one clean build of our program I saw roughly:

Go 1.2.2 real 4.8s
Go 1.3 real 3.1s

This is not a controlled toolchain benchmark. Installed package archives, the filesystem, machine load, and exact source all affect it. Repeating alternating builds from the same installed state still showed enough difference that it was not merely optimism generated by a new version number.

Linking a Go program is substantial work. The linker reads object data for the program and imported packages, resolves symbol references, applies relocations, lays out executable sections, emits metadata used by reflection and the runtime, and writes the target executable. It also removes unreachable code and data. A statically linked Go binary may contain a great deal of runtime and type information, so representation overhead inside the linker matters.

The refactor is interesting because it improves a tool by making its internal model less elaborate. No language feature was needed. Existing builds became faster and used less memory because the implementation stopped turning every byte and symbol into quite so much bookkeeping.

I still avoid reading too much into final binary size. Importing a package can pull in initialization and reachable symbols, reflection can retain type metadata, and debug information contributes too. A smaller linker heap is not the same thing as a smaller output file.

Cross-compilation makes architecture-specific tool names visible in this toolchain. The compiler and linker remain separate programs selected for the target architecture, even though go build normally hides the sequence. When investigating a failure I use go build -x to print invoked commands rather than reconstructing them from folklore.

That output also shows whether the build uses installed archives from GOROOT/pkg and GOPATH/pkg or recompiles packages. Without it, I had occasionally blamed the linker for time actually spent rebuilding a dependency after a build-tag change.

Benchmark builds with and without the relevant installed archives separately. Archives installed for the standard library under GOROOT/pkg and for workspace packages under GOPATH/pkg can hide compilation work; release builders often expose linker time more clearly. Go 1.3 improved both my build and my attention span, though only one claim has numbers behind it.

What Go 1.3 did to GC pauses

Tue, 15 Jul 2014 13:20:00 +0000

I upgraded a memory-heavy worker to Go 1.3 expecting the garbage collector to become invisible. It did not. What changed was more specific and more useful than my expectation.

Go 1.3 performs stack scanning precisely. The collector knows which stack slots contain pointers instead of treating pointer-shaped integers as possible references. This avoids keeping some dead heap objects alive accidentally and enables the runtime to copy growing stacks while correcting pointers.

The release also adds concurrent sweeping. After marking determines which heap objects remain reachable, reclaiming unmarked spans can overlap with application execution rather than keeping all sweep work inside one stop-the-world interval. Allocation may assist or encounter sweep work later, so the cost has moved and changed shape rather than vanished.

I compared GC traces from the same batch input:

Go 1.2: collections less regular, several pauses near 20ms
Go 1.3: shorter observed pauses, sweep work visible during execution

Those are observations from one program, not a portable benchmark. Heap layout, processor count, allocation pattern, and runtime revisions all matter. The collector still stops the world for portions of collection. Go 1.3 is not the future fully concurrent collector some discussions seem to have installed by enthusiasm.

One accidental retention did disappear after the upgrade because a pointer-looking value on a stack no longer kept an object alive. I did not count that as permission to ignore memory profiles. The worker also retained a 16 MB byte slice through a 40-byte subslice, and precise scanning quite correctly kept the whole backing array reachable.

The fix for that case remained a copy:

key := append([]byte(nil), largeBuffer[start:end]...)

Copying 40 bytes allowed the large buffer to die. Runtime precision can distinguish pointers from integers; it cannot infer that I only care about a tiny section of a live array.

Marking begins from roots such as globals and goroutine stacks and follows pointers through reachable heap objects. Precise stack maps improve the root set, while concurrent sweeping deals with memory already proven dead. They improve different phases; calling both simply “the new GC” hides why a workload changes.

I keep pause percentiles and total CPU in view. Moving sweep work out of a pause can reduce tail latency while leaving processor work to be paid during ordinary execution. That is usually a good trade for my service, but a batch program may value throughput differently.

My practical conclusion is to upgrade for the collector improvements, then measure again. Lower pauses do not make allocation free, concurrent sweep does not eliminate all latency, and a precise collector will faithfully preserve every object my program still references, including the embarrassingly large ones.

Growing stacks without segments

Tue, 24 Jun 2014 15:45:00 +0000

After installing Go 1.3 I reran a recursive parser benchmark that had always behaved oddly at certain depths. The new runtime changed how goroutine stacks grow, and the result finally matched the shape I expected.

Before 1.3, a goroutine stack could grow by adding another segment. Function entry checked available space and called runtime machinery when a split was needed. Repeatedly crossing a boundary in a call pattern could produce the notorious “hot stack split”: allocate a segment, return and release it, then need it again on the next iteration.

Go 1.3 replaces segmented growth with a contiguous stack that is copied to a larger allocation. Pointers into the old stack must be adjusted to point into the copy. Stack addresses therefore are not permanent, which is one reason converting a Go pointer to an integer and treating it as a stable address is a rotten idea.

My synthetic test recursively visited a nested expression:

func depth(n *node) int {
	if n == nil {
		return 0
	}
	return 1 + depth(n.child)
}

The benchmark result on my machine was illustrative rather than universal:

Go 1.2: 18400 ns/op
Go 1.3: 12100 ns/op

Normal request depths showed little difference. The improvement appeared near stack-growth behavior, exactly where a microbenchmark can exaggerate it. I am not claiming every recursive function became one third faster.

The copying design depends on knowing which stack words are pointers. Go 1.3 has precise stack scanning, allowing the runtime to update real pointers rather than conservatively treating any pointer-looking word as one. This also helps garbage collection avoid retaining objects merely because an integer happens to resemble an address.

Contiguous does not mean fixed. A goroutine begins with a modest stack and grows when a function’s stack check says more room is required. Growth now entails allocation and copying, so unexpectedly enormous stacks still cost memory and time. Deep recursion can still fail; it merely has a better growth mechanism on the way there.

Stack copying happens at safe points where the runtime understands frame layout. Code that passes Go pointers through unsafe integer storage defeats assumptions needed to relocate them. unsafe was never a promise that undocumented runtime addresses would survive stack growth; Go 1.3 merely makes that bad bargain fail more creatively.

I removed a manual recursion limit that existed solely to dodge segmented-stack performance, but kept the limit that rejects maliciously deep input. A runtime improvement is not an input-validation policy. It is, however, a welcome deletion of one workaround.