Runtime on Roberto Selbach

Go 1.4 moves the runtime toward Go

Tue, 16 Dec 2014 17:40:00 +0000

I installed Go 1.4 to test a service and found the most interesting changes below my source code. Parts of the runtime and tools are being translated from C into Go. That work is not a promise that applications suddenly run faster; it is groundwork for making the implementation easier to evolve with one type system and toolchain.

The collector is now fully precise. It identifies pointers in stacks and the heap rather than retaining objects because non-pointer data happens to resemble an address. Precision also requires cooperation from writes: write barriers record pointer updates that the collector must not miss while maintaining its view of the object graph. This is infrastructure for collector work, not the concurrent collector people keep casually attributing to this release. Go 1.4 still has stop-the-world collection behavior worth measuring.

Goroutine stacks now start at 2 KiB, down from the larger starting size in previous releases, and grow by copying when needed. This materially reduces initial memory for programs with many mostly shallow goroutines. It does not make deep stacks free; growth allocates a larger contiguous stack and copies live contents using precise pointer information.

Two source-level tools are immediately practical. go generate lets a package record commands that generate source or other build inputs:

//go:generate stringer -type=State

Running go generate is explicit; go build does not run generators automatically. I like that separation because builds should not unexpectedly require every generator or rewrite the tree. Generated files still need a clear policy in the repository, and the directive is a command for developers, not a dependency manager.

The new internal directory convention has an important 1.4-sized footnote. In this release, the restriction is enforced for packages in the main Go source repository, keeping implementation packages inside the standard library and toolchain. The intended rule is that code under a/b/internal/x may be imported only by packages rooted within a/b, but the go command does not yet enforce that boundary for general repositories. Broader enforcement is future work.

example.com/project/internal/wire
example.com/project/server

I can use the same layout in my own repository to advertise intent, but in Go 1.4 that part is still convention rather than a boundary enforced by the tool. It is worth adopting without pretending the lock is already on the door.

After the upgrade I reran tests, allocation benchmarks, and latency measurements rather than assuming runtime changes were universally beneficial. The service used less memory with thousands of idle goroutines, while its GC tail latency remained visible. That is a believable improvement: smaller starting stacks and more precise tracing reduce waste, but write barriers and collection still have costs.

Go 1.4 mostly removes future constraints. More runtime code in Go, complete pointer precision, and barriers prepare the implementation for deeper collector changes. Meanwhile, go generate is useful now, and internal shows where package boundaries are headed.

Growing stacks without segments

Tue, 24 Jun 2014 15:45:00 +0000

After installing Go 1.3 I reran a recursive parser benchmark that had always behaved oddly at certain depths. The new runtime changed how goroutine stacks grow, and the result finally matched the shape I expected.

Before 1.3, a goroutine stack could grow by adding another segment. Function entry checked available space and called runtime machinery when a split was needed. Repeatedly crossing a boundary in a call pattern could produce the notorious “hot stack split”: allocate a segment, return and release it, then need it again on the next iteration.

Go 1.3 replaces segmented growth with a contiguous stack that is copied to a larger allocation. Pointers into the old stack must be adjusted to point into the copy. Stack addresses therefore are not permanent, which is one reason converting a Go pointer to an integer and treating it as a stable address is a rotten idea.

My synthetic test recursively visited a nested expression:

func depth(n *node) int {
	if n == nil {
		return 0
	}
	return 1 + depth(n.child)
}

The benchmark result on my machine was illustrative rather than universal:

Go 1.2: 18400 ns/op
Go 1.3: 12100 ns/op

Normal request depths showed little difference. The improvement appeared near stack-growth behavior, exactly where a microbenchmark can exaggerate it. I am not claiming every recursive function became one third faster.

The copying design depends on knowing which stack words are pointers. Go 1.3 has precise stack scanning, allowing the runtime to update real pointers rather than conservatively treating any pointer-looking word as one. This also helps garbage collection avoid retaining objects merely because an integer happens to resemble an address.

Contiguous does not mean fixed. A goroutine begins with a modest stack and grows when a function’s stack check says more room is required. Growth now entails allocation and copying, so unexpectedly enormous stacks still cost memory and time. Deep recursion can still fail; it merely has a better growth mechanism on the way there.

Stack copying happens at safe points where the runtime understands frame layout. Code that passes Go pointers through unsafe integer storage defeats assumptions needed to relocate them. unsafe was never a promise that undocumented runtime addresses would survive stack growth; Go 1.3 merely makes that bad bargain fail more creatively.

I removed a manual recursion limit that existed solely to dodge segmented-stack performance, but kept the limit that rejects maliciously deep input. A runtime improvement is not an input-validation policy. It is, however, a welcome deletion of one workaround.

The two words inside an interface

Thu, 29 May 2014 14:15:00 +0000

I changed a formatter from a concrete parameter to an interface and expected the call to behave like a generic wrapper around the value. The program worked, but looking at method dispatch made the cost and behavior less mysterious.

Conceptually, an interface value is two machine words: information describing the dynamic type and a word referring to the dynamic value. For a non-empty interface, the type information also leads to the method table needed to dispatch calls. An empty interface needs no method table, but it still carries dynamic type and value.

Copying an interface copies those two words, not the object they describe:

var src io.Reader = bytes.NewBufferString("hello")
dst := src

Both interfaces refer to the same buffer. Reading through dst advances the state later observed through src; the assignment did not clone the buffer. This is easy to miss because the interface itself is a small value while the dynamic value may be shared mutable state.

Calls through an interface use the implementation selected by the dynamic type. Go’s implicit satisfaction means the concrete type does not declare which interfaces it implements; the compiler checks the method set at assignment or call sites. Pointer and value method sets still matter, so T and *T need not satisfy the same interface.

The two-word representation does not promise a two-word operation. A call still dispatches to the dynamic type’s method, and whether a value escapes depends on how the compiler can follow it through that call. Interface size alone says little about allocation.

An interface-to-interface assignment can change the static contract while preserving the same dynamic value. A type assertion asks whether that dynamic type satisfies another interface; it does not convert the object into a new representation. I use the two-result form when failure is ordinary:

c, ok := w.(io.Closer)

The one-result form panics on failure, which belongs only where the invariant is truly guaranteed.

I kept the interface where it represented a real behavioral boundary and restored the concrete parameter in the private formatter. Interfaces are contracts, not performance annotations or automatic copies of the values inside them.

What Go 1.2 changed under my goroutines

Thu, 19 Dec 2013 18:10:00 +0000

One of my test programs had a rude goroutine: it performed a long loop containing function calls while another goroutine waited to print progress. With one processor, older Go releases could let the loop dominate longer than I liked.

func spin(n int) int {
	total := 0
	for i := 0; i < n; i++ {
		total += step(i)
	}
	return total
}

func step(i int) int { return i & 7 }

Go 1.2 introduces partial preemption when entering non-inlined functions. The scheduler can arrange for a goroutine to yield when it reaches one of these safe points. I built this experiment with -gcflags=-l so the tiny step call would remain a call. It does not make arbitrary Go code preemptible at every instruction. A tight loop with no calls can still be an unpleasant neighbor, and long cgo calls remain a different matter.

That distinction matters. I first described the change to a colleague as “preemptive scheduling,” then wrote a no-call loop and disproved my own sentence. Progress became much fairer when the loop called step, but not because the runtime had acquired a general-purpose interruptible stack map for every program counter.

The other runtime change I can actually feel is stack size. New goroutine stacks in Go 1.2 start at 8 KiB rather than the smaller segments used before. That spends more virtual and physical memory up front, but avoids repeated stack splitting in modest call chains. In a crude test creating 100,000 blocked goroutines, memory is therefore not simply “100,000 times a tiny number.” Goroutines are cheap, not free.

The segmented stacks still grow by linking stack segments. This is why a function prologue checks whether enough stack remains before proceeding. The check and occasional split are part of the cost hidden behind an ordinary call.

I confirmed the scheduling effect with GOMAXPROCS=1, because multiple operating-system threads otherwise conceal starvation by running both goroutines simultaneously. That is a useful test setting, not necessarily a production recommendation. With more processors, synchronization and cache traffic enter the result; the single-processor run isolates whether one runnable goroutine gets a chance to yield to another.

Channel sends, receives, and system calls introduce other scheduling boundaries. My example intentionally removed those ordinary blocking points so the function-call behavior could be seen without a busy server around it.

My practical rule remains boring: do not add runtime.Gosched to every loop as seasoning. First determine whether the loop really starves useful work, then put natural blocking or chunking boundaries into the algorithm. Go 1.2 improves the worst behavior around calls, but it does not repeal scheduling.