Go1.1 on Roberto Selbach

When a system call takes a thread

Thu, 24 Oct 2013 16:39:00 +0000

I assumed GOMAXPROCS(1) meant my program would use one operating-system thread. A blocking system call corrected me without displaying much tact.

In Go 1.1’s scheduler, GOMAXPROCS controls Ps, the runtime resources required to execute Go code in parallel. It does not impose a hard limit on Ms, the operating-system threads. The runtime may need more threads because one can become stuck in a system call while runnable goroutines remain.

Suppose G1 is running on M1 with P1 and enters a blocking call the network poller cannot handle asynchronously. The runtime marks the transition and detaches P1. Another thread, M2, can acquire P1 and execute G2. There is still only one P executing Go code at a time, but two operating-system threads exist because M1 is inside the kernel.

When M1 returns, its goroutine must rejoin runnable work. If no P is immediately available, the runtime can queue the goroutine and park or reuse the thread. The M does not permanently own the P, which is precisely why useful execution can continue during the call.

Runtime-managed network operations often take a cheaper path. Non-blocking sockets are registered with the network poller, and the goroutine is parked while its M continues with other work. Ordinary blocking system calls and calls into C do not necessarily have that integration, so the thread itself may be occupied.

This matters when a program makes many concurrent calls that block unpredictably. The scheduler can preserve progress by creating threads, but threads are not free. They consume kernel resources and stack address space, and excessive creation adds scheduling overhead. Wrapping a slow C function in ten thousand goroutines does not convert the function into scalable asynchronous I/O. It converts optimism into threads.

A bounded worker group is appropriate when calling a blocking interface with limited capacity:

jobs := make(chan Job)
for i := 0; i < workers; i++ {
	go func() {
		for job := range jobs {
			runBlocking(job)
		}
	}()
}

The useful bound should come from measured service capacity, not a decorative constant selected because eight feels parallel. Backpressure at the job channel also makes overload visible instead of allowing an unbounded pile of blocked goroutines and threads.

The distinction among G, M, and P is practical here. Counting goroutines does not reveal thread use. Counting threads does not reveal Go parallelism. GOMAXPROCS says how many Ps may run Go code; it does not promise a process with exactly that many threads.

I now treat blocking foreign calls and unusual system calls as resources to measure and usually bound. The scheduler is very good at preventing one blocked thread from freezing unrelated Go work. It is not obliged to make an unlimited number of blocked threads inexpensive.

How the race detector knows

Fri, 13 Sep 2013 18:21:00 +0000

After the race detector found a real bug, I wanted to know whether it was merely running the program many times and hoping for an unfortunate schedule. It is doing something much more useful.

A data race exists when two goroutines access the same memory concurrently, at least one access writes, and synchronization does not establish the required ordering. The detector instruments loads and stores so they can be recorded in shadow state associated with application memory. It also observes synchronization operations such as channel communication and mutex locking.

The key idea is causality rather than wall-clock time. Each goroutine’s operations have an order. Synchronization joins those orders: a channel send and corresponding receive, for example, establish a happens-before edge. The detector maintains enough logical-clock information to determine whether conflicting memory accesses are ordered by such edges.

Consider this broken counter:

var n int

func increment(done chan bool) {
	n++
	done <- true
}

Two goroutines executing n++ each perform a read and a write. Sending completion values afterward does not order the increments with respect to each other. The detector sees conflicting accesses without a happens-before path and reports both stacks.

Putting a mutex around the increment changes more than timing:

mu.Lock()
n++
mu.Unlock()

Unlock and a later lock establish ordering, so accesses in the critical sections no longer race. Sleeping between increments might make overlap less likely, but it adds no synchronization relation and therefore is not a fix.

Instrumentation explains the overhead of -race. Every relevant access carries bookkeeping, and shadow memory plus goroutine histories consume substantial memory. This is the cost of seeing relationships that ordinary execution discards. I run race-enabled tests separately rather than using their timings as application benchmarks.

The detector is dynamic. If a branch is never executed, its accesses are never observed. If a test uses one worker while production uses twenty, the most interesting paths may remain untouched. Good concurrent tests still matter: vary inputs, create contention, repeat operations, and run realistic integration workloads when possible.

There can also be benign-looking races that do not crash today, such as an approximate statistics counter. They remain races under the memory model. The compiler is not required to preserve the comforting behaviour I happened to observe. Use synchronization or documented atomic operations when approximation is acceptable but racing is not.

Understanding the mechanism changed how I read reports. I no longer ask only whether the two lines could execute at the exact same nanosecond. I ask what synchronization orders them. If I cannot point to that edge, the detector has probably found a hole rather than developed an artistic difference of opinion.

Method values remember the receiver

Thu, 29 Aug 2013 12:16:00 +0000

Go 1.1 permits a useful expression I kept trying to write before it existed: a method value.

type counter struct{ n int }

func (c *counter) add(n int) { c.n += n }

c := &counter{}
add := c.add
add(3)
add(4)
fmt.Println(c.n) // 7

The expression c.add produces a function value with c bound as its receiver. Its type is func(int). This is handy when an API wants a callback and the work naturally belongs to an object.

It differs from a method expression:

add := (*counter).add
add(c, 3)

Here the receiver is explicit, so the function type is func(*counter, int). The distinction is captured receiver versus receiver as the first argument.

The receiver is evaluated when the method value is created. For a pointer receiver, the saved pointer continues to identify the same object. For a value receiver, the receiver value is copied, just as it is for an ordinary value-receiver call. This can matter for large values and for anyone expecting later replacement of a variable to retarget an already-created callback.

Method values may require storage for the bound receiver, so I would measure before putting their creation in a very hot loop. Their main virtue is not speed. It is that callback code can say server.handle instead of wrapping the same call in an otherwise pointless closure.

Counting allocations in Go 1.1

Fri, 09 Aug 2013 15:46:00 +0000

I had two versions of a formatter with nearly identical benchmark times. One looked cleaner, so naturally I distrusted it.

Go 1.1’s benchmark allocation reporting made the difference visible. Benchmarks can call b.ReportAllocs(), or the test command can request memory statistics for benchmarks. The output includes allocations per operation and bytes allocated per operation alongside timing.

func BenchmarkLabel(b *testing.B) {
	b.ReportAllocs()
	for i := 0; i < b.N; i++ {
		_ = label(42)
	}
}

My first formatter repeatedly concatenated strings:

func join(parts []string) string {
	var out string
	for _, part := range parts {
		out += part
	}
	return out
}

As out grows, concatenation creates new strings and copies prior contents. A bytes.Buffer version reduced allocation for larger inputs. For two tiny pieces, however, the direct expression remained clearer and entirely adequate. Allocation counts are measurements, not commandments delivered from a heap-shaped mountain.

The numbers need careful interpretation. “Bytes per operation” is an average over repeated benchmark iterations. Setup performed inside the timed loop counts against the operation, including test data accidentally rebuilt every time. Use b.ResetTimer() after setup when the setup is not part of what should be measured.

The benchmark result also depends on escape analysis and compiler decisions. A value that remains on the stack does not appear as a heap allocation merely because source code takes its address. Conversely, converting values to interfaces or retaining pointers can move data to the heap in ways that are not obvious from syntax. Compiler diagnostics can explain candidates after the benchmark shows there is a problem.

Allocation count and allocated bytes tell different stories. Many tiny objects increase allocator and collector work. One large object can dominate memory traffic while counting as a single allocation. Live heap is different again: an object retained for minutes affects collection even if it was allocated only once.

I now include allocation reports for benchmarks on hot parsing, formatting, and request paths. I first verify the output cannot be optimized away, keep setup honest, and compare behaviour as well as speed. Reducing allocations often improves performance, but an obscure zero-allocation function can still be a bad bargain.

I keep the benchmark input representative as well. A formatter tested only with an empty string can truthfully report zero allocations while answering a question no caller was likely to ask.

In this case the cleaner formatter also allocated less after a small adjustment. I was forced to accept the pleasant result. Software occasionally lacks respect for a good suspicion.

Reading lines with bufio.Scanner

Tue, 16 Jul 2013 17:33:00 +0000

Every small text-processing program I write begins with the same lie: “I only need to read a file line by line.” Five minutes later I am handling buffering, end-of-file, and the final line without a newline.

Go 1.1 added bufio.Scanner, which packages that loop neatly:

scanner := bufio.NewScanner(r)
for scanner.Scan() {
	line := scanner.Text()
	fmt.Println(line)
}
if err := scanner.Err(); err != nil {
	return err
}

By default Scanner splits input into lines and removes the line ending. Scan advances to the next token. Text returns that token as a string; Bytes returns the token as bytes. After the loop, Err distinguishes a clean end-of-input from a read or tokenization error.

That final check is easy to omit because the loop looks complete without it. It is not. An I/O error halfway through a configuration file should not quietly turn the remaining configuration into an unusually convincing empty section.

Scanner is more general than lines. Split accepts a split function, and the package supplies functions for words, bytes, and runes. A split function receives available data and whether the underlying reader has reached its end. It reports how far to advance, which token to return, and any error. This supports simple lexical work without writing the buffering loop again.

There is an important limit: Scanner uses a maximum token size. It is intended for reasonably sized tokens, not arbitrary enormous records. A log format that permits a single multi-megabyte line may exceed the limit and stop scanning with an error. For such input, bufio.Reader and an explicit loop provide more control. Choosing Scanner means accepting its token-oriented contract, not merely admiring its shorter spelling.

Bytes also deserves care. The returned slice may refer to storage Scanner reuses on the next call to Scan. Process it before advancing, or copy it if it must survive. Text gives a string representation that is safer to retain, with the corresponding conversion and allocation considerations.

I also avoid mixing direct reads from the underlying reader with Scanner. Scanner buffers ahead, so the reader’s apparent position need not correspond to the end of the last token I processed. One owner for the stream is a much less surprising arrangement.

I replaced my hand-written line reader with Scanner and deleted more edge-case code than I added. That is a good trade when its token limits fit the data. When they do not, the lower-level reader remains available. Go’s I/O packages are at their best when the simple case is genuinely simple and the escape hatch is still visible.

Two small Go 1.1 speedups

Tue, 25 Jun 2013 09:57:00 +0000

I rebuilt a little indexing program with Go 1.1 and it became faster without receiving any of my expert optimization, which is generally the safest kind.

The program mostly fills maps and then produces enough temporary objects to keep the garbage collector socially engaged:

for _, word := range words {
	counts[word]++
}

Go 1.1 includes a new map implementation and substantial runtime improvements. Map operations are faster, particularly for common key types, and the garbage collector does less work in several important paths. Programs heavy on either can improve simply by recompiling.

This is not a universal percentage discount. Key sizes, hit rates, map growth, allocation rate, live heap, processor count, and workload shape all matter. My toy index is evidence about my toy index.

The right comparison is to build the same source with both releases, run multiple times on an otherwise quiet machine, and look at distributions rather than choosing the friendliest result. If the program is a service, latency and pauses may matter more than total runtime.

Runtime speedups are welcome because they improve ordinary code without making it strange. I will still provide map size hints when I know them and avoid pointless allocation. A faster collector is not a licence to create garbage professionally.

The race detector found my clever cache

Tue, 04 Jun 2013 14:09:00 +0000

Go 1.1’s race detector found a bug in a cache I had described as “effectively read-only.” Adverbs remain an important source of software defects.

The cache was reduced to this:

var cached map[string]string

func lookup(key string) string {
	if cached == nil {
		cached = loadAll()
	}
	return cached[key]
}

Two handlers could observe cached == nil and assign it concurrently. Worse, one could read the map while another was publishing it. Running tests with go test -race reported the conflicting accesses and the goroutine stacks involved.

The detector instruments memory accesses in the compiled program and tracks synchronization events. When two goroutines access the same location concurrently, at least one access is a write, and no synchronization orders them, it can report a data race. The stack traces are the useful part: they connect an abstract memory-model violation to the two paths I actually wrote.

I fixed the cache with a mutex and a clear initialization path. A channel-based owner goroutine would also have worked, but building a miniature cache server around one assignment would have been enthusiasm rather than design.

The detector only sees executions that happen. A test that never drives two requests through initialization cannot reveal this race. I added a test that starts several goroutines together and exercises the path repeatedly. Even then, a clean run is evidence, not proof that every possible schedule is safe.

Race-enabled binaries are slower and use more memory because instrumentation and bookkeeping are not free. I would use them in tests and representative staging workloads rather than deciding the overhead is a mysterious production regression. The point is to make races observable, not to benchmark the detector against uninstrumented code and appear shocked.

A race report is also not permission to sprinkle locks around the named lines. The conflicting accesses may reveal confused ownership, publication without synchronization, or a larger invariant spanning several fields. Protecting one variable while leaving the invariant split can silence one report and preserve the bug.

Nor should the test ignore a report because the final value happened to be correct. The detected execution violated the memory model. A passing assertion after that is luck arriving late and asking to be mistaken for synchronization.

The race detector does not find deadlocks, logical races built from properly synchronized but wrongly ordered operations, or races in code it never executes. It finds data races, and that is already an excellent service. Mine was real, small, and previously certified by the prestigious It Seems Fine Institute.

Go 1.1 and the G-M-P scheduler

Thu, 16 May 2013 18:44:00 +0000

Go 1.1 was released this week, so naturally I ignored the pleasant surface additions and went looking for the scheduler changes. This is normal behaviour in some households.

The new scheduler is commonly described with three letters: G, M, and P. A G is a goroutine, including its stack and scheduling state. An M is an operating-system thread, the machine on which Go code eventually runs. A P is a processor-like runtime resource required to execute Go code. GOMAXPROCS controls the number of Ps and therefore how much Go code can execute simultaneously.

This extra P may seem unnecessary. Why not put runnable goroutines directly on threads? The answer becomes clearer when a thread blocks.

The naive picture

Imagine each M owns a queue of goroutines:

M0: G1 G2 G3
M1: G4 G5

If G1 enters a blocking system call, M0 blocks with it. The runtime must somehow preserve access to G2 and G3, arrange another thread, and keep global scheduling state coherent. If queues and caches belong permanently to Ms, every blocked call complicates their ownership.

In the G-M-P design, runnable work and resources are associated with P. An M must acquire a P to execute Go code. If its goroutine enters a blocking system call, the M can detach from its P. Another M acquires that P and continues running other goroutines. When the system call returns, its G becomes eligible to run again, possibly on a different M.

The identities are separate because their lifetimes and reasons for existing are separate. G represents work. M represents a kernel execution context. P represents permission and resources to execute Go work.

Local queues

Each P has a local run queue. Creating a runnable goroutine can usually place it on the current P’s queue instead of contending on one global scheduler lock. The current M draws work from its P locally, which improves locality and reduces synchronization.

Local queues create an obvious question: what happens when one P has no work while another has a long queue? Idle Ps steal runnable goroutines from busy Ps. Work stealing balances execution without forcing every enqueue and dequeue through a single shared structure.

There is still a global queue for cases that need it, and the scheduler periodically considers it so global work is not ignored. “Local queues” does not mean each P becomes an isolated principality with its own foreign policy.

This organization also gives runtime caches a natural home. Per-P allocation state, for example, can be used by whichever M currently runs that P. If such state belonged to M, blocking and replacing threads would either strand it or require more coordination.

A small scheduler experiment

I used a deliberately crude program:

func burn(done chan bool) {
	x := uint64(1)
	for i := 0; i < 200000000; i++ {
		x = x*1664525 + 1013904223
	}
	fmt.Println(x)
	done <- true
}

func main() {
	runtime.GOMAXPROCS(2)
	done := make(chan bool)
	go burn(done)
	go burn(done)
	<-done
	<-done
}

With two Ps and enough operating-system support, the two CPU-bound goroutines can execute in parallel. With one P they execute one at a time and, because these loops contain no scheduling point, one may finish before the other starts running. The distinction matters. Goroutines make concurrency cheap; GOMAXPROCS and available processors determine parallel Go execution.

This benchmark is useful for observing CPU use and almost useless for choosing an application’s GOMAXPROCS. Real programs block, allocate, communicate, and touch memory. A tight arithmetic loop mostly proves that arithmetic loops enjoy processors.

Blocking, parking, and spinning

Not all waiting has the same consequence. A goroutine waiting on a channel or runtime-managed network operation can be parked, allowing the M and P to run another G. A blocking system call may hold the M, so the P is handed off. A call into C may similarly occupy a thread the scheduler cannot use for Go work until the call returns.

The scheduler also needs to avoid excessive thread creation. It tracks spinning workers searching for work and parked workers waiting to be awakened. Waking every idle M whenever one G becomes runnable would produce a thundering herd. Waking none would leave parallel capacity unused. The details are subtle because scheduling is largely the art of replacing one kind of waste with a smaller kind.

What this means in programs

The design makes several common Go patterns scale better, but it does not absolve application code.

A goroutine that blocks while holding a mutex can prevent unrelated goroutines from making useful progress even though the scheduler itself is healthy. A producer can create runnable goroutines faster than Ps can execute them. CPU-bound goroutines can compete with latency-sensitive work. The scheduler distributes runnable work; it cannot determine which business request deserves sympathy.

The safest optimization remains architectural: block goroutines on channels or network operations when they have no work, bound queues where overload is possible, and measure latency under realistic contention. Reaching into scheduler behaviour to arrange exact execution order is not a design. It is a hostage negotiation with an implementation detail.

I like the G-M-P model because it separates concerns that were tangled in earlier schedulers. It gives runnable work local queues, lets processor resources survive blocked threads, and creates better places for per-execution caches. Go 1.1 should spend less time fighting a global scheduler lock and more time running the program.

Scheduler improvements cannot make sequential code parallel, remove lock contention, or turn infinite work into finite work. They do make the straightforward goroutine model a better foundation, which beats an application-level scheduler I would have to debug myself.