Internals on Roberto Selbach

When a system call takes a thread

Thu, 24 Oct 2013 16:39:00 +0000

I assumed GOMAXPROCS(1) meant my program would use one operating-system thread. A blocking system call corrected me without displaying much tact.

In Go 1.1’s scheduler, GOMAXPROCS controls Ps, the runtime resources required to execute Go code in parallel. It does not impose a hard limit on Ms, the operating-system threads. The runtime may need more threads because one can become stuck in a system call while runnable goroutines remain.

Suppose G1 is running on M1 with P1 and enters a blocking call the network poller cannot handle asynchronously. The runtime marks the transition and detaches P1. Another thread, M2, can acquire P1 and execute G2. There is still only one P executing Go code at a time, but two operating-system threads exist because M1 is inside the kernel.

When M1 returns, its goroutine must rejoin runnable work. If no P is immediately available, the runtime can queue the goroutine and park or reuse the thread. The M does not permanently own the P, which is precisely why useful execution can continue during the call.

Runtime-managed network operations often take a cheaper path. Non-blocking sockets are registered with the network poller, and the goroutine is parked while its M continues with other work. Ordinary blocking system calls and calls into C do not necessarily have that integration, so the thread itself may be occupied.

This matters when a program makes many concurrent calls that block unpredictably. The scheduler can preserve progress by creating threads, but threads are not free. They consume kernel resources and stack address space, and excessive creation adds scheduling overhead. Wrapping a slow C function in ten thousand goroutines does not convert the function into scalable asynchronous I/O. It converts optimism into threads.

A bounded worker group is appropriate when calling a blocking interface with limited capacity:

jobs := make(chan Job)
for i := 0; i < workers; i++ {
	go func() {
		for job := range jobs {
			runBlocking(job)
		}
	}()
}

The useful bound should come from measured service capacity, not a decorative constant selected because eight feels parallel. Backpressure at the job channel also makes overload visible instead of allowing an unbounded pile of blocked goroutines and threads.

The distinction among G, M, and P is practical here. Counting goroutines does not reveal thread use. Counting threads does not reveal Go parallelism. GOMAXPROCS says how many Ps may run Go code; it does not promise a process with exactly that many threads.

I now treat blocking foreign calls and unusual system calls as resources to measure and usually bound. The scheduler is very good at preventing one blocked thread from freezing unrelated Go work. It is not obliged to make an unlimited number of blocked threads inexpensive.

How the race detector knows

Fri, 13 Sep 2013 18:21:00 +0000

After the race detector found a real bug, I wanted to know whether it was merely running the program many times and hoping for an unfortunate schedule. It is doing something much more useful.

A data race exists when two goroutines access the same memory concurrently, at least one access writes, and synchronization does not establish the required ordering. The detector instruments loads and stores so they can be recorded in shadow state associated with application memory. It also observes synchronization operations such as channel communication and mutex locking.

The key idea is causality rather than wall-clock time. Each goroutine’s operations have an order. Synchronization joins those orders: a channel send and corresponding receive, for example, establish a happens-before edge. The detector maintains enough logical-clock information to determine whether conflicting memory accesses are ordered by such edges.

Consider this broken counter:

var n int

func increment(done chan bool) {
	n++
	done <- true
}

Two goroutines executing n++ each perform a read and a write. Sending completion values afterward does not order the increments with respect to each other. The detector sees conflicting accesses without a happens-before path and reports both stacks.

Putting a mutex around the increment changes more than timing:

mu.Lock()
n++
mu.Unlock()

Unlock and a later lock establish ordering, so accesses in the critical sections no longer race. Sleeping between increments might make overlap less likely, but it adds no synchronization relation and therefore is not a fix.

Instrumentation explains the overhead of -race. Every relevant access carries bookkeeping, and shadow memory plus goroutine histories consume substantial memory. This is the cost of seeing relationships that ordinary execution discards. I run race-enabled tests separately rather than using their timings as application benchmarks.

The detector is dynamic. If a branch is never executed, its accesses are never observed. If a test uses one worker while production uses twenty, the most interesting paths may remain untouched. Good concurrent tests still matter: vary inputs, create contention, repeat operations, and run realistic integration workloads when possible.

There can also be benign-looking races that do not crash today, such as an approximate statistics counter. They remain races under the memory model. The compiler is not required to preserve the comforting behaviour I happened to observe. Use synchronization or documented atomic operations when approximation is acceptable but racing is not.

Understanding the mechanism changed how I read reports. I no longer ask only whether the two lines could execute at the exact same nanosecond. I ask what synchronization orders them. If I cannot point to that edge, the detector has probably found a hole rather than developed an artistic difference of opinion.

Go 1.1 and the G-M-P scheduler

Thu, 16 May 2013 18:44:00 +0000

Go 1.1 was released this week, so naturally I ignored the pleasant surface additions and went looking for the scheduler changes. This is normal behaviour in some households.

The new scheduler is commonly described with three letters: G, M, and P. A G is a goroutine, including its stack and scheduling state. An M is an operating-system thread, the machine on which Go code eventually runs. A P is a processor-like runtime resource required to execute Go code. GOMAXPROCS controls the number of Ps and therefore how much Go code can execute simultaneously.

This extra P may seem unnecessary. Why not put runnable goroutines directly on threads? The answer becomes clearer when a thread blocks.

The naive picture

Imagine each M owns a queue of goroutines:

M0: G1 G2 G3
M1: G4 G5

If G1 enters a blocking system call, M0 blocks with it. The runtime must somehow preserve access to G2 and G3, arrange another thread, and keep global scheduling state coherent. If queues and caches belong permanently to Ms, every blocked call complicates their ownership.

In the G-M-P design, runnable work and resources are associated with P. An M must acquire a P to execute Go code. If its goroutine enters a blocking system call, the M can detach from its P. Another M acquires that P and continues running other goroutines. When the system call returns, its G becomes eligible to run again, possibly on a different M.

The identities are separate because their lifetimes and reasons for existing are separate. G represents work. M represents a kernel execution context. P represents permission and resources to execute Go work.

Local queues

Each P has a local run queue. Creating a runnable goroutine can usually place it on the current P’s queue instead of contending on one global scheduler lock. The current M draws work from its P locally, which improves locality and reduces synchronization.

Local queues create an obvious question: what happens when one P has no work while another has a long queue? Idle Ps steal runnable goroutines from busy Ps. Work stealing balances execution without forcing every enqueue and dequeue through a single shared structure.

There is still a global queue for cases that need it, and the scheduler periodically considers it so global work is not ignored. “Local queues” does not mean each P becomes an isolated principality with its own foreign policy.

This organization also gives runtime caches a natural home. Per-P allocation state, for example, can be used by whichever M currently runs that P. If such state belonged to M, blocking and replacing threads would either strand it or require more coordination.

A small scheduler experiment

I used a deliberately crude program:

func burn(done chan bool) {
	x := uint64(1)
	for i := 0; i < 200000000; i++ {
		x = x*1664525 + 1013904223
	}
	fmt.Println(x)
	done <- true
}

func main() {
	runtime.GOMAXPROCS(2)
	done := make(chan bool)
	go burn(done)
	go burn(done)
	<-done
	<-done
}

With two Ps and enough operating-system support, the two CPU-bound goroutines can execute in parallel. With one P they execute one at a time and, because these loops contain no scheduling point, one may finish before the other starts running. The distinction matters. Goroutines make concurrency cheap; GOMAXPROCS and available processors determine parallel Go execution.

This benchmark is useful for observing CPU use and almost useless for choosing an application’s GOMAXPROCS. Real programs block, allocate, communicate, and touch memory. A tight arithmetic loop mostly proves that arithmetic loops enjoy processors.

Blocking, parking, and spinning

Not all waiting has the same consequence. A goroutine waiting on a channel or runtime-managed network operation can be parked, allowing the M and P to run another G. A blocking system call may hold the M, so the P is handed off. A call into C may similarly occupy a thread the scheduler cannot use for Go work until the call returns.

The scheduler also needs to avoid excessive thread creation. It tracks spinning workers searching for work and parked workers waiting to be awakened. Waking every idle M whenever one G becomes runnable would produce a thundering herd. Waking none would leave parallel capacity unused. The details are subtle because scheduling is largely the art of replacing one kind of waste with a smaller kind.

What this means in programs

The design makes several common Go patterns scale better, but it does not absolve application code.

A goroutine that blocks while holding a mutex can prevent unrelated goroutines from making useful progress even though the scheduler itself is healthy. A producer can create runnable goroutines faster than Ps can execute them. CPU-bound goroutines can compete with latency-sensitive work. The scheduler distributes runnable work; it cannot determine which business request deserves sympathy.

The safest optimization remains architectural: block goroutines on channels or network operations when they have no work, bound queues where overload is possible, and measure latency under realistic contention. Reaching into scheduler behaviour to arrange exact execution order is not a design. It is a hostage negotiation with an implementation detail.

I like the G-M-P model because it separates concerns that were tangled in earlier schedulers. It gives runnable work local queues, lets processor resources survive blocked threads, and creates better places for per-execution caches. Go 1.1 should spend less time fighting a global scheduler lock and more time running the program.

Scheduler improvements cannot make sequential code parallel, remove lock contention, or turn infinite work into finite work. They do make the straightforward goroutine model a better foundation, which beats an application-level scheduler I would have to debug myself.

How Go waits for the network

Fri, 12 Apr 2013 16:52:00 +0000

I wrote an echo server with one goroutine per connection and then wondered where all the operating-system threads were. I had accepted the pleasant API before understanding the trick beneath it.

The simple server is almost suspiciously simple:

func serve(c net.Conn) {
	defer c.Close()
	buf := make([]byte, 4096)
	for {
		n, err := c.Read(buf)
		if err != nil {
			return
		}
		if _, err := c.Write(buf[:n]); err != nil {
			return
		}
	}
}

func main() {
	l, err := net.Listen("tcp", ":9000")
	if err != nil {
		panic(err)
	}
	for {
		c, err := l.Accept()
		if err != nil {
			return
		}
		go serve(c)
	}
}

Read appears to block the goroutine until bytes arrive. If every blocked goroutine required a blocked kernel thread, this design would approach the old thread-per-connection model with nicer syntax but similar scaling limits. The runtime instead integrates network descriptors with an operating-system readiness notification facility.

The toy server exits on an Accept error. A real daemon may retry temporary errors with a delay, but blindly continuing on every error turns a closed listener or exhausted descriptor table into a very efficient busy loop.

On Linux that facility is epoll; other systems provide equivalents such as kqueue. The exact backend is platform-specific, but the shape is similar. The socket is placed in non-blocking mode. A read attempts the system call. If the kernel says the operation would block, the runtime records that the goroutine is waiting for readiness on that descriptor and parks it. The underlying thread is then free to run another goroutine.

Parking is not polling

Parking a goroutine removes it from runnable work. It does not spin around asking the socket whether it feels conversational yet. The runtime’s network poller waits for the kernel to report descriptors that may now make progress. When an event arrives, the corresponding goroutine is made runnable and eventually tries the operation again.

Readiness is not completion. A descriptor reported readable may yield some bytes, an end-of-file condition, or another transient result by the time code runs. The runtime and net package retain the usual loop around non-blocking operations. The poller says “worth trying,” not “your complete application message has arrived on a silver tray.”

This distinction also explains partial reads. TCP is a byte stream. One Write at the sender does not imply one equally sized Read at the receiver. The poller reports available progress; framing remains the protocol’s responsibility. My echo loop can return each chunk because its protocol is deliberately trivial. A real message parser must accumulate and delimit data.

Deadlines join the machinery

Network code also needs a way to stop waiting. SetReadDeadline and SetWriteDeadline associate times with operations on a connection. The runtime combines descriptor waiting with timers so a parked goroutine can become runnable because the socket is ready or because its deadline expired.

Without deadlines, a slow or vanished peer can retain a goroutine and everything reachable from its stack indefinitely. Small goroutine stacks make this cheaper than a thread leak, not desirable. A server that accepts untrusted connections should make an explicit decision about idle time rather than interpreting silence as a lifelong subscription.

Closing a connection is another wake-up path. Code blocked in an operation must be released with an error rather than left attached to a descriptor that no longer has meaning. This is one reason descriptor state belongs in runtime-coordinated structures instead of being only an integer passed casually among goroutines.

What still blocks a thread

The network poller helps operations the runtime knows how to put into non-blocking mode and register. It does not transform every possible system call into asynchronous work. A call into C, an ordinary blocking file operation, or an unfamiliar kernel interface may occupy an operating-system thread. The scheduler can create or use another thread so runnable goroutines continue, but blocked threads still have a cost.

Regular disk files are particularly different from sockets. Readiness interfaces commonly report them ready even when the operation may wait on storage. Treating every Read method as equivalent because it satisfies the same interface would be tidy and wrong.

There is a nice Linux comparison here. An application written in C can build an event loop directly around epoll, maintain a state machine per connection, and achieve excellent performance. It also has to preserve that state across every partial operation and error path. Go’s runtime uses the same broad kernel capability, then presents the waiting state as an ordinary goroutine stack. The state machine has not vanished; the runtime and compiler help store it in a form humans generally prefer reading.

A practical test

I connected many idle clients to the echo server and watched thread count and memory rather than merely measuring requests per second. The number of goroutines rose with connections. Threads did not rise one-for-one. Memory grew, because every connection and goroutine is real, but not as though each connection had received a traditional large thread stack.

This does not prove every Go server scales automatically. File descriptors have process limits. Buffers consume memory. A goroutine can retain a large object graph. Handlers can block on locks, allocate furiously, or perform CPU work that has nothing to do with the poller. The mechanism solves efficient waiting, not careless architecture.

Still, it changes how I write network services. I can begin with one goroutine per connection because it is clear and maps directly to the protocol conversation. Then I measure descriptor use, memory, scheduler behaviour, and latency. I do not begin by translating the protocol into callback soup merely because the C implementation had to.

The net package’s blocking appearance is therefore honest from the goroutine’s perspective. That goroutine really does stop until it can proceed. The useful deception is that the operating-system thread need not stop with it.

Reflection is a polite form of unsafe

Tue, 12 Mar 2013 12:24:00 +0000

I needed to print arbitrary structs for a debugging tool. After writing the third type switch I reached for reflect, with the same cautious expression normally reserved for an unfamiliar electrical panel.

Reflection starts from interfaces. reflect.TypeOf(x) reports the dynamic type stored in the interface, while reflect.ValueOf(x) represents its dynamic value. This distinction is the same type/value pair that explains why an interface containing a nil pointer is not itself nil.

The first trap is that a reflect.Value has its own validity and kind. A nil interface produces an invalid Value. A typed nil pointer produces a valid Value whose kind is Ptr and for which IsNil is true. Calling methods that do not apply to a Value’s kind panics. Reflection replaces compile-time checks with a detailed list of ways to be wrong at runtime.

A modest struct printer looks like this:

func fields(x interface{}) {
	v := reflect.ValueOf(x)
	if v.Kind() == reflect.Ptr {
		v = v.Elem()
	}
	if !v.IsValid() || v.Kind() != reflect.Struct {
		return
	}

	t := v.Type()
	for i := 0; i < v.NumField(); i++ {
		fmt.Println(t.Field(i).Name, v.Field(i).Interface())
	}
}

Even this needs qualifications. Elem on a nil pointer yields an invalid Value. Access restrictions apply to unexported fields, and asking for an interface representation where it is not permitted can panic. Production code must decide whether to skip, report, or reject these cases rather than treating every struct as an unlocked cupboard.

Settable values are another common surprise. reflect.ValueOf(n) contains a copy and is not settable. To modify n, pass &n, call Elem, verify CanSet, and then use a setter appropriate to its kind. The pointer is not ritual; it gives reflection access to the original storage.

Reflection is invaluable at boundaries where types genuinely vary: encoders, decoders, formatters, and tools that inspect user-defined structures. It is much less convincing as a way to avoid writing an interface for ordinary application behaviour. A reflected method call loses static checking, is harder to read, and performs more runtime work than a direct call.

Calling reflection “unsafe” is unfair in one important sense. The package checks its rules and panics instead of letting arbitrary memory corruption proceed. It is polite. It knocks before ruining the evening.

For my debugging printer, reflection was exactly right. I kept it at the edge, converted the inspected data into a simpler representation, and let the rest of the program remain statically typed. That boundary is where I find reflection most useful: a small dynamic vestibule leading into an otherwise ordinary Go house.

Measuring and tuning Go GC pauses

Fri, 28 Dec 2012 14:36:00 +0000

One request in a test server occasionally took much longer than its neighbours. Rather than convict the collector from one latency graph, I recorded request times and the runtime’s pause samples on the same workload.

The current collector stops application work while it traces the reachable heap. runtime.ReadMemStats exposes NumGC, PauseTotalNs, and a ring of recent PauseNs values. I sampled before and after a fixed five-minute run, then compared only collections added during that window. Mixing in old pauses made short tests look wonderfully stable or mysteriously awful depending on what had run first.

That pause is easy to describe and harder to judge. A short command-line tool may never care. A busy service with latency requirements can care very much, even if its average throughput looks fine. Averages are talented at concealing the request somebody was actually waiting for.

My first table kept four numbers for each run: requests per second, the 50th and 99th percentile request latency, collection count, and the largest observed GC pause. Average latency alone hid the event I cared about.

The naive handler allocated a buffer for every item:

func checksum(items [][]byte) uint32 {
	var sum uint32
	for _, item := range items {
		buf := make([]byte, len(item))
		copy(buf, item)
		for _, b := range buf {
			sum += uint32(b)
		}
	}
	return sum
}

The copy served no purpose. Removing it reduced collections and improved the long tail. This was not clever tuning; I had simply stopped manufacturing garbage.

The collector’s job depends on both allocated volume and live data. Producing temporary objects quickly causes collections to happen more often. Retaining a large graph gives each marking pass more reachable memory to inspect. These are different problems: one concerns allocation rate, the other heap size and object reachability.

After that fix I tried GOGC=50, the default 100, and 200, restarting the process for each run. Lowering it collected more often with a smaller heap. Raising it collected less often but used more memory. On this service, 200 helped the tail a little and increased memory enough that I kept the default. That is a result, not a disappointment.

Manual runtime.GC() calls were useful to prove that the latency recorder could see a pause. They did not belong in the handler. Forcing collection chooses when to pay and can easily turn an occasional pause into a dependable stream of them.

The useful knob came last. First came a repeatable load, pause deltas from the same time window, and removal of an accidental allocation. Otherwise GOGC is just a dial with excellent opportunities for superstition.

Measuring a hot stack split

Mon, 03 Dec 2012 17:11:00 +0000

One benchmark had a strange cliff: adding an innocent-looking local array made a short recursive walk much slower. I suspected cache effects first. The CPU profile pointed at the runtime’s stack-growth path instead.

A goroutine begins with a small stack. Before a function uses its frame, generated code checks whether enough stack remains. If not, the runtime grows the stack by adding another segment and resumes the call. A special stack-splitting path does the unpleasant bookkeeping so ordinary Go code can continue pretending stacks are infinite, at least until memory becomes considerably less theoretical.

The benchmark was more useful as a depth sweep than as one headline number. I ran the same walk at depths 8, 16, 32, 64, and 128, then repeated it with a slightly larger local frame. Most points scaled smoothly. One narrow range jumped when calls repeatedly reached the end of a segment.

The segmented arrangement makes the initial cost of a goroutine small, but it is not magic. Each goroutine still has scheduler state and a stack segment. Every additional segment consumes memory. Deep recursion can grow a stack dramatically, and keeping the goroutine alive keeps its stack alive as well.

That narrow range is the interesting bit. If a function arrives with almost, but not quite, enough room, it grows the stack; returning can release the segment, and the next call grows it again. This “hot split” makes a particular call depth look much worse than its neighbours. A single benchmark depth can therefore blame the algorithm for standing on an implementation seam.

I checked the profile at the slow depth, then moved the benchmark one call shallower and deeper. The runtime samples faded at both neighbours. That was better evidence than merely noticing that recursion was involved. Changing the benchmark’s local array size moved the cliff too.

I did not contort the application to dodge one segment boundary. The runtime implementation will change, and ordinary code should not depend on today’s segment size. I did keep the depth sweep: it tells me whether a future optimization improves the work or merely moves the cliff.

Small stacks still make many goroutines practical. They also leave fingerprints in microbenchmarks. When one input size falls off a ledge, profile it and test both sides before rewriting the loop.

Where did that Go value escape to?

Wed, 14 Nov 2012 15:27:00 +0000

I changed a small parser to reuse a helper and made it slower. This was irritating because the new version looked cleaner, performed the same amount of useful work, and did not contain the traditional performance feature known as “a very silly loop.”

The reduced example looked like this:

type point struct {
	x, y int
}

func origin() *point {
	p := point{}
	return &p
}

Anyone arriving from C may feel a brief alarm here. p is local, yet the function returns its address. Go makes this safe by deciding where the value must live. The important distinction is not whether the source declares a value inside a function. It is whether the compiler can prove that the value stops being reachable when that function returns.

If it cannot, the value escapes and is allocated somewhere with a lifetime long enough for its uses, normally the heap. If it can prove the value remains local, it can use the goroutine’s stack. This analysis is why Go can permit returning a pointer to a local without turning every local variable into a heap allocation.

A small experiment

I wrote two deliberately uninteresting benchmarks:

var sink *point

func local() int {
	p := point{x: 20, y: 22}
	return p.x + p.y
}

func escaping() int {
	p := point{x: 20, y: 22}
	sink = &p
	return p.x + p.y
}

The arithmetic is identical. The assignment to the package variable is not. After escaping returns, another function can read sink, so p must survive. The compiler cannot put it in a dead stack frame and hope nobody notices.

The compiler can print its decisions. Building with the compiler’s escape-analysis diagnostics enabled is considerably more useful than staring at new(point) and guessing. The exact wording and flags are compiler details and may change, but reports identifying values moved to the heap make a good starting point.

This matters because source syntax alone is a poor allocation detector. new(T) does not decree that T lives on the heap. Returning a value does not decree that a copy must be heap allocated. Conversely, innocent-looking interface conversions, closures, or calls through code the compiler cannot fully inspect may cause values to escape.

Interfaces and hidden addresses

Suppose I replace a direct call with this:

func printAny(v interface{}) {
	fmt.Println(v)
}

func report() {
	p := point{x: 20, y: 22}
	printAny(p)
}

Putting p into an interface constructs an interface value containing dynamic type information and the value’s data. Depending on what the compiler knows about the call and how that data is represented, it may need storage beyond the current frame. The right answer is not “interfaces allocate.” The right answer is to inspect this particular program with this compiler, then measure it. Blanket rules age badly.

Closures offer a more obvious case:

func counter() func() int {
	n := 0
	return func() int {
		n++
		return n
	}
}

The returned function still uses n, so n cannot disappear when counter returns. The compiler arranges storage for the captured variable. This is not a leak. It is the semantics I asked for, delivered with a bill attached.

Stack is not a moral achievement

It is tempting to read an escape report as a list of compiler failures. That is not helpful. Heap allocation is necessary whenever data outlives a call, and Go’s garbage collector exists precisely because many useful objects do. The report says where the compiler could prove a lifetime, not whether the code is virtuous.

Still, allocations have costs. Allocating requires runtime work. More live heap data gives the garbage collector more to consider. A short-lived object may be cheap, but a hot loop producing millions of them deserves attention. I use benchmarks to establish that the path matters, allocation diagnostics to locate candidates, and another benchmark to verify the change. Reversing this order produces beautifully optimized code nobody needed.

Sometimes a small API change keeps a value local. A function can fill a caller-owned value instead of returning a pointer:

func setOrigin(p *point) {
	p.x = 0
	p.y = 0
}

func distance() int {
	var p point
	setOrigin(&p)
	return p.x + p.y
}

Passing an address does not automatically force escape. If the compiler can see that setOrigin does not retain it, p may remain on the caller’s stack. Again, the analysis follows lifetime, not punctuation.

I fixed my parser by removing one retained pointer from a frequently called helper. The result allocated less and recovered the lost time. I did not redesign every function to avoid pointers, because that would exchange a measured problem for a readability problem.

Escape analysis is best treated as an explanation facility. It tells me why the runtime is allocating when a profile says allocation matters. That is much better than the old C technique of declaring stack allocation “fast,” then accidentally returning its address and discovering a more exciting category of performance bug.

Looking under a Go map

Thu, 18 Oct 2012 18:05:00 +0000

I was timing a small word counter and noticed that making the map with a rough size helped more than I expected.

func count(words []string) map[string]int {
	m := make(map[string]int, len(words))
	for _, word := range words {
		m[word]++
	}
	return m
}

Of course len(words) is an overestimate when words repeat. It was still cheaper than repeatedly growing a map created with make(map[string]int).

A Go map is a hash table arranged as buckets. A hash of the key chooses a bucket, then the runtime examines entries in that bucket for a matching key. The bucket stores several keys and values rather than allocating a separate node for each item. That is a useful contrast with the traditional chained hash table I first learned, where every collision grew a little linked list and the allocator got invited to dinner.

When too many entries accumulate, the table grows. Growth is not free: keys and values must end up in buckets appropriate for the larger table. Supplying a reasonable hint lets the runtime start closer to the required size. It is a hint, not a promise about exact allocation, and writing make(map[K]V, oneMillion) because perhaps one day there may be a million entries is not optimization. It is interior decorating for an empty warehouse.

The representation explains a few visible rules. Map elements are not addressable. If the table grows and moves entries among buckets, a pointer to an element would become troublesome. This is illegal:

p := &m["answer"]

For a struct value, retrieve it, change the copy, and assign it back; or store pointers as the map values when that ownership model makes sense.

Iteration order is another thing not to build upon. The language does not specify it. Bucket layout and table state are implementation details, so code that happens to print keys in a pleasant order today has merely received a temporary kindness. Sort the keys when order matters.

There is also no general protection for concurrent access. A map operation changes more than the apparent key/value pair when it triggers growth or updates bucket metadata. I put a mutex around shared maps rather than trying to reason that two writers probably hash to different buckets. “Probably” is a peculiar synchronization primitive.

My practical rule is now simple: give make an honest size estimate when one is already available, never depend on iteration order, and make ownership of a map boringly clear. Knowing about buckets is useful. Trying to outsmart them from application code generally is not.

Debugging slice capacity aliasing

Fri, 28 Sep 2012 11:42:00 +0000

Two rows in a parser changed at once. I had stored each row by appending to a scratch slice, and enough spare capacity let the next row reuse the first row’s backing array.

scratch := make([]byte, 0, 64)
first := append(scratch, "Troy"...)
scratch = scratch[:0]
second := append(scratch, "Abed"...)

fmt.Printf("%q %q\n", first, second)

Both printed "Abed". No assignment to first appeared in the debugger, which sent me looking in entirely the wrong place.

The quickest diagnostic was printing lengths, capacities, and the first-element addresses at the point each slice was saved:

fmt.Printf("first len=%d cap=%d p=%p\n", len(first), cap(first), &first[0])
fmt.Printf("second len=%d cap=%d p=%p\n", len(second), cap(second), &second[0])

Matching addresses exposed the alias. Capacity was the clue: resetting the length to zero did not discard the storage. The next append was allowed to write there.

The saved row needed ownership, so I copied it:

row := make([]byte, len(scratch))
copy(row, scratch)
rows = append(rows, row)

For scratch data that is consumed before reuse, sharing capacity is exactly the optimization I want. For data stored beyond that reuse, it is an ownership bug. I now inspect cap and backing addresses before staring at the append line as if it had betrayed me.

One caution: taking &s[0] requires a non-empty slice. In this parser the rows are non-empty; a general diagnostic needs to check len(s) first. That tiny check is cheaper than another afternoon blaming the debugger.

What is inside a Go interface?

Fri, 07 Sep 2012 16:18:00 +0000

I lost half an hour today to an interface that was not nil even though the pointer I had put in it most certainly was. This is the smallest version of my mistake:

type File struct{}

func open() *File { return nil }

func main() {
	var f interface{} = open()
	fmt.Println(f == nil) // false
}

At first this looks like the language is being difficult merely to keep programmers alert. It is not. An interface value contains two things: a description of the concrete type and the concrete value itself. The interface above is approximately (*File, nil). A nil interface is (nil, nil). Those are not the same pair.

This is also why assigning an integer or a pointer to an interface is more than painting a different type on the same bits. The runtime needs enough information to identify the dynamic type, copy the value, compare it where permitted, and find methods. For an empty interface that amounts to a type descriptor and data. A non-empty interface also needs a table connecting its method set to implementations for the concrete type.

The table is prepared for a particular concrete-type/interface-type combination. A call such as

type Stringer interface { String() string }

func print(s Stringer) {
	fmt.Println(s.String())
}

can therefore dispatch through a known slot in that table. It does not have to search every method by name on every call. The implicit satisfaction of interfaces feels dynamic at the source level, but the machinery is quite disciplined.

There are practical consequences besides the nil trap. Copying an interface copies its pair of words, not necessarily an independent copy of everything reachable through the value. Comparing two interfaces first considers their dynamic types, then compares their dynamic values; putting a slice in an interface and comparing it will still panic because slices are not comparable. An interface does not confer diplomatic immunity on its contents.

Type assertions fit the same model. v, ok := x.(*File) asks whether the dynamic type in x is *File; it does not inspect the pointer and reject it merely because its value is nil. If the assertion succeeds, v may still be nil. The two-result form is useful because a failed assertion then reports ok == false instead of panicking and turning a routine check into theatre.

I like this representation because it explains several language rules at once. It also gives me a better debugging question. Instead of asking “is this interface nil?”, I now ask “what type and value did I put in it?” Usually that is the question I should have asked before blaming the compiler.

Batching calls across cgo

Wed, 29 Aug 2012 18:45:00 +0000

I wrapped a C checksum function and called it once per 32-byte record. The C loop won its isolated benchmark, yet the complete program lost. I had measured the work and ignored the doorway.

The fix was not exotic:

// One crossing for the whole buffer.
sum := C.checksum((*C.uchar)(unsafe.Pointer(&buf[0])), C.size_t(len(buf)))

I benchmarked batches of 1, 16, 256, and 4096 records from the same input. On my machine the one-record version managed about 7 MB/s; batches of 16 reached 58 MB/s, 256 reached 170 MB/s, and 4096 changed little at 174 MB/s. These are not portable constants. The flattening was the useful result: by 256 records, the checksum was work again and the bridge was no longer the benchmark.

I also timed the pure Go checksum in the same harness. It reached 112 MB/s. That made the decision less silly: either stay in Go and keep the simple boundary, or batch enough for C’s faster loop to pay its admission fee. Calling C per record was the one clearly bad option.

The batch API takes a contiguous buffer plus record metadata and returns all checksums at once. It also makes ownership easy to state: the buffer remains valid during the call, and C retains no Go pointer afterward.

The caveat is the empty slice: &buf[0] is invalid when len(buf) == 0, so real code checks length first. C memory needs C lifetime rules, callbacks need even more care, and a blocking C call affects runtime scheduling differently from an ordinary Go function.

I picked 256 records because larger batches bought almost nothing and delayed results. The benchmark supplied both halves of that choice; “fewer cgo calls” by itself would have pushed the batch size toward absurdity.

Go 1 stop-the-world garbage collection

Wed, 01 Aug 2012 20:20:00 +0000

I graphed request times for a Go 1 service and found narrow spikes hiding behind a good average. CPU was not saturated and the network was quiet. Allocation tracing pointed toward the garbage collector.

The reduced workload simply creates and drops temporary objects:

for _, line := range lines {
	fields := bytes.Split(line, []byte(","))
	consume(fields)
}

In Go 1, garbage collection stops ordinary Go execution while the collector identifies reachable heap objects and reclaims unreachable storage. The runtime coordinates goroutines at safe points, considers roots including stacks and global data, traces pointers through live objects, and sweeps storage that is no longer reachable. Those pauses are a direct property of the collector in today’s release.

The surprise was the difference between throughput and latency. The total collection work can be a tolerable fraction of a ten-second run while one request arriving during a stop sees the pause directly. An average conceals where time is concentrated. Percentiles and a timeline expose it.

Live heap size and allocation rate both matter, but differently. More live objects give the marker more reachable data to examine. A high rate of short-lived allocation fills available heap space quickly and brings the next collection nearer. Retaining a tiny pointer into a large structure can keep the structure reachable even when the programme considers most of it finished.

The collector does not compact the heap into one tidy block. That avoids rewriting every pointer but leaves allocation and free-space management to the runtime. Details of size classes, metadata, and precisely how particular regions are scanned are implementation-specific. The safe source-level rule is reachability, not a guessed address or object movement policy.

I prefer reducing obvious temporary allocation at a clear boundary. In this parser, reusing a scratch slice and avoiding repeated separator construction improved the latency graph without turning the package into an object pool. A profile before and after keeps that change honest.

The caveat is ownership. Reusing a buffer while another goroutine still reads it trades collection pauses for data races and corrupted requests, an impressively bad bargain. Pools can also retain far more memory than expected. Simpler allocation remains the right default until measurement says otherwise.

Go’s collector will certainly evolve, but application explanations should not arrive from the future. For Go 1 today, stop-the-world pauses are part of the runtime’s cost model. My service became steadier after allocating less, and the graph became considerably less interesting. Production graphs are at their best when they are poor conversationalists.

How Go 1 goroutine stacks grow

Wed, 04 Jul 2012 19:50:00 +0000

I launched one hundred thousand sleeping goroutines and expected memory use to resemble one hundred thousand operating-system threads. It did not. The programme was not free, but it was possible, which made the stack implementation more interesting than the programme.

The trigger can be reduced to this:

ch := make(chan bool)
for i := 0; i < 100000; i++ {
	go func() {
		<-ch
	}()
}

An operating-system thread traditionally reserves a comparatively large contiguous stack. Multiplying that reservation by one hundred thousand would end the experiment quickly. A Go 1 goroutine starts with a small stack segment and grows as calls require more space. Most of these goroutines call very little before blocking, so most never need a large stack.

The mechanism begins in compiled function entry code. Before a function consumes its frame, generated code checks whether the current stack segment has sufficient room. If it does, execution continues normally. If not, control enters the runtime, which arranges more stack space and resumes the call. The programme sees an ordinary function call; the compiler and runtime cooperate to make the stack conditional.

In this Go 1 implementation, growth uses segmented stacks. A new stack segment can be allocated and linked to the older segment rather than requiring one giant contiguous reservation from the start. The runtime records enough information to return across the segment boundary later. As calls unwind, it can move back to the preceding segment.

My surprise was that “small stacks” really means “small initial commitment plus machinery.” Each goroutine still needs a descriptor, scheduling state, stack memory, and any heap data referenced by its closure. One hundred thousand blocked goroutines consume real memory. They simply avoid paying in advance for deep call chains they may never execute.

Recursion demonstrates growth more directly:

func descend(n int) int {
	var pad [128]byte
	pad[0] = byte(n)
	if n == 0 {
		return int(pad[0])
	}
	return int(pad[0]) + descend(n-1)
}

Each call needs a frame, so eventually a stack check fails and the runtime grows the goroutine’s stack. The exact initial size, check sequence, and segment bookkeeping are implementation details. They are useful when reading assembly or runtime source, but they are not values a Go 1 programme should encode.

Stack growth interacts with pointers. The garbage collector and runtime must know where active frames and pointers live while goroutines stop, resume, and cross segments. A goroutine blocked in a channel operation still has reachable values on its stack. Those values cannot be reclaimed merely because the goroutine is not currently executing.

The scheduler is another separate layer. Cheap stacks make large goroutine counts feasible, but they do not create one kernel thread per goroutine. The old scheduler multiplexes runnable goroutines onto operating-system threads. A blocked goroutine can be parked while another runs. Later scheduler designs should not be projected backward onto the runtime installed here.

Segmented growth has a cost pattern worth noting. If execution repeatedly crosses a stack boundary near a hot call, allocating and discarding segments can become expensive. This is sometimes called a hot split. It is a runtime implementation concern rather than permission to avoid functions, but it explains why stack strategy can appear in profiles of otherwise ordinary code.

I prefer goroutines because they express independent activities and communication clearly, not because a benchmark proves I can manufacture six figures of them. A goroutine should still have a lifetime, an owner for cancellation, and a reason to exist. Leaking a cheap goroutine only makes the leak more affordable, not correct.

The caveat in my test is the closure and channel. Every goroutine waits forever unless the channel is closed, and closing it releases a runnable herd that has to be scheduled. Measuring only the quiet blocked state ignores startup and wake-up costs. A realistic service also has per-request buffers, queues, and other data that may dwarf the initial stack.

The result is a useful division of labour. I write straightforward calls and let stacks grow when necessary. The compiler inserts checks, the runtime supplies segments, and the scheduler runs whichever goroutines are ready. It is not magic; it is several carefully placed pieces of bookkeeping pretending not to interrupt the function. That is close enough to magic for a Wednesday, but much easier to profile.

Inside Go interfaces, slices, and maps

Wed, 06 Jun 2012 21:05:00 +0000

I passed a slice to a helper, changed an element, and watched the caller’s data change. Then I appended one more element and watched the helper apparently wander off with its own copy. The language rules explain both results, but the runtime representation makes them difficult to forget.

Start with the smallest example:

func alter(s []int) {
	s[0] = 9
	s = append(s, 4)
}

func main() {
	a := []int{1, 2, 3}
	alter(a)
	fmt.Println(a)
}

A slice value is a small descriptor: a pointer to an underlying array, a length, and a capacity. Passing the slice copies that descriptor, not all the elements. Both descriptors initially point into the same array, so assigning s[0] changes an element visible through a.

append is conditional. If the existing array has enough unused capacity, it can place new elements there and return a descriptor with a larger length. If capacity is exhausted, it allocates another array, copies elements, and returns a descriptor pointing to the new storage. Assigning that returned descriptor only to local s does not update the caller’s descriptor. This is why useful functions return the result of append.

My surprise was that a slice is neither an array nor a conventional pointer to a container object. It is a value describing a window onto array storage. Two slices can overlap, have different lengths, and still share elements. Capacity records how far that particular window may grow before another allocation is needed.

Interfaces add a different descriptor. Conceptually, an empty interface carries a dynamic type and a data word. A non-empty interface also needs information connecting the dynamic concrete type to the interface’s required methods. In the Go 1 runtime this is represented with internal tables and data pointers, but those names and layouts are implementation details, not structures application code may safely reproduce.

The crucial rule is that an interface value has both dynamic type and dynamic value. Assigning a typed nil pointer to an interface records the pointer’s concrete type even though the pointer value is nil:

var p *bytes.Buffer
var x interface{} = p
fmt.Println(x == nil) // false

The interface itself is not nil because its type component is populated. Calls and type assertions use that dynamic type information. This also explains why putting a value into an interface is not the same operation as erasing every fact about it.

Maps look simplest in source and hide the most machinery. A map value refers to a runtime-managed hash table. The runtime hashes a key, selects storage containing possible entries, and compares candidate keys. As entries accumulate, it allocates and organizes more bucket storage. The exact bucket layout, growth thresholds, and hash details belong to this Go implementation and can change without violating Go 1 compatibility.

Copying a map value therefore does not copy every entry. Both values refer to the same underlying map data:

a := map[string]int{"one": 1}
b := a
b["two"] = 2
fmt.Println(a["two"])

That prints 2. A nil map can be read and ranged over, but insertion requires an initialized map because there is no table to receive the entry. make creates the runtime map structure; it does not pre-create a fixed array with one slot per possible key.

Map iteration order is deliberately unspecified. The hash representation is designed for lookup, not for presenting keys in a stable narrative. If output order matters, I collect keys, sort them, and iterate over the sorted slice. Depending on the order observed in one run turns an internal accident into an external protocol.

All three types are cheap to pass as values compared with copying their referenced contents, but “cheap” is not “free” and “references storage” is not “is a reference variable.” The descriptor itself is copied. Reassigning a local slice or map variable does not reassign the caller’s variable. Mutating shared underlying storage may still be visible.

I prefer using these semantic rules in ordinary code and keeping the word-level model for debugging and performance questions. The model explains aliasing, nil interfaces, append, and map copying. The caveat is that runtime source is not the language specification. Code using unsafe to forge these headers ties itself to one compiler version, architecture, and a generous interpretation of luck.

My helper now returns the appended slice, and its caller assigns the result. No reflection and no unsafe arithmetic were required. Internals did their best work by making the simple rule memorable, then staying internal.