Programming

Go 1.1 was released this week, so naturally I ignored the pleasant surface additions and went looking for the scheduler changes. This is normal behaviour in some households.

The new scheduler is commonly described with three letters: G, M, and P. A G is a goroutine, including its stack and scheduling state. An M is an operating-system thread, the machine on which Go code eventually runs. A P is a processor-like runtime resource required to execute Go code. GOMAXPROCS controls the number of Ps and therefore how much Go code can execute simultaneously.

This extra P may seem unnecessary. Why not put runnable goroutines directly on threads? The answer becomes clearer when a thread blocks.

The naive picture

Imagine each M owns a queue of goroutines:

M0: G1 G2 G3
M1: G4 G5

If G1 enters a blocking system call, M0 blocks with it. The runtime must somehow preserve access to G2 and G3, arrange another thread, and keep global scheduling state coherent. If queues and caches belong permanently to Ms, every blocked call complicates their ownership.

In the G-M-P design, runnable work and resources are associated with P. An M must acquire a P to execute Go code. If its goroutine enters a blocking system call, the M can detach from its P. Another M acquires that P and continues running other goroutines. When the system call returns, its G becomes eligible to run again, possibly on a different M.

The identities are separate because their lifetimes and reasons for existing are separate. G represents work. M represents a kernel execution context. P represents permission and resources to execute Go work.

Local queues

Each P has a local run queue. Creating a runnable goroutine can usually place it on the current P’s queue instead of contending on one global scheduler lock. The current M draws work from its P locally, which improves locality and reduces synchronization.

Local queues create an obvious question: what happens when one P has no work while another has a long queue? Idle Ps steal runnable goroutines from busy Ps. Work stealing balances execution without forcing every enqueue and dequeue through a single shared structure.

There is still a global queue for cases that need it, and the scheduler periodically considers it so global work is not ignored. “Local queues” does not mean each P becomes an isolated principality with its own foreign policy.

This organization also gives runtime caches a natural home. Per-P allocation state, for example, can be used by whichever M currently runs that P. If such state belonged to M, blocking and replacing threads would either strand it or require more coordination.

A small scheduler experiment

I used a deliberately crude program:

func burn(done chan bool) {
	x := uint64(1)
	for i := 0; i < 200000000; i++ {
		x = x*1664525 + 1013904223
	}
	fmt.Println(x)
	done <- true
}

func main() {
	runtime.GOMAXPROCS(2)
	done := make(chan bool)
	go burn(done)
	go burn(done)
	<-done
	<-done
}

With two Ps and enough operating-system support, the two CPU-bound goroutines can execute in parallel. With one P they execute one at a time and, because these loops contain no scheduling point, one may finish before the other starts running. The distinction matters. Goroutines make concurrency cheap; GOMAXPROCS and available processors determine parallel Go execution.

This benchmark is useful for observing CPU use and almost useless for choosing an application’s GOMAXPROCS. Real programs block, allocate, communicate, and touch memory. A tight arithmetic loop mostly proves that arithmetic loops enjoy processors.

Blocking, parking, and spinning

Not all waiting has the same consequence. A goroutine waiting on a channel or runtime-managed network operation can be parked, allowing the M and P to run another G. A blocking system call may hold the M, so the P is handed off. A call into C may similarly occupy a thread the scheduler cannot use for Go work until the call returns.

The scheduler also needs to avoid excessive thread creation. It tracks spinning workers searching for work and parked workers waiting to be awakened. Waking every idle M whenever one G becomes runnable would produce a thundering herd. Waking none would leave parallel capacity unused. The details are subtle because scheduling is largely the art of replacing one kind of waste with a smaller kind.

What this means in programs

The design makes several common Go patterns scale better, but it does not absolve application code.

A goroutine that blocks while holding a mutex can prevent unrelated goroutines from making useful progress even though the scheduler itself is healthy. A producer can create runnable goroutines faster than Ps can execute them. CPU-bound goroutines can compete with latency-sensitive work. The scheduler distributes runnable work; it cannot determine which business request deserves sympathy.

The safest optimization remains architectural: block goroutines on channels or network operations when they have no work, bound queues where overload is possible, and measure latency under realistic contention. Reaching into scheduler behaviour to arrange exact execution order is not a design. It is a hostage negotiation with an implementation detail.

I like the G-M-P model because it separates concerns that were tangled in earlier schedulers. It gives runnable work local queues, lets processor resources survive blocked threads, and creates better places for per-execution caches. Go 1.1 should spend less time fighting a global scheduler lock and more time running the program.

Scheduler improvements cannot make sequential code parallel, remove lock contention, or turn infinite work into finite work. They do make the straightforward goroutine model a better foundation, which beats an application-level scheduler I would have to debug myself.

I put sql.Open inside an HTTP handler because the function was named Open and I wanted to open the database. The code worked, which delayed the useful realization that *sql.DB is not one database connection.

sql.Open creates a database handle using a registered driver and its data source name. It may not contact the server immediately. The first operation can therefore be where a bad password, missing server, or malformed configuration finally becomes visible. When startup must verify the database, run a trivial query and handle its error.

The handle manages a pool of underlying connections. It is intended to be long-lived and shared by concurrent goroutines. Opening one per request creates needless pool machinery and closing each one prevents useful connection reuse.

The practical shape is:

db, err := sql.Open("driver-name", dataSource)
if err != nil {
	log.Fatal(err)
}
defer db.Close()

var one int
if err := db.QueryRow("select 1").Scan(&one); err != nil {
	log.Fatal(err)
}

The driver import and data source are database-specific. database/sql deliberately supplies common operations while drivers deal with wire protocols and server peculiarities. This boundary is useful, but it does not make SQL dialects identical or transactions portable by divine intervention.

Rows carry a connection resource until they are exhausted or closed. That makes this pattern important:

rows, err := db.Query("select name from people")
if err != nil {
	return err
}
defer rows.Close()

for rows.Next() {
	var name string
	if err := rows.Scan(&name); err != nil {
		return err
	}
}
return rows.Err()

Checking only the initial Query error misses errors encountered while iterating. Forgetting Close can keep the underlying connection unavailable to other work. A leak in a pooled abstraction tends to present itself as an unrelated request mysteriously waiting, which is a thoughtful way for a bug to meet new people.

Transactions bind operations to the transaction’s connection. Once Begin succeeds, calls that belong to the transaction must use tx.Query, tx.Exec, and friends, not the original db handle. Calling db.Exec in the middle may use another connection and therefore sit outside the transaction entirely.

Statements and transactions need closing too. I keep their lifetime in the smallest useful scope and arrange cleanup immediately after successful creation. This is less about tidiness than returning scarce server and connection resources on every error path.

I now create one *sql.DB for a configured database, verify it during startup when appropriate, pass it to code that needs it, close rows promptly, and inspect iteration errors. Boring database plumbing is good database plumbing.

I wrote an echo server with one goroutine per connection and then wondered where all the operating-system threads were. I had accepted the pleasant API before understanding the trick beneath it.

The simple server is almost suspiciously simple:

func serve(c net.Conn) {
	defer c.Close()
	buf := make([]byte, 4096)
	for {
		n, err := c.Read(buf)
		if err != nil {
			return
		}
		if _, err := c.Write(buf[:n]); err != nil {
			return
		}
	}
}

func main() {
	l, err := net.Listen("tcp", ":9000")
	if err != nil {
		panic(err)
	}
	for {
		c, err := l.Accept()
		if err != nil {
			return
		}
		go serve(c)
	}
}

Read appears to block the goroutine until bytes arrive. If every blocked goroutine required a blocked kernel thread, this design would approach the old thread-per-connection model with nicer syntax but similar scaling limits. The runtime instead integrates network descriptors with an operating-system readiness notification facility.

The toy server exits on an Accept error. A real daemon may retry temporary errors with a delay, but blindly continuing on every error turns a closed listener or exhausted descriptor table into a very efficient busy loop.

On Linux that facility is epoll; other systems provide equivalents such as kqueue. The exact backend is platform-specific, but the shape is similar. The socket is placed in non-blocking mode. A read attempts the system call. If the kernel says the operation would block, the runtime records that the goroutine is waiting for readiness on that descriptor and parks it. The underlying thread is then free to run another goroutine.

Parking is not polling

Parking a goroutine removes it from runnable work. It does not spin around asking the socket whether it feels conversational yet. The runtime’s network poller waits for the kernel to report descriptors that may now make progress. When an event arrives, the corresponding goroutine is made runnable and eventually tries the operation again.

Readiness is not completion. A descriptor reported readable may yield some bytes, an end-of-file condition, or another transient result by the time code runs. The runtime and net package retain the usual loop around non-blocking operations. The poller says “worth trying,” not “your complete application message has arrived on a silver tray.”

This distinction also explains partial reads. TCP is a byte stream. One Write at the sender does not imply one equally sized Read at the receiver. The poller reports available progress; framing remains the protocol’s responsibility. My echo loop can return each chunk because its protocol is deliberately trivial. A real message parser must accumulate and delimit data.

Deadlines join the machinery

Network code also needs a way to stop waiting. SetReadDeadline and SetWriteDeadline associate times with operations on a connection. The runtime combines descriptor waiting with timers so a parked goroutine can become runnable because the socket is ready or because its deadline expired.

Without deadlines, a slow or vanished peer can retain a goroutine and everything reachable from its stack indefinitely. Small goroutine stacks make this cheaper than a thread leak, not desirable. A server that accepts untrusted connections should make an explicit decision about idle time rather than interpreting silence as a lifelong subscription.

Closing a connection is another wake-up path. Code blocked in an operation must be released with an error rather than left attached to a descriptor that no longer has meaning. This is one reason descriptor state belongs in runtime-coordinated structures instead of being only an integer passed casually among goroutines.

What still blocks a thread

The network poller helps operations the runtime knows how to put into non-blocking mode and register. It does not transform every possible system call into asynchronous work. A call into C, an ordinary blocking file operation, or an unfamiliar kernel interface may occupy an operating-system thread. The scheduler can create or use another thread so runnable goroutines continue, but blocked threads still have a cost.

Regular disk files are particularly different from sockets. Readiness interfaces commonly report them ready even when the operation may wait on storage. Treating every Read method as equivalent because it satisfies the same interface would be tidy and wrong.

There is a nice Linux comparison here. An application written in C can build an event loop directly around epoll, maintain a state machine per connection, and achieve excellent performance. It also has to preserve that state across every partial operation and error path. Go’s runtime uses the same broad kernel capability, then presents the waiting state as an ordinary goroutine stack. The state machine has not vanished; the runtime and compiler help store it in a form humans generally prefer reading.

A practical test

I connected many idle clients to the echo server and watched thread count and memory rather than merely measuring requests per second. The number of goroutines rose with connections. Threads did not rise one-for-one. Memory grew, because every connection and goroutine is real, but not as though each connection had received a traditional large thread stack.

This does not prove every Go server scales automatically. File descriptors have process limits. Buffers consume memory. A goroutine can retain a large object graph. Handlers can block on locks, allocate furiously, or perform CPU work that has nothing to do with the poller. The mechanism solves efficient waiting, not careless architecture.

Still, it changes how I write network services. I can begin with one goroutine per connection because it is clear and maps directly to the protocol conversation. Then I measure descriptor use, memory, scheduler behaviour, and latency. I do not begin by translating the protocol into callback soup merely because the C implementation had to.

The net package’s blocking appearance is therefore honest from the goroutine’s perspective. That goroutine really does stop until it can proceed. The useful deception is that the operating-system thread need not stop with it.

I needed to print arbitrary structs for a debugging tool. After writing the third type switch I reached for reflect, with the same cautious expression normally reserved for an unfamiliar electrical panel.

Reflection starts from interfaces. reflect.TypeOf(x) reports the dynamic type stored in the interface, while reflect.ValueOf(x) represents its dynamic value. This distinction is the same type/value pair that explains why an interface containing a nil pointer is not itself nil.

The first trap is that a reflect.Value has its own validity and kind. A nil interface produces an invalid Value. A typed nil pointer produces a valid Value whose kind is Ptr and for which IsNil is true. Calling methods that do not apply to a Value’s kind panics. Reflection replaces compile-time checks with a detailed list of ways to be wrong at runtime.

A modest struct printer looks like this:

func fields(x interface{}) {
	v := reflect.ValueOf(x)
	if v.Kind() == reflect.Ptr {
		v = v.Elem()
	}
	if !v.IsValid() || v.Kind() != reflect.Struct {
		return
	}

	t := v.Type()
	for i := 0; i < v.NumField(); i++ {
		fmt.Println(t.Field(i).Name, v.Field(i).Interface())
	}
}

Even this needs qualifications. Elem on a nil pointer yields an invalid Value. Access restrictions apply to unexported fields, and asking for an interface representation where it is not permitted can panic. Production code must decide whether to skip, report, or reject these cases rather than treating every struct as an unlocked cupboard.

Settable values are another common surprise. reflect.ValueOf(n) contains a copy and is not settable. To modify n, pass &n, call Elem, verify CanSet, and then use a setter appropriate to its kind. The pointer is not ritual; it gives reflection access to the original storage.

Reflection is invaluable at boundaries where types genuinely vary: encoders, decoders, formatters, and tools that inspect user-defined structures. It is much less convincing as a way to avoid writing an interface for ordinary application behaviour. A reflected method call loses static checking, is harder to read, and performs more runtime work than a direct call.

Calling reflection “unsafe” is unfair in one important sense. The package checks its rules and panics instead of letting arbitrary memory corruption proceed. It is polite. It knocks before ruining the evening.

For my debugging printer, reflection was exactly right. I kept it at the edge, converted the inspected data into a simpler representation, and let the rest of the program remain statically typed. That boundary is where I find reflection most useful: a small dynamic vestibule leading into an otherwise ordinary Go house.

I wrote the concurrency equivalent of checking whether the refrigerator light is off by repeatedly opening the door:

var ready bool
var value string

func load() {
	value = "finished"
	ready = true
}

func main() {
	go load()
	for !ready {
	}
	fmt.Println(value)
}

It worked on my machine, which is information about my machine rather than evidence that the program is correct.

Two goroutines access ready, one writes it, and there is no synchronization. The same is true for value: seeing ready does not create a language guarantee that the write to value is visible. Compilers and processors may reorder operations where a single goroutine cannot observe the difference. Caches and registers add further opportunities for my intuition to be wrong.

The Go memory model describes which operations establish a “happens before” relationship. If one event happens before another, the effects required by that relationship are visible in the defined order. Goroutine creation, channel communication, mutex operations, and package initialization provide useful ordering guarantees. A hopeful loop around a boolean does not.

The channel version says what I meant:

func load(done chan bool) {
	value = "finished"
	done <- true
}

func main() {
	done := make(chan bool)
	go load(done)
	<-done
	fmt.Println(value)
}

The send happens before the corresponding receive completes. Since value is written before the send, reading it after the receive is properly ordered. Better still, the channel documents the handoff instead of hiding it behind a global flag.

A mutex would also work. Unlocking a mutex and subsequently locking it establishes the required ordering around protected data. The choice is not that channels are holy and mutexes are regrettable. Channels fit ownership transfer and signalling; mutexes fit shared state with a clear invariant. Using the mechanism that expresses the relationship makes the program easier to audit.

This is not merely a question of whether reading and writing a machine-sized boolean is individually atomic. Even if a load cannot tear, atomicity of that one load does not order the separate write to value. “But the flag is only one byte” answers the wrong question with admirable confidence.

The safe rule is pleasantly boring: when multiple goroutines access data and at least one modifies it, arrange synchronization unless the operations use a specifically documented atomic mechanism. Do not infer ordering from sleep calls, observed scheduling, processor brand, or the phase of the moon.

I like Go’s concurrency syntax because it makes the correct version short. That does not make every short concurrent program correct. Goroutines remove ceremony from concurrency, not causality.