Go 1.1 was released this week, so naturally I ignored the pleasant surface additions and went looking for the scheduler changes. This is normal behaviour in some households.
The new scheduler is commonly described with three letters: G, M, and P. A G is a goroutine, including its stack and scheduling state. An M is an operating-system thread, the machine on which Go code eventually runs. A P is a processor-like runtime resource required to execute Go code. GOMAXPROCS controls the number of Ps and therefore how much Go code can execute simultaneously.
This extra P may seem unnecessary. Why not put runnable goroutines directly on threads? The answer becomes clearer when a thread blocks.
The naive picture
Imagine each M owns a queue of goroutines:
If G1 enters a blocking system call, M0 blocks with it. The runtime must somehow preserve access to G2 and G3, arrange another thread, and keep global scheduling state coherent. If queues and caches belong permanently to Ms, every blocked call complicates their ownership.
In the G-M-P design, runnable work and resources are associated with P. An M must acquire a P to execute Go code. If its goroutine enters a blocking system call, the M can detach from its P. Another M acquires that P and continues running other goroutines. When the system call returns, its G becomes eligible to run again, possibly on a different M.
The identities are separate because their lifetimes and reasons for existing are separate. G represents work. M represents a kernel execution context. P represents permission and resources to execute Go work.
Local queues
Each P has a local run queue. Creating a runnable goroutine can usually place it on the current P’s queue instead of contending on one global scheduler lock. The current M draws work from its P locally, which improves locality and reduces synchronization.
Local queues create an obvious question: what happens when one P has no work while another has a long queue? Idle Ps steal runnable goroutines from busy Ps. Work stealing balances execution without forcing every enqueue and dequeue through a single shared structure.
There is still a global queue for cases that need it, and the scheduler periodically considers it so global work is not ignored. “Local queues” does not mean each P becomes an isolated principality with its own foreign policy.
This organization also gives runtime caches a natural home. Per-P allocation state, for example, can be used by whichever M currently runs that P. If such state belonged to M, blocking and replacing threads would either strand it or require more coordination.
A small scheduler experiment
I used a deliberately crude program:
func burn(done chan bool) {
x := uint64(1)
for i := 0; i < 200000000; i++ {
x = x*1664525 + 1013904223
}
fmt.Println(x)
done <- true
}
func main() {
runtime.GOMAXPROCS(2)
done := make(chan bool)
go burn(done)
go burn(done)
<-done
<-done
}
With two Ps and enough operating-system support, the two CPU-bound goroutines can execute in parallel. With one P they execute one at a time and, because these loops contain no scheduling point, one may finish before the other starts running. The distinction matters. Goroutines make concurrency cheap; GOMAXPROCS and available processors determine parallel Go execution.
This benchmark is useful for observing CPU use and almost useless for choosing an application’s GOMAXPROCS. Real programs block, allocate, communicate, and touch memory. A tight arithmetic loop mostly proves that arithmetic loops enjoy processors.
Blocking, parking, and spinning
Not all waiting has the same consequence. A goroutine waiting on a channel or runtime-managed network operation can be parked, allowing the M and P to run another G. A blocking system call may hold the M, so the P is handed off. A call into C may similarly occupy a thread the scheduler cannot use for Go work until the call returns.
The scheduler also needs to avoid excessive thread creation. It tracks spinning workers searching for work and parked workers waiting to be awakened. Waking every idle M whenever one G becomes runnable would produce a thundering herd. Waking none would leave parallel capacity unused. The details are subtle because scheduling is largely the art of replacing one kind of waste with a smaller kind.
What this means in programs
The design makes several common Go patterns scale better, but it does not absolve application code.
A goroutine that blocks while holding a mutex can prevent unrelated goroutines from making useful progress even though the scheduler itself is healthy. A producer can create runnable goroutines faster than Ps can execute them. CPU-bound goroutines can compete with latency-sensitive work. The scheduler distributes runnable work; it cannot determine which business request deserves sympathy.
The safest optimization remains architectural: block goroutines on channels or network operations when they have no work, bound queues where overload is possible, and measure latency under realistic contention. Reaching into scheduler behaviour to arrange exact execution order is not a design. It is a hostage negotiation with an implementation detail.
I like the G-M-P model because it separates concerns that were tangled in earlier schedulers. It gives runnable work local queues, lets processor resources survive blocked threads, and creates better places for per-execution caches. Go 1.1 should spend less time fighting a global scheduler lock and more time running the program.
Scheduler improvements cannot make sequential code parallel, remove lock contention, or turn infinite work into finite work. They do make the straightforward goroutine model a better foundation, which beats an application-level scheduler I would have to debug myself.