Concurrency

One benchmark had a strange cliff: adding an innocent-looking local array made a short recursive walk much slower. I suspected cache effects first. The CPU profile pointed at the runtime’s stack-growth path instead.

A goroutine begins with a small stack. Before a function uses its frame, generated code checks whether enough stack remains. If not, the runtime grows the stack by adding another segment and resumes the call. A special stack-splitting path does the unpleasant bookkeeping so ordinary Go code can continue pretending stacks are infinite, at least until memory becomes considerably less theoretical.

The benchmark was more useful as a depth sweep than as one headline number. I ran the same walk at depths 8, 16, 32, 64, and 128, then repeated it with a slightly larger local frame. Most points scaled smoothly. One narrow range jumped when calls repeatedly reached the end of a segment.

The segmented arrangement makes the initial cost of a goroutine small, but it is not magic. Each goroutine still has scheduler state and a stack segment. Every additional segment consumes memory. Deep recursion can grow a stack dramatically, and keeping the goroutine alive keeps its stack alive as well.

That narrow range is the interesting bit. If a function arrives with almost, but not quite, enough room, it grows the stack; returning can release the segment, and the next call grows it again. This “hot split” makes a particular call depth look much worse than its neighbours. A single benchmark depth can therefore blame the algorithm for standing on an implementation seam.

I checked the profile at the slow depth, then moved the benchmark one call shallower and deeper. The runtime samples faded at both neighbours. That was better evidence than merely noticing that recursion was involved. Changing the benchmark’s local array size moved the cliff too.

I did not contort the application to dodge one segment boundary. The runtime implementation will change, and ordinary code should not depend on today’s segment size. I did keep the depth sweep: it tells me whether a future optimization improves the work or merely moves the cliff.

Small stacks still make many goroutines practical. They also leave fingerprints in microbenchmarks. When one input size falls off a ledge, profile it and test both sides before rewriting the loop.

I wrote a small fan-out test and expected twice as many goroutines to mean twice as much parallel work. The result barely moved. That sent me away from the programme and into the runtime scheduler.

The test looked innocent:

for i := 0; i < 4; i++ {
	go crunch(work[i])
}

Goroutines are cheap concurrent activities, but concurrency does not itself promise simultaneous execution on every processor. In the current runtime, scheduling is still comparatively simple, and processor use depends on runtime settings, operating-system threads, blocking calls, and where the scheduler gets a chance to run.

The surprise was how many layers my little loop depended upon. A goroutine begins with a small stack and runtime bookkeeping. Runnable goroutines wait for the scheduler. The scheduler places work onto operating-system threads, and the kernel finally decides when those threads run. Linux 3.0 may schedule the threads beautifully, but it cannot create parallel Go execution the runtime never offered it.

This scheduler predates the more ambitious designs being discussed for later Go. It should not be described using machinery that does not exist yet. There is no reason to draw a modern diagram over an old runtime and then congratulate the diagram for explaining it.

Blocking is the interesting case. If a goroutine enters a system call, the runtime may need another thread so runnable Go code can continue. Channel operations and synchronization also move goroutines between runnable and waiting states. The scheduler therefore coordinates cheap user-level goroutines with more expensive kernel threads rather than mapping one permanently to the other.

I prefer to write synchronization for correctness first and treat parallel speedup as a measured property. A channel can make ownership clear even on one processor. If CPU parallelism matters, I set the runtime’s processor allowance explicitly, run a long enough workload, and compare against a serial version.

The caveat is that scheduler behaviour is an implementation detail in active development. A trick that coaxes today’s scheduler may become useless or harmful after a snapshot. Explicit communication and coarse useful work are safer investments than depending on an accidental scheduling order.

My test eventually did improve, but only after I made each unit of work large enough to outweigh coordination and enabled the processors I intended to use. Four goroutines were not four tiny employees waiting inside the computer. This was disappointing for payroll, but helpful for the benchmark.

Scheduler discussions become folklore quickly, so I built a few experiments that force observable events instead of inferring everything from total runtime.

The first launches workers that announce when they start, wait on a gate, perform fixed CPU work, and announce completion. A channel carries each announcement to one logger goroutine. That gives me a trace without concurrent writes making the output resemble ransom typography.

func worker(id int, gate <-chan bool, events chan<- Event) {
    events <- Event{id, "ready"}
    <-gate
    burnCPU()
    events <- Event{id, "done"}
}

I run the same executable with different GOMAXPROCS values. With one processor thread allowed, CPU-bound completion is serialized even though all workers are runnable. With two, completions overlap in wall time on my dual-core machine. More than two adds runnable work but no additional cores, and eventually loses to scheduling overhead.

The second experiment replaces CPU work with pipe reads. Many goroutines can wait for I/O while the runtime arranges for other work to proceed. Increasing the processor allowance barely changes throughput because the external producer is the limit. This is why one benchmark cannot characterize “goroutine performance.”

The third experiment is intentionally rude: one goroutine runs a tight arithmetic loop with no communication while another tries to report periodically. On this early runtime, the reporter can be delayed noticeably. Scheduling behavior is under active development, and I should not assume the preemption properties of a mature operating-system scheduler.

Adding explicit communication points makes the test cooperative and reflects real pipeline code better. I do not sprinkle meaningless calls into production loops merely to influence today’s runtime; I break large work into units with natural channel operations or function calls, then measure again.

The event channel affects the experiment, of course. Observing a scheduler changes timing by adding synchronization. I keep messages outside the inner loop and compare against an uninstrumented wall-clock run. The trace explains ordering; it does not provide cycle-accurate truth.

These results belong to the May 2011 snapshot and my machine. They are not a specification of future Go scheduling, and I am deliberately avoiding an internal diagram that will expire before the ink dries. The practical findings are modest: concurrency is not parallelism, blocking and CPU work stress different paths, and processor settings should follow measurements. Schedulers are quite willing to show their work, but only if the experiment asks a specific question.

I ran two independent CPU loops in goroutines and expected my dual-core machine to halve the time. It did not. Expectations remain the cheapest profiler available and the least accurate.

This runtime defaults to limited processor use. Setting GOMAXPROCS=2 for the experiment allowed Go work to execute on two operating-system threads:

$ time GOMAXPROCS=1 ./spin
real  0m3.84s

$ time GOMAXPROCS=2 ./spin
real  0m2.17s

The result is not exactly half. Both loops touch memory, the scheduler has work, and the rest of the machine has declined to disappear for my benchmark.

Goroutines and parallel execution are separate ideas. Thousands of goroutines can make a concurrent design clear while one thread runs them. Increasing the thread allowance can provide parallelism when there is independent CPU work and hardware to execute it.

The early scheduler also has rough edges. A tight loop with no calls may delay other goroutines more than I expect from mature thread schedulers. I add real synchronization rather than relying on hopeful fairness, and I test the weekly runtime I intend to use.

For I/O-heavy programs, two processor threads may change little. For CPU loops, it can help until memory bandwidth or coordination dominates. The variable is a knob, not a turbo button. Turbo buttons at least had a satisfying light.

I rewrote a small Python log processor as three stages: read lines, parse records, and aggregate counters. The Python version used queues and worker threads. The Go version uses channels, mostly because I wanted to discover where they stop being cute.

The parser looks like this in the weekly snapshot I have installed:

func parse(in <-chan string, out chan<- Record) {
    for line := range in {
        r, ok := parseLine(line)
        if ok {
            out <- r
        }
    }
    close(out)
}

Directional channel types document who sends and who receives. They also let the compiler reject an accidental send in a consumer. The reader closes its output, the parser drains that input and closes the next channel, and the aggregator stops when its input is closed.

An unbuffered channel forces a rendezvous. If aggregation falls behind, parsing blocks, then reading blocks. That is back pressure without a separate queue-length policy. For this tool it is desirable because there is no value in reading a gigabyte ahead merely to admire it in memory.

A buffered channel changes the timing, not the ownership:

records := make(chan Record, 64)

Now the parser can get 64 records ahead. I tried capacities from zero to 4096. Small buffers smoothed disk and parser timing; large ones mostly increased memory use. The best number depended on record size and machine, so naturally 64 is now a sacred constant until the next measurement.

Closing requires discipline. The sender should close a channel when no more values will arrive. A receiver closing it is guessing about other senders and can provoke a panic. Multiple producers therefore need one coordinator to wait for them and close the shared output.

The current implementation is experimental, as are some channel syntax and library details around it. This is not a claim that channels beat every queue or lock. A shared read-mostly table may be clearer behind a mutex. But for a pipeline, channels put flow control, synchronization, and the handoff of values in one visible operation. I removed two condition variables and, more importantly, stopped having to prove that I signaled both of them on every path.