I wanted to rename a field across a pile of Go files. My first impulse was a regular expression. My second, considerably healthier impulse was to remember that Go ships with a parser.

Source code is text, but edits usually concern syntax. The go/parser package reads source into nodes from go/ast. An identifier is an *ast.Ident; a function declaration is an *ast.FuncDecl; an expression has structure beyond where its parentheses happened to land. Once the source is a tree, a tool can ask precise questions instead of hoping a comment does not look executable.

Here is a tiny program that lists function names:

package main

import (
	"fmt"
	"go/ast"
	"go/parser"
	"go/token"
)

func main() {
	fset := token.NewFileSet()
	f, err := parser.ParseFile(fset, "sample.go", nil, 0)
	if err != nil {
		panic(err)
	}
	ast.Inspect(f, func(n ast.Node) bool {
		if fn, ok := n.(*ast.FuncDecl); ok {
			fmt.Println(fn.Name.Name)
		}
		return true
	})
}

token.FileSet connects tree positions to files, lines, and columns. It looks like administrative furniture until an error must say exactly where it found something. Then it becomes the only chair in the room.

After changing a tree, go/printer can write Go source in the standard layout. This is the important lesson of gofmt: parse syntax, then print it consistently. It is not lining up text with an especially ambitious set of regular expressions.

Comments make transformations subtler. Parsing with parser.ParseComments retains comment groups, but moving nodes while preserving the intended comment association still requires care. Imports are another edge. A rename may change whether an import is used, and blindly printing the tree does not make semantic mistakes disappear.

There is also a limit to what the AST alone knows. The syntax tree can tell me that x.Name is a selector expression. It cannot, by itself, tell me which package or type supplied Name. That needs type information. For a local mechanical transformation with an unambiguous syntactic shape, the AST is often enough. For a semantic rename, it may not be.

My field change ended up being a dozen lines around ast.Inspect, followed by formatting and a review of the diff. That was more code than a one-line substitution and much less code than repairing a one-line substitution after it renamed examples, comments, and an unrelated field with the same spelling.

I used to think gofmt was mainly a successful ceasefire in the whitespace wars. It also shows the value of shipping language tools as ordinary libraries: the compiler is not the only program allowed to understand Go source.

I appreciate beautiful, readable code. And if someone were to ask me for an example of beautiful code, I’ve always had the answer ready: the linked list implementation in the Linux kernel.

The code is gorgeous in its simplicity, clarity, and amazing flexibility. If there’s ever a museum for code, this belongs there. It is a masterpiece of the craft.

I was just telling a friend about it while we talked about beautiful code and he found this piece that I share here: Linux Kernel Linked List Explained.

One request in a test server occasionally took much longer than its neighbours. Rather than convict the collector from one latency graph, I recorded request times and the runtime’s pause samples on the same workload.

The current collector stops application work while it traces the reachable heap. runtime.ReadMemStats exposes NumGC, PauseTotalNs, and a ring of recent PauseNs values. I sampled before and after a fixed five-minute run, then compared only collections added during that window. Mixing in old pauses made short tests look wonderfully stable or mysteriously awful depending on what had run first.

That pause is easy to describe and harder to judge. A short command-line tool may never care. A busy service with latency requirements can care very much, even if its average throughput looks fine. Averages are talented at concealing the request somebody was actually waiting for.

My first table kept four numbers for each run: requests per second, the 50th and 99th percentile request latency, collection count, and the largest observed GC pause. Average latency alone hid the event I cared about.

The naive handler allocated a buffer for every item:

func checksum(items [][]byte) uint32 {
	var sum uint32
	for _, item := range items {
		buf := make([]byte, len(item))
		copy(buf, item)
		for _, b := range buf {
			sum += uint32(b)
		}
	}
	return sum
}

The copy served no purpose. Removing it reduced collections and improved the long tail. This was not clever tuning; I had simply stopped manufacturing garbage.

The collector’s job depends on both allocated volume and live data. Producing temporary objects quickly causes collections to happen more often. Retaining a large graph gives each marking pass more reachable memory to inspect. These are different problems: one concerns allocation rate, the other heap size and object reachability.

After that fix I tried GOGC=50, the default 100, and 200, restarting the process for each run. Lowering it collected more often with a smaller heap. Raising it collected less often but used more memory. On this service, 200 helped the tail a little and increased memory enough that I kept the default. That is a result, not a disappointment.

Manual runtime.GC() calls were useful to prove that the latency recorder could see a pause. They did not belong in the handler. Forcing collection chooses when to pay and can easily turn an occasional pause into a dependable stream of them.

The useful knob came last. First came a repeatable load, pause deltas from the same time window, and removal of an accidental allocation. Otherwise GOGC is just a dial with excellent opportunities for superstition.

The recently approved C++11 standard brings a lot of welcome changes to C++ that modernize the language a little bit. Among the many changes, we find that containers have received some special love.

Initialization

C++ was long behind modern languages when it came to initializing containers. While you could do

[cpp]int a[] = {1, 2, 3};[/cpp]

for simple arrays, things tended to get more verbose for more complex containers:

[cpp]
vector v;
v.push_back(“One”);
v.push_back(“Two”);
v.push_back(“Three”);
[/cpp]

C++11 has introduced an easier, simpler way to initialize this:

[cpp]
vector v = {“One”, “Two”, “Three”};
[/cpp]

The effects of the changes are even better for things like maps, which could get cumbersome quickly:

[cpp]
map<string, vector > m;
vector v1;
v1.push_back(“A”);
v1.push_back(“B”);
v1.push_back(“C”);

vector v2;
v2.push_back(“A”);
v2.push_back(“B”);
v2.push_back(“C”);

m[“One”] = v1;
m[“Two”] = v2;
[/cpp]

This can now be expressed as:

[cpp]
map<string, vector> m = One,

                             {"Two", {"Z", "Y", "X"}}};

[/cpp]

Much simpler and in line with most modern languages. As an aside, there’s another change in C++11 that would be easy to miss in the code above. The declaration

[cpp]map<string, vector> m;[/cpp]

was illegal until now due to » always being evaluated to the right-shift operator; a space would always be required, like

[cpp]map<string, vector > m[/cpp]

No longer the case.

Iterating

Iterating through containers was also inconvenient. Iterating the simple vector v above:

[cpp]
for (vector::iterator i = v.begin();

 i != v.end(); i++)
cout << i << endl;[/cpp]

Modern languages have long had some foreach equivalent that allowed us easier ways to iterate through these structures without having to explicitly worry about iterators types. C++11 is finally catching up:

[cpp]
for (string s : v)

cout << s << endl;

[/cpp]

As well, C++11 brings in a new keyword, auto, that will evaluate to a type in compile-type. So instead of

[cpp]
for (map<string, vector >::iterator i = m.begin();

 i != m.end(); i++) {

[/cpp]

we can now write

[cpp]
for (auto i = m.begin(); i != m.end(); i++) {
[/cpp]

and auto will evaluate to map<string, vector>::iterator.

Combining these changes, we move from the horrendous

[cpp]
for (map<string, vector >::iterator i = m.begin();

 i != m.end(); i++)
for (vector<string>::iterator j = i->second.begin();
     j != i->second.end(); j++)
    cout << i->first << ': ' << *j << endl;

[/cpp]

to the much simpler

[cpp]
for (auto i : m)

for (auto j : i.second)
    cout << i.first << ': ' << j << endl;

[/cpp]

Not bad.

C++11 support varies a lot from compiler to compiler, but all of the changes above are already supported in the latest versions of GCC, LLVM, and MSVC compilers.

One benchmark had a strange cliff: adding an innocent-looking local array made a short recursive walk much slower. I suspected cache effects first. The CPU profile pointed at the runtime’s stack-growth path instead.

A goroutine begins with a small stack. Before a function uses its frame, generated code checks whether enough stack remains. If not, the runtime grows the stack by adding another segment and resumes the call. A special stack-splitting path does the unpleasant bookkeeping so ordinary Go code can continue pretending stacks are infinite, at least until memory becomes considerably less theoretical.

The benchmark was more useful as a depth sweep than as one headline number. I ran the same walk at depths 8, 16, 32, 64, and 128, then repeated it with a slightly larger local frame. Most points scaled smoothly. One narrow range jumped when calls repeatedly reached the end of a segment.

The segmented arrangement makes the initial cost of a goroutine small, but it is not magic. Each goroutine still has scheduler state and a stack segment. Every additional segment consumes memory. Deep recursion can grow a stack dramatically, and keeping the goroutine alive keeps its stack alive as well.

That narrow range is the interesting bit. If a function arrives with almost, but not quite, enough room, it grows the stack; returning can release the segment, and the next call grows it again. This “hot split” makes a particular call depth look much worse than its neighbours. A single benchmark depth can therefore blame the algorithm for standing on an implementation seam.

I checked the profile at the slow depth, then moved the benchmark one call shallower and deeper. The runtime samples faded at both neighbours. That was better evidence than merely noticing that recursion was involved. Changing the benchmark’s local array size moved the cliff too.

I did not contort the application to dodge one segment boundary. The runtime implementation will change, and ordinary code should not depend on today’s segment size. I did keep the depth sweep: it tells me whether a future optimization improves the work or merely moves the cliff.

Small stacks still make many goroutines practical. They also leave fingerprints in microbenchmarks. When one input size falls off a ledge, profile it and test both sides before rewriting the loop.