Programming

I blamed the garbage collector for a slow parser this week. It was a satisfying theory because it required no evidence and made the runtime somebody else’s problem. Then I collected a CPU profile.

The programme only needed a profile file and a bounded run:

f, err := os.Create("cpu.prof")
if err != nil {
	return err
}
defer f.Close()

pprof.StartCPUProfile(f)
defer pprof.StopCPUProfile()

parseAll(files)

The exact profiling API and command-line tools are still moving with the snapshots, but the important method is stable enough: measure a representative workload, inspect where samples accumulate, change one thing, and measure again.

My surprise was a tiny character-class helper near the top of the profile. It performed a linear search through a string for every input byte. The allocator appeared, but it was not remotely the villain I had prepared a speech about. Replacing the repeated search with a direct table lookup changed the run far more than any speculative memory trick.

A CPU profile is sampling, not a complete diary. At intervals, the runtime records where execution is spending time. Enough samples reveal hot stacks without instrumenting every function call. That means very short runs can mislead, and blocked time is not the same as CPU time. A programme waiting on disk may feel slow while producing an uninteresting CPU profile.

Heap information answers another question: what remains live or what has been allocated, depending on the profile and tool. I have learned not to blur those together. A high allocation rate can increase collection work even if the live heap stays modest. A large live heap can be perfectly reasonable if the programme genuinely needs the data.

I prefer profiles to stopwatch calls scattered through the source. The profile keeps context: not just how long a region took, but which call paths led there. A small benchmark is still useful after the profile identifies the narrow piece worth isolating.

The caveat is representativeness. Profiling a toy input optimizes the toy. Debug output, cold caches, and a different machine can also distort results. I record the snapshot version and input beside the profile because both are part of the result.

The parser is faster now. More importantly, the garbage collector has been acquitted for lack of evidence, which is the closest a runtime component gets to an apology.

I reached for reflection after writing the same diagnostic print twice. The obvious first experiment was to inspect one value:

func describe(x interface{}) {
	v := reflect.ValueOf(x)
	fmt.Println(v.Type(), v.Kind())
}

What surprised me was that type and kind answer different questions. A named integer type has its own type, but its kind is still integer. The type preserves identity; the kind tells reflective code which family of operations is available.

An interface value passed to describe carries a dynamic type and data. reflect.ValueOf exposes a view of that pair. From there, operations are checked at run time. Asking for an integer from a string value is not a clever conversion; it is a panic with good timing.

Settable values were the second lesson. Reflecting on a copied value lets me inspect it, not rewrite the caller’s variable. To change the original I must pass a pointer, obtain the pointed-to value, and ensure it is settable. Reflection follows the same addressability rules as ordinary Go, only with more opportunities to discover mistakes while running.

I prefer ordinary interfaces whenever the required behaviour can be named. They give compile-time checking and explain intent better than a tour through Kind. Reflection earns its place when the types themselves are the input, as in formatting or decoding.

The caveat is that a generic-looking helper can become a private, badly documented type system. My diagnostic printer stayed reflective. The business logic did not. One magician in the programme is plenty.

I wrapped a C function that opens a file. The call itself was easy; deciding where a C string stopped and a Go os.Error began took the rest of the afternoon.

The first useful program was deliberately dull:

package main

/*
#include <fcntl.h>
#include <stdlib.h>

static int open_read_only(const char *name) {
	return open(name, O_RDONLY);
}
*/
import "C"

import (
	"os"
	"unsafe"
)

func open(name string) (int, os.Error) {
	cname := C.CString(name)
	defer C.free(unsafe.Pointer(cname))

	fd, err := C.open_read_only(cname)
	if fd < 0 {
		return -1, err
	}
	return int(fd), nil
}

The comment immediately before import "C" is cgo’s preamble, not decoration. The little C wrapper is necessary because open is variadic and cgo cannot call it directly. C.CString allocates and copies, so the matching C.free stays next to it. The Go string may contain a zero byte, too; a C API will see that as the end of the name. My wrapper rejects such names before this call rather than quietly opening something else.

In a two-result cgo call, cgo captures C’s errno as the os.Error used by this Go snapshot. I test the function’s documented failure result first; a stale nonzero errno after a successful call is not an error. This also keeps C’s zero-terminated error text on the C side of the wrapper rather than exposing a borrowed character pointer to callers.

That translation belongs right here. Returning a bare integer status would make every caller know C’s failure sentinel, when errno is meaningful, and how long the message storage lives. One wrapper is enough bureaucracy.

There are wrinkles beyond this toy. Some functions return errors directly instead of using errno; some return allocated strings that need a library-specific free function; and a successful result can still require cleanup. The C header, not habit, decides each case.

There are practical caveats. The build now needs a C compiler and the library’s headers and linker flags. Types that merely look alike are not automatically interchangeable. Memory allocated by C must be released according to C’s rules, and Go pointers should not be tucked away by C code for later use.

The wrapper is dull now: Go string in, file descriptor or os.Error out. Dull borders are much easier to debug.

I wanted to test a function that wrote a report without letting the test create files. My first instinct, trained by larger object systems, was to invent a report hierarchy. Go let me stop much earlier.

The function only needs one operation:

type writer interface {
	Write([]byte) (int, os.Error)
}

func report(w writer, name string) os.Error {
	_, err := w.Write([]byte("hello, " + name + "\n"))
	return err
}

A file can satisfy that interface. So can a network connection or a little buffer in a test. None of those types has to announce that it implements writer; the method set is enough.

That last detail was my surprise. I am used to a type naming its interfaces at the type declaration. Here the relationship can be discovered where it is needed. The package containing report can define the narrow interface, even when the concrete type belongs to another package and cannot be edited.

The mechanism is structural. An interface describes a set of methods. A value is assignable to it when its type has those methods with the right signatures. The compiler checks the assignment, so this is not hopeful late binding. Calls through the interface still carry dynamic type information, but the agreement itself needs no explicit declaration.

Pointer methods add one wrinkle. If a method has a pointer receiver, a pointer to the type has that method; the plain value does not. That distinction matters when a method modifies state, and it explains several otherwise puzzling assignment errors.

I now prefer interfaces declared by the code that consumes them, and I keep them as small as the operation permits. A one-method interface is not a toy. It is often a precise seam between a useful function and everything it does not need to know.

The caveat is that an interface should represent behaviour, not merely hide every concrete type on principle. Returning an interface too early can discard useful methods and make code harder to follow. If there is only one implementation and no boundary to protect, the concrete type is frequently clearer.

Here, one method bought a test with no temporary file and no elaborate framework. I will take that trade.

This weekend I took the family to celebrate Father’s Day away from town. We went around getting to know parts of the province we live in and never been to.

Since English isn’t my first language, I have a limited vocabulary, which leaves me at a disadvantage against my English-speaking friends. I can handle the “hangman” part of the game where I have to guess the word my friends come up with; but when it becomes “Scrabble” and I’ve got to form words using only a given set of letters and still make them difficult enough that a native English speaker will have problems figuring them out, then it’s tough.

An itch that needed some scratching. Enter Anagramizer.

When I woke up this morning, I decided to write a little program to help me. You call it cheating, I call it having a bit of nerd fun.

Being that I’m currently in love with Go, I decided to write in that language and it was really easy and quick to do it. It took me about half an hour to write the program that did what I needed. But then…

I succumbed to the temptation and started adding bells and whistles. Admittedly it was mostly for my own amusement and trying stuff in Go, but by the time we were leaving for lunch, the program had more options than the KDE audio volume utility (see what I did just there?)

I decided to make it available to anyone who wants to play with it. It served its purpose of entertaining me for about half a day 🙂

It’s now available on Github and released under a BSD licence.