Go

I wrote a small fan-out test and expected twice as many goroutines to mean twice as much parallel work. The result barely moved. That sent me away from the programme and into the runtime scheduler.

The test looked innocent:

for i := 0; i < 4; i++ {
	go crunch(work[i])
}

Goroutines are cheap concurrent activities, but concurrency does not itself promise simultaneous execution on every processor. In the current runtime, scheduling is still comparatively simple, and processor use depends on runtime settings, operating-system threads, blocking calls, and where the scheduler gets a chance to run.

The surprise was how many layers my little loop depended upon. A goroutine begins with a small stack and runtime bookkeeping. Runnable goroutines wait for the scheduler. The scheduler places work onto operating-system threads, and the kernel finally decides when those threads run. Linux 3.0 may schedule the threads beautifully, but it cannot create parallel Go execution the runtime never offered it.

This scheduler predates the more ambitious designs being discussed for later Go. It should not be described using machinery that does not exist yet. There is no reason to draw a modern diagram over an old runtime and then congratulate the diagram for explaining it.

Blocking is the interesting case. If a goroutine enters a system call, the runtime may need another thread so runnable Go code can continue. Channel operations and synchronization also move goroutines between runnable and waiting states. The scheduler therefore coordinates cheap user-level goroutines with more expensive kernel threads rather than mapping one permanently to the other.

I prefer to write synchronization for correctness first and treat parallel speedup as a measured property. A channel can make ownership clear even on one processor. If CPU parallelism matters, I set the runtime’s processor allowance explicitly, run a long enough workload, and compare against a serial version.

The caveat is that scheduler behaviour is an implementation detail in active development. A trick that coaxes today’s scheduler may become useless or harmful after a snapshot. Explicit communication and coarse useful work are safer investments than depending on an accidental scheduling order.

My test eventually did improve, but only after I made each unit of work large enough to outweigh coordination and enabled the processors I intended to use. Four goroutines were not four tiny employees waiting inside the computer. This was disappointing for payroll, but helpful for the benchmark.

I blamed the garbage collector for a slow parser this week. It was a satisfying theory because it required no evidence and made the runtime somebody else’s problem. Then I collected a CPU profile.

The programme only needed a profile file and a bounded run:

f, err := os.Create("cpu.prof")
if err != nil {
	return err
}
defer f.Close()

pprof.StartCPUProfile(f)
defer pprof.StopCPUProfile()

parseAll(files)

The exact profiling API and command-line tools are still moving with the snapshots, but the important method is stable enough: measure a representative workload, inspect where samples accumulate, change one thing, and measure again.

My surprise was a tiny character-class helper near the top of the profile. It performed a linear search through a string for every input byte. The allocator appeared, but it was not remotely the villain I had prepared a speech about. Replacing the repeated search with a direct table lookup changed the run far more than any speculative memory trick.

A CPU profile is sampling, not a complete diary. At intervals, the runtime records where execution is spending time. Enough samples reveal hot stacks without instrumenting every function call. That means very short runs can mislead, and blocked time is not the same as CPU time. A programme waiting on disk may feel slow while producing an uninteresting CPU profile.

Heap information answers another question: what remains live or what has been allocated, depending on the profile and tool. I have learned not to blur those together. A high allocation rate can increase collection work even if the live heap stays modest. A large live heap can be perfectly reasonable if the programme genuinely needs the data.

I prefer profiles to stopwatch calls scattered through the source. The profile keeps context: not just how long a region took, but which call paths led there. A small benchmark is still useful after the profile identifies the narrow piece worth isolating.

The caveat is representativeness. Profiling a toy input optimizes the toy. Debug output, cold caches, and a different machine can also distort results. I record the snapshot version and input beside the profile because both are part of the result.

The parser is faster now. More importantly, the garbage collector has been acquitted for lack of evidence, which is the closest a runtime component gets to an apology.

I reached for reflection after writing the same diagnostic print twice. The obvious first experiment was to inspect one value:

func describe(x interface{}) {
	v := reflect.ValueOf(x)
	fmt.Println(v.Type(), v.Kind())
}

What surprised me was that type and kind answer different questions. A named integer type has its own type, but its kind is still integer. The type preserves identity; the kind tells reflective code which family of operations is available.

An interface value passed to describe carries a dynamic type and data. reflect.ValueOf exposes a view of that pair. From there, operations are checked at run time. Asking for an integer from a string value is not a clever conversion; it is a panic with good timing.

Settable values were the second lesson. Reflecting on a copied value lets me inspect it, not rewrite the caller’s variable. To change the original I must pass a pointer, obtain the pointed-to value, and ensure it is settable. Reflection follows the same addressability rules as ordinary Go, only with more opportunities to discover mistakes while running.

I prefer ordinary interfaces whenever the required behaviour can be named. They give compile-time checking and explain intent better than a tour through Kind. Reflection earns its place when the types themselves are the input, as in formatting or decoding.

The caveat is that a generic-looking helper can become a private, badly documented type system. My diagnostic printer stayed reflective. The business logic did not. One magician in the programme is plenty.

I wrapped a C function that opens a file. The call itself was easy; deciding where a C string stopped and a Go os.Error began took the rest of the afternoon.

The first useful program was deliberately dull:

package main

/*
#include <fcntl.h>
#include <stdlib.h>

static int open_read_only(const char *name) {
	return open(name, O_RDONLY);
}
*/
import "C"

import (
	"os"
	"unsafe"
)

func open(name string) (int, os.Error) {
	cname := C.CString(name)
	defer C.free(unsafe.Pointer(cname))

	fd, err := C.open_read_only(cname)
	if fd < 0 {
		return -1, err
	}
	return int(fd), nil
}

The comment immediately before import "C" is cgo’s preamble, not decoration. The little C wrapper is necessary because open is variadic and cgo cannot call it directly. C.CString allocates and copies, so the matching C.free stays next to it. The Go string may contain a zero byte, too; a C API will see that as the end of the name. My wrapper rejects such names before this call rather than quietly opening something else.

In a two-result cgo call, cgo captures C’s errno as the os.Error used by this Go snapshot. I test the function’s documented failure result first; a stale nonzero errno after a successful call is not an error. This also keeps C’s zero-terminated error text on the C side of the wrapper rather than exposing a borrowed character pointer to callers.

That translation belongs right here. Returning a bare integer status would make every caller know C’s failure sentinel, when errno is meaningful, and how long the message storage lives. One wrapper is enough bureaucracy.

There are wrinkles beyond this toy. Some functions return errors directly instead of using errno; some return allocated strings that need a library-specific free function; and a successful result can still require cleanup. The C header, not habit, decides each case.

There are practical caveats. The build now needs a C compiler and the library’s headers and linker flags. Types that merely look alike are not automatically interchangeable. Memory allocated by C must be released according to C’s rules, and Go pointers should not be tucked away by C code for later use.

The wrapper is dull now: Go string in, file descriptor or os.Error out. Dull borders are much easier to debug.

I wanted to test a function that wrote a report without letting the test create files. My first instinct, trained by larger object systems, was to invent a report hierarchy. Go let me stop much earlier.

The function only needs one operation:

type writer interface {
	Write([]byte) (int, os.Error)
}

func report(w writer, name string) os.Error {
	_, err := w.Write([]byte("hello, " + name + "\n"))
	return err
}

A file can satisfy that interface. So can a network connection or a little buffer in a test. None of those types has to announce that it implements writer; the method set is enough.

That last detail was my surprise. I am used to a type naming its interfaces at the type declaration. Here the relationship can be discovered where it is needed. The package containing report can define the narrow interface, even when the concrete type belongs to another package and cannot be edited.

The mechanism is structural. An interface describes a set of methods. A value is assignable to it when its type has those methods with the right signatures. The compiler checks the assignment, so this is not hopeful late binding. Calls through the interface still carry dynamic type information, but the agreement itself needs no explicit declaration.

Pointer methods add one wrinkle. If a method has a pointer receiver, a pointer to the type has that method; the plain value does not. That distinction matters when a method modifies state, and it explains several otherwise puzzling assignment errors.

I now prefer interfaces declared by the code that consumes them, and I keep them as small as the operation permits. A one-method interface is not a toy. It is often a precise seam between a useful function and everything it does not need to know.

The caveat is that an interface should represent behaviour, not merely hide every concrete type on principle. Returning an interface too early can discard useful methods and make code harder to follow. If there is only one implementation and no boundary to protect, the concrete type is frequently clearer.

Here, one method bought a test with no temporary file and no elaborate framework. I will take that trade.