Go

The new goinstall tool removes some Makefile work from small Go packages. Given an import path on a host it recognizes, it fetches the source, builds dependencies, and installs the package in the Go tree under GOROOT. This predates the workspace layout I keep wishing into existence.

For a package hosted on GitHub I can run:

$ goinstall github.com/rselbach/lines.git

GitHub is one of the repository hosts this version knows how to fetch, provided the repository name includes the .git suffix it expects. The import path is also the package identity: source lands below $GOROOT/src/pkg/github.com/rselbach/lines.git, and the installed archive goes into the corresponding package tree below $GOROOT/pkg. There is no GOPATH here yet.

This is early machinery. Repository conventions, supported fetch methods, command flags, and even package locations are changing between weekly snapshots. I would not encode today’s behavior into a company-wide build system without pinning the toolchain and keeping the escape hatch labeled.

It also does not solve version selection. Installing the latest source from a remote location is convenient for experiments and rather less comforting for reproducing a build six months later. For serious work I still record the exact revisions of the compiler and imported code.

Even with those caveats, goinstall saves the old fetching, directory arranging, dependency chasing, and archive copying dance. The machine can have that ritual; it will invent another soon enough.

I rewrote a small Python log processor as three stages: read lines, parse records, and aggregate counters. The Python version used queues and worker threads. The Go version uses channels, mostly because I wanted to discover where they stop being cute.

The parser looks like this in the weekly snapshot I have installed:

func parse(in <-chan string, out chan<- Record) {
    for line := range in {
        r, ok := parseLine(line)
        if ok {
            out <- r
        }
    }
    close(out)
}

Directional channel types document who sends and who receives. They also let the compiler reject an accidental send in a consumer. The reader closes its output, the parser drains that input and closes the next channel, and the aggregator stops when its input is closed.

An unbuffered channel forces a rendezvous. If aggregation falls behind, parsing blocks, then reading blocks. That is back pressure without a separate queue-length policy. For this tool it is desirable because there is no value in reading a gigabyte ahead merely to admire it in memory.

A buffered channel changes the timing, not the ownership:

records := make(chan Record, 64)

Now the parser can get 64 records ahead. I tried capacities from zero to 4096. Small buffers smoothed disk and parser timing; large ones mostly increased memory use. The best number depended on record size and machine, so naturally 64 is now a sacred constant until the next measurement.

Closing requires discipline. The sender should close a channel when no more values will arrive. A receiver closing it is guessing about other senders and can provoke a panic. Multiple producers therefore need one coordinator to wait for them and close the shared output.

The current implementation is experimental, as are some channel syntax and library details around it. This is not a claim that channels beat every queue or lock. A shared read-mostly table may be clearer behind a mutex. But for a pipeline, channels put flow control, synchronization, and the handoff of values in one visible operation. I removed two condition variables and, more importantly, stopped having to prove that I signaled both of them on every path.

I usually notice a missing cleanup path during review, which is a dignified way of saying after I wrote it.

Go’s defer lets me put cleanup beside acquisition:

f, err := os.Open(name, os.O_RDONLY, 0)
if err != nil {
    return err
}
defer f.Close()

The call runs when the surrounding function returns, including an early return. Several deferred calls run in last-in, first-out order. That order fits nested resources rather nicely.

Arguments are evaluated when the defer statement executes, not later. This matters in loops and when variables change:

for i := 0; i < 3; i++ {
    defer fmt.Println(i)
}

The output is 2, 1, 0. It is a stack of already-prepared calls, not three promises to inspect i at the end.

I would not put defer in a hot inner loop without measuring it. This compiler and runtime are young, and convenience has a cost even when the syntax hides the invoice. A helper function can give each iteration its own cleanup boundary.

The larger win is boring correctness. Files close on every return, locks can be paired with unlocks, and the cleanup is visible before the function accumulates three branches. C’s goto out remains perfectly respectable, but defer makes the common case harder to omit. This note matches my current weekly snapshot; the surrounding library calls are still moving.

My reflex when seeing go f() was to translate it into “start a thread.” That translation is convenient and wrong enough to cause bad designs.

A goroutine is a language-level activity scheduled by the Go runtime. Its initial stack is small and can grow, so creating thousands is practical in cases where creating thousands of pthreads would be a conversation with both the kernel and the person carrying the pager.

I tested this with a deliberately dull program:

package main

import "fmt"

func wait(done chan bool) {
    done <- true
}

func main() {
    done := make(chan bool)
    for i := 0; i < 10000; i++ {
        go wait(done)
    }
    for i := 0; i < 10000; i++ {
        <-done
    }
    fmt.Println("finished")
}

The channel is doing two jobs: carrying a value and synchronizing sender with receiver. With an unbuffered channel, the send waits for a receive. The program does not need a mutex around a shared completion counter.

That does not make races impossible. If all those goroutines update the same map, I still have a race. “Do not communicate by sharing memory” is good direction, not a force field.

The early runtime is another caveat. On the snapshot I used, scheduling and stack management are active work. Timing results vary, and a tight goroutine that never calls into the runtime can be an unfriendly neighbor. I would not build a latency promise from this experiment.

The cheap creation cost also does not excuse leaving goroutines blocked forever. Their stacks are small, not imaginary, and their work still needs a defined end.

What I do like is the change in scale. In C, a thread is sufficiently costly and awkward that I first ask how to avoid one. In Go, I can model each independent operation directly, then decide where synchronization belongs. The runtime may multiplex many goroutines over fewer operating-system threads; that implementation is precisely why “goroutine equals pthread” is a poor mental model.

This is not magic parallelism either. More runnable work does not create more processors, and today’s defaults may use only one. It does make concurrency cheap enough to express before I optimize it. For server and pipeline code, that changes the first sketch substantially.

Qt taught me to appreciate explicit contracts. It also taught me that an object system can acquire enough machinery to qualify for municipal government.

Go’s interfaces are much smaller. A type satisfies an interface by having the required methods; it does not announce the relationship in its declaration. This took a moment to trust because I am used to inheritance lists serving as signposts.

Here is the complete contract for something that can write bytes in my current snapshot:

type Writer interface {
    Write(p []byte) (n int, err os.Error)
}

A file can satisfy it. So can a network connection or a tiny test buffer. The producer only needs to accept a Writer:

func emit(w Writer, p []byte) os.Error {
    _, err := w.Write(p)
    return err
}

There is no implements Writer marker on those concrete types. The advantage is that I can define an interface around behavior already provided by a package. The package author did not need to predict my abstraction.

The cost is discoverability. Looking at a type alone does not list every interface it happens to satisfy, and a method signature changed by one type is a different method. Compiler diagnostics are doing more of the navigation work than they do in my C++ code.

I tried the idea on a log collector. The original Python version accepted anything with a write method because Python accepts almost anything until it does not. The Go version gets similar flexibility, but verifies the method set when values are connected. That is a useful middle ground.

Interfaces also contain a pair of things internally: a dynamic type and a dynamic value. An interface value can therefore be non-empty even when the pointer stored inside it is nil. I managed to manufacture that surprise within an hour, which must be some kind of efficiency record.

These details reflect a weekly build, not a stable release. Method-set and library conventions may still shift. Even so, structural satisfaction feels important. It encourages narrow contracts at the point of use rather than grand interface families designed before the program has done anything useful.