Blog

I had a polling client that became steadily slower. The server was fine, DNS was fine, and my diagnostic method of staring at it was producing limited returns.

The client checked only the status:

resp, err := http.Get(url)
if err != nil {
	return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
	return fmt.Errorf("status: %s", resp.Status)
}

Closing the body is necessary, but for persistent HTTP connections it is not always sufficient. If I do not read the response body to EOF, the transport may be unable to reuse that TCP connection. My server returned a small explanatory body for an error status; I ignored it and paid for a new connection on the next poll.

For responses whose contents I deliberately discard, I now do this:

defer resp.Body.Close()
if _, err := io.Copy(ioutil.Discard, resp.Body); err != nil {
	return err
}

The transport owns a pool of persistent connections keyed by destination. Reading through EOF lets its body wrapper observe completion and return the connection to the idle pool. Close still matters because it releases resources on every exit path.

This is not permission to drain an unbounded hostile response. If I only need to preserve reuse for a known-small error body, I put a limit around it and accept that an oversized body may force the connection closed:

r := io.LimitReader(resp.Body, 4<<10)
_, err = io.Copy(ioutil.Discard, r)

The same ownership rule applies on success. The caller that receives *http.Response generally owns Body and must close it. A helper that consumes the body should close it itself and return decoded data instead. Splitting those responsibilities is how bodies remain open.

I verified reuse by wrapping Transport.Dial with a counter. Ten requests to the same server produced ten dials before the fix and one after it. That is a crude instrument, but better than inferring connection behavior from elapsed time. A server can close persistent connections itself, so one dial is not a universal expected value.

There is a related server-side rule: a handler should not assume unread request bytes are harmless. net/http often manages request bodies for handlers, but large or malformed input still needs explicit size limits. Connection reuse is valuable; reading arbitrary quantities merely to obtain it is not.

After draining the tiny responses, repeated requests settled onto the same few connections instead of continually dialing. It was not an exotic kernel problem. It was one unread paragraph of error text, patiently converting itself into sockets.

I wrote a small validator that accepted interface{} and walked fields marked validate:"required". The happy path worked, so I naturally declared victory before trying a pointer, a nil pointer, or anything that was not a struct.

The first version began with this:

v := reflect.ValueOf(x)
for i := 0; i < v.NumField(); i++ {
	// inspect v.Field(i)
}

Passing &User{} produced reflect: call of reflect.Value.NumField on ptr Value. Recovering the panic would hide the real question: what inputs does this validator promise to accept? I chose structs and non-nil pointers to structs, with ordinary errors for everything else.

func validate(x interface{}) error {
	v := reflect.ValueOf(x)
	if !v.IsValid() {
		return errors.New("cannot validate nil")
	}
	if v.Kind() == reflect.Ptr {
		if v.IsNil() {
			return errors.New("cannot validate nil pointer")
		}
		v = v.Elem()
	}
	if v.Kind() != reflect.Struct {
		return fmt.Errorf("cannot validate %s", v.Kind())
	}

	t := v.Type()
	for i := 0; i < v.NumField(); i++ {
		if t.Field(i).Tag.Get("validate") != "required" {
			continue
		}
		if isZero(v.Field(i)) {
			return fmt.Errorf("%s is required", t.Field(i).Name)
		}
	}
	return nil
}

The harder policy is what “required” means. For this validator it rejects empty strings, numeric zero, false, and nil pointers, maps, slices, and interfaces. That makes false invalid, so a required boolean is usually a design smell; a pointer can represent the distinct states absent, false, and true.

I ignore unexported fields. A general validator should not turn another package’s private representation into public input policy.

Nested structs are not automatically validated. A field gets nested validation only when its tag asks for it; otherwise adding a private detail to a child type could unexpectedly change the parent’s contract. Nil optional pointers remain acceptable, while a required nil pointer fails at its own field path.

Errors report the Go field path as well as the failed rule. A bare “required” is compact but useless once validation descends into two nested structs with similarly named fields.

Reflection removes field-by-field plumbing, but the useful part is still the contract: accepted top-level shapes, the meaning of “required,” whether nesting is opt-in, and useful error paths. Once those choices are explicit, the reflective code can stay small and unsurprising.

I wanted one struct to describe both its JSON name and a tiny validation rule. The obvious declaration was:

type User struct {
	Name string `json:"name" validate:"required"`
}

My first parser split the raw tag on spaces and colons. It worked until quoting became interesting, which took approximately seven minutes. reflect.StructTag already understands the convention:

t := reflect.TypeOf(User{})
f, _ := t.FieldByName("Name")
fmt.Println(f.Tag.Get("json"))
fmt.Println(f.Tag.Get("validate"))

Output:

name
required

Tags are metadata attached to fields in the type. Reflection exposes them; the compiler does not enforce what validate:"required" means. Misspelling the key gives an empty string, indistinguishable from an explicitly empty value when using Get.

Only exported fields are useful to packages such as encoding/json, because reflection cannot generally set unexported fields from another package. I also avoid turning tags into a miniature programming language. Once a rule needs conditions, database access, or cross-field state, ordinary Go is clearer.

One more trap: reflect.TypeOf on a pointer yields a pointer type. To inspect its fields I use t.Elem(). Reflection is precise, but it is not especially interested in guessing what I meant.

On the way to the Moon, the Apollo crew and ground control needed precise information about the position, speed, and acceleration of the stack. Getting that information was a feat of engineering involving very clever solutions that deserve a post of their own, but the easiest piece of the puzzle was the position.

To figure out where exactly the Apollo was at any given moment, the crew used a guidance computer designed by the MIT along with a sextant, not much unlike the ones used for centuries by sea captains.

The CM pilot would pick a specific star (or planet sometimes) that could be placed within view of the sextant optics, he’d then align the sextant viewport with the chosen star, and then tell the guidance computer to calculate the position by telling it which star he had just pointed the sextant to.

The astronauts had a table of stars they could use, each accompanied by an octal code that could be fed to the computer. In this list, along with well-known stars—such as Sirius (15), Rigel (12), and the Sun (46)—were three oddly named stars.

While helping with the development of this system, the crew of Apollo I decided to have a little fun and added references to themselves by changing the names of some stars to parts of their own names spelled backwards:

• Navi (03) for Gus Grisson’s middle name (Ivan)
• Regor (17) for Roger Chaffee
• Dnoces (20) for Edward White II.

In reality, Navi was γ Cassiopeiae, Regor was γ Velorum, and Dnoces was ι Ursae Majoris.

After the crew lost their lives in a tragic fire—the first fallen in the American space manned program—the people at NASA kept those names in all their documentation and literature.

Three Go 1.2 additions fixed three unrelated annoyances in one week. None is glamorous, which is usually a sign that I will use it.

First, go test now measures statement coverage directly:

The cover tool is still distributed separately in go.tools, so I installed it first:

$ go get code.google.com/p/go.tools/cmd/cover

$ go test -cover
PASS
coverage: 71.4% of statements
ok      example/parser   0.012s

For detail I use go test -coverprofile=cover.out, followed by go tool cover -func=cover.out or go tool cover -html=cover.out. Coverage works by rewriting source with counters. It tells me which statements ran, not whether the assertions were intelligent. I can reach 100 percent while testing almost nothing; I have demonstrated this skill repeatedly.

Second, crypto/tls gained better control over protocol and cipher behavior, including TLS 1.2 support. For a client that must reject obsolete protocol versions I can set the minimum explicitly:

tr := &http.Transport{
	TLSClientConfig: &tls.Config{
		MinVersion: tls.VersionTLS11,
	},
}
client := &http.Client{Transport: tr}

I still let the standard library choose cipher suites unless interoperability forces my hand. A handwritten list ages, and a secure-looking list copied from a weblog ages especially well.

Finally, sync/atomic now has swap operations. I used one to replace a numeric generation and retrieve the previous value as a single atomic action:

old := atomic.SwapUint64(&generation, next)
log.Printf("generation %d -> %d", old, next)

The important phrase is “single atomic action.” Loading and then storing separately leaves a window in which another goroutine can intervene. A swap closes that window, but it does not magically protect related fields or establish a larger invariant. For that I still use a mutex.

The default coverage mode counts whether statements execute. When tests exercise code concurrently, -covermode=atomic makes counter updates safe, at additional cost. I do not compare benchmark timings from an instrumented coverage build with normal timings. The inserted counters are deliberately observable work.

Atomics also require aligned, correctly typed storage and a design whose state really fits in one word. My generation counter does. A configuration containing a counter, pointer, and status does not become coherent because each field can be manipulated atomically. Readers could observe a combination that never represented one published configuration.

I keep the atomic operation beside a comment describing that single-word invariant. Otherwise the next maintenance change tends to add a companion field and quietly invalidate the reasoning.

These features share a useful property: they turn homemade approximations into explicit operations. I deleted a coverage script, avoided a custom TLS dialer, and replaced a dubious load/store pair. Upgrades are pleasant when the final diff is mostly subtraction.