Testing on Roberto Selbach

Parallel benchmarks and wandering maps

Thu, 09 Oct 2014 13:10:00 +0000

I tested a locked lookup table with Go 1.3’s parallel benchmark support:

func BenchmarkLookupParallel(b *testing.B) {
	b.RunParallel(func(pb *testing.PB) {
		for pb.Next() {
			_ = table.Lookup("troy")
		}
	})
}

RunParallel creates goroutines and distributes benchmark iterations among them. This exposed lock contention that a serial benchmark politely concealed. It is still synthetic: all workers hammering one key may be less realistic than production, so I use representative key distributions too.

The same upgrade broke a test that compared map traversal output to a fixed string. Go 1.3 randomizes iteration order for small maps, making an assumption that was always invalid fail more reliably.

for k := range m {
	keys = append(keys, k)
}
sort.Strings(keys)

Sorting is correct when output order is part of the result. For equality, I compare contents instead of rendering iteration order.

Random order is not cryptographic randomness, nor should a program depend on getting a different order every pass. It is an implementation defense against accidental ordering assumptions, not a shuffling API.

I approve of the randomization. A runtime that accidentally rewards a bad assumption lets the assumption become an API. My test was not flaky; it was finally honest.

Coverage, TLS, and atomic swaps in Go 1.2

Thu, 09 Jan 2014 15:35:00 +0000

Three Go 1.2 additions fixed three unrelated annoyances in one week. None is glamorous, which is usually a sign that I will use it.

First, go test now measures statement coverage directly:

The cover tool is still distributed separately in go.tools, so I installed it first:

$ go get code.google.com/p/go.tools/cmd/cover

$ go test -cover
PASS
coverage: 71.4% of statements
ok example/parser 0.012s

For detail I use go test -coverprofile=cover.out, followed by go tool cover -func=cover.out or go tool cover -html=cover.out. Coverage works by rewriting source with counters. It tells me which statements ran, not whether the assertions were intelligent. I can reach 100 percent while testing almost nothing; I have demonstrated this skill repeatedly.

Second, crypto/tls gained better control over protocol and cipher behavior, including TLS 1.2 support. For a client that must reject obsolete protocol versions I can set the minimum explicitly:

tr := &http.Transport{
	TLSClientConfig: &tls.Config{
		MinVersion: tls.VersionTLS11,
	},
}
client := &http.Client{Transport: tr}

I still let the standard library choose cipher suites unless interoperability forces my hand. A handwritten list ages, and a secure-looking list copied from a weblog ages especially well.

Finally, sync/atomic now has swap operations. I used one to replace a numeric generation and retrieve the previous value as a single atomic action:

old := atomic.SwapUint64(&generation, next)
log.Printf("generation %d -> %d", old, next)

The important phrase is “single atomic action.” Loading and then storing separately leaves a window in which another goroutine can intervene. A swap closes that window, but it does not magically protect related fields or establish a larger invariant. For that I still use a mutex.

The default coverage mode counts whether statements execute. When tests exercise code concurrently, -covermode=atomic makes counter updates safe, at additional cost. I do not compare benchmark timings from an instrumented coverage build with normal timings. The inserted counters are deliberately observable work.

Atomics also require aligned, correctly typed storage and a design whose state really fits in one word. My generation counter does. A configuration containing a counter, pointer, and status does not become coherent because each field can be manipulated atomically. Readers could observe a combination that never represented one published configuration.

I keep the atomic operation beside a comment describing that single-word invariant. Otherwise the next maintenance change tends to add a companion field and quietly invalidate the reasoning.

These features share a useful property: they turn homemade approximations into explicit operations. I deleted a coverage script, avoided a custom TLS dialer, and replaced a dubious load/store pair. Upgrades are pleasant when the final diff is mostly subtraction.

Testing KDE 3.5 With Real Work

Fri, 04 Nov 2005 19:25:00 +0000

I’ve been testing KDE 3.5’s KHTML changes ahead of the final release. Passing a standards test is good news, but I wanted to know whether the same engine still handled the untidy pages I use for work.

I started with a small directory of local pages: nested floats, a wide preformatted block, a form inside a table, transparent PNGs, and one page with deliberately broken markup. Each isolates a behavior well enough that a rendering change isn’t just “this site looks odd.”

Then I used Konqueror on documentation and project sites with style sheets enabled, disabled, and replaced by a tiny user sheet. Keyboard focus still had to move through links and form controls in a sensible order. Better CSS support isn’t much help if a search field becomes unreachable without a mouse.

One layout did regress. Instead of filing the full page, I removed blocks until only a float, a cleared heading, and six lines of CSS remained. That reduction changed the report from a screenshot of somebody else’s site into a local file that showed the bug every time.

I kept the exact revision and compared the reduced page with the previous build. “Konqueror acted funny” is emotionally accurate and technically ornamental; a twelve-line attachment with a before-and-after result is useful.

KHTML’s shared role raises the stakes. A rendering fix reaches Konqueror and any application embedding the part, while a regression travels just as efficiently. Small local pages make that shared behavior testable without depending on a live site changing underneath the report.

The standards progress in 3.5 is visible, especially on layouts that used to need browser-specific nudges. The useful release behavior isn’t the badge, though; it’s rendering those pages more correctly without breaking forms, focus, and rough old HTML.

My little page collection now lives beside the build scripts. It’s less glamorous than a browser shootout, but much better at explaining the next crooked heading.