sync.Pool is not a cache

Thu, 18 Sep 2014 16:25:00 +0000

I used Go 1.3’s new sync.Pool to reuse temporary buffers in an HTTP formatter. Then I almost used it to cache parsed templates. The first idea reduced garbage; the second would have randomly forgotten application data.

A pool holds temporary values that may be removed at any time. Garbage collection is explicitly allowed to clear it. That makes it suitable for amortizing allocation of scratch objects, not for storing anything whose presence affects correctness.

My buffer pool looks like this:

var buffers = sync.Pool{
	New: func() interface{} {
		return new(bytes.Buffer)
	},
}

func render(v value) []byte {
	b := buffers.Get().(*bytes.Buffer)
	b.Reset()

	writeValue(b, v)
	out := append([]byte(nil), b.Bytes()...)
	if b.Len() < 64<<10 {
		b.Reset()
		buffers.Put(b)
	}
	return out
}

The copy before Put is essential. b.Bytes() aliases the buffer’s storage. Returning that slice and putting the buffer back would allow another goroutine to overwrite the caller’s result. Pool safety does not transfer ownership safety to pooled objects.

The pool synchronizes Get and Put, but an object obtained from it belongs to the caller until returned. Two goroutines must not use the same buffer concurrently. Go 1.3’s bytes.Buffer does not expose its capacity, so I use the rendered length as a simple retention policy: a response that grows a buffer past 64 KiB does not return that buffer to the pool. It is only a proxy for capacity, but it keeps a freshly oversized allocation out of the pool.

In a benchmark of the formatter, pooling reduced allocation volume:

BenchmarkRender-4 500000 4100 ns/op 2304 B/op 12 allocs/op
BenchmarkRenderPool-4 500000 3000 ns/op 768 B/op 5 allocs/op

The output copy accounts for necessary ownership. If the function wrote directly to an io.Writer, even that copy could disappear, probably a better API than ever more pooling.

New should return the expected type consistently. A pool may omit New, in which case Get can return nil and every caller must handle that. I define it when all callers need the same scratch type; this keeps allocation policy together without pretending values are permanent.

Copying a sync.Pool after first use is invalid because its synchronization state and local storage are not value-like application data. I keep pools as package variables or pointer-owned fields initialized once.

I would not add sync.Pool without allocation profiles and a benchmark. It obscures object lifetime and may provide little benefit when allocations are already short and cheap. A cache promises retrieval; a pool merely offers to rummage in a box the collector is free to empty.

Sync on Roberto Selbach

sync.Pool is not a cache