Go

I put sql.Open inside an HTTP handler because the function was named Open and I wanted to open the database. The code worked, which delayed the useful realization that *sql.DB is not one database connection.

sql.Open creates a database handle using a registered driver and its data source name. It may not contact the server immediately. The first operation can therefore be where a bad password, missing server, or malformed configuration finally becomes visible. When startup must verify the database, run a trivial query and handle its error.

The handle manages a pool of underlying connections. It is intended to be long-lived and shared by concurrent goroutines. Opening one per request creates needless pool machinery and closing each one prevents useful connection reuse.

The practical shape is:

db, err := sql.Open("driver-name", dataSource)
if err != nil {
	log.Fatal(err)
}
defer db.Close()

var one int
if err := db.QueryRow("select 1").Scan(&one); err != nil {
	log.Fatal(err)
}

The driver import and data source are database-specific. database/sql deliberately supplies common operations while drivers deal with wire protocols and server peculiarities. This boundary is useful, but it does not make SQL dialects identical or transactions portable by divine intervention.

Rows carry a connection resource until they are exhausted or closed. That makes this pattern important:

rows, err := db.Query("select name from people")
if err != nil {
	return err
}
defer rows.Close()

for rows.Next() {
	var name string
	if err := rows.Scan(&name); err != nil {
		return err
	}
}
return rows.Err()

Checking only the initial Query error misses errors encountered while iterating. Forgetting Close can keep the underlying connection unavailable to other work. A leak in a pooled abstraction tends to present itself as an unrelated request mysteriously waiting, which is a thoughtful way for a bug to meet new people.

Transactions bind operations to the transaction’s connection. Once Begin succeeds, calls that belong to the transaction must use tx.Query, tx.Exec, and friends, not the original db handle. Calling db.Exec in the middle may use another connection and therefore sit outside the transaction entirely.

Statements and transactions need closing too. I keep their lifetime in the smallest useful scope and arrange cleanup immediately after successful creation. This is less about tidiness than returning scarce server and connection resources on every error path.

I now create one *sql.DB for a configured database, verify it during startup when appropriate, pass it to code that needs it, close rows promptly, and inspect iteration errors. Boring database plumbing is good database plumbing.

I wrote an echo server with one goroutine per connection and then wondered where all the operating-system threads were. I had accepted the pleasant API before understanding the trick beneath it.

The simple server is almost suspiciously simple:

func serve(c net.Conn) {
	defer c.Close()
	buf := make([]byte, 4096)
	for {
		n, err := c.Read(buf)
		if err != nil {
			return
		}
		if _, err := c.Write(buf[:n]); err != nil {
			return
		}
	}
}

func main() {
	l, err := net.Listen("tcp", ":9000")
	if err != nil {
		panic(err)
	}
	for {
		c, err := l.Accept()
		if err != nil {
			return
		}
		go serve(c)
	}
}

Read appears to block the goroutine until bytes arrive. If every blocked goroutine required a blocked kernel thread, this design would approach the old thread-per-connection model with nicer syntax but similar scaling limits. The runtime instead integrates network descriptors with an operating-system readiness notification facility.

The toy server exits on an Accept error. A real daemon may retry temporary errors with a delay, but blindly continuing on every error turns a closed listener or exhausted descriptor table into a very efficient busy loop.

On Linux that facility is epoll; other systems provide equivalents such as kqueue. The exact backend is platform-specific, but the shape is similar. The socket is placed in non-blocking mode. A read attempts the system call. If the kernel says the operation would block, the runtime records that the goroutine is waiting for readiness on that descriptor and parks it. The underlying thread is then free to run another goroutine.

Parking is not polling

Parking a goroutine removes it from runnable work. It does not spin around asking the socket whether it feels conversational yet. The runtime’s network poller waits for the kernel to report descriptors that may now make progress. When an event arrives, the corresponding goroutine is made runnable and eventually tries the operation again.

Readiness is not completion. A descriptor reported readable may yield some bytes, an end-of-file condition, or another transient result by the time code runs. The runtime and net package retain the usual loop around non-blocking operations. The poller says “worth trying,” not “your complete application message has arrived on a silver tray.”

This distinction also explains partial reads. TCP is a byte stream. One Write at the sender does not imply one equally sized Read at the receiver. The poller reports available progress; framing remains the protocol’s responsibility. My echo loop can return each chunk because its protocol is deliberately trivial. A real message parser must accumulate and delimit data.

Deadlines join the machinery

Network code also needs a way to stop waiting. SetReadDeadline and SetWriteDeadline associate times with operations on a connection. The runtime combines descriptor waiting with timers so a parked goroutine can become runnable because the socket is ready or because its deadline expired.

Without deadlines, a slow or vanished peer can retain a goroutine and everything reachable from its stack indefinitely. Small goroutine stacks make this cheaper than a thread leak, not desirable. A server that accepts untrusted connections should make an explicit decision about idle time rather than interpreting silence as a lifelong subscription.

Closing a connection is another wake-up path. Code blocked in an operation must be released with an error rather than left attached to a descriptor that no longer has meaning. This is one reason descriptor state belongs in runtime-coordinated structures instead of being only an integer passed casually among goroutines.

What still blocks a thread

The network poller helps operations the runtime knows how to put into non-blocking mode and register. It does not transform every possible system call into asynchronous work. A call into C, an ordinary blocking file operation, or an unfamiliar kernel interface may occupy an operating-system thread. The scheduler can create or use another thread so runnable goroutines continue, but blocked threads still have a cost.

Regular disk files are particularly different from sockets. Readiness interfaces commonly report them ready even when the operation may wait on storage. Treating every Read method as equivalent because it satisfies the same interface would be tidy and wrong.

There is a nice Linux comparison here. An application written in C can build an event loop directly around epoll, maintain a state machine per connection, and achieve excellent performance. It also has to preserve that state across every partial operation and error path. Go’s runtime uses the same broad kernel capability, then presents the waiting state as an ordinary goroutine stack. The state machine has not vanished; the runtime and compiler help store it in a form humans generally prefer reading.

A practical test

I connected many idle clients to the echo server and watched thread count and memory rather than merely measuring requests per second. The number of goroutines rose with connections. Threads did not rise one-for-one. Memory grew, because every connection and goroutine is real, but not as though each connection had received a traditional large thread stack.

This does not prove every Go server scales automatically. File descriptors have process limits. Buffers consume memory. A goroutine can retain a large object graph. Handlers can block on locks, allocate furiously, or perform CPU work that has nothing to do with the poller. The mechanism solves efficient waiting, not careless architecture.

Still, it changes how I write network services. I can begin with one goroutine per connection because it is clear and maps directly to the protocol conversation. Then I measure descriptor use, memory, scheduler behaviour, and latency. I do not begin by translating the protocol into callback soup merely because the C implementation had to.

The net package’s blocking appearance is therefore honest from the goroutine’s perspective. That goroutine really does stop until it can proceed. The useful deception is that the operating-system thread need not stop with it.

I needed to print arbitrary structs for a debugging tool. After writing the third type switch I reached for reflect, with the same cautious expression normally reserved for an unfamiliar electrical panel.

Reflection starts from interfaces. reflect.TypeOf(x) reports the dynamic type stored in the interface, while reflect.ValueOf(x) represents its dynamic value. This distinction is the same type/value pair that explains why an interface containing a nil pointer is not itself nil.

The first trap is that a reflect.Value has its own validity and kind. A nil interface produces an invalid Value. A typed nil pointer produces a valid Value whose kind is Ptr and for which IsNil is true. Calling methods that do not apply to a Value’s kind panics. Reflection replaces compile-time checks with a detailed list of ways to be wrong at runtime.

A modest struct printer looks like this:

func fields(x interface{}) {
	v := reflect.ValueOf(x)
	if v.Kind() == reflect.Ptr {
		v = v.Elem()
	}
	if !v.IsValid() || v.Kind() != reflect.Struct {
		return
	}

	t := v.Type()
	for i := 0; i < v.NumField(); i++ {
		fmt.Println(t.Field(i).Name, v.Field(i).Interface())
	}
}

Even this needs qualifications. Elem on a nil pointer yields an invalid Value. Access restrictions apply to unexported fields, and asking for an interface representation where it is not permitted can panic. Production code must decide whether to skip, report, or reject these cases rather than treating every struct as an unlocked cupboard.

Settable values are another common surprise. reflect.ValueOf(n) contains a copy and is not settable. To modify n, pass &n, call Elem, verify CanSet, and then use a setter appropriate to its kind. The pointer is not ritual; it gives reflection access to the original storage.

Reflection is invaluable at boundaries where types genuinely vary: encoders, decoders, formatters, and tools that inspect user-defined structures. It is much less convincing as a way to avoid writing an interface for ordinary application behaviour. A reflected method call loses static checking, is harder to read, and performs more runtime work than a direct call.

Calling reflection “unsafe” is unfair in one important sense. The package checks its rules and panics instead of letting arbitrary memory corruption proceed. It is polite. It knocks before ruining the evening.

For my debugging printer, reflection was exactly right. I kept it at the edge, converted the inspected data into a simpler representation, and let the rest of the program remain statically typed. That boundary is where I find reflection most useful: a small dynamic vestibule leading into an otherwise ordinary Go house.

I wrote the concurrency equivalent of checking whether the refrigerator light is off by repeatedly opening the door:

var ready bool
var value string

func load() {
	value = "finished"
	ready = true
}

func main() {
	go load()
	for !ready {
	}
	fmt.Println(value)
}

It worked on my machine, which is information about my machine rather than evidence that the program is correct.

Two goroutines access ready, one writes it, and there is no synchronization. The same is true for value: seeing ready does not create a language guarantee that the write to value is visible. Compilers and processors may reorder operations where a single goroutine cannot observe the difference. Caches and registers add further opportunities for my intuition to be wrong.

The Go memory model describes which operations establish a “happens before” relationship. If one event happens before another, the effects required by that relationship are visible in the defined order. Goroutine creation, channel communication, mutex operations, and package initialization provide useful ordering guarantees. A hopeful loop around a boolean does not.

The channel version says what I meant:

func load(done chan bool) {
	value = "finished"
	done <- true
}

func main() {
	done := make(chan bool)
	go load(done)
	<-done
	fmt.Println(value)
}

The send happens before the corresponding receive completes. Since value is written before the send, reading it after the receive is properly ordered. Better still, the channel documents the handoff instead of hiding it behind a global flag.

A mutex would also work. Unlocking a mutex and subsequently locking it establishes the required ordering around protected data. The choice is not that channels are holy and mutexes are regrettable. Channels fit ownership transfer and signalling; mutexes fit shared state with a clear invariant. Using the mechanism that expresses the relationship makes the program easier to audit.

This is not merely a question of whether reading and writing a machine-sized boolean is individually atomic. Even if a load cannot tear, atomicity of that one load does not order the separate write to value. “But the flag is only one byte” answers the wrong question with admirable confidence.

The safe rule is pleasantly boring: when multiple goroutines access data and at least one modifies it, arrange synchronization unless the operations use a specifically documented atomic mechanism. Do not infer ordering from sleep calls, observed scheduling, processor brand, or the phase of the moon.

I like Go’s concurrency syntax because it makes the correct version short. That does not make every short concurrent program correct. Goroutines remove ceremony from concurrency, not causality.

I wanted to rename a field across a pile of Go files. My first impulse was a regular expression. My second, considerably healthier impulse was to remember that Go ships with a parser.

Source code is text, but edits usually concern syntax. The go/parser package reads source into nodes from go/ast. An identifier is an *ast.Ident; a function declaration is an *ast.FuncDecl; an expression has structure beyond where its parentheses happened to land. Once the source is a tree, a tool can ask precise questions instead of hoping a comment does not look executable.

Here is a tiny program that lists function names:

package main

import (
	"fmt"
	"go/ast"
	"go/parser"
	"go/token"
)

func main() {
	fset := token.NewFileSet()
	f, err := parser.ParseFile(fset, "sample.go", nil, 0)
	if err != nil {
		panic(err)
	}
	ast.Inspect(f, func(n ast.Node) bool {
		if fn, ok := n.(*ast.FuncDecl); ok {
			fmt.Println(fn.Name.Name)
		}
		return true
	})
}

token.FileSet connects tree positions to files, lines, and columns. It looks like administrative furniture until an error must say exactly where it found something. Then it becomes the only chair in the room.

After changing a tree, go/printer can write Go source in the standard layout. This is the important lesson of gofmt: parse syntax, then print it consistently. It is not lining up text with an especially ambitious set of regular expressions.

Comments make transformations subtler. Parsing with parser.ParseComments retains comment groups, but moving nodes while preserving the intended comment association still requires care. Imports are another edge. A rename may change whether an import is used, and blindly printing the tree does not make semantic mistakes disappear.

There is also a limit to what the AST alone knows. The syntax tree can tell me that x.Name is a selector expression. It cannot, by itself, tell me which package or type supplied Name. That needs type information. For a local mechanical transformation with an unambiguous syntactic shape, the AST is often enough. For a semantic rename, it may not be.

My field change ended up being a dozen lines around ast.Inspect, followed by formatting and a review of the diff. That was more code than a one-line substitution and much less code than repairing a one-line substitution after it renamed examples, comments, and an unrelated field with the same spelling.

I used to think gofmt was mainly a successful ceasefire in the whitespace wars. It also shows the value of shipping language tools as ordinary libraries: the compiler is not the only program allowed to understand Go source.