I wrote a small TCP service using one thread per connection. It was the obvious design: accept a socket, start a thread, perform blocking reads and write the response. Each connection looked like a normal sequential program, and the first version was finished before lunch. This is an excellent property in prototypes and an alarming one in architecture.

The service was fine with a few clients. With many mostly idle connections, however, it accumulated threads whose principal activity was owning stacks and waiting. Shutdown required waking them all, shared statistics needed locks, and one slow operation could hold a mutex needed by several innocent connections. I had converted network concurrency into scheduler concurrency whether the work needed it or not.

An event loop starts from a different observation: most connections are not executing. They are waiting for a socket to become readable or writable, a timer to expire, or another operation to complete. One thread can wait for many such events, dispatch a small handler for each ready source, then wait again.

State instead of stacks

The thread-per-connection version stores progress in the thread’s call stack. A function reads a header, calls another function to read a body, and eventually writes a reply. When a read blocks, the stack quietly remembers where execution should continue.

With an event loop, handlers must not block. The connection’s progress becomes explicit state: reading headers, reading a body, preparing a response, or writing remaining bytes. A read handler consumes whatever is available, updates the state and returns. If the message is incomplete, the loop will call it again after the socket becomes readable.

This looks more complicated because it is more explicit. Partial reads and writes were always possible; blocking calls merely kept the unfinished operation on a private stack. The state machine makes those boundaries visible. That visibility is annoying for the first implementation and useful for every timeout, cancellation and protocol error added later.

A connection object in my revised service holds the socket, input and output buffers, parser state and deadlines. The loop owns the objects and invokes them serially. Because one loop thread mutates this state, most of it needs no locking. Ownership becomes a rule I can state rather than a hope expressed through mutexes.

Readiness is not completion

The operating system reports that an operation is likely to make progress. A readable socket may provide only part of a request. A writable socket may accept only part of a response. Handlers therefore loop until the operation would block, preserve what remains, and return control.

Nonblocking mode is essential. If one handler performs a blocking disk lookup or name resolution, every connection assigned to that loop waits behind it. Event-driven code does not eliminate blocking; it makes accidental blocking more expensive and easier to notice when the entire service pauses at once. Very democratic, if not especially helpful.

Timers belong in the same model. The loop computes the next deadline, limits its wait accordingly, then expires idle connections or retries work. I originally used one timer thread that sent messages to connection threads. Replacing that arrangement with a deadline queue removed both threads and a surprising amount of code.

Where threads still fit

An event loop is not an argument for one thread in the entire program. CPU-heavy work can be sent to a bounded worker pool. Blocking interfaces that cannot be integrated with readiness may need workers too. The important constraint is that results return to the owning loop as events, rather than allowing workers to mutate connection state directly.

Multiple loops can run on multiple processors, each owning a set of connections. This keeps the single-owner rule while using more than one core. Distribution and load balancing then become concerns, but they are explicit and usually less tangled than allowing every connection thread to touch global state.

Threads remain attractive when there are few concurrent operations, the blocking flow closely matches the problem, or the platform’s asynchronous facilities are poor. A thread can also be a good unit of isolation for independent subsystems. I do not want to replace clear blocking code with callbacks merely to win a concurrency argument on the Internet.

The event-driven version has its own failure modes. A handler that performs too much work increases latency for everything behind it. Unbounded input or output buffers merely move resource exhaustion from stacks to heaps. Reentrant callbacks can produce transitions the state machine did not expect. Fairness must be designed: a permanently busy socket should not monopolize one pass through the loop.

My current preference

For a service with many mostly waiting connections, I now strongly prefer an event loop with explicit ownership and a small worker pool for unavoidable blocking or CPU work. It uses threads according to available parallelism, not according to the number of clients. It also gives timeouts, cancellation and backpressure natural places in the design.

That preference is qualified. The state machine must be kept small, protocol parsing should be separated from readiness plumbing, and every handler must have a bounded amount of work. If those rules cannot be maintained, a modest number of threads may be safer than one ingenious loop nobody can debug.

My first threaded service was easier to write. The event-loop version is easier to reason about under load, which is a different and more valuable form of easy. It took me several pages of notes to discover that sleeping threads are still resources. The kernel, rather rudely, knew this already.