Blog

My event-driven server stayed responsive under load, but its memory use climbed steadily. I had removed blocking and accidentally built an extremely efficient machine for accepting work it could not finish.

Each connection had an output buffer. Producers appended responses faster than the network drained them, so those buffers became an unbounded queue distributed across clients. Nothing was technically stuck. The server was merely saving enough unfinished work to become stuck later.

Backpressure means carrying limited downstream capacity toward the producer. When a connection’s output crosses a high-water mark, I stop reading more requests from it or stop scheduling new work. Once buffered output falls below a lower mark, reading resumes. Separate thresholds avoid switching state on every small write.

The same rule applies between the event loop and worker pool. A bounded job queue makes overload visible. If it is full, the loop must defer input, reject work or shed a connection according to policy. Adding another unbounded queue does not increase capacity; it increases the delay before admitting there is none.

I prefer bounded queues and explicit overload behavior, even when rejection feels impolite. The exact limits require measurement, and short bursts deserve some room. But a service that says “not now” can recover. One that accepts everything may respond to nobody, which is very accommodating in principle and less so in practice.

Two things have repeatedly made Qt harder to suggest: licensing confusion and the lack of an obvious first development environment. Qt 4.5 improved both.

Qt is now available under the LGPL as well as its existing licenses. That gives more applications a straightforward way to use the library while keeping their own licensing choices, provided they comply with the LGPL’s requirements. It is not a declaration that licenses no longer need reading. It merely makes the common case less theatrical.

Qt Creator 1.0 supplies the other missing front door. It understands Qt projects, code navigation, building and debugging without trying to become an operating system disguised as an editor. I opened an existing project and was productive quickly, despite initially searching for several commands where my usual editor keeps them.

Creator is still young, and experienced users with carefully constructed environments may not gain much. I am not abandoning my editor after one pleasant afternoon. For newcomers, however, a focused tool removes a pile of setup from the first experiment.

I strongly prefer this combination: a capable cross-platform toolkit with a less restrictive entry point and an IDE that teaches its normal workflow. Neither guarantees good applications. They simply let people reach the part where application design can go wrong, which is where programmers traditionally demonstrate real creativity.

Btrfs entered the mainline kernel earlier this year, so naturally I created a filesystem and started putting unimportant data on it. Filesystems are fascinating, provided the nouns “only copy” and “family photographs” remain elsewhere in the sentence.

The central structure is a copy-on-write B-tree. When metadata changes, Btrfs writes new blocks rather than overwriting the existing path in place. Data can also be copy-on-write. This makes snapshots and clones natural consequences of the storage model instead of layers awkwardly attached above a filesystem.

Snapshots are cheap at creation because they initially share existing extents. As either side changes, new extents represent the differences. That is attractive for test trees: I can capture a known state, run an unpleasant upgrade and return without copying every unchanged file. Shared blocks also mean deletion and free-space accounting are less obvious than in a simple independent copy.

Checksums cover data and metadata, allowing corruption to be detected rather than quietly delivered. With redundant storage, the design can potentially use a valid copy to repair damaged data. Detection alone remains valuable, although it cannot reconstruct bytes that exist in only one damaged place. A checksum is an alarm, not a backup wearing mathematical spectacles.

Btrfs also aims to manage multiple devices and provide subvolumes, online growth, compression and other facilities usually assembled from several layers. Integrating them allows the filesystem to understand which blocks contain live data, which can help operations that would otherwise see only an undifferentiated block device.

The ambitious scope is exactly why I am cautious. Mainline inclusion makes development and testing easier; it does not make the format or implementation mature overnight. Repair tools, failure behavior and operational knowledge matter more than a successful benchmark. A filesystem earns trust through years of dull recoveries, not a feature table.

My current test uses build outputs and source trees already stored elsewhere. I exercise snapshots, fill the filesystem, delete subvolumes and simulate untidy shutdowns. Performance is interesting, but I care more about whether every operation has understandable results and whether tools can explain the state afterward.

I prefer Btrfs’s integrated copy-on-write direction for experiments that benefit from snapshots. I do not yet prefer it for important production data. ext4 is the conservative choice today, and even that choice requires backups. Btrfs is early and explicitly experimental; the most useful contribution ordinary testers can make is finding failures without turning those failures into personal disasters.

My test laptop changes display modes several times while booting: firmware text, a console mode, the X server’s preferred resolution, and sometimes another mode after resume. Each transition blanks the screen long enough to make me wonder whether the machine is considering a different profession.

Kernel mode setting moves responsibility for configuring display modes into the kernel graphics driver. Traditionally the X server performed that setup from user space. With KMS, the kernel knows about connectors, CRTCs, encoders, framebuffers and modes early enough to establish the display before X starts.

That early ownership makes boot transitions smoother, but the architectural benefit is larger. Suspend and resume already require kernel coordination with devices. If display state is also managed there, restoration does not depend on a particular user-space server reconstructing everything after the fact. Panic and virtual-terminal paths can retain a usable framebuffer as well.

The Direct Rendering Manager interface exposes the resources to user space. A display server can discover connectors and available modes, choose how framebuffers map to scanout hardware, and request changes through a common kernel boundary. The kernel arbitrates access to the device rather than allowing unrelated programs to reprogram it optimistically.

KMS also fits the broader memory-management work in graphics drivers. Rendering buffers and scanout buffers share hardware constraints, and transitions between them need synchronization. Putting mode setting beside DRM’s device and buffer management reduces the number of components that must agree through private arrangements.

My Intel machine with a recent kernel now reaches its native mode earlier and switches to X with less flashing. Resume has also become more predictable. This is not universal evidence. Driver support varies, hardware contains creative surprises, and a failed early mode set can replace a merely ugly boot with a completely dark one.

Debugging changes too. Problems that once belonged entirely to the X log may now begin in kernel messages and driver parameters. Distributions need rescue paths and conservative defaults while support matures. The mechanism is better placed, but moving code into the kernel also raises the cost of mistakes.

I strongly prefer KMS as the long-term division of responsibility. Display hardware is a shared device whose state matters before, during and after the graphical session; the kernel is the natural owner. For current machines I enable it where the driver is known to work and keep a boot option that disables it. Architectural conviction is comforting, but a visible console is more immediately useful.

I was chasing a latency spike in a kernel path and began adding printk() calls. This is the kernel equivalent of looking for a gas leak with a candle: familiar, illuminating and likely to influence the situation under investigation.

Recent kernels include ftrace, with controls under debugfs, normally in /sys/kernel/debug/tracing. It started as a function tracer and has grown useful tracing modes and events. The useful bit is that tracing can often be selected and controlled at runtime rather than compiled into a private forest of diagnostic messages.

The function tracer records kernel function activity in per-CPU ring buffers. Filters can restrict collection to relevant functions, which matters because tracing everything creates enormous output and changes timing. The function-graph tracer adds entry and return relationships, making nested execution and duration easier to see.

For my problem, the scheduler tracing facilities were more revealing than raw function calls. They showed when the task stopped running, which task replaced it and when it returned. The delay I had blamed on one function was mostly time spent waiting to be scheduled. A timestamp around the function could show elapsed time, but not explain where that time went.

Ring buffers are a sensible mechanism for this job. Each CPU can record events with less cross-CPU contention, and old entries can be overwritten when configured as a rolling trace. I can leave tracing ready, reproduce the spike and stop collection soon afterward. The final seconds are usually more valuable than a complete account of the machine booting and becoming bored.

Tracing still has overhead. Function tracing a hot path produces work, filters cost something, and reading the trace while collecting it can perturb results. I begin with the narrowest event set that can answer the question and compare behavior with tracing disabled. A trace is evidence gathered by an instrument, not the unobserved universe itself.

printk() remains appropriate for durable diagnostics that users or administrators need. It is poor as an ad hoc high-volume performance recorder: messages contend for shared facilities, flood logs and require source edits to move the probe. Temporary tracing and permanent logging solve different problems.

For kernel execution and scheduling questions, I try ftrace first. A carefully placed message can fill in semantic information the trace lacks. This is faster and leaves less debris in the source, including my traditional final patch removing forty-seven increasingly desperate print statements.