C

Now that Linux 2.6.13 includes inotify, I replaced a timer in a small file-watching utility. The timer scanned a directory every second, which was simple, portable, and wasteful. It also introduced the charming choice between delayed updates and more frequent useless work.

My first inotify version assumed each read() returned one complete event. It worked during gentle testing and became nonsense when several files changed quickly. The interface returns a byte stream containing one or more variable-length struct inotify_event records. Correct code walks the buffer by each fixed header plus its len field.

union {
        struct inotify_event event;
        char bytes[4096];
} buffer;
ssize_t count;
size_t offset = 0;

do {
        count = read(fd, buffer.bytes, sizeof(buffer.bytes));
} while (count < 0 && errno == EINTR);

if (count < 0) {
        perror("read");
        exit(EXIT_FAILURE);
}
if (count == 0) {
        fprintf(stderr, "inotify descriptor reached EOF\n");
        exit(EXIT_FAILURE);
}

while (offset < (size_t)count) {
        struct inotify_event *event;
        size_t event_size;

        if ((size_t)count - offset < sizeof(*event)) {
                fprintf(stderr, "short inotify event header\n");
                exit(EXIT_FAILURE);
        }
        event = (struct inotify_event *)(buffer.bytes + offset);
        event_size = sizeof(*event) + event->len;
        if (event_size > (size_t)count - offset) {
                fprintf(stderr, "short inotify event name\n");
                exit(EXIT_FAILURE);
        }
        handle_event(event);
        offset += event_size;
}

The union gives the byte storage suitable alignment for struct inotify_event; a plain character array doesn’t promise that. The code also deals with interruption, read failure, EOF, and truncated records before treating bytes as an event. A watch identifies a pathname to the kernel, and events report changes associated with a watch descriptor.

The largest conceptual correction was learning that notifications are hints about state changes, not a private journal guaranteed to preserve my worldview. Events can arrive in bursts. A queue can overflow. A watched object can move or disappear. On overflow, the safe response is to rescan and rebuild known state rather than guess what was missed.

Renames deserve similar care. Related move events carry a cookie that can help pair them, but the application still needs sensible behavior when only one side is visible. Watching a directory also does not automatically mean recursively watching every directory below it.

I kept one function that performs a complete scan and made event handling update the same internal model. That gives the program a recovery path and keeps startup behavior consistent with overflow recovery. The event loop is an optimization over known state, not the only source of truth.

Compared with polling, inotify is immediate and avoids repeated directory walks when nothing changes. It also exposes more edge cases because the kernel reports actual operations rather than letting a periodic snapshot blur them together.

I prefer the new mechanism, with one reservation: event-driven doesn’t mean infallible. Read complete buffers, parse every record, expect overflow, and keep a rescan path. The kernel says something happened; deciding what the application should now believe is still our job.

A driver crashed several calls after an object had been freed. The final bad pointer looked plausible, and ordinary logs only showed that teardown had happened sometime earlier. I needed the stale access to leave a clearer fingerprint.

I rebuilt the test kernel with CONFIG_DEBUG_SLAB. The slab allocator’s debugging checks and poisoning replace freed storage with a recognizable pattern instead of leaving yesterday’s fields looking valid. The next failure stopped looking like random corruption: the pointer led into an object full of poison bytes.

That didn’t identify the owner by itself. I decoded the complete oops against the matching unstripped vmlinux, then followed the call chain back to a timer callback. The timer retained the private pointer after the remove path had called kfree().

The repair was pleasantly unoriginal: prevent rearming, delete the timer with the synchronizing form, and only then free the object. I ran repeated load, activity, and unload cycles with slab debugging still enabled. I also forced probe failures after each acquisition to make sure partially built objects followed the same lifetime rules.

Poisoning is a diagnostic, not memory safety. It works when the freed bytes haven’t already been reused, and it makes a debug kernel slower. Here it turned a believable stale structure into an unmistakably dead one, which was exactly the clue I needed.

I ported a small driver experiment to a newer 2.6 kernel this week. The old code compiled after a few mechanical edits, so I congratulated myself much too early. Loading the module produced an oops during device removal, which is the kernel’s way of reviewing code without softening the language.

My first attempt was to stare at the final faulting line and add a null check. That stopped one crash but left the design wrong. The pointer was not unexpectedly null; it referred to an object whose lifetime had ended. A defensive condition would merely make the race less visible.

The 2.6 device model asks drivers to express relationships among buses, devices, and drivers. Registration is not just bookkeeping. It establishes ownership and determines which callbacks can run as devices appear and disappear. I had treated remove as the reverse spelling of probe, but not as the reverse sequence of acquired resources.

I wrote down each successful step in probe: enable the device, reserve its region, allocate private state, initialize synchronization, and expose the interface. Then I made every failure path unwind only the steps already completed. remove performs the same cleanup in reverse order after preventing new work from entering.

static int sample_probe(struct device *device)
{
        struct sample *sample;
        int error;

        sample = kmalloc(sizeof(*sample), GFP_KERNEL);
        if (!sample)
                return -ENOMEM;

        memset(sample, 0, sizeof(*sample));
        error = reserve_hardware(sample);
        if (error)
                goto free_sample;

        /* Published callbacks must be able to find fully initialized state. */
        dev_set_drvdata(device, sample);

        error = register_interface(sample);
        if (error)
                goto clear_drvdata;

        return 0;

clear_drvdata:
        dev_set_drvdata(device, NULL);
release_hardware:
        release_hardware(sample);
free_sample:
        kfree(sample);
        return error;
}

This is shortened code, but the order matters: private data is attached before register_interface() makes callbacks possible. If publication fails, the failure path clears that pointer before releasing and freeing the state. In C, a linear chain of labels is often clearer than nested cleanup conditionals.

Removal uses the other half of the same rule:

static int sample_remove(struct device *device)
{
        struct sample *sample = dev_get_drvdata(device);

        unregister_interface(sample);
        stop_hardware(sample);
        cancel_deferred_work(sample);
        dev_set_drvdata(device, NULL);
        release_hardware(sample);
        kfree(sample);
        return 0;
}

unregister_interface() first prevents new user callbacks. The hardware is then stopped and outstanding work is drained while drvdata still points to live memory. Only after no callback can use it do I detach and free it.

For evidence, I enabled the kernel debugging options relevant to memory and locking and built a kernel specifically for this work. They cost performance, but a development kernel is a measuring instrument, not a racing bicycle. I also kept the unstripped vmlinux and the exact .config beside the test build.

When the next oops appeared, the symbolic trace identified both the callback and its caller. printk() markers around acquisition and release confirmed that queued work could still run after teardown began. The repair was to stop new requests, cancel or drain deferred activity, and only then free the state it could reference.

The order now looks like this:

block new work
stop device activity
wait for deferred work
remove user-visible interface
release hardware resources
free private state

I tested failure paths too. Returning an artificial error after each acquisition showed whether cleanup restored the previous state. Module load and unload in a loop then exercised the unremarkable path repeatedly. Repetition is valuable because races dislike appointments.

I included reference counts in the review even though the first crash did not mention them. A reference is a claim that an object must remain alive, not merely an integer to increment near a pointer. Every path acquiring one needs a visible release, and teardown must prevent new claims before waiting for old ones. Counting without a shutdown rule only measures how confused the lifetime has become.

Lock choice followed the same reasoning. I did not replace every lock with a larger one to make the warning disappear. I listed which fields formed one invariant and which contexts accessed them. A spinlock suited the short state transition shared with interrupt context; work that could sleep stayed outside it. Narrow critical sections made both the rules and the traces easier to read.

One subtle lesson was to distrust logs as timing-neutral. A printk() can alter scheduling enough to hide a race. I used it to establish broad ordering, then relied on assertions, debug checks, and repeated tests rather than declaring victory when extra logging made the crash vanish.

An oops after testing still required a disciplined capture. I saved the complete report, not only the final address, and matched it to the exact kernel image. A decoded call chain is context: it shows how execution arrived, which often matters more than the instruction that finally touched invalid memory.

The kernel APIs will continue to evolve, and copying a driver from an older tree will continue to be tempting. The durable knowledge is not a particular field name. It is the discipline of checking the current in-tree callers, understanding callback context, defining object lifetime, and making cleanup the exact inverse of setup.

I still prefer user-space debugging, where a bad pointer usually ruins one process rather than the afternoon. The kernel does leave a trail if the build preserves symbols and the code preserves invariants. That’s better than adding null checks until the machine goes quiet.

A driver experiment needed to do a little cleanup after an interrupt. I put the cleanup directly in the interrupt handler because it was only a little work. Then that little work called something that could sleep, and the kernel objected with considerably more confidence than I had shown.

The naive model says an interrupt handler is just a function the kernel calls quickly. The missing detail is context. Hard interrupt context cannot sleep: there is no process there to suspend in the normal sense, and holding up interrupt handling harms the rest of the machine.

The useful pattern is to split the operation. The handler acknowledges the device, records the minimal state under the right lock, and schedules deferred work. A workqueue function later runs in process context and may use operations that can sleep.

static void device_work(void *data)
{
        struct sample_device *dev = data;

        /* Slow cleanup is safe in process context. */
        finish_request(dev);
}

The exact initialization APIs deserve a check against the headers for the kernel being built; this area has changed before and surely will again. The invariant matters more than memorizing one macro: know which context executes the callback, what locks are held, and whether every function it calls may sleep.

Allocation flags make the distinction concrete. Code in a context that cannot sleep must not request an allocation mode that may block. Moving slow work to a workqueue lets that callback use normal process-context operations, but it does not make shared state safe automatically.

Teardown is the other half. Before freeing the device state, the driver must stop scheduling work and ensure already queued callbacks have finished. Otherwise the deferred function merely postpones the crash until after the object is gone.

I added an explicit comment at the scheduling point and reduced the handler to the smallest useful unit. Latency improved, but the larger gain was conceptual. Once context is part of the function’s contract, calls that looked harmless become obviously wrong.

Kernel C is not difficult because the syntax is exotic. It is difficult because an ordinary-looking call carries constraints about context, ownership, and concurrency. Ignoring those constraints produces bugs that wait patiently for the least convenient machine. Mine at least had the courtesy to complain immediately.