Linux

I needed to answer a simple question on a 2.6 test machine: which driver owns this network device? My old habit was to rummage through boot messages, /proc, and module output until the answer surrendered. The new device model gives us a less archaeological method.

First, mount sysfs if the installation has not done it already:

mount -t sysfs none /sys

The interesting part is not that /sys contains files. Linux can turn almost anything into files if left unattended. The useful part is the structure. Devices appear according to their physical or logical parentage under /sys/devices. Buses, such as PCI and USB, have views under /sys/bus, while /sys/class groups devices by function, such as network interfaces or block devices.

These are views of the same kernel objects, joined by symbolic links. For an Ethernet interface named eth0, this is a useful start:

readlink /sys/class/net/eth0/device
ls -l /sys/class/net/eth0/device/driver

The first link leads back into the device hierarchy. The driver link, when present, identifies the bound driver. Following parent links tells me which PCI device contains the interface and where that device sits.

This differs from /proc. Procfs grew as a place for process information and gradually acquired assorted kernel knobs and device facts. Sysfs is an exported representation of the kernel’s device model. One attribute per file is the usual convention, which makes simple scripts possible without parsing a decorative essay from the kernel.

I am deliberately cautious about those scripts. The 2.6 series is not final yet, and sysfs details can change as interfaces settle. Scripts should consume documented attributes, not depend on directory ordering or infer meaning from every internal-looking name. A readable tree is still an interface, not an invitation to marry the first implementation detail one encounters.

The larger implication is device management in userspace. If the kernel exposes device identity and events cleanly, a userspace program can create names and permissions according to policy rather than a huge static /dev or a kernel naming scheme. That work is still young, but sysfs supplies the map it needs.

For now, it has already replaced several minutes of log-diving with two symbolic links. I consider that progress, even if it has added another virtual filesystem to my vocabulary.

I upgraded a test machine to a recent glibc with NPTL and immediately thought several processes had disappeared. An old LinuxThreads program used to make ps look as if every thread had invited itself to the process list. With NPTL and updated tools, the display is much closer to what the programmer meant: one process containing several threads.

The old implementation was clever, but its compromises leaked. LinuxThreads created tasks with clone() and used a manager thread for coordination. Thread IDs and process IDs did not line up neatly with POSIX expectations, signal delivery had awkward corners, and tools often presented threads as peculiar sibling processes.

NPTL also builds threads on kernel tasks, but newer kernel facilities let it implement POSIX semantics much more directly. Thread groups identify tasks belonging to one process. Futexes provide a fast synchronization primitive: uncontended locking can happen in userspace, while the kernel becomes involved when a thread must sleep or wake another waiter. That avoids a system call for every successful mutex operation.

The practical lesson is to upgrade the tools with the library. A stale ps, debugger, or monitoring script can report technically available data in a thoroughly unhelpful way. On this machine I check both the process view and the individual tasks under /proc/PID/task when debugging:

ls /proc/1234/task

Each directory there is a kernel task ID. The process has a thread-group leader, and the other tasks share resources according to the flags used when they were created. “Thread” is not a completely separate kernel creature. It is a task sharing an address space and other state with its group.

Compatibility deserves some attention too. Programs should use the POSIX thread API and not depend on LinuxThreads accidents such as each thread appearing to have an unrelated process identity. Code that sends signals to guessed numeric IDs is especially suspicious. Correct code generally needs no source change, but incorrect assumptions have enjoyed years to become tradition.

NPTL is faster in useful places, particularly thread creation and synchronization, but I like the semantic cleanup more. Better benchmarks are welcome; fewer explanations beginning with “well, on Linux a thread looks like…” are better. The implementation is finally making the common abstraction less dishonest.

I have been testing the new kernel on a machine that does two incompatible jobs: compile software and remain pleasant enough to use while compiling software. Under load, the 2.6 test kernel feels less sticky than my usual 2.4 setup. The O(1) scheduler is a large part of that.

The name describes a property, not a performance promise. Choosing the next runnable task takes constant time rather than getting more expensive as the run queue grows. The scheduler keeps arrays of queues indexed by priority, plus a bitmap showing which priorities contain runnable tasks. Finding work becomes a matter of locating the first set bit and taking a task from that queue.

There are actually two priority arrays per processor: active and expired. A normal task runs for its time slice and then moves to the expired array. When active becomes empty, the kernel swaps the arrays. No grand tour through every process is required.

That design also avoids one global run queue becoming a lock everybody fights over on an SMP machine. Each processor has its own run queue, and load balancing periodically moves work when the queues become uneven. Per-processor data makes the common path cheaper, although balancing is still real work and certainly not magic.

Interactive behaviour comes from dynamic priorities. A task that sleeps often, as an editor or terminal usually does while waiting for me, receives favourable treatment when it wakes. A compiler chewing CPU continuously tends to use its slice and wait in the expired array. The scheduler is estimating interactivity from sleep behaviour; applications do not announce that they are important because a human is glaring at them.

There is a caveat. Heuristics can be fooled, and “feels faster” is not a benchmark. A database server and a desktop have very different ideas of fairness. I am watching throughput as well as latency and checking for tasks that receive odd priority treatment.

Still, the mechanism is refreshingly concrete. Constant-time selection, per-CPU queues, and explicit expired work explain the observed behaviour without fairy dust. The desktop remains responsive during a build, the build still finishes, and neither needs a special incantation. That is a scheduler improvement I can actually notice.

Linus released 2.6.0-test1 this week, which means the 2.5 development series has finally put on a tie and is pretending to be suitable for polite company. It is still a test kernel. I would not install it on the machine that pays the bills, but I do want to know what will break before 2.6 becomes ordinary.

The obvious approach is to keep the current 2.4 kernel installed, add 2.6 beside it, and make the boot loader offer both. Do not replace the known-good entry. This sounds insultingly basic until the new kernel cannot find its root filesystem and the old entry is the difference between five minutes and a rescue disk.

My first checklist is mostly userspace:

module-init-tools
recent modutils-compatible configuration
procps that understands the newer /proc output
filesystem tools matching every filesystem in use
working boot loader configuration

The module change is the one most likely to surprise people. Linux 2.6 uses a new module format and the module-init-tools programs. They can coexist with the old tools, so one installation can boot both 2.4 and 2.6. Check this before rebooting, not while staring at an emergency shell.

I also build the drivers needed to reach the root filesystem directly into the kernel for the first attempt. That means the disk controller, root filesystem, and anything else between the kernel and /sbin/init. An initial ramdisk is useful, but removing it from the first experiment makes failures much easier to understand.

The device model has changed substantially. A mounted sysfs, normally at /sys, exposes devices and drivers in a structured tree. Old scripts that scrape /proc or assume particular module names deserve suspicion. Sound users should note that ALSA is now in the kernel tree; OSS compatibility exists, but that does not make every mixer setup identical.

Finally, save the exact .config and the complete build output. make oldconfig is a convenient starting point, not proof that old choices still mean the same thing. Read every new question that affects storage, filesystems, networking, or the console.

The kernel has many attractive improvements: preemption, the O(1) scheduler, better threading support, and a much saner device model. None of them will comfort you if the machine does not boot. My plan is boring: test hardware, test userspace, test rollback, then become adventurous. Adventure is much nicer with a working fallback kernel.

I put packet bookkeeping into a network device’s interrupt handler because the information was conveniently available there. Under load, the machine reminded me that convenience inside interrupt context is borrowed at unreasonable interest.

The interrupt handler should acknowledge the device, collect enough status to know what happened, move completed descriptors along, and arrange deferred processing where appropriate. It cannot sleep, and time spent there delays other interrupt work. Long parsing, allocation loops, or diagnostic printing are especially poor guests.

Linux 2.4 offers deferred mechanisms including tasklets and the network receive path’s softirq processing. A driver can prepare received skbs and pass them to netif_rx(), allowing the protocol stack to process packets outside the hard interrupt handler. For driver-specific deferred work, I put the tasklet in the device’s private state:

struct sample_dev {
        struct tasklet_struct rx_tasklet;
        /* rings, locks, and device state */
};

static void sample_tasklet(unsigned long data)
{
        struct sample_dev *dev = (struct sample_dev *)data;

        service_completed_packets(dev);
}

static void sample_init_tasklet(struct sample_dev *dev)
{
        tasklet_init(&dev->rx_tasklet, sample_tasklet,
                     (unsigned long)dev);
}

sample_init_tasklet() runs after dev is allocated and before the interrupt can schedule dev->rx_tasklet. Teardown disables that interrupt source, calls tasklet_kill(&dev->rx_tasklet), and only then frees dev. The callback therefore never receives the null value that a static DECLARE_TASKLET(..., 0) would have supplied.

Shared rings and status fields need protection across the interrupt, deferred function, transmit entry point, and process-context control operations. I use spinlocks where the data can be touched in interrupt context and choose the interrupt-disabling variant when the same processor could otherwise interrupt a lock holder. I keep the protected section short and never call a function that can sleep while holding it.

There is a balance. Deferring every tiny operation adds overhead and can complicate ordering, while doing everything in the handler harms latency. I measure under sustained receive and transmit load, watch dropped packets, and check that another device’s interrupts are not starved.

Interrupt acknowledgement order is device-specific and worth documenting beside the handler. A level-triggered interrupt that remains asserted can immediately enter again; acknowledging too early on other hardware may lose status that has not been copied. I read the device manual rather than deriving this order from whichever experiment happened not to hang.

Shutdown receives the same scrutiny. The driver disables the device’s interrupt source, prevents new deferred work, frees the IRQ with the required synchronization, and only then releases rings and private data. Otherwise a late tasklet can faithfully process memory that has already acquired a new purpose.

I want the hard interrupt path to be visibly bounded. One caveat: printk can completely distort timing here. A hundred helpful messages per second soon become the performance problem, a promotion few debug statements deserve.