Cgroups Are Accounting Before Containment

Thu, 22 Jan 2009 20:16:00 +0000

One of two batch jobs was eating the workstation, but their worker processes came and went too quickly for top to give me a useful total. Watching individual PIDs was the wrong level. I wanted the CPU bill for each job and all of its children.

The cpuacct cgroup controller does that accounting. I put each launcher in its own group before it forked workers. The children inherited the group, so short-lived compiler and helper processes charged CPU time to the same job without a polling script trying to catch them.

cpuacct.usage gives accumulated CPU time in nanoseconds. Reading it before and after a run gave me a simple total for the whole process tree. cpuacct.stat split the charge into user and system time in clock ticks, which showed that the troublesome job was spending most of its time in user code rather than hammering the kernel.

The counters are cumulative, so I record a starting value or create a fresh group for a run. A single reading is not a percentage. Dividing the delta by elapsed wall time gives a rough average, and a multithreaded job can legitimately consume more than one CPU-second per second on a multicore machine.

This also avoids a trap in process snapshots. If a parent exits after launching workers, summing only its current descendants misses work already completed. The group’s counter keeps the charge after those tasks are gone.

Accounting did not slow either job or reserve a processor. That was fine. It showed that the job I had blamed was innocent and the other one used nearly three times the CPU I expected. Only then did scheduler policy seem worth discussing.

For now I am leaving the groups as meters. A reliable number is already an improvement over glaring at whichever PID happens to be at the top of top.

Trying Control Groups After 2.6.24

Fri, 04 Apr 2008 19:15:00 +0000

I wanted a compile job and its helper processes to stop overwhelming everything else on my test machine. Renicing the shell helped until the build launched processes with behavior I had not accounted for, and it did nothing about memory consumption.

Linux 2.6.24 includes the control groups framework, so I enabled the relevant options and mounted a control-group filesystem for testing. My first attempt was to write the compiler’s process ID into a group after the build had started. Some children were already elsewhere, and the result was inconsistent enough to look supernatural.

A control group is a hierarchy used to classify tasks. Controllers attach particular resource policies or accounting to that hierarchy. The framework itself is not one universal “make this process small” knob. CPU scheduling, processor sets and other controllers expose different files and semantics. A task belongs to a group in a hierarchy, and children normally inherit membership, so classification should happen before launching the workload.

I created a group, moved my test shell into it, and then started the build from that shell. Its descendants appeared in the same group. Applying CPU scheduling controls made the machine more responsive under contention without changing every process individually. Memory control is a separate matter and depends on kernel configuration; enabling it also has overhead that should not be waved away.

The hierarchy is powerful but easy to design badly. Groups may represent users, services or workloads, and those choices do not always nest neatly. Multiple hierarchies can attach different controllers, which provides flexibility at the cost of another opportunity to create an administrative puzzle.

For now I regard cgroups as kernel machinery for building resource-management policy, not as a finished desktop feature. The interface is low-level and sharp-edged. Tools will need to create groups, place tasks early, apply limits coherently and clean up afterward.

My practical conclusion is to start with one measured problem and one controller. Verify membership before interpreting results, and do not confuse accounting with enforcement. Cgroups can organize a family of processes in a way nice alone cannot. They can also organize one’s mistakes into a tidy hierarchy, which is progress of a sort.

Cgroups on Roberto Selbach

Cgroups Are Accounting Before Containment

Trying Control Groups After 2.6.24