Re: [patch 04/41] cpu ops: Core piece for generic atomic per cpu operations

Christoph Lameter
2008-05-30T18:00:54Z

On Fri, 30 May 2008, Rusty Russell wrote:

> > No its not! In order to increment a per cpu value you need to calculate
> > the per cpu pointer address in the current per cpu segment.
>
> Christoph, you just missed it, that's all. Look at cpu_local_read et al in
> include/asm-i386/local.h (ie. before the x86 mergers chose the lowest common
> denominator one).

There is no doubt that local_t does perform an atomic vs. interrupt inc
for example. But its not usable. Because you need to determine the address
of the local_t belonging to the current processor first. As soon as you
have loaded a processor specific address you can no longer be preempted
because that may change the processor and then the wrong address may be
increment (and then we have a race again since now we are incrementing
counters belonging to other processors). So local_t at mininum requires
disabling preempt.

Believe me I have tried to use local_t repeatedly for vm statistics etc.
It always fails on that issue. See f.e. the patch that converts vmstat to
cpu alloc and compare with my initial local_t based implementation 2 years
ago that bombed out because I assumed that local_t would work right.

cpu ops does both

1. The determination of the address of the object belonging to the local
processor.

and

2. The RMW

in one instruction. That avoids having to disable preemption or interrupts
and it shortens the instructions significantly.