NMIs, Locks, linked lists, barriers, and rculists, oh my!

[arlie@worldash.org (Arlie Stephens)]

Hi Michi,

These all look like excellent points. Thank you for responding. 

On Apr 04 2013, michi1 at michaelblizek.twilightparadox.com wrote:
> On 19:08 Wed 03 Apr     , Arlie Stephens wrote:
> ...
> > Put another way, my problem is that I want the readers to be able to
> > do their thing regardless of the state of the lock that serializes
> > writers. 
> > 
> > In all cases, writers are protected from each other with a mutex. 
> > 
> > Here are some choices for making sure readers see something sensible:
> > 
> > 1) Use an rculist, and hope that it really does cover this
> > situation. 
> 
> rculist might cover this case. The question is whether you can aquire read
> locks inside panic(), NMI, ... However, you should be able to skip the read
> lock, if you never free the contents.

If I understand correctly, if you want to deal with the NMI case, you
are even more restricted than if you simply want to be called in
interrupt context. The protection from interrupts that come with
spinlocks doesn't help you in the case of NMI. So all you can use are
trylock primitives - get the lock if available, otherwise fail. 

Fortunately I believe the rcu system doesn't need any locks on the
reader side. (If I'm wrong about that, than I'd better stop reading
the docs and start reading the code :-) I haven't actually looked at
code any lower than rculist.h) 
> 
> > 2) Be thankful I'm only running on x86,
> 
> Is there any good reason for limiting your code to x86? "My machine is x86 and
> I do not care about yours" will probably not convince maintainers.

That's part of my working context. I'm writing code for a family of
network appliances running scientific linux 6.1 and earlier
versions. They are all x86. And the code I'm attempting to enhance
addresses a problem that's been addressed upstream with a rather
different design. 

If we didn't already have working code, I'd convince the upstream
patches to apply to 2.6.32 and earlier kernels (possibly painful),
add support for our hardware, fix whichever of its issues bit us, 
add the additional features we need, and then try to upstream the
lot - preferably in small patches. 

And if the upstream code handled NMIs well I'd copy its design. 

> > hope that the linux kernel isn't built with aggressive optimizations that
> > reorder writes to pointers,
> 
> There is no guarantee for that.
> 
> > fix __list_add() so it updates pointers in the new entry before updating
> > pointers in its neighbours,
> 
> IMHO, this is like asking for trouble. Somebody else might change the order
> in the future without knowing it might break your stuff. Also, if you have bad
> luck, somebody else might already depend on the current order without you
> knowing.

It's notable that __list_add_rcu(), from rculist.h has the order the
way I want it, even in our elderly kernel version. 

> You should at least do atomic_read()+atomic_set(). RCU is exactly that plus
> a mechanism to free memory after all readers have finished.

Now I'm confused. It seems to me that part of RCU's contribution is
use of barrier instructions (compiler and run-time) at appropriate
points. Are you saying that atomic_read() and atomic_set() have
implicit barriers? 

--
Arlie