Re: [rfc][patch 3/3] use SLAB_ALIGN_SMP

Eric Dumazet
2008-03-03T13:01:17Z

Nick Piggin a écrit :
> On Mon, Mar 03, 2008 at 10:53:52AM +0100, Eric Dumazet wrote:
>
>> Nick Piggin a écrit :
>>
>>> Use SLAB_SMP_ALIGN in a few places.
>>>
>>>
>>>
>> I dont understand why you added SLAB_SMP_ALIGN, without removing
>> SLAB_HWCACHE_ALIGN on these places.
>>
>
> Because I thought that in most of the cases, we also want some cacheline
> alignment on UP systems as well because we care about the layout of the
> structure WRT the cachelines for the mandatory/capacity miss cases, as
> well as wanting to avoid false sharing misses on SMP.
>
> Actually I didn't think _too_ hard about them, possibly some could be
> removed. But the problem is that these things do require careful
> thought so I should not change them unless I have done that ;)
>
> I guess there are some basic guidelines -- if size is a problem (ie if
> there can be lots of these structures), then that is going to be a
> factor; if the total pool of objects is likely to be fairly densely
> resident in cache, then it will start to favour dense packing rather
> than good alignment.
>
>
Well, if a kmem_cache_create() is used, this is probably because number
of objects can be large, so kmalloc() power-of-two granularity could
waste lot of ram.

But yes, you are right that SLAB_SMP_ALIGN doesnt imply SLAB_HWCACHE_ALIGN

- SMP_ALIGN is a hint about false sharing (when object contains a refcnt
for example), that is a concern only if

num_possible_cpus() > 1

While HWCACHE_ALIGN might be a hint saying :
- The writer carefully designed the structure so that max performance is
obtained when all objects starts on a cache line boundary, even on
Uniprocessor.

But I suspect some uses of HWCACHE_ALIGN are not a hint but a strong
requirement.

Maybe we need to use three flags to separate the meanings ?

SLAB_HINT_SMP_ALIGN
SLAB_HINT_HWCACHE_ALIGN
SLAB_HWCACHE_ALIGN /* strong requirement that two objects dont share a
cache line */