Re: [PATCH net-next] netdevice: define and allocate &net_device _properly_

[Alexander Lobakin <aleksander.lobakin@intel.com>]

From: Eric Dumazet <edumazet@google.com>
Date: Tue, 7 May 2024 20:21:58 +0200

> On Tue, May 7, 2024 at 2:40 PM Alexander Lobakin
> <aleksander.lobakin@intel.com> wrote:
>>
>> In fact, this structure contains a flexible array at the end, but
>> historically its size, alignment etc., is calculated manually.
>> There are several instances of the structure embedded into other
>> structures, but also there's ongoing effort to remove them and we
>> could in the meantime declare &net_device properly.
>> Declare the array explicitly, use struct_size() and store the array
>> size inside the structure, so that __counted_by() can be applied.
>> Don't use PTR_ALIGN(), as SLUB itself tries its best to ensure the
>> allocated buffer is aligned to what the user expects.
>> Also, change its alignment from %NETDEV_ALIGN to the cacheline size
>> as per several suggestions on the netdev ML.
>>
>> bloat-o-meter for vmlinux:
>>
>> free_netdev                                  445     440      -5
>> netdev_freemem                                24       -     -24
>> alloc_netdev_mqs                            1481    1450     -31
>>
>> On x86_64 with several NICs of different vendors, I was never able to
>> get a &net_device pointer not aligned to the cacheline size after the
>> change.
>>
>> Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com>
>> ---
> 
> ...
> 
>> -       p = kvzalloc(alloc_size, GFP_KERNEL_ACCOUNT | __GFP_RETRY_MAYFAIL);
>> -       if (!p)
>> +       sizeof_priv = ALIGN(sizeof_priv, SMP_CACHE_BYTES);
> 
> If we have a __counted_by(priv_len), why do you ALIGN(sizeof_priv,
> SMP_CACHE_BYTES) ?

To have the whole block of &net_device + private part aligned to a
cacheline size.

> 
> If a driver pretends its private part is 4 bytes, we should get a
> warning if 20 bytes are used instead.

Ah okay, so this should be

	p = kvzalloc(struct_size(ALIGN(sizeof_priv, SMP_CACHE_BYTES)));
	p->priv_len = sizeof_priv;

> 
> You added two ____cacheline_aligned already in net_device already.

The whole size passed to kvzalloc() must be cacheline-aligned, otherwise
the MM layer can miscalculate the pointer alignment.

Thanks,
Olek