Re: [PATCH 1/3] net: introduce a list of device addresses dev_addr_list (v3)

Eric Dumazet
2009-04-18T08:09:00Z

Jiri Pirko a écrit :
> Sat, Apr 18, 2009 at 09:35:32AM CEST, dada1@cosmosbay.com wrote:
>> Jiri Pirko a écrit :
>>> Fri, Apr 17, 2009 at 05:33:15PM CEST, shemminger@vyatta.com wrote:
>>>
>>> <snip>
>>>
>>>>> +struct netdev_hw_addr {
>>>>> + struct list_head list;
>>>>> + unsigned char addr[MAX_ADDR_LEN];
>>>>> + int refcount;
>>>>> + struct rcu_head rcu_head;
>>>>> +};
>>>> Minor nit, the ordering of elements cause holes that might not be
>>>> needed.
>>> Agree that ordering might be done better. Will do.
>>>> Space saving? is rcu_head needed or would using synchronize_net
>>>> make code cleaner and save space.
>>>>
>>> Well I originaly had this done by synchronize_rcu(). Eric argued that it might
>>> cause especially __hw_addr_del_multiple_ii() to run long and suggested to use
>>> call_rcu() instead. I plan to switch this to kfree_rcu() (or whatever it's
>>> called) once it hits the tree.
>>>
>> Yes, and dont forget we wont save space, as we allocate a full
>> cache line to hold a 'struct netdev_hw_addr', since we dont want this
>> critical and read_mostly object polluted by a hot spot elsewhere in kernel...
>>
>> Considering this, letting 'rcu_head' at the end of structure, even if we
>> have an eventual hole on 64 bit arches is not really a problem, and IMHO
>> the best thing to do, as rcu_head is only used at dismantle time.
>
> I will order the struct better, there are archs with small cache line size where
> it makes sense.

How exactly ?

If you consider a 32bit arch with 16 or 32 bytes cache line,
sizeof(struct_list_dead) is 8
sizeof(addr) = 32 (but we really use 6 bytes for ethernet)

struct netdev_hw_addr {
unsigned char addr[MAX_ADDR_LEN];
struct list_head list;
int refcount;
struct rcu_head rcu_head;
};

would cost more at lookup time, since we would use two cache lines

struct netdev_hw_addr {
struct list_head list;
unsigned char addr[MAX_ADDR_LEN];
int refcount;
struct rcu_head rcu_head;
};

Is nicer, because at least 8 bytes of addr share the same cache line
than list. So direct dev->dev_addr would be fast (for devices with one
address), and is_etherdev_addr() would still use one cache line per
item.