Re: [PATCH net-next] net-device: move gso_partial_features to net_device_read_tx

[Simon Horman <horms@kernel.org>]

On Thu, Dec 21, 2023 at 02:07:47PM +0000, Eric Dumazet wrote:
> dev->gso_partial_features is read from tx fast path for GSO packets.
> 
> Move it to appropriate section to avoid a cache line miss.
> 
> Fixes: 43a71cd66b9c ("net-device: reorganize net_device fast path variables")
> Signed-off-by: Eric Dumazet <edumazet@google.com>
> Cc: Coco Li <lixiaoyan@google.com>
> Cc: David Ahern <dsahern@kernel.org>

Thanks Eric,

FWIIW, this change looks good to me.

Reviewed-by: Simon Horman <horms@kernel.org>

I have a follow-up question below.

...

> diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> index 75c7725e5e4fdf59da55923cd803e084956b0fa0..5d1ec780122919c31e4215358d736aef3f8a0acd 100644
> --- a/include/linux/netdevice.h
> +++ b/include/linux/netdevice.h
> @@ -2114,6 +2114,7 @@ struct net_device {
>  	const struct net_device_ops *netdev_ops;
>  	const struct header_ops *header_ops;
>  	struct netdev_queue	*_tx;
> +	netdev_features_t	gso_partial_features;
>  	unsigned int		real_num_tx_queues;
>  	unsigned int		gso_max_size;
>  	unsigned int		gso_ipv4_max_size;

While looking at this I came to wonder if it would
be worth adding a 16bit pad a little below this hunk
so that tc_to_txq sits on it's own cacheline.

I'm unsure if the access pattern of tc_to_txq makes this worthwhile.
But if so it would be a simple tweak.

With such a change in place, on top of your patch, the diff of pahole output
on x86_64 is:

@@ -7432,10 +7432,9 @@
        s16                        num_tc;               /*    54     2 */
        unsigned int               mtu;                  /*    56     4 */
        short unsigned int         needed_headroom;      /*    60     2 */
-       struct netdev_tc_txq       tc_to_txq[16];        /*    62    64 */
-
-       /* XXX 2 bytes hole, try to pack */
-
+       u16                        pad1;                 /*    62     2 */
+       /* --- cacheline 1 boundary (64 bytes) --- */
+       struct netdev_tc_txq       tc_to_txq[16];        /*    64    64 */
        /* --- cacheline 2 boundary (128 bytes) --- */
        struct xps_dev_maps *      xps_maps[2];          /*   128    16 */
        struct nf_hook_entries *   nf_hooks_egress;      /*   144     8 */