* Re: [PATCH net-next] virtio-net: Set needed_headroom for virtio-net when VIRTIO_F_ANY_LAYOUT is true
2014-04-29 10:43 [PATCH net-next] virtio-net: Set needed_headroom for virtio-net when VIRTIO_F_ANY_LAYOUT is true Zhangjie (HZ)
@ 2014-04-30 1:44 ` Rusty Russell
2014-04-30 9:38 ` Jason Wang
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Rusty Russell @ 2014-04-30 1:44 UTC (permalink / raw)
To: Zhangjie (HZ), netdev, linux-kernel
Cc: davem, linville, jeffrey.t.kirsher, mst, jasowang, qinchuanyu,
liuyongan, zhangjie14, zhaoshenglong
"Zhangjie (HZ)" <zhangjie14@huawei.com> writes:
> This is a small supplement for commit e7428e95a06fb516fac1308bd0e176e27c0b9287
> ("virtio-net: put virtio-net header inline with data"). TCP packages have
> enough room to put virtio-net header in, but UDP packages do not. By
> setting dev->needed_headroom for virtio-net device, UDP packages could have
> enough room.
>
> For UDP packages, sk_buff is alloced in fun __ip_append_data. The size is
> "alloclen + hh_len + 15", and "hh_len = LL_RESERVED_SPACE(rt-dst.dev);".
> The Macro is defined as follows:
> #define LL_RESERVED_SPACE(dev) \
> ((((dev)->hard_header_len+(dev)->needed_headroom)\
> &~(HH_DATA_MOD - 1)) + HH_DATA_MOD)
> By default, for UDP packages, after skb is allocated, only 16 bytes
> reserved. And 2 bytes remained after mac header is set. That is not enough
> to put virtio-net header in. If we set dev->needed_headroom to 12 or 10
> (according to mergeable_rx_bufs is on or off ), more room can be reserved.
> Then there is enough room for UDP packages to put the header in.
Seems like a no-brainer. Good find!
Acked-by: Rusty Russell <rusty@rustcorp.com.au>
Thanks,
Rusty.
>
> test result list as below:
> guest and host: suse11sp3, netperf, intel 2.4GHz
> +-------+---------+---------+---------+---------+
> | | old | new |
> +-------+---------+---------+---------+---------+
> | UDP | Gbit/s | pps | Gbit/s | pps |
> | 64 | 0.57 | 692232 | 0.61 | 742420 |
> | 256 | 1.60 | 686860 | 1.71 | 733331 |
> | 512 | 2.92 | 674576 | 3.07 | 710446 |
> | 1024 | 4.99 | 598977 | 5.17 | 620821 |
> | 1460 | 5.68 | 483757 | 7.16 | 610519 |
> | 4096 | 6.98 | 637468 | 7.21 | 658471 |
> +-------+---------+---------+---------+---------+
>
> Signed-off-by: Zhang Jie <zhangjie14@huawei.com>
> ---
> drivers/net/virtio_net.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index 8a852b5..b852bb3 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -1724,6 +1724,13 @@ static int virtnet_probe(struct virtio_device *vdev)
> if (virtio_has_feature(vdev, VIRTIO_NET_F_CTRL_VQ))
> vi->has_cvq = true;
>
> + if (vi->any_header_sg) {
> + if (vi->mergeable_rx_bufs)
> + dev->needed_headroom = sizeof(struct virtio_net_hdr_mrg_rxbuf);
> + else
> + dev->needed_headroom = sizeof(struct virtio_net_hdr);
> + }
> +
> /* Use single tx/rx queue pair as default */
> vi->curr_queue_pairs = 1;
> vi->max_queue_pairs = max_queue_pairs;
> --
> 1.9.0.msysgit.0
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH net-next] virtio-net: Set needed_headroom for virtio-net when VIRTIO_F_ANY_LAYOUT is true
2014-04-29 10:43 [PATCH net-next] virtio-net: Set needed_headroom for virtio-net when VIRTIO_F_ANY_LAYOUT is true Zhangjie (HZ)
2014-04-30 1:44 ` Rusty Russell
@ 2014-04-30 9:38 ` Jason Wang
2014-04-30 10:19 ` Michael S. Tsirkin
2014-04-30 17:32 ` David Miller
3 siblings, 0 replies; 5+ messages in thread
From: Jason Wang @ 2014-04-30 9:38 UTC (permalink / raw)
To: Zhangjie (HZ), netdev, linux-kernel
Cc: davem, linville, jeffrey.t.kirsher, rusty, mst, qinchuanyu,
liuyongan, zhaoshenglong
On 04/29/2014 06:43 PM, Zhangjie (HZ) wrote:
> This is a small supplement for commit e7428e95a06fb516fac1308bd0e176e27c0b9287
> ("virtio-net: put virtio-net header inline with data"). TCP packages have
> enough room to put virtio-net header in, but UDP packages do not. By
> setting dev->needed_headroom for virtio-net device, UDP packages could have
> enough room.
>
> For UDP packages, sk_buff is alloced in fun __ip_append_data. The size is
> "alloclen + hh_len + 15", and "hh_len = LL_RESERVED_SPACE(rt-dst.dev);".
> The Macro is defined as follows:
> #define LL_RESERVED_SPACE(dev) \
> ((((dev)->hard_header_len+(dev)->needed_headroom)\
> &~(HH_DATA_MOD - 1)) + HH_DATA_MOD)
> By default, for UDP packages, after skb is allocated, only 16 bytes
> reserved. And 2 bytes remained after mac header is set. That is not enough
> to put virtio-net header in. If we set dev->needed_headroom to 12 or 10
> (according to mergeable_rx_bufs is on or off ), more room can be reserved.
> Then there is enough room for UDP packages to put the header in.
>
> test result list as below:
> guest and host: suse11sp3, netperf, intel 2.4GHz
> +-------+---------+---------+---------+---------+
> | | old | new |
> +-------+---------+---------+---------+---------+
> | UDP | Gbit/s | pps | Gbit/s | pps |
> | 64 | 0.57 | 692232 | 0.61 | 742420 |
> | 256 | 1.60 | 686860 | 1.71 | 733331 |
> | 512 | 2.92 | 674576 | 3.07 | 710446 |
> | 1024 | 4.99 | 598977 | 5.17 | 620821 |
> | 1460 | 5.68 | 483757 | 7.16 | 610519 |
> | 4096 | 6.98 | 637468 | 7.21 | 658471 |
> +-------+---------+---------+---------+---------+
>
> Signed-off-by: Zhang Jie <zhangjie14@huawei.com>
> ---
> drivers/net/virtio_net.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index 8a852b5..b852bb3 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -1724,6 +1724,13 @@ static int virtnet_probe(struct virtio_device *vdev)
> if (virtio_has_feature(vdev, VIRTIO_NET_F_CTRL_VQ))
> vi->has_cvq = true;
>
> + if (vi->any_header_sg) {
> + if (vi->mergeable_rx_bufs)
> + dev->needed_headroom = sizeof(struct virtio_net_hdr_mrg_rxbuf);
> + else
> + dev->needed_headroom = sizeof(struct virtio_net_hdr);
> + }
> +
> /* Use single tx/rx queue pair as default */
> vi->curr_queue_pairs = 1;
> vi->max_queue_pairs = max_queue_pairs;
Acked-by: Jason Wang <jasowang@redhat.com>
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH net-next] virtio-net: Set needed_headroom for virtio-net when VIRTIO_F_ANY_LAYOUT is true
2014-04-29 10:43 [PATCH net-next] virtio-net: Set needed_headroom for virtio-net when VIRTIO_F_ANY_LAYOUT is true Zhangjie (HZ)
2014-04-30 1:44 ` Rusty Russell
2014-04-30 9:38 ` Jason Wang
@ 2014-04-30 10:19 ` Michael S. Tsirkin
2014-04-30 17:32 ` David Miller
3 siblings, 0 replies; 5+ messages in thread
From: Michael S. Tsirkin @ 2014-04-30 10:19 UTC (permalink / raw)
To: Zhangjie (HZ)
Cc: netdev, linux-kernel, davem, linville, jeffrey.t.kirsher, rusty,
jasowang, qinchuanyu, liuyongan, zhaoshenglong
On Tue, Apr 29, 2014 at 06:43:22PM +0800, Zhangjie (HZ) wrote:
> This is a small supplement for commit e7428e95a06fb516fac1308bd0e176e27c0b9287
> ("virtio-net: put virtio-net header inline with data"). TCP packages have
> enough room to put virtio-net header in, but UDP packages do not. By
> setting dev->needed_headroom for virtio-net device, UDP packages could have
> enough room.
>
> For UDP packages, sk_buff is alloced in fun __ip_append_data. The size is
> "alloclen + hh_len + 15", and "hh_len = LL_RESERVED_SPACE(rt-dst.dev);".
> The Macro is defined as follows:
> #define LL_RESERVED_SPACE(dev) \
> ((((dev)->hard_header_len+(dev)->needed_headroom)\
> &~(HH_DATA_MOD - 1)) + HH_DATA_MOD)
> By default, for UDP packages, after skb is allocated, only 16 bytes
> reserved. And 2 bytes remained after mac header is set. That is not enough
> to put virtio-net header in. If we set dev->needed_headroom to 12 or 10
> (according to mergeable_rx_bufs is on or off ), more room can be reserved.
> Then there is enough room for UDP packages to put the header in.
>
> test result list as below:
> guest and host: suse11sp3, netperf, intel 2.4GHz
> +-------+---------+---------+---------+---------+
> | | old | new |
> +-------+---------+---------+---------+---------+
> | UDP | Gbit/s | pps | Gbit/s | pps |
> | 64 | 0.57 | 692232 | 0.61 | 742420 |
> | 256 | 1.60 | 686860 | 1.71 | 733331 |
> | 512 | 2.92 | 674576 | 3.07 | 710446 |
> | 1024 | 4.99 | 598977 | 5.17 | 620821 |
> | 1460 | 5.68 | 483757 | 7.16 | 610519 |
> | 4096 | 6.98 | 637468 | 7.21 | 658471 |
> +-------+---------+---------+---------+---------+
>
> Signed-off-by: Zhang Jie <zhangjie14@huawei.com>
Thanks for your patience!
Acked-by: Michael S. Tsirkin <mst@redhat.com>
> ---
> drivers/net/virtio_net.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index 8a852b5..b852bb3 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -1724,6 +1724,13 @@ static int virtnet_probe(struct virtio_device *vdev)
> if (virtio_has_feature(vdev, VIRTIO_NET_F_CTRL_VQ))
> vi->has_cvq = true;
>
> + if (vi->any_header_sg) {
> + if (vi->mergeable_rx_bufs)
> + dev->needed_headroom = sizeof(struct virtio_net_hdr_mrg_rxbuf);
> + else
> + dev->needed_headroom = sizeof(struct virtio_net_hdr);
> + }
> +
> /* Use single tx/rx queue pair as default */
> vi->curr_queue_pairs = 1;
> vi->max_queue_pairs = max_queue_pairs;
> --
> 1.9.0.msysgit.0
>
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH net-next] virtio-net: Set needed_headroom for virtio-net when VIRTIO_F_ANY_LAYOUT is true
2014-04-29 10:43 [PATCH net-next] virtio-net: Set needed_headroom for virtio-net when VIRTIO_F_ANY_LAYOUT is true Zhangjie (HZ)
` (2 preceding siblings ...)
2014-04-30 10:19 ` Michael S. Tsirkin
@ 2014-04-30 17:32 ` David Miller
3 siblings, 0 replies; 5+ messages in thread
From: David Miller @ 2014-04-30 17:32 UTC (permalink / raw)
To: zhangjie14
Cc: netdev, linux-kernel, linville, jeffrey.t.kirsher, rusty, mst,
jasowang, qinchuanyu, liuyongan, zhaoshenglong
From: "Zhangjie (HZ)" <zhangjie14@huawei.com>
Date: Tue, 29 Apr 2014 18:43:22 +0800
> This is a small supplement for commit e7428e95a06fb516fac1308bd0e176e27c0b9287
> ("virtio-net: put virtio-net header inline with data"). TCP packages have
> enough room to put virtio-net header in, but UDP packages do not. By
> setting dev->needed_headroom for virtio-net device, UDP packages could have
> enough room.
>
> For UDP packages, sk_buff is alloced in fun __ip_append_data. The size is
> "alloclen + hh_len + 15", and "hh_len = LL_RESERVED_SPACE(rt-dst.dev);".
> The Macro is defined as follows:
> #define LL_RESERVED_SPACE(dev) \
> ((((dev)->hard_header_len+(dev)->needed_headroom)\
> &~(HH_DATA_MOD - 1)) + HH_DATA_MOD)
> By default, for UDP packages, after skb is allocated, only 16 bytes
> reserved. And 2 bytes remained after mac header is set. That is not enough
> to put virtio-net header in. If we set dev->needed_headroom to 12 or 10
> (according to mergeable_rx_bufs is on or off ), more room can be reserved.
> Then there is enough room for UDP packages to put the header in.
>
> test result list as below:
> guest and host: suse11sp3, netperf, intel 2.4GHz
> +-------+---------+---------+---------+---------+
> | | old | new |
> +-------+---------+---------+---------+---------+
> | UDP | Gbit/s | pps | Gbit/s | pps |
> | 64 | 0.57 | 692232 | 0.61 | 742420 |
> | 256 | 1.60 | 686860 | 1.71 | 733331 |
> | 512 | 2.92 | 674576 | 3.07 | 710446 |
> | 1024 | 4.99 | 598977 | 5.17 | 620821 |
> | 1460 | 5.68 | 483757 | 7.16 | 610519 |
> | 4096 | 6.98 | 637468 | 7.21 | 658471 |
> +-------+---------+---------+---------+---------+
>
> Signed-off-by: Zhang Jie <zhangjie14@huawei.com>
Applied.
^ permalink raw reply [flat|nested] 5+ messages in thread