* Re: [PATCH bpf-next v2 0/3] xsk: build skb by page [not found] <cover.1611048724.git.xuanzhuo@linux.alibaba.com> @ 2021-01-19 9:50 ` Michael S. Tsirkin [not found] ` <1611053609.502882-1-xuanzhuo@linux.alibaba.com> [not found] ` <30ae1c94b5c26919bd90bb251761c526edfbaf56.1611048724.git.xuanzhuo@linux.alibaba.com> 1 sibling, 1 reply; 3+ messages in thread From: Michael S. Tsirkin @ 2021-01-19 9:50 UTC (permalink / raw) To: Xuan Zhuo Cc: Song Liu, Martin KaFai Lau, Jesper Dangaard Brouer, Daniel Borkmann, netdev, John Fastabend, Alexei Starovoitov, Andrii Nakryiko, Yonghong Song, Jonathan Lemon, KP Singh, Jakub Kicinski, bpf, Björn Töpel, virtualization, David S. Miller, Magnus Karlsson On Tue, Jan 19, 2021 at 05:45:09PM +0800, Xuan Zhuo wrote: > v2: > 1. add priv_flags IFF_TX_SKB_NO_LINEAR instead of netdev_feature > 2. split the patch to three: > a. add priv_flags IFF_TX_SKB_NO_LINEAR > b. virtio net add priv_flags IFF_TX_SKB_NO_LINEAR > c. When there is support this flag, construct skb without linear space > 3. use ERR_PTR() and PTR_ERR() to handle the err > > > v1 message log: > --------------- > > This patch is used to construct skb based on page to save memory copy > overhead. > > This has one problem: > > We construct the skb by fill the data page as a frag into the skb. In > this way, the linear space is empty, and the header information is also > in the frag, not in the linear space, which is not allowed for some > network cards. For example, Mellanox Technologies MT27710 Family > [ConnectX-4 Lx] will get the following error message: > > mlx5_core 0000:3b:00.1 eth1: Error cqe on cqn 0x817, ci 0x8, qn 0x1dbb, opcode 0xd, syndrome 0x1, vendor syndrome 0x68 > 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > 00000030: 00 00 00 00 60 10 68 01 0a 00 1d bb 00 0f 9f d2 > WQE DUMP: WQ size 1024 WQ cur size 0, WQE index 0xf, len: 64 > 00000000: 00 00 0f 0a 00 1d bb 03 00 00 00 08 00 00 00 00 > 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > 00000020: 00 00 00 2b 00 08 00 00 00 00 00 05 9e e3 08 00 > 00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > mlx5_core 0000:3b:00.1 eth1: ERR CQE on SQ: 0x1dbb > > I also tried to use build_skb to construct skb, but because of the > existence of skb_shinfo, it must be behind the linear space, so this > method is not working. We can't put skb_shinfo on desc->addr, it will be > exposed to users, this is not safe. > > Finally, I added a feature NETIF_F_SKB_NO_LINEAR to identify whether the > network card supports the header information of the packet in the frag > and not in the linear space. > > ---------------- Performance Testing ------------ > > The test environment is Aliyun ECS server. > Test cmd: > ``` > xdpsock -i eth0 -t -S -s <msg size> > ``` > > Test result data: > > size 64 512 1024 1500 > copy 1916747 1775988 1600203 1440054 > page 1974058 1953655 1945463 1904478 > percent 3.0% 10.0% 21.58% 32.3% Just making sure, are these test results with v2? > > Xuan Zhuo (3): > net: add priv_flags for allow tx skb without linear > virtio-net: support IFF_TX_SKB_NO_LINEAR > xsk: build skb by page > > drivers/net/virtio_net.c | 3 +- > include/linux/netdevice.h | 3 ++ > net/xdp/xsk.c | 112 ++++++++++++++++++++++++++++++++++++++-------- > 3 files changed, 99 insertions(+), 19 deletions(-) > > -- > 1.8.3.1 _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 3+ messages in thread
[parent not found: <1611053609.502882-1-xuanzhuo@linux.alibaba.com>]
* Re: [PATCH bpf-next v2 0/3] xsk: build skb by page [not found] ` <1611053609.502882-1-xuanzhuo@linux.alibaba.com> @ 2021-01-19 11:02 ` Michael S. Tsirkin 0 siblings, 0 replies; 3+ messages in thread From: Michael S. Tsirkin @ 2021-01-19 11:02 UTC (permalink / raw) To: Xuan Zhuo Cc: Song Liu, Martin KaFai Lau, Jesper Dangaard Brouer, Daniel Borkmann, netdev, John Fastabend, Alexei Starovoitov, Andrii Nakryiko, Yonghong Song, Jonathan Lemon, KP Singh, Jakub Kicinski, bpf, Björn Töpel, virtualization, David S. Miller, Magnus Karlsson On Tue, Jan 19, 2021 at 06:53:29PM +0800, Xuan Zhuo wrote: > On Tue, 19 Jan 2021 04:50:30 -0500, Michael S. Tsirkin <mst@redhat.com> wrote: > > On Tue, Jan 19, 2021 at 05:45:09PM +0800, Xuan Zhuo wrote: > > > v2: > > > 1. add priv_flags IFF_TX_SKB_NO_LINEAR instead of netdev_feature > > > 2. split the patch to three: > > > a. add priv_flags IFF_TX_SKB_NO_LINEAR > > > b. virtio net add priv_flags IFF_TX_SKB_NO_LINEAR > > > c. When there is support this flag, construct skb without linear space > > > 3. use ERR_PTR() and PTR_ERR() to handle the err > > > > > > > > > v1 message log: > > > --------------- > > > > > > This patch is used to construct skb based on page to save memory copy > > > overhead. > > > > > > This has one problem: > > > > > > We construct the skb by fill the data page as a frag into the skb. In > > > this way, the linear space is empty, and the header information is also > > > in the frag, not in the linear space, which is not allowed for some > > > network cards. For example, Mellanox Technologies MT27710 Family > > > [ConnectX-4 Lx] will get the following error message: > > > > > > mlx5_core 0000:3b:00.1 eth1: Error cqe on cqn 0x817, ci 0x8, qn 0x1dbb, opcode 0xd, syndrome 0x1, vendor syndrome 0x68 > > > 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > > > 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > > > 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > > > 00000030: 00 00 00 00 60 10 68 01 0a 00 1d bb 00 0f 9f d2 > > > WQE DUMP: WQ size 1024 WQ cur size 0, WQE index 0xf, len: 64 > > > 00000000: 00 00 0f 0a 00 1d bb 03 00 00 00 08 00 00 00 00 > > > 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > > > 00000020: 00 00 00 2b 00 08 00 00 00 00 00 05 9e e3 08 00 > > > 00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > > > mlx5_core 0000:3b:00.1 eth1: ERR CQE on SQ: 0x1dbb > > > > > > I also tried to use build_skb to construct skb, but because of the > > > existence of skb_shinfo, it must be behind the linear space, so this > > > method is not working. We can't put skb_shinfo on desc->addr, it will be > > > exposed to users, this is not safe. > > > > > > Finally, I added a feature NETIF_F_SKB_NO_LINEAR to identify whether the > > > network card supports the header information of the packet in the frag > > > and not in the linear space. > > > > > > ---------------- Performance Testing ------------ > > > > > > The test environment is Aliyun ECS server. > > > Test cmd: > > > ``` > > > xdpsock -i eth0 -t -S -s <msg size> > > > ``` > > > > > > Test result data: > > > > > > size 64 512 1024 1500 > > > copy 1916747 1775988 1600203 1440054 > > > page 1974058 1953655 1945463 1904478 > > > percent 3.0% 10.0% 21.58% 32.3% > > > > Just making sure, are these test results with v2? > > The data was tested at v1, > but v2 did not modify the performance-related code. > > Thanks. Looks like v1 wouldn't even build, or did I miss anything? It would be nicer if you retested it ... > > > > > > > > > Xuan Zhuo (3): > > > net: add priv_flags for allow tx skb without linear > > > virtio-net: support IFF_TX_SKB_NO_LINEAR > > > xsk: build skb by page > > > > > > drivers/net/virtio_net.c | 3 +- > > > include/linux/netdevice.h | 3 ++ > > > net/xdp/xsk.c | 112 ++++++++++++++++++++++++++++++++++++++-------- > > > 3 files changed, 99 insertions(+), 19 deletions(-) > > > > > > -- > > > 1.8.3.1 > > _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 3+ messages in thread
[parent not found: <30ae1c94b5c26919bd90bb251761c526edfbaf56.1611048724.git.xuanzhuo@linux.alibaba.com>]
* Re: [PATCH bpf-next v2 1/3] net: add priv_flags for allow tx skb without linear [not found] ` <30ae1c94b5c26919bd90bb251761c526edfbaf56.1611048724.git.xuanzhuo@linux.alibaba.com> @ 2021-01-20 3:01 ` Jason Wang 0 siblings, 0 replies; 3+ messages in thread From: Jason Wang @ 2021-01-20 3:01 UTC (permalink / raw) To: Xuan Zhuo, netdev Cc: Song Liu, Martin KaFai Lau, Jesper Dangaard Brouer, Daniel Borkmann, Michael S. Tsirkin, Yonghong Song, John Fastabend, Alexei Starovoitov, Andrii Nakryiko, Jonathan Lemon, KP Singh, Jakub Kicinski, bpf, Björn Töpel, virtualization, David S. Miller, Magnus Karlsson On 2021/1/19 下午5:45, Xuan Zhuo wrote: > In some cases, we hope to construct skb directly based on the existing > memory without copying data. In this case, the page will be placed > directly in the skb, and the linear space of skb is empty. But > unfortunately, many the network card does not support this operation. > For example Mellanox Technologies MT27710 Family [ConnectX-4 Lx] will > get the following error message: > > mlx5_core 0000:3b:00.1 eth1: Error cqe on cqn 0x817, ci 0x8, qn 0x1dbb, opcode 0xd, syndrome 0x1, vendor syndrome 0x68 > 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > 00000030: 00 00 00 00 60 10 68 01 0a 00 1d bb 00 0f 9f d2 > WQE DUMP: WQ size 1024 WQ cur size 0, WQE index 0xf, len: 64 > 00000000: 00 00 0f 0a 00 1d bb 03 00 00 00 08 00 00 00 00 > 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > 00000020: 00 00 00 2b 00 08 00 00 00 00 00 05 9e e3 08 00 > 00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > mlx5_core 0000:3b:00.1 eth1: ERR CQE on SQ: 0x1dbb > > So a priv_flag is added here to indicate whether the network card > supports this feature. I don't see Mellanox engineers are copied. I wonder if we need their confirmation on whether it's a bug or hardware limitation. Thanks _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-01-20 3:01 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <cover.1611048724.git.xuanzhuo@linux.alibaba.com>
2021-01-19 9:50 ` [PATCH bpf-next v2 0/3] xsk: build skb by page Michael S. Tsirkin
[not found] ` <1611053609.502882-1-xuanzhuo@linux.alibaba.com>
2021-01-19 11:02 ` Michael S. Tsirkin
[not found] ` <30ae1c94b5c26919bd90bb251761c526edfbaf56.1611048724.git.xuanzhuo@linux.alibaba.com>
2021-01-20 3:01 ` [PATCH bpf-next v2 1/3] net: add priv_flags for allow tx skb without linear Jason Wang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).