* Re: [PATCH bpf-next v3] virtio_net: add XDP meta data support
From: Daniel Borkmann @ 2019-07-08 22:38 UTC (permalink / raw)
To: Yuya Kusakabe, Jason Wang
Cc: ast, davem, hawk, jakub.kicinski, john.fastabend, kafai, mst,
netdev, songliubraving, yhs
In-Reply-To: <52e3fc0d-bdd7-83ee-58e6-488e2b91cc83@gmail.com>
On 07/02/2019 04:11 PM, Yuya Kusakabe wrote:
> On 7/2/19 5:33 PM, Jason Wang wrote:
>> On 2019/7/2 下午4:16, Yuya Kusakabe wrote:
>>> This adds XDP meta data support to both receive_small() and
>>> receive_mergeable().
>>>
>>> Fixes: de8f3a83b0a0 ("bpf: add meta pointer for direct access")
>>> Signed-off-by: Yuya Kusakabe <yuya.kusakabe@gmail.com>
>>> ---
>>> v3:
>>> - fix preserve the vnet header in receive_small().
>>> v2:
>>> - keep copy untouched in page_to_skb().
>>> - preserve the vnet header in receive_small().
>>> - fix indentation.
>>> ---
>>> drivers/net/virtio_net.c | 45 +++++++++++++++++++++++++++-------------
>>> 1 file changed, 31 insertions(+), 14 deletions(-)
>>>
>>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
>>> index 4f3de0ac8b0b..03a1ae6fe267 100644
>>> --- a/drivers/net/virtio_net.c
>>> +++ b/drivers/net/virtio_net.c
>>> @@ -371,7 +371,7 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi,
>>> struct receive_queue *rq,
>>> struct page *page, unsigned int offset,
>>> unsigned int len, unsigned int truesize,
>>> - bool hdr_valid)
>>> + bool hdr_valid, unsigned int metasize)
>>> {
>>> struct sk_buff *skb;
>>> struct virtio_net_hdr_mrg_rxbuf *hdr;
>>> @@ -393,7 +393,7 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi,
>>> else
>>> hdr_padded_len = sizeof(struct padded_vnet_hdr);
>>> - if (hdr_valid)
>>> + if (hdr_valid && !metasize)
>>> memcpy(hdr, p, hdr_len);
>>> len -= hdr_len;
>>> @@ -405,6 +405,11 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi,
>>> copy = skb_tailroom(skb);
>>> skb_put_data(skb, p, copy);
>>> + if (metasize) {
>>> + __skb_pull(skb, metasize);
>>> + skb_metadata_set(skb, metasize);
>>> + }
>>> +
>>> len -= copy;
>>> offset += copy;
>>> @@ -644,6 +649,7 @@ static struct sk_buff *receive_small(struct net_device *dev,
>>> unsigned int delta = 0;
>>> struct page *xdp_page;
>>> int err;
>>> + unsigned int metasize = 0;
>>> len -= vi->hdr_len;
>>> stats->bytes += len;
>>> @@ -683,10 +689,13 @@ static struct sk_buff *receive_small(struct net_device *dev,
>>> xdp.data_hard_start = buf + VIRTNET_RX_PAD + vi->hdr_len;
>>> xdp.data = xdp.data_hard_start + xdp_headroom;
>>> - xdp_set_data_meta_invalid(&xdp);
>>> xdp.data_end = xdp.data + len;
>>> + xdp.data_meta = xdp.data;
>>> xdp.rxq = &rq->xdp_rxq;
>>> orig_data = xdp.data;
>>> + /* Copy the vnet header to the front of data_hard_start to avoid
>>> + * overwriting by XDP meta data */
>>> + memcpy(xdp.data_hard_start - vi->hdr_len, xdp.data - vi->hdr_len, vi->hdr_len);
I'm not fully sure if I'm following this one correctly, probably just missing
something. Isn't the vnet header based on how we set up xdp.data_hard_start
earlier already in front of it? Wouldn't we copy invalid data from xdp.data -
vi->hdr_len into the vnet header at that point (given there can be up to 256
bytes of headroom between the two)? If it's relative to xdp.data and headroom
is >0, then BPF prog could otherwise mangle this; something doesn't add up to
me here. Could you clarify? Thx
>> What happens if we have a large metadata that occupies all headroom here?
>>
>> Thanks
>
> Do you mean a large "XDP" metadata? If a large metadata is a large "XDP" metadata, I think we can not use a metadata that occupies all headroom. The size of metadata limited by bpf_xdp_adjust_meta() as below.
> bpf_xdp_adjust_meta() in net/core/filter.c:
> if (unlikely((metalen & (sizeof(__u32) - 1)) ||
> (metalen > 32)))
> return -EACCES;
>
> Thanks.
>
>>
>>
>>> act = bpf_prog_run_xdp(xdp_prog, &xdp);
>>> stats->xdp_packets++;
>>> @@ -695,9 +704,11 @@ static struct sk_buff *receive_small(struct net_device *dev,
>>> /* Recalculate length in case bpf program changed it */
>>> delta = orig_data - xdp.data;
>>> len = xdp.data_end - xdp.data;
>>> + metasize = xdp.data - xdp.data_meta;
>>> break;
>>> case XDP_TX:
>>> stats->xdp_tx++;
>>> + xdp.data_meta = xdp.data;
>>> xdpf = convert_to_xdp_frame(&xdp);
>>> if (unlikely(!xdpf))
>>> goto err_xdp;
>>> @@ -736,10 +747,12 @@ static struct sk_buff *receive_small(struct net_device *dev,
>>> skb_reserve(skb, headroom - delta);
>>> skb_put(skb, len);
>>> if (!delta) {
>>> - buf += header_offset;
>>> - memcpy(skb_vnet_hdr(skb), buf, vi->hdr_len);
>>> + memcpy(skb_vnet_hdr(skb), buf + VIRTNET_RX_PAD, vi->hdr_len);
>>> } /* keep zeroed vnet hdr since packet was changed by bpf */
>>> + if (metasize)
>>> + skb_metadata_set(skb, metasize);
>>> +
>>> err:
>>> return skb;
>>> @@ -760,8 +773,8 @@ static struct sk_buff *receive_big(struct net_device *dev,
>>> struct virtnet_rq_stats *stats)
>>> {
>>> struct page *page = buf;
>>> - struct sk_buff *skb = page_to_skb(vi, rq, page, 0, len,
>>> - PAGE_SIZE, true);
>>> + struct sk_buff *skb =
>>> + page_to_skb(vi, rq, page, 0, len, PAGE_SIZE, true, 0);
>>> stats->bytes += len - vi->hdr_len;
>>> if (unlikely(!skb))
>>> @@ -793,6 +806,7 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
>>> unsigned int truesize;
>>> unsigned int headroom = mergeable_ctx_to_headroom(ctx);
>>> int err;
>>> + unsigned int metasize = 0;
>>> head_skb = NULL;
>>> stats->bytes += len - vi->hdr_len;
>>> @@ -839,8 +853,8 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
>>> data = page_address(xdp_page) + offset;
>>> xdp.data_hard_start = data - VIRTIO_XDP_HEADROOM + vi->hdr_len;
>>> xdp.data = data + vi->hdr_len;
>>> - xdp_set_data_meta_invalid(&xdp);
>>> xdp.data_end = xdp.data + (len - vi->hdr_len);
>>> + xdp.data_meta = xdp.data;
>>> xdp.rxq = &rq->xdp_rxq;
>>> act = bpf_prog_run_xdp(xdp_prog, &xdp);
>>> @@ -852,8 +866,9 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
>>> * adjustments. Note other cases do not build an
>>> * skb and avoid using offset
>>> */
>>> - offset = xdp.data -
>>> - page_address(xdp_page) - vi->hdr_len;
>>> + metasize = xdp.data - xdp.data_meta;
>>> + offset = xdp.data - page_address(xdp_page) -
>>> + vi->hdr_len - metasize;
>>> /* recalculate len if xdp.data or xdp.data_end were
>>> * adjusted
>>> @@ -863,14 +878,15 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
>>> if (unlikely(xdp_page != page)) {
>>> rcu_read_unlock();
>>> put_page(page);
>>> - head_skb = page_to_skb(vi, rq, xdp_page,
>>> - offset, len,
>>> - PAGE_SIZE, false);
>>> + head_skb = page_to_skb(vi, rq, xdp_page, offset,
>>> + len, PAGE_SIZE, false,
>>> + metasize);
>>> return head_skb;
>>> }
>>> break;
>>> case XDP_TX:
>>> stats->xdp_tx++;
>>> + xdp.data_meta = xdp.data;
>>> xdpf = convert_to_xdp_frame(&xdp);
>>> if (unlikely(!xdpf))
>>> goto err_xdp;
>>> @@ -921,7 +937,8 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
>>> goto err_skb;
>>> }
>>> - head_skb = page_to_skb(vi, rq, page, offset, len, truesize, !xdp_prog);
>>> + head_skb = page_to_skb(vi, rq, page, offset, len, truesize, !xdp_prog,
>>> + metasize);
>>> curr_skb = head_skb;
>>> if (unlikely(!curr_skb))
^ permalink raw reply
* Re: [PATCH v3 0/3] vsock/virtio: several fixes in the .probe() and .remove()
From: David Miller @ 2019-07-08 22:35 UTC (permalink / raw)
To: sgarzare; +Cc: netdev, kvm, mst, linux-kernel, jasowang, virtualization,
stefanha
In-Reply-To: <20190705110454.95302-1-sgarzare@redhat.com>
From: Stefano Garzarella <sgarzare@redhat.com>
Date: Fri, 5 Jul 2019 13:04:51 +0200
> During the review of "[PATCH] vsock/virtio: Initialize core virtio vsock
> before registering the driver", Stefan pointed out some possible issues
> in the .probe() and .remove() callbacks of the virtio-vsock driver.
...
Series applied to net-next, thanks.
^ permalink raw reply
* Re: [PATCH v3 0/2] Document the configuration of b53
From: David Miller @ 2019-07-08 22:30 UTC (permalink / raw)
To: b.spranger; +Cc: f.fainelli, netdev, bigeasy, kurt, andrew, vivien.didelot
In-Reply-To: <20190705095719.24095-1-b.spranger@linutronix.de>
From: Benedikt Spranger <b.spranger@linutronix.de>
Date: Fri, 5 Jul 2019 11:57:17 +0200
> this is the third round to document the configuration of a b53 supported
> switch.
Series applied.
There was some trailing whitespace which I took care of for you this
time.
Thanks.
^ permalink raw reply
* Re: [PATCH 0/2] forcedeth: recv cache support
From: David Miller @ 2019-07-08 22:23 UTC (permalink / raw)
To: yanjun.zhu; +Cc: netdev
In-Reply-To: <1562307568-21549-1-git-send-email-yanjun.zhu@oracle.com>
From: Zhu Yanjun <yanjun.zhu@oracle.com>
Date: Fri, 5 Jul 2019 02:19:26 -0400
> This recv cache is to make NIC work steadily when the system memory is
> not enough.
The system is supposed to hold onto enough atomic memory to absorb all
reasonable situations like this.
If anything a solution to this problem belongs generically somewhere,
not in a driver. And furthermore looping over an allocation attempt
with a delay is strongly discouraged.
^ permalink raw reply
* Re: [PATCH net-next v2 0/4] bnxt_en: Add XDP_REDIRECT support.
From: David Miller @ 2019-07-08 22:20 UTC (permalink / raw)
To: michael.chan; +Cc: gospo, netdev, hawk, ast, ilias.apalodimas
In-Reply-To: <1562622784-29918-1-git-send-email-michael.chan@broadcom.com>
From: Michael Chan <michael.chan@broadcom.com>
Date: Mon, 8 Jul 2019 17:53:00 -0400
> This patch series adds XDP_REDIRECT support by Andy Gospodarek.
Series applied, thanks everyone.
^ permalink raw reply
* Re: [PATCH v9 net-next 0/5] net: ethernet: ti: cpsw: Add XDP support
From: David Miller @ 2019-07-08 22:12 UTC (permalink / raw)
To: ivan.khoronzhuk
Cc: grygorii.strashko, hawk, ast, linux-kernel, linux-omap,
xdp-newbies, ilias.apalodimas, netdev, daniel, jakub.kicinski,
john.fastabend
In-Reply-To: <20190708213432.8525-1-ivan.khoronzhuk@linaro.org>
From: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
Date: Tue, 9 Jul 2019 00:34:27 +0300
> This patchset adds XDP support for TI cpsw driver and base it on
> page_pool allocator. It was verified on af_xdp socket drop,
> af_xdp l2f, ebpf XDP_DROP, XDP_REDIRECT, XDP_PASS, XDP_TX.
...
Series applied, thanks Ivan!
^ permalink raw reply
* Re: [PATCH v3 net-next 13/19] ionic: Add initial ethtool support
From: Andrew Lunn @ 2019-07-08 22:04 UTC (permalink / raw)
To: Shannon Nelson; +Cc: netdev
In-Reply-To: <20190708192532.27420-14-snelson@pensando.io>
> +static int ionic_get_link_ksettings(struct net_device *netdev,
> + struct ethtool_link_ksettings *ks)
> +{
> + struct lif *lif = netdev_priv(netdev);
> + struct ionic_dev *idev = &lif->ionic->idev;
> + int copper_seen = 0;
> +
> + ethtool_link_ksettings_zero_link_mode(ks, supported);
> + ethtool_link_ksettings_zero_link_mode(ks, advertising);
> +
> + switch (le16_to_cpu(idev->port_info->status.xcvr.pid)) {
> + /* Copper */
> + case XCVR_PID_QSFP_100G_CR4:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 100000baseCR4_Full);
> + copper_seen++;
> + break;
> + case XCVR_PID_QSFP_40GBASE_CR4:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 40000baseCR4_Full);
> + copper_seen++;
> + break;
> + case XCVR_PID_SFP_25GBASE_CR_S:
> + case XCVR_PID_SFP_25GBASE_CR_L:
> + case XCVR_PID_SFP_25GBASE_CR_N:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 25000baseCR_Full);
> + copper_seen++;
> + break;
> + case XCVR_PID_SFP_10GBASE_AOC:
> + case XCVR_PID_SFP_10GBASE_CU:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 10000baseCR_Full);
> + copper_seen++;
> + break;
> +
> + /* Fibre */
> + case XCVR_PID_QSFP_100G_SR4:
> + case XCVR_PID_QSFP_100G_AOC:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 100000baseSR4_Full);
> + break;
> + case XCVR_PID_QSFP_100G_LR4:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 100000baseLR4_ER4_Full);
> + break;
> + case XCVR_PID_QSFP_100G_ER4:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 100000baseLR4_ER4_Full);
> + break;
> + case XCVR_PID_QSFP_40GBASE_SR4:
> + case XCVR_PID_QSFP_40GBASE_AOC:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 40000baseSR4_Full);
> + break;
> + case XCVR_PID_QSFP_40GBASE_LR4:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 40000baseLR4_Full);
> + break;
> + case XCVR_PID_SFP_25GBASE_SR:
> + case XCVR_PID_SFP_25GBASE_AOC:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 25000baseSR_Full);
> + break;
> + case XCVR_PID_SFP_10GBASE_SR:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 10000baseSR_Full);
> + break;
> + case XCVR_PID_SFP_10GBASE_LR:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 10000baseLR_Full);
> + break;
> + case XCVR_PID_SFP_10GBASE_LRM:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 10000baseLRM_Full);
> + break;
> + case XCVR_PID_SFP_10GBASE_ER:
> + ethtool_link_ksettings_add_link_mode(ks, supported,
> + 10000baseER_Full);
> + break;
I don't know these link modes too well. But only setting a single bit
seems odd. What i do know is that an SFP which supports 2500BaseX
should also be able to support 1000BaseX. So should a 100G SFP also
support 40G, 25G, 10G etc? The SERDES just runs a slower bitstream
over the basic bitpipe?
> + case XCVR_PID_QSFP_100G_ACC:
> + case XCVR_PID_QSFP_40GBASE_ER4:
> + case XCVR_PID_SFP_25GBASE_LR:
> + case XCVR_PID_SFP_25GBASE_ER:
> + dev_info(lif->ionic->dev, "no decode bits for xcvr type pid=%d / 0x%x\n",
> + idev->port_info->status.xcvr.pid,
> + idev->port_info->status.xcvr.pid);
> + break;
Why not add them?
> + memcpy(ks->link_modes.advertising, ks->link_modes.supported,
> + sizeof(ks->link_modes.advertising));
bitmap_copy() would be a better way to do this. You could consider
adding a helper to ethtool.h.
Andrew
^ permalink raw reply
* [PATCH net-next v2 4/4] bnxt_en: add page_pool support
From: Michael Chan @ 2019-07-08 21:53 UTC (permalink / raw)
To: davem, gospo; +Cc: netdev, hawk, ast, ilias.apalodimas
In-Reply-To: <1562622784-29918-1-git-send-email-michael.chan@broadcom.com>
From: Andy Gospodarek <gospo@broadcom.com>
This removes contention over page allocation for XDP_REDIRECT actions by
adding page_pool support per queue for the driver. The performance for
XDP_REDIRECT actions scales linearly with the number of cores performing
redirect actions when using the page pools instead of the standard page
allocator.
v2: Fix up the error path from XDP registration, noted by Ilias Apalodimas.
Signed-off-by: Andy Gospodarek <gospo@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/Kconfig | 1 +
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 47 +++++++++++++++++++++++----
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 3 ++
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 3 +-
4 files changed, 47 insertions(+), 7 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/Kconfig b/drivers/net/ethernet/broadcom/Kconfig
index 2e4a8c7..e9017ca 100644
--- a/drivers/net/ethernet/broadcom/Kconfig
+++ b/drivers/net/ethernet/broadcom/Kconfig
@@ -199,6 +199,7 @@ config BNXT
select FW_LOADER
select LIBCRC32C
select NET_DEVLINK
+ select PAGE_POOL
---help---
This driver supports Broadcom NetXtreme-C/E 10/25/40/50 gigabit
Ethernet cards. To compile this driver as a module, choose M here:
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index d8f0846..d25bb38 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -54,6 +54,7 @@
#include <net/pkt_cls.h>
#include <linux/hwmon.h>
#include <linux/hwmon-sysfs.h>
+#include <net/page_pool.h>
#include "bnxt_hsi.h"
#include "bnxt.h"
@@ -668,19 +669,20 @@ static void bnxt_tx_int(struct bnxt *bp, struct bnxt_napi *bnapi, int nr_pkts)
}
static struct page *__bnxt_alloc_rx_page(struct bnxt *bp, dma_addr_t *mapping,
+ struct bnxt_rx_ring_info *rxr,
gfp_t gfp)
{
struct device *dev = &bp->pdev->dev;
struct page *page;
- page = alloc_page(gfp);
+ page = page_pool_dev_alloc_pages(rxr->page_pool);
if (!page)
return NULL;
*mapping = dma_map_page_attrs(dev, page, 0, PAGE_SIZE, bp->rx_dir,
DMA_ATTR_WEAK_ORDERING);
if (dma_mapping_error(dev, *mapping)) {
- __free_page(page);
+ page_pool_recycle_direct(rxr->page_pool, page);
return NULL;
}
*mapping += bp->rx_dma_offset;
@@ -716,7 +718,8 @@ int bnxt_alloc_rx_data(struct bnxt *bp, struct bnxt_rx_ring_info *rxr,
dma_addr_t mapping;
if (BNXT_RX_PAGE_MODE(bp)) {
- struct page *page = __bnxt_alloc_rx_page(bp, &mapping, gfp);
+ struct page *page =
+ __bnxt_alloc_rx_page(bp, &mapping, rxr, gfp);
if (!page)
return -ENOMEM;
@@ -2360,7 +2363,7 @@ static void bnxt_free_rx_skbs(struct bnxt *bp)
dma_unmap_page_attrs(&pdev->dev, mapping,
PAGE_SIZE, bp->rx_dir,
DMA_ATTR_WEAK_ORDERING);
- __free_page(data);
+ page_pool_recycle_direct(rxr->page_pool, data);
} else {
dma_unmap_single_attrs(&pdev->dev, mapping,
bp->rx_buf_use_size,
@@ -2497,6 +2500,8 @@ static void bnxt_free_rx_rings(struct bnxt *bp)
if (xdp_rxq_info_is_reg(&rxr->xdp_rxq))
xdp_rxq_info_unreg(&rxr->xdp_rxq);
+ rxr->page_pool = NULL;
+
kfree(rxr->rx_tpa);
rxr->rx_tpa = NULL;
@@ -2511,6 +2516,26 @@ static void bnxt_free_rx_rings(struct bnxt *bp)
}
}
+static int bnxt_alloc_rx_page_pool(struct bnxt *bp,
+ struct bnxt_rx_ring_info *rxr)
+{
+ struct page_pool_params pp = { 0 };
+
+ pp.pool_size = bp->rx_ring_size;
+ pp.nid = dev_to_node(&bp->pdev->dev);
+ pp.dev = &bp->pdev->dev;
+ pp.dma_dir = DMA_BIDIRECTIONAL;
+
+ rxr->page_pool = page_pool_create(&pp);
+ if (IS_ERR(rxr->page_pool)) {
+ int err = PTR_ERR(rxr->page_pool);
+
+ rxr->page_pool = NULL;
+ return err;
+ }
+ return 0;
+}
+
static int bnxt_alloc_rx_rings(struct bnxt *bp)
{
int i, rc, agg_rings = 0, tpa_rings = 0;
@@ -2530,14 +2555,24 @@ static int bnxt_alloc_rx_rings(struct bnxt *bp)
ring = &rxr->rx_ring_struct;
+ rc = bnxt_alloc_rx_page_pool(bp, rxr);
+ if (rc)
+ return rc;
+
rc = xdp_rxq_info_reg(&rxr->xdp_rxq, bp->dev, i);
- if (rc < 0)
+ if (rc < 0) {
+ page_pool_free(rxr->page_pool);
+ rxr->page_pool = NULL;
return rc;
+ }
rc = xdp_rxq_info_reg_mem_model(&rxr->xdp_rxq,
- MEM_TYPE_PAGE_SHARED, NULL);
+ MEM_TYPE_PAGE_POOL,
+ rxr->page_pool);
if (rc) {
xdp_rxq_info_unreg(&rxr->xdp_rxq);
+ page_pool_free(rxr->page_pool);
+ rxr->page_pool = NULL;
return rc;
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 8ac51fa..16694b7 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -26,6 +26,8 @@
#include <net/xdp.h>
#include <linux/dim.h>
+struct page_pool;
+
struct tx_bd {
__le32 tx_bd_len_flags_type;
#define TX_BD_TYPE (0x3f << 0)
@@ -799,6 +801,7 @@ struct bnxt_rx_ring_info {
struct bnxt_ring_struct rx_ring_struct;
struct bnxt_ring_struct rx_agg_ring_struct;
struct xdp_rxq_info xdp_rxq;
+ struct page_pool *page_pool;
};
struct bnxt_cp_ring_info {
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
index 12489d2..c6f6f20 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
@@ -15,6 +15,7 @@
#include <linux/bpf.h>
#include <linux/bpf_trace.h>
#include <linux/filter.h>
+#include <net/page_pool.h>
#include "bnxt_hsi.h"
#include "bnxt.h"
#include "bnxt_xdp.h"
@@ -191,7 +192,7 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
if (xdp_do_redirect(bp->dev, &xdp, xdp_prog)) {
trace_xdp_exception(bp->dev, xdp_prog, act);
- __free_page(page);
+ page_pool_recycle_direct(rxr->page_pool, page);
return true;
}
--
2.5.1
^ permalink raw reply related
* [PATCH net-next v2 3/4] bnxt_en: optimized XDP_REDIRECT support
From: Michael Chan @ 2019-07-08 21:53 UTC (permalink / raw)
To: davem, gospo; +Cc: netdev, hawk, ast, ilias.apalodimas
In-Reply-To: <1562622784-29918-1-git-send-email-michael.chan@broadcom.com>
From: Andy Gospodarek <gospo@broadcom.com>
This adds basic support for XDP_REDIRECT in the bnxt_en driver. Next
patch adds the more optimized page pool support.
Signed-off-by: Andy Gospodarek <gospo@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 27 ++++++-
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 13 +++-
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 108 ++++++++++++++++++++++++--
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h | 2 +
4 files changed, 140 insertions(+), 10 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index b7b6227..d8f0846 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -1989,6 +1989,9 @@ static int __bnxt_poll_work(struct bnxt *bp, struct bnxt_cp_ring_info *cpr,
}
}
+ if (event & BNXT_REDIRECT_EVENT)
+ xdp_do_flush_map();
+
if (event & BNXT_TX_EVENT) {
struct bnxt_tx_ring_info *txr = bnapi->tx_ring;
u16 prod = txr->tx_prod;
@@ -2254,9 +2257,23 @@ static void bnxt_free_tx_skbs(struct bnxt *bp)
for (j = 0; j < max_idx;) {
struct bnxt_sw_tx_bd *tx_buf = &txr->tx_buf_ring[j];
- struct sk_buff *skb = tx_buf->skb;
+ struct sk_buff *skb;
int k, last;
+ if (i < bp->tx_nr_rings_xdp &&
+ tx_buf->action == XDP_REDIRECT) {
+ dma_unmap_single(&pdev->dev,
+ dma_unmap_addr(tx_buf, mapping),
+ dma_unmap_len(tx_buf, len),
+ PCI_DMA_TODEVICE);
+ xdp_return_frame(tx_buf->xdpf);
+ tx_buf->action = 0;
+ tx_buf->xdpf = NULL;
+ j++;
+ continue;
+ }
+
+ skb = tx_buf->skb;
if (!skb) {
j++;
continue;
@@ -2517,6 +2534,13 @@ static int bnxt_alloc_rx_rings(struct bnxt *bp)
if (rc < 0)
return rc;
+ rc = xdp_rxq_info_reg_mem_model(&rxr->xdp_rxq,
+ MEM_TYPE_PAGE_SHARED, NULL);
+ if (rc) {
+ xdp_rxq_info_unreg(&rxr->xdp_rxq);
+ return rc;
+ }
+
rc = bnxt_alloc_ring(bp, &ring->ring_mem);
if (rc)
return rc;
@@ -10233,6 +10257,7 @@ static const struct net_device_ops bnxt_netdev_ops = {
.ndo_udp_tunnel_add = bnxt_udp_tunnel_add,
.ndo_udp_tunnel_del = bnxt_udp_tunnel_del,
.ndo_bpf = bnxt_xdp,
+ .ndo_xdp_xmit = bnxt_xdp_xmit,
.ndo_bridge_getlink = bnxt_bridge_getlink,
.ndo_bridge_setlink = bnxt_bridge_setlink,
.ndo_get_devlink_port = bnxt_get_devlink_port,
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index bf12cfc..8ac51fa 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -587,13 +587,18 @@ struct nqe_cn {
#define BNXT_HWRM_CHNL_CHIMP 0
#define BNXT_HWRM_CHNL_KONG 1
-#define BNXT_RX_EVENT 1
-#define BNXT_AGG_EVENT 2
-#define BNXT_TX_EVENT 4
+#define BNXT_RX_EVENT 1
+#define BNXT_AGG_EVENT 2
+#define BNXT_TX_EVENT 4
+#define BNXT_REDIRECT_EVENT 8
struct bnxt_sw_tx_bd {
- struct sk_buff *skb;
+ union {
+ struct sk_buff *skb;
+ struct xdp_frame *xdpf;
+ };
DEFINE_DMA_UNMAP_ADDR(mapping);
+ DEFINE_DMA_UNMAP_LEN(len);
u8 is_gso;
u8 is_push;
u8 action;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
index 41e232e..12489d2 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
@@ -53,6 +53,20 @@ static void __bnxt_xmit_xdp(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
tx_buf->action = XDP_TX;
}
+static void __bnxt_xmit_xdp_redirect(struct bnxt *bp,
+ struct bnxt_tx_ring_info *txr,
+ dma_addr_t mapping, u32 len,
+ struct xdp_frame *xdpf)
+{
+ struct bnxt_sw_tx_bd *tx_buf;
+
+ tx_buf = bnxt_xmit_bd(bp, txr, mapping, len);
+ tx_buf->action = XDP_REDIRECT;
+ tx_buf->xdpf = xdpf;
+ dma_unmap_addr_set(tx_buf, mapping, mapping);
+ dma_unmap_len_set(tx_buf, len, 0);
+}
+
void bnxt_tx_int_xdp(struct bnxt *bp, struct bnxt_napi *bnapi, int nr_pkts)
{
struct bnxt_tx_ring_info *txr = bnapi->tx_ring;
@@ -66,7 +80,17 @@ void bnxt_tx_int_xdp(struct bnxt *bp, struct bnxt_napi *bnapi, int nr_pkts)
for (i = 0; i < nr_pkts; i++) {
tx_buf = &txr->tx_buf_ring[tx_cons];
- if (tx_buf->action == XDP_TX) {
+ if (tx_buf->action == XDP_REDIRECT) {
+ struct pci_dev *pdev = bp->pdev;
+
+ dma_unmap_single(&pdev->dev,
+ dma_unmap_addr(tx_buf, mapping),
+ dma_unmap_len(tx_buf, len),
+ PCI_DMA_TODEVICE);
+ xdp_return_frame(tx_buf->xdpf);
+ tx_buf->action = 0;
+ tx_buf->xdpf = NULL;
+ } else if (tx_buf->action == XDP_TX) {
rx_doorbell_needed = true;
last_tx_cons = tx_cons;
}
@@ -101,19 +125,19 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
return false;
pdev = bp->pdev;
- txr = rxr->bnapi->tx_ring;
rx_buf = &rxr->rx_buf_ring[cons];
offset = bp->rx_offset;
+ mapping = rx_buf->mapping - bp->rx_dma_offset;
+ dma_sync_single_for_cpu(&pdev->dev, mapping + offset, *len, bp->rx_dir);
+
+ txr = rxr->bnapi->tx_ring;
xdp.data_hard_start = *data_ptr - offset;
xdp.data = *data_ptr;
xdp_set_data_meta_invalid(&xdp);
xdp.data_end = *data_ptr + *len;
xdp.rxq = &rxr->xdp_rxq;
orig_data = xdp.data;
- mapping = rx_buf->mapping - bp->rx_dma_offset;
-
- dma_sync_single_for_cpu(&pdev->dev, mapping + offset, *len, bp->rx_dir);
rcu_read_lock();
act = bpf_prog_run_xdp(xdp_prog, &xdp);
@@ -149,6 +173,30 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
NEXT_RX(rxr->rx_prod));
bnxt_reuse_rx_data(rxr, cons, page);
return true;
+ case XDP_REDIRECT:
+ /* if we are calling this here then we know that the
+ * redirect is coming from a frame received by the
+ * bnxt_en driver.
+ */
+ dma_unmap_page_attrs(&pdev->dev, mapping,
+ PAGE_SIZE, bp->rx_dir,
+ DMA_ATTR_WEAK_ORDERING);
+
+ /* if we are unable to allocate a new buffer, abort and reuse */
+ if (bnxt_alloc_rx_data(bp, rxr, rxr->rx_prod, GFP_ATOMIC)) {
+ trace_xdp_exception(bp->dev, xdp_prog, act);
+ bnxt_reuse_rx_data(rxr, cons, page);
+ return true;
+ }
+
+ if (xdp_do_redirect(bp->dev, &xdp, xdp_prog)) {
+ trace_xdp_exception(bp->dev, xdp_prog, act);
+ __free_page(page);
+ return true;
+ }
+
+ *event |= BNXT_REDIRECT_EVENT;
+ break;
default:
bpf_warn_invalid_xdp_action(act);
/* Fall thru */
@@ -162,6 +210,56 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
return true;
}
+int bnxt_xdp_xmit(struct net_device *dev, int num_frames,
+ struct xdp_frame **frames, u32 flags)
+{
+ struct bnxt *bp = netdev_priv(dev);
+ struct bpf_prog *xdp_prog = READ_ONCE(bp->xdp_prog);
+ struct pci_dev *pdev = bp->pdev;
+ struct bnxt_tx_ring_info *txr;
+ dma_addr_t mapping;
+ int drops = 0;
+ int ring;
+ int i;
+
+ if (!test_bit(BNXT_STATE_OPEN, &bp->state) ||
+ !bp->tx_nr_rings_xdp ||
+ !xdp_prog)
+ return -EINVAL;
+
+ ring = smp_processor_id() % bp->tx_nr_rings_xdp;
+ txr = &bp->tx_ring[ring];
+
+ for (i = 0; i < num_frames; i++) {
+ struct xdp_frame *xdp = frames[i];
+
+ if (!txr || !bnxt_tx_avail(bp, txr) ||
+ !(bp->bnapi[ring]->flags & BNXT_NAPI_FLAG_XDP)) {
+ xdp_return_frame_rx_napi(xdp);
+ drops++;
+ continue;
+ }
+
+ mapping = dma_map_single(&pdev->dev, xdp->data, xdp->len,
+ DMA_TO_DEVICE);
+
+ if (dma_mapping_error(&pdev->dev, mapping)) {
+ xdp_return_frame_rx_napi(xdp);
+ drops++;
+ continue;
+ }
+ __bnxt_xmit_xdp_redirect(bp, txr, mapping, xdp->len, xdp);
+ }
+
+ if (flags & XDP_XMIT_FLUSH) {
+ /* Sync BD data before updating doorbell */
+ wmb();
+ bnxt_db_write(bp, &txr->tx_db, txr->tx_prod);
+ }
+
+ return num_frames - drops;
+}
+
/* Under rtnl_lock */
static int bnxt_xdp_set(struct bnxt *bp, struct bpf_prog *prog)
{
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h
index 20e470c..0df40c3 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h
@@ -18,5 +18,7 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
struct page *page, u8 **data_ptr, unsigned int *len,
u8 *event);
int bnxt_xdp(struct net_device *dev, struct netdev_bpf *xdp);
+int bnxt_xdp_xmit(struct net_device *dev, int num_frames,
+ struct xdp_frame **frames, u32 flags);
#endif
--
2.5.1
^ permalink raw reply related
* [PATCH net-next v2 1/4] bnxt_en: rename some xdp functions
From: Michael Chan @ 2019-07-08 21:53 UTC (permalink / raw)
To: davem, gospo; +Cc: netdev, hawk, ast, ilias.apalodimas
In-Reply-To: <1562622784-29918-1-git-send-email-michael.chan@broadcom.com>
From: Andy Gospodarek <gospo@broadcom.com>
Renaming bnxt_xmit_xdp to __bnxt_xmit_xdp to get ready for XDP_REDIRECT
support and reduce confusion/namespace collision.
Signed-off-by: Andy Gospodarek <gospo@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 2 +-
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 8 ++++----
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h | 4 ++--
3 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index a6c7baf..21a0431 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -2799,7 +2799,7 @@ static int bnxt_run_loopback(struct bnxt *bp)
dev_kfree_skb(skb);
return -EIO;
}
- bnxt_xmit_xdp(bp, txr, map, pkt_size, 0);
+ __bnxt_xmit_xdp(bp, txr, map, pkt_size, 0);
/* Sync BD data before updating doorbell */
wmb();
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
index 0184ef6..4bc9595 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
@@ -19,8 +19,8 @@
#include "bnxt.h"
#include "bnxt_xdp.h"
-void bnxt_xmit_xdp(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
- dma_addr_t mapping, u32 len, u16 rx_prod)
+void __bnxt_xmit_xdp(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ dma_addr_t mapping, u32 len, u16 rx_prod)
{
struct bnxt_sw_tx_bd *tx_buf;
struct tx_bd *txbd;
@@ -132,8 +132,8 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
*event = BNXT_TX_EVENT;
dma_sync_single_for_device(&pdev->dev, mapping + offset, *len,
bp->rx_dir);
- bnxt_xmit_xdp(bp, txr, mapping + offset, *len,
- NEXT_RX(rxr->rx_prod));
+ __bnxt_xmit_xdp(bp, txr, mapping + offset, *len,
+ NEXT_RX(rxr->rx_prod));
bnxt_reuse_rx_data(rxr, cons, page);
return true;
default:
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h
index 414b748..b36087b 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h
@@ -10,8 +10,8 @@
#ifndef BNXT_XDP_H
#define BNXT_XDP_H
-void bnxt_xmit_xdp(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
- dma_addr_t mapping, u32 len, u16 rx_prod);
+void __bnxt_xmit_xdp(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ dma_addr_t mapping, u32 len, u16 rx_prod);
void bnxt_tx_int_xdp(struct bnxt *bp, struct bnxt_napi *bnapi, int nr_pkts);
bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
struct page *page, u8 **data_ptr, unsigned int *len,
--
2.5.1
^ permalink raw reply related
* [PATCH net-next v2 2/4] bnxt_en: Refactor __bnxt_xmit_xdp().
From: Michael Chan @ 2019-07-08 21:53 UTC (permalink / raw)
To: davem, gospo; +Cc: netdev, hawk, ast, ilias.apalodimas
In-Reply-To: <1562622784-29918-1-git-send-email-michael.chan@broadcom.com>
__bnxt_xmit_xdp() is used by XDP_TX and ethtool loopback packet transmit.
Refactor it so that it can be re-used by the XDP_REDIRECT logic.
Restructure the TX interrupt handler logic to cleanly separate XDP_TX
logic in preparation for XDP_REDIRECT.
Acked-by: Andy Gospodarek <gospo@broadcom.com>
Signed-off-by: Michael Chan <michael.chan@broadcom.com>
---
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 +
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 2 +-
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 33 ++++++++++++++++-------
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h | 5 ++--
4 files changed, 28 insertions(+), 13 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 4b3ae92..bf12cfc 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -596,6 +596,7 @@ struct bnxt_sw_tx_bd {
DEFINE_DMA_UNMAP_ADDR(mapping);
u8 is_gso;
u8 is_push;
+ u8 action;
union {
unsigned short nr_frags;
u16 rx_prod;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
index 21a0431..a0f3277 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c
@@ -2799,7 +2799,7 @@ static int bnxt_run_loopback(struct bnxt *bp)
dev_kfree_skb(skb);
return -EIO;
}
- __bnxt_xmit_xdp(bp, txr, map, pkt_size, 0);
+ bnxt_xmit_bd(bp, txr, map, pkt_size);
/* Sync BD data before updating doorbell */
wmb();
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
index 4bc9595..41e232e 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
@@ -19,8 +19,9 @@
#include "bnxt.h"
#include "bnxt_xdp.h"
-void __bnxt_xmit_xdp(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
- dma_addr_t mapping, u32 len, u16 rx_prod)
+struct bnxt_sw_tx_bd *bnxt_xmit_bd(struct bnxt *bp,
+ struct bnxt_tx_ring_info *txr,
+ dma_addr_t mapping, u32 len)
{
struct bnxt_sw_tx_bd *tx_buf;
struct tx_bd *txbd;
@@ -29,7 +30,6 @@ void __bnxt_xmit_xdp(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
prod = txr->tx_prod;
tx_buf = &txr->tx_buf_ring[prod];
- tx_buf->rx_prod = rx_prod;
txbd = &txr->tx_desc_ring[TX_RING(prod)][TX_IDX(prod)];
flags = (len << TX_BD_LEN_SHIFT) | (1 << TX_BD_FLAGS_BD_CNT_SHIFT) |
@@ -40,30 +40,43 @@ void __bnxt_xmit_xdp(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
prod = NEXT_TX(prod);
txr->tx_prod = prod;
+ return tx_buf;
+}
+
+static void __bnxt_xmit_xdp(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
+ dma_addr_t mapping, u32 len, u16 rx_prod)
+{
+ struct bnxt_sw_tx_bd *tx_buf;
+
+ tx_buf = bnxt_xmit_bd(bp, txr, mapping, len);
+ tx_buf->rx_prod = rx_prod;
+ tx_buf->action = XDP_TX;
}
void bnxt_tx_int_xdp(struct bnxt *bp, struct bnxt_napi *bnapi, int nr_pkts)
{
struct bnxt_tx_ring_info *txr = bnapi->tx_ring;
struct bnxt_rx_ring_info *rxr = bnapi->rx_ring;
+ bool rx_doorbell_needed = false;
struct bnxt_sw_tx_bd *tx_buf;
u16 tx_cons = txr->tx_cons;
u16 last_tx_cons = tx_cons;
- u16 rx_prod;
int i;
for (i = 0; i < nr_pkts; i++) {
- last_tx_cons = tx_cons;
+ tx_buf = &txr->tx_buf_ring[tx_cons];
+
+ if (tx_buf->action == XDP_TX) {
+ rx_doorbell_needed = true;
+ last_tx_cons = tx_cons;
+ }
tx_cons = NEXT_TX(tx_cons);
}
txr->tx_cons = tx_cons;
- if (bnxt_tx_avail(bp, txr) == bp->tx_ring_size) {
- rx_prod = rxr->rx_prod;
- } else {
+ if (rx_doorbell_needed) {
tx_buf = &txr->tx_buf_ring[last_tx_cons];
- rx_prod = tx_buf->rx_prod;
+ bnxt_db_write(bp, &rxr->rx_db, tx_buf->rx_prod);
}
- bnxt_db_write(bp, &rxr->rx_db, rx_prod);
}
/* returns the following:
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h
index b36087b..20e470c 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h
@@ -10,8 +10,9 @@
#ifndef BNXT_XDP_H
#define BNXT_XDP_H
-void __bnxt_xmit_xdp(struct bnxt *bp, struct bnxt_tx_ring_info *txr,
- dma_addr_t mapping, u32 len, u16 rx_prod);
+struct bnxt_sw_tx_bd *bnxt_xmit_bd(struct bnxt *bp,
+ struct bnxt_tx_ring_info *txr,
+ dma_addr_t mapping, u32 len);
void bnxt_tx_int_xdp(struct bnxt *bp, struct bnxt_napi *bnapi, int nr_pkts);
bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
struct page *page, u8 **data_ptr, unsigned int *len,
--
2.5.1
^ permalink raw reply related
* [PATCH net-next v2 0/4] bnxt_en: Add XDP_REDIRECT support.
From: Michael Chan @ 2019-07-08 21:53 UTC (permalink / raw)
To: davem, gospo; +Cc: netdev, hawk, ast, ilias.apalodimas
This patch series adds XDP_REDIRECT support by Andy Gospodarek.
Andy Gospodarek (3):
bnxt_en: rename some xdp functions
bnxt_en: optimized XDP_REDIRECT support
bnxt_en: add page_pool support
Michael Chan (1):
bnxt_en: Refactor __bnxt_xmit_xdp().
drivers/net/ethernet/broadcom/Kconfig | 1 +
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 72 ++++++++++-
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 17 ++-
drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 2 +-
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 144 +++++++++++++++++++---
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.h | 7 +-
6 files changed, 214 insertions(+), 29 deletions(-)
--
2.5.1
^ permalink raw reply
* Re: [EXT] Re: [PATCH net-next v2 4/4] qed*: Add devlink support for configuration attributes.
From: Jakub Kicinski @ 2019-07-08 21:47 UTC (permalink / raw)
To: Sudarsana Reddy Kalluru
Cc: davem@davemloft.net, netdev@vger.kernel.org, Michal Kalderon,
Ariel Elior, Jiri Pirko
In-Reply-To: <MN2PR18MB25280224F5DDDFE8D86B234CD3F60@MN2PR18MB2528.namprd18.prod.outlook.com>
On Mon, 8 Jul 2019 02:31:15 +0000, Sudarsana Reddy Kalluru wrote:
> > > > > + Type: u8
> > > > > + Configuration mode: Permanent
> > > > > +
> > > > > +dcbx_mode [PORT, DRIVER-SPECIFIC]
> > > > > + Configure DCBX mode for the device.
> > > > > + Supported dcbx modes are,
> > > > > + Disabled(0), IEEE(1), CEE(2) and
> > > > > Dynamic(3)
> > > > > + Type: u8
> > > > > + Configuration mode: Permanent
> > > >
> > > > Why is this a permanent parameter?
> > > >
> > > This specifies the dcbx_mode to be configured in non-volatile memory.
> > > The value is persistent and is used in the next load of OS or the mfw.
> >
> > And it can't be changed at runtime?
>
> Run time dcbx params are not affected via this interface, it only
> updates config on permanent storage of the port.
IOW it affects the defaults after boot? It'd be preferable to have
the DCB uAPI handle "persistent"/default settings if that's the case.
^ permalink raw reply
* Re: [PATCH bpf-next] selftests/bpf: make verifier loop tests arch independent
From: Ilya Leoshkevich @ 2019-07-08 21:44 UTC (permalink / raw)
To: Stanislav Fomichev
Cc: Y Song, Stanislav Fomichev, netdev, bpf, David Miller,
Alexei Starovoitov, Daniel Borkmann
In-Reply-To: <20190708212012.GA9509@mini-arch>
> Am 08.07.2019 um 23:20 schrieb Stanislav Fomichev <sdf@fomichev.me>:
>
> On 07/08, Ilya Leoshkevich wrote:
>>
>>
>>> Am 08.07.2019 um 18:13 schrieb Stanislav Fomichev <sdf@fomichev.me>:
>>>
>>> On 07/03, Y Song wrote:
>>>> On Wed, Jul 3, 2019 at 1:51 PM Stanislav Fomichev <sdf@google.com> wrote:
>>>>>
>>>>> Take the first x bytes of pt_regs for scalability tests, there is
>>>>> no real reason we need x86 specific rax.
>>>>>
>>>>> Signed-off-by: Stanislav Fomichev <sdf@google.com>
>>>>> ---
>>>>> tools/testing/selftests/bpf/progs/loop1.c | 3 ++-
>>>>> tools/testing/selftests/bpf/progs/loop2.c | 3 ++-
>>>>> tools/testing/selftests/bpf/progs/loop3.c | 3 ++-
>>>>> 3 files changed, 6 insertions(+), 3 deletions(-)
>>>>>
>>>>> diff --git a/tools/testing/selftests/bpf/progs/loop1.c b/tools/testing/selftests/bpf/progs/loop1.c
>>>>> index dea395af9ea9..d530c61d2517 100644
>>>>> --- a/tools/testing/selftests/bpf/progs/loop1.c
>>>>> +++ b/tools/testing/selftests/bpf/progs/loop1.c
>>>>> @@ -14,11 +14,12 @@ SEC("raw_tracepoint/kfree_skb")
>>>>> int nested_loops(volatile struct pt_regs* ctx)
>>>>> {
>>>>> int i, j, sum = 0, m;
>>>>> + volatile int *any_reg = (volatile int *)ctx;
>>>>>
>>>>> for (j = 0; j < 300; j++)
>>>>> for (i = 0; i < j; i++) {
>>>>> if (j & 1)
>>>>> - m = ctx->rax;
>>>>> + m = *any_reg;
>>>>
>>>> I agree. ctx->rax here is only to generate some operations, which
>>>> cannot be optimized away by the compiler. dereferencing a volatile
>>>> pointee may just serve that purpose.
>>>>
>>>> Comparing the byte code generated with ctx->rax and *any_reg, they are
>>>> slightly different. Using *any_reg is slighly worse, but this should
>>>> be still okay for the test.
>>>>
>>>>> else
>>>>> m = j;
>>>>> sum += i * m;
>>>>> diff --git a/tools/testing/selftests/bpf/progs/loop2.c b/tools/testing/selftests/bpf/progs/loop2.c
>>>>> index 0637bd8e8bcf..91bb89d901e3 100644
>>>>> --- a/tools/testing/selftests/bpf/progs/loop2.c
>>>>> +++ b/tools/testing/selftests/bpf/progs/loop2.c
>>>>> @@ -14,9 +14,10 @@ SEC("raw_tracepoint/consume_skb")
>>>>> int while_true(volatile struct pt_regs* ctx)
>>>>> {
>>>>> int i = 0;
>>>>> + volatile int *any_reg = (volatile int *)ctx;
>>>>>
>>>>> while (true) {
>>>>> - if (ctx->rax & 1)
>>>>> + if (*any_reg & 1)
>>>>> i += 3;
>>>>> else
>>>>> i += 7;
>>>>> diff --git a/tools/testing/selftests/bpf/progs/loop3.c b/tools/testing/selftests/bpf/progs/loop3.c
>>>>> index 30a0f6cba080..3a7f12d7186c 100644
>>>>> --- a/tools/testing/selftests/bpf/progs/loop3.c
>>>>> +++ b/tools/testing/selftests/bpf/progs/loop3.c
>>>>> @@ -14,9 +14,10 @@ SEC("raw_tracepoint/consume_skb")
>>>>> int while_true(volatile struct pt_regs* ctx)
>>>>> {
>>>>> __u64 i = 0, sum = 0;
>>>>> + volatile __u64 *any_reg = (volatile __u64 *)ctx;
>>>>> do {
>>>>> i++;
>>>>> - sum += ctx->rax;
>>>>> + sum += *any_reg;
>>>>> } while (i < 0x100000000ULL);
>>>>> return sum;
>>>>> }
>>>>> --
>>>>> 2.22.0.410.gd8fdbe21b5-goog
>>>>
>>>> Ilya Leoshkevich (iii@linux.ibm.com, cc'ed) has another patch set
>>>> trying to solve this problem by introducing s360 arch register access
>>>> macros. I guess for now that patch set is not needed any more?
>>> Oh, I missed them. Do they fix the tests for other (non-s360) arches as
>>> well? I was trying to fix the issue by not depending on any arch
>>> specific stuff because the test really doesn't care :-)
>>
>> They are supposed to work for everything that defines PT_REGS_RC in
>> bpf_helpers.h, but I have to admit I tested only x86_64 and s390.
>>
>> The main source of problems with my approach were mismatching definitions
>> of struct pt_regs for userspace and kernel, and because of that there was
>> some tweaking required for both arches. I will double check how it looks
>> for others (arm, mips, ppc, sparc) tomorrow.
> Thanks, I've tested your patches and they fix my issue as well. So you
> can have my Tested-by if we'd go with your approach.
>
> One thing I don't understand is: why do you add 'ifdef __KERNEL__' to
> the bpf_helpers.h for x86 case? Who is using bpf_helpers.h with
> __KERNEL__ defined? Is it perf?
That’s samples/bpf. Also, there is a modified copy of it in bcc
(src/cc/export/helpers.h), which also gets built with __KERNEL__.
Best regards,
Ilya
^ permalink raw reply
* Re: [PATCH net-next,v3 00/11] netfilter: add hardware offload infrastructure
From: David Miller @ 2019-07-08 21:39 UTC (permalink / raw)
To: pablo
Cc: netdev, thomas.lendacky, f.fainelli, ariel.elior, michael.chan,
madalin.bucur, yisen.zhuang, salil.mehta, jeffrey.t.kirsher,
tariqt, saeedm, jiri, idosch, jakub.kicinski, peppe.cavallaro,
grygorii.strashko, andrew, vivien.didelot, alexandre.torgue,
joabreu, linux-net-drivers, ogerlitz, Manish.Chopra,
marcelo.leitner, mkubecek, venkatkumar.duvvuru, maxime.chevallier,
cphealy, netfilter-devel
In-Reply-To: <20190708160614.2226-1-pablo@netfilter.org>
From: Pablo Neira Ayuso <pablo@netfilter.org>
Date: Mon, 8 Jul 2019 18:06:02 +0200
> This patchset adds support for Netfilter hardware offloads.
Pablo if you can address Jiri's feedback fully and repost this by
Tuesday (PST timezone) I will allow it into this merge window.
Thank you.
^ permalink raw reply
* Re: [PATCH V5 net-next 4/6] dt-bindings: ptp: Introduce MII time stamping devices.
From: Rob Herring @ 2019-07-08 21:38 UTC (permalink / raw)
To: Richard Cochran
Cc: netdev, David Miller, devicetree, Andrew Lunn, Florian Fainelli,
Jacob Keller, Mark Rutland, Miroslav Lichvar, Willem de Bruijn
In-Reply-To: <d786656435c64160d50014beb3d3d9d1aaf6f22d.1559281985.git.richardcochran@gmail.com>
On Thu, May 30, 2019 at 10:56:24PM -0700, Richard Cochran wrote:
> This patch add a new binding that allows non-PHY MII time stamping
> devices to find their buses. The new documentation covers both the
> generic binding and one upcoming user.
>
> Signed-off-by: Richard Cochran <richardcochran@gmail.com>
> Reviewed-by: Andrew Lunn <andrew@lunn.ch>
> ---
> Documentation/devicetree/bindings/ptp/ptp-ines.txt | 35 ++++++++++++++++++
> .../devicetree/bindings/ptp/timestamper.txt | 41 ++++++++++++++++++++++
> 2 files changed, 76 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/ptp/ptp-ines.txt
> create mode 100644 Documentation/devicetree/bindings/ptp/timestamper.txt
>
> diff --git a/Documentation/devicetree/bindings/ptp/ptp-ines.txt b/Documentation/devicetree/bindings/ptp/ptp-ines.txt
> new file mode 100644
> index 000000000000..4dee9eb89455
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/ptp/ptp-ines.txt
> @@ -0,0 +1,35 @@
> +ZHAW InES PTP time stamping IP core
> +
> +The IP core needs two different kinds of nodes. The control node
> +lives somewhere in the memory map and specifies the address of the
> +control registers. There can be up to three port handles placed as
> +attributes of PHY nodes. These associate a particular MII bus with a
> +port index within the IP core.
> +
> +Required properties of the control node:
> +
> +- compatible: "ines,ptp-ctrl"
This is an IP block that gets integrated into SoCs? It's not very
specific given that there could be different versions of the IP block
and SoC vendors can integrate various versions of the IP block in their
own unique (i.e. buggy) way.
> +- reg: physical address and size of the register bank
> +
> +Required format of the port handle within the PHY node:
> +
> +- timestamper: provides control node reference and
> + the port channel within the IP core
> +
> +Example:
> +
> + tstamper: timestamper@60000000 {
> + compatible = "ines,ptp-ctrl";
> + reg = <0x60000000 0x80>;
> + };
> +
> + ethernet@80000000 {
> + ...
> + mdio {
> + ...
> + phy@3 {
ethernet-phy is the correct node name.
> + ...
> + timestamper = <&tstamper 0>;
> + };
> + };
> + };
> diff --git a/Documentation/devicetree/bindings/ptp/timestamper.txt b/Documentation/devicetree/bindings/ptp/timestamper.txt
> new file mode 100644
> index 000000000000..88ea0bc7d662
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/ptp/timestamper.txt
> @@ -0,0 +1,41 @@
> +Time stamps from MII bus snooping devices
> +
> +This binding supports non-PHY devices that snoop the MII bus and
> +provide time stamps. In contrast to PHY time stamping drivers (which
> +can simply attach their interface directly to the PHY instance), stand
> +alone MII time stamping drivers use this binding to specify the
> +connection between the snooping device and a given network interface.
> +
> +Non-PHY MII time stamping drivers typically talk to the control
> +interface over another bus like I2C, SPI, UART, or via a memory mapped
> +peripheral. This controller device is associated with one or more
> +time stamping channels, each of which snoops on a MII bus.
> +
> +The "timestamper" property lives in a phy node and links a time
> +stamping channel from the controller device to that phy's MII bus.
> +
> +Example:
> +
> + tstamper: timestamper@10000000 {
> + compatible = "bigcorp,ts-ctrl";
Would be better to use a real example here.
> + };
> +
> + ethernet@20000000 {
> + mdio {
> + phy@1 {
> + timestamper = <&tstamper 0>;
> + };
> + };
> + };
> +
> + ethernet@30000000 {
> + mdio {
> + phy@2 {
> + timestamper = <&tstamper 1>;
> + };
> + };
> + };
> +
> +In this example, time stamps from the MII bus attached to phy@1 will
> +appear on time stamp channel 0 (zero), and those from phy@2 appear on
> +channel 1.
> --
> 2.11.0
>
^ permalink raw reply
* [PATCH v9 net-next 1/5] net: core: page_pool: add user refcnt and reintroduce page_pool_destroy
From: Ivan Khoronzhuk @ 2019-07-08 21:34 UTC (permalink / raw)
To: grygorii.strashko, hawk, davem
Cc: ast, linux-kernel, linux-omap, xdp-newbies, ilias.apalodimas,
netdev, daniel, jakub.kicinski, john.fastabend, Ivan Khoronzhuk,
Jesper Dangaard Brouer
In-Reply-To: <20190708213432.8525-1-ivan.khoronzhuk@linaro.org>
Jesper recently removed page_pool_destroy() (from driver invocation)
and moved shutdown and free of page_pool into xdp_rxq_info_unreg(),
in-order to handle in-flight packets/pages. This created an asymmetry
in drivers create/destroy pairs.
This patch reintroduce page_pool_destroy and add page_pool user
refcnt. This serves the purpose to simplify drivers error handling as
driver now drivers always calls page_pool_destroy() and don't need to
track if xdp_rxq_info_reg_mem_model() was unsuccessful.
This could be used for a special cases where a single RX-queue (with a
single page_pool) provides packets for two net_device'es, and thus
needs to register the same page_pool twice with two xdp_rxq_info
structures.
This patch is primarily to ease API usage for drivers. The recently
merged netsec driver, actually have a bug in this area, which is
solved by this API change.
This patch is a modified version of Ivan Khoronzhuk's original patch.
Link: https://lore.kernel.org/netdev/20190625175948.24771-2-ivan.khoronzhuk@linaro.org/
Fixes: 5c67bf0ec4d0 ("net: netsec: Use page_pool API")
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Reviewed-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
Reviewed-by: Saeed Mahameed <saeedm@mellanox.com>
Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
---
.../net/ethernet/mellanox/mlx5/core/en_main.c | 4 +--
drivers/net/ethernet/socionext/netsec.c | 8 ++----
include/net/page_pool.h | 25 +++++++++++++++++++
net/core/page_pool.c | 8 ++++++
net/core/xdp.c | 3 +++
5 files changed, 40 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 83194d56434d..10efd69de7ef 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -577,8 +577,6 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c,
}
err = xdp_rxq_info_reg_mem_model(&rq->xdp_rxq,
MEM_TYPE_PAGE_POOL, rq->page_pool);
- if (err)
- page_pool_free(rq->page_pool);
}
if (err)
goto err_free;
@@ -646,6 +644,7 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c,
if (rq->xdp_prog)
bpf_prog_put(rq->xdp_prog);
xdp_rxq_info_unreg(&rq->xdp_rxq);
+ page_pool_destroy(rq->page_pool);
mlx5_wq_destroy(&rq->wq_ctrl);
return err;
@@ -680,6 +679,7 @@ static void mlx5e_free_rq(struct mlx5e_rq *rq)
}
xdp_rxq_info_unreg(&rq->xdp_rxq);
+ page_pool_destroy(rq->page_pool);
mlx5_wq_destroy(&rq->wq_ctrl);
}
diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c
index 460777449cd9..d7307ab90d74 100644
--- a/drivers/net/ethernet/socionext/netsec.c
+++ b/drivers/net/ethernet/socionext/netsec.c
@@ -1212,15 +1212,11 @@ static void netsec_uninit_pkt_dring(struct netsec_priv *priv, int id)
}
}
- /* Rx is currently using page_pool
- * since the pool is created during netsec_setup_rx_dring(), we need to
- * free the pool manually if the registration failed
- */
+ /* Rx is currently using page_pool */
if (id == NETSEC_RING_RX) {
if (xdp_rxq_info_is_reg(&dring->xdp_rxq))
xdp_rxq_info_unreg(&dring->xdp_rxq);
- else
- page_pool_free(dring->page_pool);
+ page_pool_destroy(dring->page_pool);
}
memset(dring->desc, 0, sizeof(struct netsec_desc) * DESC_NUM);
diff --git a/include/net/page_pool.h b/include/net/page_pool.h
index ee9c871d2043..2cbcdbdec254 100644
--- a/include/net/page_pool.h
+++ b/include/net/page_pool.h
@@ -101,6 +101,12 @@ struct page_pool {
struct ptr_ring ring;
atomic_t pages_state_release_cnt;
+
+ /* A page_pool is strictly tied to a single RX-queue being
+ * protected by NAPI, due to above pp_alloc_cache. This
+ * refcnt serves purpose is to simplify drivers error handling.
+ */
+ refcount_t user_cnt;
};
struct page *page_pool_alloc_pages(struct page_pool *pool, gfp_t gfp);
@@ -134,6 +140,15 @@ static inline void page_pool_free(struct page_pool *pool)
#endif
}
+/* Drivers use this instead of page_pool_free */
+static inline void page_pool_destroy(struct page_pool *pool)
+{
+ if (!pool)
+ return;
+
+ page_pool_free(pool);
+}
+
/* Never call this directly, use helpers below */
void __page_pool_put_page(struct page_pool *pool,
struct page *page, bool allow_direct);
@@ -201,4 +216,14 @@ static inline bool is_page_pool_compiled_in(void)
#endif
}
+static inline void page_pool_get(struct page_pool *pool)
+{
+ refcount_inc(&pool->user_cnt);
+}
+
+static inline bool page_pool_put(struct page_pool *pool)
+{
+ return refcount_dec_and_test(&pool->user_cnt);
+}
+
#endif /* _NET_PAGE_POOL_H */
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index b366f59885c1..3272dc7a8c81 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -49,6 +49,9 @@ static int page_pool_init(struct page_pool *pool,
atomic_set(&pool->pages_state_release_cnt, 0);
+ /* Driver calling page_pool_create() also call page_pool_destroy() */
+ refcount_set(&pool->user_cnt, 1);
+
if (pool->p.flags & PP_FLAG_DMA_MAP)
get_device(pool->p.dev);
@@ -70,6 +73,7 @@ struct page_pool *page_pool_create(const struct page_pool_params *params)
kfree(pool);
return ERR_PTR(err);
}
+
return pool;
}
EXPORT_SYMBOL(page_pool_create);
@@ -356,6 +360,10 @@ static void __warn_in_flight(struct page_pool *pool)
void __page_pool_free(struct page_pool *pool)
{
+ /* Only last user actually free/release resources */
+ if (!page_pool_put(pool))
+ return;
+
WARN(pool->alloc.count, "API usage violation");
WARN(!ptr_ring_empty(&pool->ring), "ptr_ring is not empty");
diff --git a/net/core/xdp.c b/net/core/xdp.c
index 829377cc83db..d7bf62ffbb5e 100644
--- a/net/core/xdp.c
+++ b/net/core/xdp.c
@@ -370,6 +370,9 @@ int xdp_rxq_info_reg_mem_model(struct xdp_rxq_info *xdp_rxq,
goto err;
}
+ if (type == MEM_TYPE_PAGE_POOL)
+ page_pool_get(xdp_alloc->page_pool);
+
mutex_unlock(&mem_id_lock);
trace_mem_connect(xdp_alloc, xdp_rxq);
--
2.17.1
^ permalink raw reply related
* [PATCH v9 net-next 3/5] net: ethernet: ti: davinci_cpdma: allow desc split while down
From: Ivan Khoronzhuk @ 2019-07-08 21:34 UTC (permalink / raw)
To: grygorii.strashko, hawk, davem
Cc: ast, linux-kernel, linux-omap, xdp-newbies, ilias.apalodimas,
netdev, daniel, jakub.kicinski, john.fastabend, Ivan Khoronzhuk
In-Reply-To: <20190708213432.8525-1-ivan.khoronzhuk@linaro.org>
That's possible to set ring params while interfaces are down. When
interface gets up it uses number of descs to fill rx queue and on
later on changes to create rx pools. Usually, this resplit can happen
after phy is up, but it can be needed before this, so allow it to
happen while setting number of rx descs, when interfaces are down.
Also, if no dependency on intf state, move it to cpdma layer, where
it should be.
Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
---
drivers/net/ethernet/ti/cpsw_ethtool.c | 17 +++++++++++------
drivers/net/ethernet/ti/davinci_cpdma.c | 17 ++++++++++++++++-
drivers/net/ethernet/ti/davinci_cpdma.h | 3 +--
3 files changed, 28 insertions(+), 9 deletions(-)
diff --git a/drivers/net/ethernet/ti/cpsw_ethtool.c b/drivers/net/ethernet/ti/cpsw_ethtool.c
index f60dc1dfc443..c477e6b620d6 100644
--- a/drivers/net/ethernet/ti/cpsw_ethtool.c
+++ b/drivers/net/ethernet/ti/cpsw_ethtool.c
@@ -650,7 +650,7 @@ int cpsw_set_ringparam(struct net_device *ndev,
{
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
- int ret;
+ int descs_num, ret;
/* ignore ering->tx_pending - only rx_pending adjustment is supported */
@@ -659,20 +659,25 @@ int cpsw_set_ringparam(struct net_device *ndev,
ering->rx_pending > (cpsw->descs_pool_size - CPSW_MAX_QUEUES))
return -EINVAL;
- if (ering->rx_pending == cpdma_get_num_rx_descs(cpsw->dma))
+ descs_num = cpdma_get_num_rx_descs(cpsw->dma);
+ if (ering->rx_pending == descs_num)
return 0;
cpsw_suspend_data_pass(ndev);
- cpdma_set_num_rx_descs(cpsw->dma, ering->rx_pending);
+ ret = cpdma_set_num_rx_descs(cpsw->dma, ering->rx_pending);
+ if (ret) {
+ if (cpsw_resume_data_pass(ndev))
+ goto err;
- if (cpsw->usage_count)
- cpdma_chan_split_pool(cpsw->dma);
+ return ret;
+ }
ret = cpsw_resume_data_pass(ndev);
if (!ret)
return 0;
-
+err:
+ cpdma_set_num_rx_descs(cpsw->dma, descs_num);
dev_err(cpsw->dev, "cannot set ring params, closing device\n");
dev_close(ndev);
return ret;
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.c b/drivers/net/ethernet/ti/davinci_cpdma.c
index 4e693c3aab27..0ca2a1a254de 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.c
+++ b/drivers/net/ethernet/ti/davinci_cpdma.c
@@ -1423,8 +1423,23 @@ int cpdma_get_num_tx_descs(struct cpdma_ctlr *ctlr)
return ctlr->num_tx_desc;
}
-void cpdma_set_num_rx_descs(struct cpdma_ctlr *ctlr, int num_rx_desc)
+int cpdma_set_num_rx_descs(struct cpdma_ctlr *ctlr, int num_rx_desc)
{
+ unsigned long flags;
+ int temp, ret;
+
+ spin_lock_irqsave(&ctlr->lock, flags);
+
+ temp = ctlr->num_rx_desc;
ctlr->num_rx_desc = num_rx_desc;
ctlr->num_tx_desc = ctlr->pool->num_desc - ctlr->num_rx_desc;
+ ret = cpdma_chan_split_pool(ctlr);
+ if (ret) {
+ ctlr->num_rx_desc = temp;
+ ctlr->num_tx_desc = ctlr->pool->num_desc - ctlr->num_rx_desc;
+ }
+
+ spin_unlock_irqrestore(&ctlr->lock, flags);
+
+ return ret;
}
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.h b/drivers/net/ethernet/ti/davinci_cpdma.h
index 0271a20c2e09..d3cfe234d16a 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.h
+++ b/drivers/net/ethernet/ti/davinci_cpdma.h
@@ -116,8 +116,7 @@ enum cpdma_control {
int cpdma_control_get(struct cpdma_ctlr *ctlr, int control);
int cpdma_control_set(struct cpdma_ctlr *ctlr, int control, int value);
int cpdma_get_num_rx_descs(struct cpdma_ctlr *ctlr);
-void cpdma_set_num_rx_descs(struct cpdma_ctlr *ctlr, int num_rx_desc);
+int cpdma_set_num_rx_descs(struct cpdma_ctlr *ctlr, int num_rx_desc);
int cpdma_get_num_tx_descs(struct cpdma_ctlr *ctlr);
-int cpdma_chan_split_pool(struct cpdma_ctlr *ctlr);
#endif
--
2.17.1
^ permalink raw reply related
* [PATCH v9 net-next 2/5] net: ethernet: ti: davinci_cpdma: add dma mapped submit
From: Ivan Khoronzhuk @ 2019-07-08 21:34 UTC (permalink / raw)
To: grygorii.strashko, hawk, davem
Cc: ast, linux-kernel, linux-omap, xdp-newbies, ilias.apalodimas,
netdev, daniel, jakub.kicinski, john.fastabend, Ivan Khoronzhuk
In-Reply-To: <20190708213432.8525-1-ivan.khoronzhuk@linaro.org>
In case if dma mapped packet needs to be sent, like with XDP
page pool, the "mapped" submit can be used. This patch adds dma
mapped submit based on regular one.
Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
---
v8..v9:
- fix warnings on arm64 caused by typos in type casting
drivers/net/ethernet/ti/davinci_cpdma.c | 89 ++++++++++++++++++++++---
drivers/net/ethernet/ti/davinci_cpdma.h | 4 ++
2 files changed, 83 insertions(+), 10 deletions(-)
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.c b/drivers/net/ethernet/ti/davinci_cpdma.c
index 5cf1758d425b..4e693c3aab27 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.c
+++ b/drivers/net/ethernet/ti/davinci_cpdma.c
@@ -139,6 +139,7 @@ struct submit_info {
int directed;
void *token;
void *data;
+ int flags;
int len;
};
@@ -184,6 +185,8 @@ static struct cpdma_control_info controls[] = {
(directed << CPDMA_TO_PORT_SHIFT)); \
} while (0)
+#define CPDMA_DMA_EXT_MAP BIT(16)
+
static void cpdma_desc_pool_destroy(struct cpdma_ctlr *ctlr)
{
struct cpdma_desc_pool *pool = ctlr->pool;
@@ -1015,6 +1018,7 @@ static int cpdma_chan_submit_si(struct submit_info *si)
struct cpdma_chan *chan = si->chan;
struct cpdma_ctlr *ctlr = chan->ctlr;
int len = si->len;
+ int swlen = len;
struct cpdma_desc __iomem *desc;
dma_addr_t buffer;
u32 mode;
@@ -1036,16 +1040,22 @@ static int cpdma_chan_submit_si(struct submit_info *si)
chan->stats.runt_transmit_buff++;
}
- buffer = dma_map_single(ctlr->dev, si->data, len, chan->dir);
- ret = dma_mapping_error(ctlr->dev, buffer);
- if (ret) {
- cpdma_desc_free(ctlr->pool, desc, 1);
- return -EINVAL;
- }
-
mode = CPDMA_DESC_OWNER | CPDMA_DESC_SOP | CPDMA_DESC_EOP;
cpdma_desc_to_port(chan, mode, si->directed);
+ if (si->flags & CPDMA_DMA_EXT_MAP) {
+ buffer = (dma_addr_t)si->data;
+ dma_sync_single_for_device(ctlr->dev, buffer, len, chan->dir);
+ swlen |= CPDMA_DMA_EXT_MAP;
+ } else {
+ buffer = dma_map_single(ctlr->dev, si->data, len, chan->dir);
+ ret = dma_mapping_error(ctlr->dev, buffer);
+ if (ret) {
+ cpdma_desc_free(ctlr->pool, desc, 1);
+ return -EINVAL;
+ }
+ }
+
/* Relaxed IO accessors can be used here as there is read barrier
* at the end of write sequence.
*/
@@ -1055,7 +1065,7 @@ static int cpdma_chan_submit_si(struct submit_info *si)
writel_relaxed(mode | len, &desc->hw_mode);
writel_relaxed((uintptr_t)si->token, &desc->sw_token);
writel_relaxed(buffer, &desc->sw_buffer);
- writel_relaxed(len, &desc->sw_len);
+ writel_relaxed(swlen, &desc->sw_len);
desc_read(desc, sw_len);
__cpdma_chan_submit(chan, desc);
@@ -1079,6 +1089,32 @@ int cpdma_chan_idle_submit(struct cpdma_chan *chan, void *token, void *data,
si.data = data;
si.len = len;
si.directed = directed;
+ si.flags = 0;
+
+ spin_lock_irqsave(&chan->lock, flags);
+ if (chan->state == CPDMA_STATE_TEARDOWN) {
+ spin_unlock_irqrestore(&chan->lock, flags);
+ return -EINVAL;
+ }
+
+ ret = cpdma_chan_submit_si(&si);
+ spin_unlock_irqrestore(&chan->lock, flags);
+ return ret;
+}
+
+int cpdma_chan_idle_submit_mapped(struct cpdma_chan *chan, void *token,
+ dma_addr_t data, int len, int directed)
+{
+ struct submit_info si;
+ unsigned long flags;
+ int ret;
+
+ si.chan = chan;
+ si.token = token;
+ si.data = (void *)data;
+ si.len = len;
+ si.directed = directed;
+ si.flags = CPDMA_DMA_EXT_MAP;
spin_lock_irqsave(&chan->lock, flags);
if (chan->state == CPDMA_STATE_TEARDOWN) {
@@ -1103,6 +1139,32 @@ int cpdma_chan_submit(struct cpdma_chan *chan, void *token, void *data,
si.data = data;
si.len = len;
si.directed = directed;
+ si.flags = 0;
+
+ spin_lock_irqsave(&chan->lock, flags);
+ if (chan->state != CPDMA_STATE_ACTIVE) {
+ spin_unlock_irqrestore(&chan->lock, flags);
+ return -EINVAL;
+ }
+
+ ret = cpdma_chan_submit_si(&si);
+ spin_unlock_irqrestore(&chan->lock, flags);
+ return ret;
+}
+
+int cpdma_chan_submit_mapped(struct cpdma_chan *chan, void *token,
+ dma_addr_t data, int len, int directed)
+{
+ struct submit_info si;
+ unsigned long flags;
+ int ret;
+
+ si.chan = chan;
+ si.token = token;
+ si.data = (void *)data;
+ si.len = len;
+ si.directed = directed;
+ si.flags = CPDMA_DMA_EXT_MAP;
spin_lock_irqsave(&chan->lock, flags);
if (chan->state != CPDMA_STATE_ACTIVE) {
@@ -1140,10 +1202,17 @@ static void __cpdma_chan_free(struct cpdma_chan *chan,
uintptr_t token;
token = desc_read(desc, sw_token);
- buff_dma = desc_read(desc, sw_buffer);
origlen = desc_read(desc, sw_len);
- dma_unmap_single(ctlr->dev, buff_dma, origlen, chan->dir);
+ buff_dma = desc_read(desc, sw_buffer);
+ if (origlen & CPDMA_DMA_EXT_MAP) {
+ origlen &= ~CPDMA_DMA_EXT_MAP;
+ dma_sync_single_for_cpu(ctlr->dev, buff_dma, origlen,
+ chan->dir);
+ } else {
+ dma_unmap_single(ctlr->dev, buff_dma, origlen, chan->dir);
+ }
+
cpdma_desc_free(pool, desc, 1);
(*chan->handler)((void *)token, outlen, status);
}
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.h b/drivers/net/ethernet/ti/davinci_cpdma.h
index 9343c8c73c1b..0271a20c2e09 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.h
+++ b/drivers/net/ethernet/ti/davinci_cpdma.h
@@ -77,8 +77,12 @@ int cpdma_chan_stop(struct cpdma_chan *chan);
int cpdma_chan_get_stats(struct cpdma_chan *chan,
struct cpdma_chan_stats *stats);
+int cpdma_chan_submit_mapped(struct cpdma_chan *chan, void *token,
+ dma_addr_t data, int len, int directed);
int cpdma_chan_submit(struct cpdma_chan *chan, void *token, void *data,
int len, int directed);
+int cpdma_chan_idle_submit_mapped(struct cpdma_chan *chan, void *token,
+ dma_addr_t data, int len, int directed);
int cpdma_chan_idle_submit(struct cpdma_chan *chan, void *token, void *data,
int len, int directed);
int cpdma_chan_process(struct cpdma_chan *chan, int quota);
--
2.17.1
^ permalink raw reply related
* [PATCH v9 net-next 4/5] net: ethernet: ti: cpsw_ethtool: allow res split while down
From: Ivan Khoronzhuk @ 2019-07-08 21:34 UTC (permalink / raw)
To: grygorii.strashko, hawk, davem
Cc: ast, linux-kernel, linux-omap, xdp-newbies, ilias.apalodimas,
netdev, daniel, jakub.kicinski, john.fastabend, Ivan Khoronzhuk
In-Reply-To: <20190708213432.8525-1-ivan.khoronzhuk@linaro.org>
That's possible to set channel num while interfaces are down. When
interface gets up it should resplit budget. This resplit can happen
after phy is up but only if speed is changed, so should be set before
this, for this allow it to happen while changing number of channels,
when interfaces are down.
Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
---
drivers/net/ethernet/ti/cpsw_ethtool.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/ti/cpsw_ethtool.c b/drivers/net/ethernet/ti/cpsw_ethtool.c
index c477e6b620d6..e4d7185fde49 100644
--- a/drivers/net/ethernet/ti/cpsw_ethtool.c
+++ b/drivers/net/ethernet/ti/cpsw_ethtool.c
@@ -620,8 +620,7 @@ int cpsw_set_channels_common(struct net_device *ndev,
}
}
- if (cpsw->usage_count)
- cpsw_split_res(cpsw);
+ cpsw_split_res(cpsw);
ret = cpsw_resume_data_pass(ndev);
if (!ret)
--
2.17.1
^ permalink raw reply related
* Re: [PATCH net-next 0/4] bnxt_en: Add XDP_REDIRECT support.
From: David Miller @ 2019-07-08 21:34 UTC (permalink / raw)
To: michael.chan; +Cc: gospo, netdev, hawk, ast
In-Reply-To: <1562398578-26020-1-git-send-email-michael.chan@broadcom.com>
From: Michael Chan <michael.chan@broadcom.com>
Date: Sat, 6 Jul 2019 03:36:14 -0400
> This patch series adds XDP_REDIRECT support by Andy Gospodarek.
>
> Andy Gospodarek (3):
> bnxt_en: rename some xdp functions
> bnxt_en: optimized XDP_REDIRECT support
> bnxt_en: add page_pool support
>
> Michael Chan (1):
> bnxt_en: Refactor __bnxt_xmit_xdp().
Andy et al. please respin this ASAP with the feedback you've received
and I will apply it.
Thank you.
^ permalink raw reply
* [PATCH v9 net-next 5/5] net: ethernet: ti: cpsw: add XDP support
From: Ivan Khoronzhuk @ 2019-07-08 21:34 UTC (permalink / raw)
To: grygorii.strashko, hawk, davem
Cc: ast, linux-kernel, linux-omap, xdp-newbies, ilias.apalodimas,
netdev, daniel, jakub.kicinski, john.fastabend, Ivan Khoronzhuk
In-Reply-To: <20190708213432.8525-1-ivan.khoronzhuk@linaro.org>
Add XDP support based on rx page_pool allocator, one frame per page.
Page pool allocator is used with assumption that only one rx_handler
is running simultaneously. DMA map/unmap is reused from page pool
despite there is no need to map whole page.
Due to specific of cpsw, the same TX/RX handler can be used by 2
network devices, so special fields in buffer are added to identify
an interface the frame is destined to. Thus XDP works for both
interfaces, that allows to test xdp redirect between two interfaces
easily. Also, each rx queue have own page pools, but common for both
netdevs.
XDP prog is common for all channels till appropriate changes are added
in XDP infrastructure. Also, once page_pool recycling becomes part of
skb netstack some simplifications can be added, like removing
page_pool_release_page() before skb receive.
In order to keep rx_dev while redirect, that can be somehow used in
future, do flush in rx_handler, that allows to keep rx dev the same
while redirect. It allows to conform with tracing rx_dev pointed
by Jesper.
Also, there is probability, that XDP generic code can be extended to
support multi ndev drivers like this one, using same rx queue for
several ndevs, based on switchdev for instance or else. In this case,
driver can be modified like exposed here:
https://lkml.org/lkml/2019/7/3/243
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
Signed-off-by: Ivan Khoronzhuk <ivan.khoronzhuk@linaro.org>
---
drivers/net/ethernet/ti/Kconfig | 1 +
drivers/net/ethernet/ti/cpsw.c | 502 ++++++++++++++++++++++---
drivers/net/ethernet/ti/cpsw_ethtool.c | 37 +-
drivers/net/ethernet/ti/cpsw_priv.h | 7 +
4 files changed, 488 insertions(+), 59 deletions(-)
diff --git a/drivers/net/ethernet/ti/Kconfig b/drivers/net/ethernet/ti/Kconfig
index a800d3417411..834afca3a019 100644
--- a/drivers/net/ethernet/ti/Kconfig
+++ b/drivers/net/ethernet/ti/Kconfig
@@ -50,6 +50,7 @@ config TI_CPSW
depends on ARCH_DAVINCI || ARCH_OMAP2PLUS || COMPILE_TEST
select TI_DAVINCI_MDIO
select MFD_SYSCON
+ select PAGE_POOL
select REGMAP
---help---
This driver supports TI's CPSW Ethernet Switch.
diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c
index 32b7b3b74a6b..f16aefd8870b 100644
--- a/drivers/net/ethernet/ti/cpsw.c
+++ b/drivers/net/ethernet/ti/cpsw.c
@@ -31,6 +31,10 @@
#include <linux/if_vlan.h>
#include <linux/kmemleak.h>
#include <linux/sys_soc.h>
+#include <net/page_pool.h>
+#include <linux/bpf.h>
+#include <linux/bpf_trace.h>
+#include <linux/filter.h>
#include <linux/pinctrl/consumer.h>
#include <net/pkt_cls.h>
@@ -60,6 +64,10 @@ static int descs_pool_size = CPSW_CPDMA_DESCS_POOL_SIZE_DEFAULT;
module_param(descs_pool_size, int, 0444);
MODULE_PARM_DESC(descs_pool_size, "Number of CPDMA CPPI descriptors in pool");
+/* The buf includes headroom compatible with both skb and xdpf */
+#define CPSW_HEADROOM_NA (max(XDP_PACKET_HEADROOM, NET_SKB_PAD) + NET_IP_ALIGN)
+#define CPSW_HEADROOM ALIGN(CPSW_HEADROOM_NA, sizeof(long))
+
#define for_each_slave(priv, func, arg...) \
do { \
struct cpsw_slave *slave; \
@@ -74,6 +82,11 @@ MODULE_PARM_DESC(descs_pool_size, "Number of CPDMA CPPI descriptors in pool");
(func)(slave++, ##arg); \
} while (0)
+#define CPSW_XMETA_OFFSET ALIGN(sizeof(struct xdp_frame), sizeof(long))
+
+#define CPSW_XDP_CONSUMED 1
+#define CPSW_XDP_PASS 0
+
static int cpsw_ndo_vlan_rx_add_vid(struct net_device *ndev,
__be16 proto, u16 vid);
@@ -337,24 +350,58 @@ void cpsw_intr_disable(struct cpsw_common *cpsw)
return;
}
+static int cpsw_is_xdpf_handle(void *handle)
+{
+ return (unsigned long)handle & BIT(0);
+}
+
+static void *cpsw_xdpf_to_handle(struct xdp_frame *xdpf)
+{
+ return (void *)((unsigned long)xdpf | BIT(0));
+}
+
+static struct xdp_frame *cpsw_handle_to_xdpf(void *handle)
+{
+ return (struct xdp_frame *)((unsigned long)handle & ~BIT(0));
+}
+
+struct __aligned(sizeof(long)) cpsw_meta_xdp {
+ struct net_device *ndev;
+ int ch;
+};
+
void cpsw_tx_handler(void *token, int len, int status)
{
+ struct cpsw_meta_xdp *xmeta;
+ struct xdp_frame *xdpf;
+ struct net_device *ndev;
struct netdev_queue *txq;
- struct sk_buff *skb = token;
- struct net_device *ndev = skb->dev;
- struct cpsw_common *cpsw = ndev_to_cpsw(ndev);
+ struct sk_buff *skb;
+ int ch;
+
+ if (cpsw_is_xdpf_handle(token)) {
+ xdpf = cpsw_handle_to_xdpf(token);
+ xmeta = (void *)xdpf + CPSW_XMETA_OFFSET;
+ ndev = xmeta->ndev;
+ ch = xmeta->ch;
+ xdp_return_frame(xdpf);
+ } else {
+ skb = token;
+ ndev = skb->dev;
+ ch = skb_get_queue_mapping(skb);
+ cpts_tx_timestamp(ndev_to_cpsw(ndev)->cpts, skb);
+ dev_kfree_skb_any(skb);
+ }
/* Check whether the queue is stopped due to stalled tx dma, if the
* queue is stopped then start the queue as we have free desc for tx
*/
- txq = netdev_get_tx_queue(ndev, skb_get_queue_mapping(skb));
+ txq = netdev_get_tx_queue(ndev, ch);
if (unlikely(netif_tx_queue_stopped(txq)))
netif_tx_wake_queue(txq);
- cpts_tx_timestamp(cpsw->cpts, skb);
ndev->stats.tx_packets++;
ndev->stats.tx_bytes += len;
- dev_kfree_skb_any(skb);
}
static void cpsw_rx_vlan_encap(struct sk_buff *skb)
@@ -400,24 +447,252 @@ static void cpsw_rx_vlan_encap(struct sk_buff *skb)
}
}
+static int cpsw_xdp_tx_frame(struct cpsw_priv *priv, struct xdp_frame *xdpf,
+ struct page *page)
+{
+ struct cpsw_common *cpsw = priv->cpsw;
+ struct cpsw_meta_xdp *xmeta;
+ struct cpdma_chan *txch;
+ dma_addr_t dma;
+ int ret, port;
+
+ xmeta = (void *)xdpf + CPSW_XMETA_OFFSET;
+ xmeta->ndev = priv->ndev;
+ xmeta->ch = 0;
+ txch = cpsw->txv[0].ch;
+
+ port = priv->emac_port + cpsw->data.dual_emac;
+ if (page) {
+ dma = page_pool_get_dma_addr(page);
+ dma += xdpf->headroom + sizeof(struct xdp_frame);
+ ret = cpdma_chan_submit_mapped(txch, cpsw_xdpf_to_handle(xdpf),
+ dma, xdpf->len, port);
+ } else {
+ if (sizeof(*xmeta) > xdpf->headroom) {
+ xdp_return_frame_rx_napi(xdpf);
+ return -EINVAL;
+ }
+
+ ret = cpdma_chan_submit(txch, cpsw_xdpf_to_handle(xdpf),
+ xdpf->data, xdpf->len, port);
+ }
+
+ if (ret) {
+ priv->ndev->stats.tx_dropped++;
+ xdp_return_frame_rx_napi(xdpf);
+ }
+
+ return ret;
+}
+
+static int cpsw_run_xdp(struct cpsw_priv *priv, int ch, struct xdp_buff *xdp,
+ struct page *page)
+{
+ struct cpsw_common *cpsw = priv->cpsw;
+ struct net_device *ndev = priv->ndev;
+ int ret = CPSW_XDP_CONSUMED;
+ struct xdp_frame *xdpf;
+ struct bpf_prog *prog;
+ u32 act;
+
+ rcu_read_lock();
+
+ prog = READ_ONCE(priv->xdp_prog);
+ if (!prog) {
+ ret = CPSW_XDP_PASS;
+ goto out;
+ }
+
+ act = bpf_prog_run_xdp(prog, xdp);
+ switch (act) {
+ case XDP_PASS:
+ ret = CPSW_XDP_PASS;
+ break;
+ case XDP_TX:
+ xdpf = convert_to_xdp_frame(xdp);
+ if (unlikely(!xdpf))
+ goto drop;
+
+ cpsw_xdp_tx_frame(priv, xdpf, page);
+ break;
+ case XDP_REDIRECT:
+ if (xdp_do_redirect(ndev, xdp, prog))
+ goto drop;
+
+ /* Have to flush here, per packet, instead of doing it in bulk
+ * at the end of the napi handler. The RX devices on this
+ * particular hardware is sharing a common queue, so the
+ * incoming device might change per packet.
+ */
+ xdp_do_flush_map();
+ break;
+ default:
+ bpf_warn_invalid_xdp_action(act);
+ /* fall through */
+ case XDP_ABORTED:
+ trace_xdp_exception(ndev, prog, act);
+ /* fall through -- handle aborts by dropping packet */
+ case XDP_DROP:
+ goto drop;
+ }
+out:
+ rcu_read_unlock();
+ return ret;
+drop:
+ rcu_read_unlock();
+ page_pool_recycle_direct(cpsw->page_pool[ch], page);
+ return ret;
+}
+
+static unsigned int cpsw_rxbuf_total_len(unsigned int len)
+{
+ len += CPSW_HEADROOM;
+ len += SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
+
+ return SKB_DATA_ALIGN(len);
+}
+
+static struct page_pool *cpsw_create_page_pool(struct cpsw_common *cpsw,
+ int size)
+{
+ struct page_pool_params pp_params;
+ struct page_pool *pool;
+
+ pp_params.order = 0;
+ pp_params.flags = PP_FLAG_DMA_MAP;
+ pp_params.pool_size = size;
+ pp_params.nid = NUMA_NO_NODE;
+ pp_params.dma_dir = DMA_BIDIRECTIONAL;
+ pp_params.dev = cpsw->dev;
+
+ pool = page_pool_create(&pp_params);
+ if (IS_ERR(pool))
+ dev_err(cpsw->dev, "cannot create rx page pool\n");
+
+ return pool;
+}
+
+static int cpsw_ndev_create_xdp_rxq(struct cpsw_priv *priv, int ch)
+{
+ struct cpsw_common *cpsw = priv->cpsw;
+ struct xdp_rxq_info *rxq;
+ struct page_pool *pool;
+ int ret;
+
+ pool = cpsw->page_pool[ch];
+ rxq = &priv->xdp_rxq[ch];
+
+ ret = xdp_rxq_info_reg(rxq, priv->ndev, ch);
+ if (ret)
+ return ret;
+
+ ret = xdp_rxq_info_reg_mem_model(rxq, MEM_TYPE_PAGE_POOL, pool);
+ if (ret)
+ xdp_rxq_info_unreg(rxq);
+
+ return ret;
+}
+
+static void cpsw_ndev_destroy_xdp_rxq(struct cpsw_priv *priv, int ch)
+{
+ struct xdp_rxq_info *rxq = &priv->xdp_rxq[ch];
+
+ if (!xdp_rxq_info_is_reg(rxq))
+ return;
+
+ xdp_rxq_info_unreg(rxq);
+}
+
+static int cpsw_create_rx_pool(struct cpsw_common *cpsw, int ch)
+{
+ struct page_pool *pool;
+ int ret = 0, pool_size;
+
+ pool_size = cpdma_chan_get_rx_buf_num(cpsw->rxv[ch].ch);
+ pool = cpsw_create_page_pool(cpsw, pool_size);
+ if (IS_ERR(pool))
+ ret = PTR_ERR(pool);
+ else
+ cpsw->page_pool[ch] = pool;
+
+ return ret;
+}
+
+void cpsw_destroy_xdp_rxqs(struct cpsw_common *cpsw)
+{
+ struct net_device *ndev;
+ int i, ch;
+
+ for (ch = 0; ch < cpsw->rx_ch_num; ch++) {
+ for (i = 0; i < cpsw->data.slaves; i++) {
+ ndev = cpsw->slaves[i].ndev;
+ if (!ndev)
+ continue;
+
+ cpsw_ndev_destroy_xdp_rxq(netdev_priv(ndev), ch);
+ }
+
+ page_pool_destroy(cpsw->page_pool[ch]);
+ cpsw->page_pool[ch] = NULL;
+ }
+}
+
+int cpsw_create_xdp_rxqs(struct cpsw_common *cpsw)
+{
+ struct net_device *ndev;
+ int i, ch, ret;
+
+ for (ch = 0; ch < cpsw->rx_ch_num; ch++) {
+ ret = cpsw_create_rx_pool(cpsw, ch);
+ if (ret)
+ goto err_cleanup;
+
+ /* using same page pool is allowed as no running rx handlers
+ * simultaneously for both ndevs
+ */
+ for (i = 0; i < cpsw->data.slaves; i++) {
+ ndev = cpsw->slaves[i].ndev;
+ if (!ndev)
+ continue;
+
+ ret = cpsw_ndev_create_xdp_rxq(netdev_priv(ndev), ch);
+ if (ret)
+ goto err_cleanup;
+ }
+ }
+
+ return 0;
+
+err_cleanup:
+ cpsw_destroy_xdp_rxqs(cpsw);
+
+ return ret;
+}
+
static void cpsw_rx_handler(void *token, int len, int status)
{
- struct cpdma_chan *ch;
- struct sk_buff *skb = token;
- struct sk_buff *new_skb;
- struct net_device *ndev = skb->dev;
- int ret = 0, port;
- struct cpsw_common *cpsw = ndev_to_cpsw(ndev);
+ struct page *new_page, *page = token;
+ void *pa = page_address(page);
+ struct cpsw_meta_xdp *xmeta = pa + CPSW_XMETA_OFFSET;
+ struct cpsw_common *cpsw = ndev_to_cpsw(xmeta->ndev);
+ int pkt_size = cpsw->rx_packet_max;
+ int ret = 0, port, ch = xmeta->ch;
+ int headroom = CPSW_HEADROOM;
+ struct net_device *ndev = xmeta->ndev;
struct cpsw_priv *priv;
+ struct page_pool *pool;
+ struct sk_buff *skb;
+ struct xdp_buff xdp;
+ dma_addr_t dma;
- if (cpsw->data.dual_emac) {
+ if (cpsw->data.dual_emac && status >= 0) {
port = CPDMA_RX_SOURCE_PORT(status);
- if (port) {
+ if (port)
ndev = cpsw->slaves[--port].ndev;
- skb->dev = ndev;
- }
}
+ priv = netdev_priv(ndev);
+ pool = cpsw->page_pool[ch];
if (unlikely(status < 0) || unlikely(!netif_running(ndev))) {
/* In dual emac mode check for all interfaces */
if (cpsw->data.dual_emac && cpsw->usage_count &&
@@ -426,43 +701,87 @@ static void cpsw_rx_handler(void *token, int len, int status)
* is already down and the other interface is up
* and running, instead of freeing which results
* in reducing of the number of rx descriptor in
- * DMA engine, requeue skb back to cpdma.
+ * DMA engine, requeue page back to cpdma.
*/
- new_skb = skb;
+ new_page = page;
goto requeue;
}
- /* the interface is going down, skbs are purged */
- dev_kfree_skb_any(skb);
+ /* the interface is going down, pages are purged */
+ page_pool_recycle_direct(pool, page);
return;
}
- new_skb = netdev_alloc_skb_ip_align(ndev, cpsw->rx_packet_max);
- if (new_skb) {
- skb_copy_queue_mapping(new_skb, skb);
- skb_put(skb, len);
- if (status & CPDMA_RX_VLAN_ENCAP)
- cpsw_rx_vlan_encap(skb);
- priv = netdev_priv(ndev);
- if (priv->rx_ts_enabled)
- cpts_rx_timestamp(cpsw->cpts, skb);
- skb->protocol = eth_type_trans(skb, ndev);
- netif_receive_skb(skb);
- ndev->stats.rx_bytes += len;
- ndev->stats.rx_packets++;
- kmemleak_not_leak(new_skb);
- } else {
+ new_page = page_pool_dev_alloc_pages(pool);
+ if (unlikely(!new_page)) {
+ new_page = page;
ndev->stats.rx_dropped++;
- new_skb = skb;
+ goto requeue;
+ }
+
+ if (priv->xdp_prog) {
+ if (status & CPDMA_RX_VLAN_ENCAP) {
+ xdp.data = pa + CPSW_HEADROOM +
+ CPSW_RX_VLAN_ENCAP_HDR_SIZE;
+ xdp.data_end = xdp.data + len -
+ CPSW_RX_VLAN_ENCAP_HDR_SIZE;
+ } else {
+ xdp.data = pa + CPSW_HEADROOM;
+ xdp.data_end = xdp.data + len;
+ }
+
+ xdp_set_data_meta_invalid(&xdp);
+
+ xdp.data_hard_start = pa;
+ xdp.rxq = &priv->xdp_rxq[ch];
+
+ ret = cpsw_run_xdp(priv, ch, &xdp, page);
+ if (ret != CPSW_XDP_PASS)
+ goto requeue;
+
+ /* XDP prog might have changed packet data and boundaries */
+ len = xdp.data_end - xdp.data;
+ headroom = xdp.data - xdp.data_hard_start;
+
+ /* XDP prog can modify vlan tag, so can't use encap header */
+ status &= ~CPDMA_RX_VLAN_ENCAP;
}
+ /* pass skb to netstack if no XDP prog or returned XDP_PASS */
+ skb = build_skb(pa, cpsw_rxbuf_total_len(pkt_size));
+ if (!skb) {
+ ndev->stats.rx_dropped++;
+ page_pool_recycle_direct(pool, page);
+ goto requeue;
+ }
+
+ skb_reserve(skb, headroom);
+ skb_put(skb, len);
+ skb->dev = ndev;
+ if (status & CPDMA_RX_VLAN_ENCAP)
+ cpsw_rx_vlan_encap(skb);
+ if (priv->rx_ts_enabled)
+ cpts_rx_timestamp(cpsw->cpts, skb);
+ skb->protocol = eth_type_trans(skb, ndev);
+
+ /* unmap page as no netstack skb page recycling */
+ page_pool_release_page(pool, page);
+ netif_receive_skb(skb);
+
+ ndev->stats.rx_bytes += len;
+ ndev->stats.rx_packets++;
+
requeue:
- ch = cpsw->rxv[skb_get_queue_mapping(new_skb)].ch;
- ret = cpdma_chan_submit(ch, new_skb, new_skb->data,
- skb_tailroom(new_skb), 0);
+ xmeta = page_address(new_page) + CPSW_XMETA_OFFSET;
+ xmeta->ndev = ndev;
+ xmeta->ch = ch;
+
+ dma = page_pool_get_dma_addr(new_page) + CPSW_HEADROOM;
+ ret = cpdma_chan_submit_mapped(cpsw->rxv[ch].ch, new_page, dma,
+ pkt_size, 0);
if (ret < 0) {
WARN_ON(ret == -ENOMEM);
- dev_kfree_skb_any(new_skb);
+ page_pool_recycle_direct(pool, new_page);
}
}
@@ -1032,33 +1351,39 @@ static void cpsw_init_host_port(struct cpsw_priv *priv)
int cpsw_fill_rx_channels(struct cpsw_priv *priv)
{
struct cpsw_common *cpsw = priv->cpsw;
- struct sk_buff *skb;
+ struct cpsw_meta_xdp *xmeta;
+ struct page_pool *pool;
+ struct page *page;
int ch_buf_num;
int ch, i, ret;
+ dma_addr_t dma;
for (ch = 0; ch < cpsw->rx_ch_num; ch++) {
+ pool = cpsw->page_pool[ch];
ch_buf_num = cpdma_chan_get_rx_buf_num(cpsw->rxv[ch].ch);
for (i = 0; i < ch_buf_num; i++) {
- skb = __netdev_alloc_skb_ip_align(priv->ndev,
- cpsw->rx_packet_max,
- GFP_KERNEL);
- if (!skb) {
- cpsw_err(priv, ifup, "cannot allocate skb\n");
+ page = page_pool_dev_alloc_pages(pool);
+ if (!page) {
+ cpsw_err(priv, ifup, "allocate rx page err\n");
return -ENOMEM;
}
- skb_set_queue_mapping(skb, ch);
- ret = cpdma_chan_idle_submit(cpsw->rxv[ch].ch, skb,
- skb->data,
- skb_tailroom(skb), 0);
+ xmeta = page_address(page) + CPSW_XMETA_OFFSET;
+ xmeta->ndev = priv->ndev;
+ xmeta->ch = ch;
+
+ dma = page_pool_get_dma_addr(page) + CPSW_HEADROOM;
+ ret = cpdma_chan_idle_submit_mapped(cpsw->rxv[ch].ch,
+ page, dma,
+ cpsw->rx_packet_max,
+ 0);
if (ret < 0) {
cpsw_err(priv, ifup,
- "cannot submit skb to channel %d rx, error %d\n",
+ "cannot submit page to channel %d rx, error %d\n",
ch, ret);
- kfree_skb(skb);
+ page_pool_recycle_direct(pool, page);
return ret;
}
- kmemleak_not_leak(skb);
}
cpsw_info(priv, ifup, "ch %d rx, submitted %d descriptors\n",
@@ -1394,6 +1719,13 @@ static int cpsw_ndo_open(struct net_device *ndev)
enable_irq(cpsw->irqs_table[0]);
}
+ /* create rxqs for both infs in dual mac as they use same pool
+ * and must be destroyed together when no users.
+ */
+ ret = cpsw_create_xdp_rxqs(cpsw);
+ if (ret < 0)
+ goto err_cleanup;
+
ret = cpsw_fill_rx_channels(priv);
if (ret < 0)
goto err_cleanup;
@@ -1422,9 +1754,10 @@ static int cpsw_ndo_open(struct net_device *ndev)
err_cleanup:
if (!cpsw->usage_count) {
cpdma_ctlr_stop(cpsw->dma);
- for_each_slave(priv, cpsw_slave_stop, cpsw);
+ cpsw_destroy_xdp_rxqs(cpsw);
}
+ for_each_slave(priv, cpsw_slave_stop, cpsw);
pm_runtime_put_sync(cpsw->dev);
netif_carrier_off(priv->ndev);
return ret;
@@ -1447,6 +1780,7 @@ static int cpsw_ndo_stop(struct net_device *ndev)
cpsw_intr_disable(cpsw);
cpdma_ctlr_stop(cpsw->dma);
cpsw_ale_stop(cpsw->ale);
+ cpsw_destroy_xdp_rxqs(cpsw);
}
for_each_slave(priv, cpsw_slave_stop, cpsw);
@@ -2004,6 +2338,64 @@ static int cpsw_ndo_setup_tc(struct net_device *ndev, enum tc_setup_type type,
}
}
+static int cpsw_xdp_prog_setup(struct cpsw_priv *priv, struct netdev_bpf *bpf)
+{
+ struct bpf_prog *prog = bpf->prog;
+
+ if (!priv->xdpi.prog && !prog)
+ return 0;
+
+ if (!xdp_attachment_flags_ok(&priv->xdpi, bpf))
+ return -EBUSY;
+
+ WRITE_ONCE(priv->xdp_prog, prog);
+
+ xdp_attachment_setup(&priv->xdpi, bpf);
+
+ return 0;
+}
+
+static int cpsw_ndo_bpf(struct net_device *ndev, struct netdev_bpf *bpf)
+{
+ struct cpsw_priv *priv = netdev_priv(ndev);
+
+ switch (bpf->command) {
+ case XDP_SETUP_PROG:
+ return cpsw_xdp_prog_setup(priv, bpf);
+
+ case XDP_QUERY_PROG:
+ return xdp_attachment_query(&priv->xdpi, bpf);
+
+ default:
+ return -EINVAL;
+ }
+}
+
+static int cpsw_ndo_xdp_xmit(struct net_device *ndev, int n,
+ struct xdp_frame **frames, u32 flags)
+{
+ struct cpsw_priv *priv = netdev_priv(ndev);
+ struct xdp_frame *xdpf;
+ int i, drops = 0;
+
+ if (unlikely(flags & ~XDP_XMIT_FLAGS_MASK))
+ return -EINVAL;
+
+ for (i = 0; i < n; i++) {
+ xdpf = frames[i];
+ if (xdpf->len < CPSW_MIN_PACKET_SIZE) {
+ xdp_return_frame_rx_napi(xdpf);
+ drops++;
+ continue;
+ }
+
+ if (cpsw_xdp_tx_frame(priv, xdpf, NULL))
+ drops++;
+ }
+
+ return n - drops;
+}
+
#ifdef CONFIG_NET_POLL_CONTROLLER
static void cpsw_ndo_poll_controller(struct net_device *ndev)
{
@@ -2032,6 +2424,8 @@ static const struct net_device_ops cpsw_netdev_ops = {
.ndo_vlan_rx_add_vid = cpsw_ndo_vlan_rx_add_vid,
.ndo_vlan_rx_kill_vid = cpsw_ndo_vlan_rx_kill_vid,
.ndo_setup_tc = cpsw_ndo_setup_tc,
+ .ndo_bpf = cpsw_ndo_bpf,
+ .ndo_xdp_xmit = cpsw_ndo_xdp_xmit,
};
static void cpsw_get_drvinfo(struct net_device *ndev,
diff --git a/drivers/net/ethernet/ti/cpsw_ethtool.c b/drivers/net/ethernet/ti/cpsw_ethtool.c
index e4d7185fde49..31248a6cc642 100644
--- a/drivers/net/ethernet/ti/cpsw_ethtool.c
+++ b/drivers/net/ethernet/ti/cpsw_ethtool.c
@@ -578,6 +578,18 @@ static int cpsw_update_channels_res(struct cpsw_priv *priv, int ch_num, int rx,
return 0;
}
+static void cpsw_fail(struct cpsw_common *cpsw)
+{
+ struct net_device *ndev;
+ int i;
+
+ for (i = 0; i < cpsw->data.slaves; i++) {
+ ndev = cpsw->slaves[i].ndev;
+ if (ndev)
+ dev_close(ndev);
+ }
+}
+
int cpsw_set_channels_common(struct net_device *ndev,
struct ethtool_channels *chs,
cpdma_handler_fn rx_handler)
@@ -585,7 +597,7 @@ int cpsw_set_channels_common(struct net_device *ndev,
struct cpsw_priv *priv = netdev_priv(ndev);
struct cpsw_common *cpsw = priv->cpsw;
struct net_device *sl_ndev;
- int i, ret;
+ int i, new_pools, ret;
ret = cpsw_check_ch_settings(cpsw, chs);
if (ret < 0)
@@ -593,6 +605,8 @@ int cpsw_set_channels_common(struct net_device *ndev,
cpsw_suspend_data_pass(ndev);
+ new_pools = (chs->rx_count != cpsw->rx_ch_num) && cpsw->usage_count;
+
ret = cpsw_update_channels_res(priv, chs->rx_count, 1, rx_handler);
if (ret)
goto err;
@@ -622,12 +636,19 @@ int cpsw_set_channels_common(struct net_device *ndev,
cpsw_split_res(cpsw);
+ if (new_pools) {
+ cpsw_destroy_xdp_rxqs(cpsw);
+ ret = cpsw_create_xdp_rxqs(cpsw);
+ if (ret)
+ goto err;
+ }
+
ret = cpsw_resume_data_pass(ndev);
if (!ret)
return 0;
err:
dev_err(priv->dev, "cannot update channels number, closing device\n");
- dev_close(ndev);
+ cpsw_fail(cpsw);
return ret;
}
@@ -647,8 +668,7 @@ void cpsw_get_ringparam(struct net_device *ndev,
int cpsw_set_ringparam(struct net_device *ndev,
struct ethtool_ringparam *ering)
{
- struct cpsw_priv *priv = netdev_priv(ndev);
- struct cpsw_common *cpsw = priv->cpsw;
+ struct cpsw_common *cpsw = ndev_to_cpsw(ndev);
int descs_num, ret;
/* ignore ering->tx_pending - only rx_pending adjustment is supported */
@@ -672,13 +692,20 @@ int cpsw_set_ringparam(struct net_device *ndev,
return ret;
}
+ if (cpsw->usage_count) {
+ cpsw_destroy_xdp_rxqs(cpsw);
+ ret = cpsw_create_xdp_rxqs(cpsw);
+ if (ret)
+ goto err;
+ }
+
ret = cpsw_resume_data_pass(ndev);
if (!ret)
return 0;
err:
cpdma_set_num_rx_descs(cpsw->dma, descs_num);
dev_err(cpsw->dev, "cannot set ring params, closing device\n");
- dev_close(ndev);
+ cpsw_fail(cpsw);
return ret;
}
diff --git a/drivers/net/ethernet/ti/cpsw_priv.h b/drivers/net/ethernet/ti/cpsw_priv.h
index 04795b97ee71..086bf38a4736 100644
--- a/drivers/net/ethernet/ti/cpsw_priv.h
+++ b/drivers/net/ethernet/ti/cpsw_priv.h
@@ -346,6 +346,7 @@ struct cpsw_common {
int rx_ch_num, tx_ch_num;
int speed;
int usage_count;
+ struct page_pool *page_pool[CPSW_MAX_QUEUES];
};
struct cpsw_priv {
@@ -360,6 +361,10 @@ struct cpsw_priv {
int shp_cfg_speed;
int tx_ts_enabled;
int rx_ts_enabled;
+ struct bpf_prog *xdp_prog;
+ struct xdp_rxq_info xdp_rxq[CPSW_MAX_QUEUES];
+ struct xdp_attachment_info xdpi;
+
u32 emac_port;
struct cpsw_common *cpsw;
};
@@ -391,6 +396,8 @@ int cpsw_fill_rx_channels(struct cpsw_priv *priv);
void cpsw_intr_enable(struct cpsw_common *cpsw);
void cpsw_intr_disable(struct cpsw_common *cpsw);
void cpsw_tx_handler(void *token, int len, int status);
+int cpsw_create_xdp_rxqs(struct cpsw_common *cpsw);
+void cpsw_destroy_xdp_rxqs(struct cpsw_common *cpsw);
/* ethtool */
u32 cpsw_get_msglevel(struct net_device *ndev);
--
2.17.1
^ permalink raw reply related
* [PATCH v9 net-next 0/5] net: ethernet: ti: cpsw: Add XDP support
From: Ivan Khoronzhuk @ 2019-07-08 21:34 UTC (permalink / raw)
To: grygorii.strashko, hawk, davem
Cc: ast, linux-kernel, linux-omap, xdp-newbies, ilias.apalodimas,
netdev, daniel, jakub.kicinski, john.fastabend, Ivan Khoronzhuk
This patchset adds XDP support for TI cpsw driver and base it on
page_pool allocator. It was verified on af_xdp socket drop,
af_xdp l2f, ebpf XDP_DROP, XDP_REDIRECT, XDP_PASS, XDP_TX.
It was verified with following configs enabled:
CONFIG_JIT=y
CONFIG_BPFILTER=y
CONFIG_BPF_SYSCALL=y
CONFIG_XDP_SOCKETS=y
CONFIG_BPF_EVENTS=y
CONFIG_HAVE_EBPF_JIT=y
CONFIG_BPF_JIT=y
CONFIG_CGROUP_BPF=y
Link on previous v7:
https://lkml.org/lkml/2019/7/4/715
Also regular tests with iperf2 were done in order to verify impact on
regular netstack performance, compared with base commit:
https://pastebin.com/JSMT0iZ4
v8..v9:
- fix warnings on arm64 caused by typos in type casting
v7..v8:
- corrected dma calculation based on headroom instead of hard start
- minor comment changes
v6..v7:
- rolled back to v4 solution but with small modification
- picked up patch:
https://www.spinics.net/lists/netdev/msg583145.html
- added changes related to netsec fix and cpsw
v5..v6:
- do changes that is rx_dev while redirect/flush cycle is kept the same
- dropped net: ethernet: ti: davinci_cpdma: return handler status
- other changes desc in patches
v4..v5:
- added two plreliminary patches:
net: ethernet: ti: davinci_cpdma: allow desc split while down
net: ethernet: ti: cpsw_ethtool: allow res split while down
- added xdp alocator refcnt on xdp level, avoiding page pool refcnt
- moved flush status as separate argument for cpdma_chan_process
- reworked cpsw code according to last changes to allocator
- added missed statistic counter
v3..v4:
- added page pool user counter
- use same pool for ndevs in dual mac
- restructured page pool create/destroy according to the last changes in API
v2..v3:
- each rxq and ndev has its own page pool
v1..v2:
- combined xdp_xmit functions
- used page allocation w/o refcnt juggle
- unmapped page for skb netstack
- moved rxq/page pool allocation to open/close pair
- added several preliminary patches:
net: page_pool: add helper function to retrieve dma addresses
net: page_pool: add helper function to unmap dma addresses
net: ethernet: ti: cpsw: use cpsw as drv data
net: ethernet: ti: cpsw_ethtool: simplify slave loops
Ivan Khoronzhuk (5):
net: core: page_pool: add user refcnt and reintroduce
page_pool_destroy
net: ethernet: ti: davinci_cpdma: add dma mapped submit
net: ethernet: ti: davinci_cpdma: allow desc split while down
net: ethernet: ti: cpsw_ethtool: allow res split while down
net: ethernet: ti: cpsw: add XDP support
.../net/ethernet/mellanox/mlx5/core/en_main.c | 4 +-
drivers/net/ethernet/socionext/netsec.c | 8 +-
drivers/net/ethernet/ti/Kconfig | 1 +
drivers/net/ethernet/ti/cpsw.c | 502 ++++++++++++++++--
drivers/net/ethernet/ti/cpsw_ethtool.c | 57 +-
drivers/net/ethernet/ti/cpsw_priv.h | 7 +
drivers/net/ethernet/ti/davinci_cpdma.c | 106 +++-
drivers/net/ethernet/ti/davinci_cpdma.h | 7 +-
include/net/page_pool.h | 25 +
net/core/page_pool.c | 8 +
net/core/xdp.c | 3 +
11 files changed, 640 insertions(+), 88 deletions(-)
--
2.17.1
^ permalink raw reply
* Re: [PATCH bpf] xdp: fix possible cq entry leak
From: William Tu @ 2019-07-08 21:25 UTC (permalink / raw)
To: Björn Töpel
Cc: Ilya Maximets, Netdev, LKML, bpf, Xdp, David S. Miller,
Björn Töpel, Magnus Karlsson, Jonathan Lemon,
Jakub Kicinski, Alexei Starovoitov, Daniel Borkmann
In-Reply-To: <CAJ+HfNi2EdLwtq9SfccZBymDMv_cW5+vxB-JLqxyvYS_TG3ScA@mail.gmail.com>
On Thu, Jul 4, 2019 at 11:49 PM Björn Töpel <bjorn.topel@gmail.com> wrote:
>
> On Thu, 4 Jul 2019 at 16:25, Ilya Maximets <i.maximets@samsung.com> wrote:
> >
> > Completion queue address reservation could not be undone.
> > In case of bad 'queue_id' or skb allocation failure, reserved entry
> > will be leaked reducing the total capacity of completion queue.
> >
> > Fix that by moving reservation to the point where failure is not
> > possible. Additionally, 'queue_id' checking moved out from the loop
> > since there is no point to check it there.
> >
>
> Good catch, Ilya! Thanks for the patch!
>
> Acked-by: Björn Töpel <bjorn.topel@intel.com>
>
Thanks
Tested-by: William Tu <u9012063@gmail.com>
^ permalink raw reply
* Re: [PATCH bpf-next] selftests/bpf: make verifier loop tests arch independent
From: Stanislav Fomichev @ 2019-07-08 21:20 UTC (permalink / raw)
To: Ilya Leoshkevich
Cc: Y Song, Stanislav Fomichev, netdev, bpf, David Miller,
Alexei Starovoitov, Daniel Borkmann
In-Reply-To: <99593C98-5DEC-4B18-AE6D-271DD8A8A7F6@linux.ibm.com>
On 07/08, Ilya Leoshkevich wrote:
>
>
> > Am 08.07.2019 um 18:13 schrieb Stanislav Fomichev <sdf@fomichev.me>:
> >
> > On 07/03, Y Song wrote:
> >> On Wed, Jul 3, 2019 at 1:51 PM Stanislav Fomichev <sdf@google.com> wrote:
> >>>
> >>> Take the first x bytes of pt_regs for scalability tests, there is
> >>> no real reason we need x86 specific rax.
> >>>
> >>> Signed-off-by: Stanislav Fomichev <sdf@google.com>
> >>> ---
> >>> tools/testing/selftests/bpf/progs/loop1.c | 3 ++-
> >>> tools/testing/selftests/bpf/progs/loop2.c | 3 ++-
> >>> tools/testing/selftests/bpf/progs/loop3.c | 3 ++-
> >>> 3 files changed, 6 insertions(+), 3 deletions(-)
> >>>
> >>> diff --git a/tools/testing/selftests/bpf/progs/loop1.c b/tools/testing/selftests/bpf/progs/loop1.c
> >>> index dea395af9ea9..d530c61d2517 100644
> >>> --- a/tools/testing/selftests/bpf/progs/loop1.c
> >>> +++ b/tools/testing/selftests/bpf/progs/loop1.c
> >>> @@ -14,11 +14,12 @@ SEC("raw_tracepoint/kfree_skb")
> >>> int nested_loops(volatile struct pt_regs* ctx)
> >>> {
> >>> int i, j, sum = 0, m;
> >>> + volatile int *any_reg = (volatile int *)ctx;
> >>>
> >>> for (j = 0; j < 300; j++)
> >>> for (i = 0; i < j; i++) {
> >>> if (j & 1)
> >>> - m = ctx->rax;
> >>> + m = *any_reg;
> >>
> >> I agree. ctx->rax here is only to generate some operations, which
> >> cannot be optimized away by the compiler. dereferencing a volatile
> >> pointee may just serve that purpose.
> >>
> >> Comparing the byte code generated with ctx->rax and *any_reg, they are
> >> slightly different. Using *any_reg is slighly worse, but this should
> >> be still okay for the test.
> >>
> >>> else
> >>> m = j;
> >>> sum += i * m;
> >>> diff --git a/tools/testing/selftests/bpf/progs/loop2.c b/tools/testing/selftests/bpf/progs/loop2.c
> >>> index 0637bd8e8bcf..91bb89d901e3 100644
> >>> --- a/tools/testing/selftests/bpf/progs/loop2.c
> >>> +++ b/tools/testing/selftests/bpf/progs/loop2.c
> >>> @@ -14,9 +14,10 @@ SEC("raw_tracepoint/consume_skb")
> >>> int while_true(volatile struct pt_regs* ctx)
> >>> {
> >>> int i = 0;
> >>> + volatile int *any_reg = (volatile int *)ctx;
> >>>
> >>> while (true) {
> >>> - if (ctx->rax & 1)
> >>> + if (*any_reg & 1)
> >>> i += 3;
> >>> else
> >>> i += 7;
> >>> diff --git a/tools/testing/selftests/bpf/progs/loop3.c b/tools/testing/selftests/bpf/progs/loop3.c
> >>> index 30a0f6cba080..3a7f12d7186c 100644
> >>> --- a/tools/testing/selftests/bpf/progs/loop3.c
> >>> +++ b/tools/testing/selftests/bpf/progs/loop3.c
> >>> @@ -14,9 +14,10 @@ SEC("raw_tracepoint/consume_skb")
> >>> int while_true(volatile struct pt_regs* ctx)
> >>> {
> >>> __u64 i = 0, sum = 0;
> >>> + volatile __u64 *any_reg = (volatile __u64 *)ctx;
> >>> do {
> >>> i++;
> >>> - sum += ctx->rax;
> >>> + sum += *any_reg;
> >>> } while (i < 0x100000000ULL);
> >>> return sum;
> >>> }
> >>> --
> >>> 2.22.0.410.gd8fdbe21b5-goog
> >>
> >> Ilya Leoshkevich (iii@linux.ibm.com, cc'ed) has another patch set
> >> trying to solve this problem by introducing s360 arch register access
> >> macros. I guess for now that patch set is not needed any more?
> > Oh, I missed them. Do they fix the tests for other (non-s360) arches as
> > well? I was trying to fix the issue by not depending on any arch
> > specific stuff because the test really doesn't care :-)
>
> They are supposed to work for everything that defines PT_REGS_RC in
> bpf_helpers.h, but I have to admit I tested only x86_64 and s390.
>
> The main source of problems with my approach were mismatching definitions
> of struct pt_regs for userspace and kernel, and because of that there was
> some tweaking required for both arches. I will double check how it looks
> for others (arm, mips, ppc, sparc) tomorrow.
Thanks, I've tested your patches and they fix my issue as well. So you
can have my Tested-by if we'd go with your approach.
One thing I don't understand is: why do you add 'ifdef __KERNEL__' to
the bpf_helpers.h for x86 case? Who is using bpf_helpers.h with
__KERNEL__ defined? Is it perf?
> Best regards,
> Ilya
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox