* [net-next 01/12] ixgbe: add status reg reads to ixgbe_check_remove
From: Jeff Kirsher @ 2018-03-23 23:16 UTC (permalink / raw)
To: davem; +Cc: Paul Greenwalt, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180323231631.32502-1-jeffrey.t.kirsher@intel.com>
From: Paul Greenwalt <paul.greenwalt@intel.com>
Add status register reads and delay between reads to ixgbe_check_remove.
Registers can read 0xFFFFFFFF during PCI reset, which causes the driver
to remove the adapter. The additional status register reads can reduce the
chance of this race condition.
If the status register is not 0xFFFFFFFF, then ixgbe_check_remove returns
the value of the register being read.
Signed-off-by: Paul Greenwalt <paul.greenwalt@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe_common.h | 1 +
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 31 ++++++++++++++++---------
2 files changed, 21 insertions(+), 11 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.h b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.h
index 67f304289fd9..2b311382167a 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.h
@@ -154,6 +154,7 @@ s32 ixgbe_setup_mac_link_multispeed_fiber(struct ixgbe_hw *hw,
void ixgbe_set_soft_rate_select_speed(struct ixgbe_hw *hw,
ixgbe_link_speed speed);
+#define IXGBE_FAILED_READ_RETRIES 5
#define IXGBE_FAILED_READ_REG 0xffffffffU
#define IXGBE_FAILED_READ_CFG_DWORD 0xffffffffU
#define IXGBE_FAILED_READ_CFG_WORD 0xffffU
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 85369423452d..faf368b14389 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -353,23 +353,32 @@ static void ixgbe_remove_adapter(struct ixgbe_hw *hw)
ixgbe_service_event_schedule(adapter);
}
-static void ixgbe_check_remove(struct ixgbe_hw *hw, u32 reg)
+static u32 ixgbe_check_remove(struct ixgbe_hw *hw, u32 reg)
{
+ u8 __iomem *reg_addr;
u32 value;
+ int i;
- /* The following check not only optimizes a bit by not
- * performing a read on the status register when the
- * register just read was a status register read that
- * returned IXGBE_FAILED_READ_REG. It also blocks any
- * potential recursion.
+ reg_addr = READ_ONCE(hw->hw_addr);
+ if (ixgbe_removed(reg_addr))
+ return IXGBE_FAILED_READ_REG;
+
+ /* Register read of 0xFFFFFFF can indicate the adapter has been removed,
+ * so perform several status register reads to determine if the adapter
+ * has been removed.
*/
- if (reg == IXGBE_STATUS) {
- ixgbe_remove_adapter(hw);
- return;
+ for (i = 0; i < IXGBE_FAILED_READ_RETRIES; i++) {
+ value = readl(reg_addr + IXGBE_STATUS);
+ if (value != IXGBE_FAILED_READ_REG)
+ break;
+ mdelay(3);
}
- value = ixgbe_read_reg(hw, IXGBE_STATUS);
+
if (value == IXGBE_FAILED_READ_REG)
ixgbe_remove_adapter(hw);
+ else
+ value = readl(reg_addr + reg);
+ return value;
}
/**
@@ -415,7 +424,7 @@ u32 ixgbe_read_reg(struct ixgbe_hw *hw, u32 reg)
writes_completed:
value = readl(reg_addr + reg);
if (unlikely(value == IXGBE_FAILED_READ_REG))
- ixgbe_check_remove(hw, reg);
+ value = ixgbe_check_remove(hw, reg);
return value;
}
--
2.14.3
^ permalink raw reply related
* [net-next 02/12] ixgbe: fix read-modify-write in x550 phy setup
From: Jeff Kirsher @ 2018-03-23 23:16 UTC (permalink / raw)
To: davem; +Cc: Paul Greenwalt, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180323231631.32502-1-jeffrey.t.kirsher@intel.com>
From: Paul Greenwalt <paul.greenwalt@intel.com>
Replaced an assignment operation with an OR operation.
The variable assignment was overwriting the value read from the PHY
register. The OR operation sets only the intended register bits.
The bits that were being overwritten are reserved, so the assignment had no
functional impact.
Reported by: Shannon Nelson <shannon.nelson@oracle.com>
Signed-off-by: Paul Greenwalt <paul.greenwalt@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c
index f470d0204771..3123267dfba9 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c
@@ -1847,9 +1847,9 @@ ixgbe_setup_mac_link_sfp_x550a(struct ixgbe_hw *hw, ixgbe_link_speed speed,
(IXGBE_CS4227_EDC_MODE_SR << 1));
if (setup_linear)
- reg_phy_ext = (IXGBE_CS4227_EDC_MODE_CX1 << 1) | 1;
+ reg_phy_ext |= (IXGBE_CS4227_EDC_MODE_CX1 << 1) | 1;
else
- reg_phy_ext = (IXGBE_CS4227_EDC_MODE_SR << 1) | 1;
+ reg_phy_ext |= (IXGBE_CS4227_EDC_MODE_SR << 1) | 1;
ret_val = hw->phy.ops.write_reg(hw, reg_slice,
IXGBE_MDIO_ZERO_DEV_TYPE, reg_phy_ext);
--
2.14.3
^ permalink raw reply related
* [net-next 03/12] ixgbe: no need for ipsec csum feature check
From: Jeff Kirsher @ 2018-03-23 23:16 UTC (permalink / raw)
To: davem; +Cc: Shannon Nelson, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180323231631.32502-1-jeffrey.t.kirsher@intel.com>
From: Shannon Nelson <shannon.nelson@oracle.com>
With the patch
commit f8aa2696b4af ("esp: check the NETIF_F_HW_ESP_TX_CSUM bit before segmenting")
we no longer need to protect ourself from checksum
offload requests on IPsec packets, so we can remove
the check in our .ndo_features_check callback.
Signed-off-by: Shannon Nelson <shannon.nelson@oracle.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 6 ------
1 file changed, 6 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index faf368b14389..0eb45d1cd0b2 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -9917,12 +9917,6 @@ ixgbe_features_check(struct sk_buff *skb, struct net_device *dev,
if (skb->encapsulation && !(features & NETIF_F_TSO_MANGLEID))
features &= ~NETIF_F_TSO;
-#ifdef CONFIG_XFRM_OFFLOAD
- /* IPsec offload doesn't get along well with others *yet* */
- if (skb->sp)
- features &= ~(NETIF_F_TSO | NETIF_F_HW_CSUM);
-#endif
-
return features;
}
--
2.14.3
^ permalink raw reply related
* [net-next 04/12] ixgbe: remove unneeded ipsec test in TX path
From: Jeff Kirsher @ 2018-03-23 23:16 UTC (permalink / raw)
To: davem; +Cc: Shannon Nelson, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180323231631.32502-1-jeffrey.t.kirsher@intel.com>
From: Shannon Nelson <shannon.nelson@oracle.com>
Since the ipsec data fields will be zero anyway in the non-ipsec
case, we can remove the conditional jump.
Suggested-by: Alexander Duyck <alexander.duyck@gmail.com>
Signed-off-by: Shannon Nelson <shannon.nelson@oracle.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 0eb45d1cd0b2..74da310378da 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -7873,10 +7873,8 @@ static void ixgbe_tx_csum(struct ixgbe_ring *tx_ring,
vlan_macip_lens |= skb_network_offset(skb) << IXGBE_ADVTXD_MACLEN_SHIFT;
vlan_macip_lens |= first->tx_flags & IXGBE_TX_FLAGS_VLAN_MASK;
- if (first->tx_flags & IXGBE_TX_FLAGS_IPSEC) {
- fceof_saidx |= itd->sa_idx;
- type_tucmd |= itd->flags | itd->trailer_len;
- }
+ fceof_saidx |= itd->sa_idx;
+ type_tucmd |= itd->flags | itd->trailer_len;
ixgbe_tx_ctxtdesc(tx_ring, vlan_macip_lens, fceof_saidx, type_tucmd, 0);
}
--
2.14.3
^ permalink raw reply related
* [net-next 06/12] ixgbe: enable TSO with IPsec offload
From: Jeff Kirsher @ 2018-03-23 23:16 UTC (permalink / raw)
To: davem; +Cc: Shannon Nelson, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180323231631.32502-1-jeffrey.t.kirsher@intel.com>
From: Shannon Nelson <shannon.nelson@oracle.com>
Fix things up to support TSO offload in conjunction
with IPsec hw offload. This raises throughput with
IPsec offload on to nearly line rate.
Signed-off-by: Shannon Nelson <shannon.nelson@oracle.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c | 9 ++++++--
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 31 ++++++++++++++++++--------
2 files changed, 29 insertions(+), 11 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c
index 5ddea43a21c2..68af127987bc 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c
@@ -929,8 +929,13 @@ void ixgbe_init_ipsec_offload(struct ixgbe_adapter *adapter)
ixgbe_ipsec_clear_hw_tables(adapter);
adapter->netdev->xfrmdev_ops = &ixgbe_xfrmdev_ops;
- adapter->netdev->features |= NETIF_F_HW_ESP;
- adapter->netdev->hw_enc_features |= NETIF_F_HW_ESP;
+
+#define IXGBE_ESP_FEATURES (NETIF_F_HW_ESP | \
+ NETIF_F_HW_ESP_TX_CSUM | \
+ NETIF_F_GSO_ESP)
+
+ adapter->netdev->features |= IXGBE_ESP_FEATURES;
+ adapter->netdev->hw_enc_features |= IXGBE_ESP_FEATURES;
return;
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 74da310378da..c0d8d7238f71 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -7730,7 +7730,8 @@ static void ixgbe_service_task(struct work_struct *work)
static int ixgbe_tso(struct ixgbe_ring *tx_ring,
struct ixgbe_tx_buffer *first,
- u8 *hdr_len)
+ u8 *hdr_len,
+ struct ixgbe_ipsec_tx_data *itd)
{
u32 vlan_macip_lens, type_tucmd, mss_l4len_idx;
struct sk_buff *skb = first->skb;
@@ -7744,6 +7745,7 @@ static int ixgbe_tso(struct ixgbe_ring *tx_ring,
unsigned char *hdr;
} l4;
u32 paylen, l4_offset;
+ u32 fceof_saidx = 0;
int err;
if (skb->ip_summed != CHECKSUM_PARTIAL)
@@ -7769,13 +7771,15 @@ static int ixgbe_tso(struct ixgbe_ring *tx_ring,
if (ip.v4->version == 4) {
unsigned char *csum_start = skb_checksum_start(skb);
unsigned char *trans_start = ip.hdr + (ip.v4->ihl * 4);
+ int len = csum_start - trans_start;
/* IP header will have to cancel out any data that
- * is not a part of the outer IP header
+ * is not a part of the outer IP header, so set to
+ * a reverse csum if needed, else init check to 0.
*/
- ip.v4->check = csum_fold(csum_partial(trans_start,
- csum_start - trans_start,
- 0));
+ ip.v4->check = (skb_shinfo(skb)->gso_type & SKB_GSO_PARTIAL) ?
+ csum_fold(csum_partial(trans_start,
+ len, 0)) : 0;
type_tucmd |= IXGBE_ADVTXD_TUCMD_IPV4;
ip.v4->tot_len = 0;
@@ -7806,12 +7810,15 @@ static int ixgbe_tso(struct ixgbe_ring *tx_ring,
mss_l4len_idx = (*hdr_len - l4_offset) << IXGBE_ADVTXD_L4LEN_SHIFT;
mss_l4len_idx |= skb_shinfo(skb)->gso_size << IXGBE_ADVTXD_MSS_SHIFT;
+ fceof_saidx |= itd->sa_idx;
+ type_tucmd |= itd->flags | itd->trailer_len;
+
/* vlan_macip_lens: HEADLEN, MACLEN, VLAN tag */
vlan_macip_lens = l4.hdr - ip.hdr;
vlan_macip_lens |= (ip.hdr - skb->data) << IXGBE_ADVTXD_MACLEN_SHIFT;
vlan_macip_lens |= first->tx_flags & IXGBE_TX_FLAGS_VLAN_MASK;
- ixgbe_tx_ctxtdesc(tx_ring, vlan_macip_lens, 0, type_tucmd,
+ ixgbe_tx_ctxtdesc(tx_ring, vlan_macip_lens, fceof_saidx, type_tucmd,
mss_l4len_idx);
return 1;
@@ -8502,7 +8509,7 @@ netdev_tx_t ixgbe_xmit_frame_ring(struct sk_buff *skb,
if (skb->sp && !ixgbe_ipsec_tx(tx_ring, first, &ipsec_tx))
goto out_drop;
#endif
- tso = ixgbe_tso(tx_ring, first, &hdr_len);
+ tso = ixgbe_tso(tx_ring, first, &hdr_len, &ipsec_tx);
if (tso < 0)
goto out_drop;
else if (!tso)
@@ -9911,9 +9918,15 @@ ixgbe_features_check(struct sk_buff *skb, struct net_device *dev,
/* We can only support IPV4 TSO in tunnels if we can mangle the
* inner IP ID field, so strip TSO if MANGLEID is not supported.
+ * IPsec offoad sets skb->encapsulation but still can handle
+ * the TSO, so it's the exception.
*/
- if (skb->encapsulation && !(features & NETIF_F_TSO_MANGLEID))
- features &= ~NETIF_F_TSO;
+ if (skb->encapsulation && !(features & NETIF_F_TSO_MANGLEID)) {
+#ifdef CONFIG_XFRM
+ if (!skb->sp)
+#endif
+ features &= ~NETIF_F_TSO;
+ }
return features;
}
--
2.14.3
^ permalink raw reply related
* [net-next 05/12] ixgbe: no need for esp trailer if GSO
From: Jeff Kirsher @ 2018-03-23 23:16 UTC (permalink / raw)
To: davem; +Cc: Shannon Nelson, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180323231631.32502-1-jeffrey.t.kirsher@intel.com>
From: Shannon Nelson <shannon.nelson@oracle.com>
There is no need to calculate the trailer length if we're doing
a GSO/TSO, as there is no trailer added to the packet data.
Also, don't bother clearing the flags field as it was already
cleared earlier.
Signed-off-by: Shannon Nelson <shannon.nelson@oracle.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c | 37 +++++++++++++++-----------
1 file changed, 21 insertions(+), 16 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c
index f2254528dcfc..5ddea43a21c2 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c
@@ -774,11 +774,7 @@ int ixgbe_ipsec_tx(struct ixgbe_ring *tx_ring,
first->tx_flags |= IXGBE_TX_FLAGS_IPSEC | IXGBE_TX_FLAGS_CC;
- itd->flags = 0;
if (xs->id.proto == IPPROTO_ESP) {
- struct sk_buff *skb = first->skb;
- int ret, authlen, trailerlen;
- u8 padlen;
itd->flags |= IXGBE_ADVTXD_TUCMD_IPSEC_TYPE_ESP |
IXGBE_ADVTXD_TUCMD_L4T_TCP;
@@ -790,19 +786,28 @@ int ixgbe_ipsec_tx(struct ixgbe_ring *tx_ring,
* padlen bytes of padding. This ends up not the same
* as the static value found in xs->props.trailer_len (21).
*
- * The "correct" way to get the auth length would be to use
- * authlen = crypto_aead_authsize(xs->data);
- * but since we know we only have one size to worry about
- * we can let the compiler use the constant and save us a
- * few CPU cycles.
+ * ... but if we're doing GSO, don't bother as the stack
+ * doesn't add a trailer for those.
*/
- authlen = IXGBE_IPSEC_AUTH_BITS / 8;
-
- ret = skb_copy_bits(skb, skb->len - (authlen + 2), &padlen, 1);
- if (unlikely(ret))
- return 0;
- trailerlen = authlen + 2 + padlen;
- itd->trailer_len = trailerlen;
+ if (!skb_is_gso(first->skb)) {
+ /* The "correct" way to get the auth length would be
+ * to use
+ * authlen = crypto_aead_authsize(xs->data);
+ * but since we know we only have one size to worry
+ * about * we can let the compiler use the constant
+ * and save us a few CPU cycles.
+ */
+ const int authlen = IXGBE_IPSEC_AUTH_BITS / 8;
+ struct sk_buff *skb = first->skb;
+ u8 padlen;
+ int ret;
+
+ ret = skb_copy_bits(skb, skb->len - (authlen + 2),
+ &padlen, 1);
+ if (unlikely(ret))
+ return 0;
+ itd->trailer_len = authlen + 2 + padlen;
+ }
}
if (tsa->encrypt)
itd->flags |= IXGBE_ADVTXD_TUCMD_IPSEC_ENCRYPT_EN;
--
2.14.3
^ permalink raw reply related
* [net-next 07/12] ixgbevf: Add XDP support for pass and drop actions
From: Jeff Kirsher @ 2018-03-23 23:16 UTC (permalink / raw)
To: davem; +Cc: Tony Nguyen, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180323231631.32502-1-jeffrey.t.kirsher@intel.com>
From: Tony Nguyen <anthony.l.nguyen@intel.com>
Implement XDP_PASS and XDP_DROP based on the ixgbe implementation.
Based largely on commit 924708081629 ("ixgbe: add XDP support for pass and
drop actions").
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
Acked-by: John Fastabend <john.fastabend@gmail.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbevf/ethtool.c | 9 +-
drivers/net/ethernet/intel/ixgbevf/ixgbevf.h | 10 +-
drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 201 ++++++++++++++++++----
3 files changed, 178 insertions(+), 42 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbevf/ethtool.c b/drivers/net/ethernet/intel/ixgbevf/ethtool.c
index e7623fed42da..4946a62c70a4 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ethtool.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ethtool.c
@@ -1,7 +1,7 @@
/*******************************************************************************
Intel 82599 Virtual Function driver
- Copyright(c) 1999 - 2015 Intel Corporation.
+ Copyright(c) 1999 - 2018 Intel Corporation.
This program is free software; you can redistribute it and/or modify it
under the terms and conditions of the GNU General Public License,
@@ -336,8 +336,13 @@ static int ixgbevf_set_ringparam(struct net_device *netdev,
for (i = 0; i < adapter->num_rx_queues; i++) {
/* clone ring and setup updated count */
rx_ring[i] = *adapter->rx_ring[i];
+
+ /* Clear copied XDP RX-queue info */
+ memset(&rx_ring[i].xdp_rxq, 0,
+ sizeof(rx_ring[i].xdp_rxq));
+
rx_ring[i].count = new_rx_count;
- err = ixgbevf_setup_rx_resources(&rx_ring[i]);
+ err = ixgbevf_setup_rx_resources(adapter, &rx_ring[i]);
if (err) {
while (i) {
i--;
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h b/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h
index c06ea4dc49a0..d41365b1b674 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h
@@ -2,7 +2,7 @@
/*******************************************************************************
Intel 82599 Virtual Function driver
- Copyright(c) 1999 - 2015 Intel Corporation.
+ Copyright(c) 1999 - 2018 Intel Corporation.
This program is free software; you can redistribute it and/or modify it
under the terms and conditions of the GNU General Public License,
@@ -35,6 +35,7 @@
#include <linux/netdevice.h>
#include <linux/if_vlan.h>
#include <linux/u64_stats_sync.h>
+#include <net/xdp.h>
#include "vf.h"
@@ -100,6 +101,7 @@ struct ixgbevf_ring {
struct ixgbevf_ring *next;
struct ixgbevf_q_vector *q_vector; /* backpointer to q_vector */
struct net_device *netdev;
+ struct bpf_prog *xdp_prog;
struct device *dev;
void *desc; /* descriptor ring memory */
dma_addr_t dma; /* phys. address of descriptor ring */
@@ -120,7 +122,7 @@ struct ixgbevf_ring {
struct ixgbevf_tx_queue_stats tx_stats;
struct ixgbevf_rx_queue_stats rx_stats;
};
-
+ struct xdp_rxq_info xdp_rxq;
u64 hw_csum_rx_error;
u8 __iomem *tail;
struct sk_buff *skb;
@@ -357,6 +359,7 @@ struct ixgbevf_adapter {
/* OS defined structs */
struct net_device *netdev;
+ struct bpf_prog *xdp_prog;
struct pci_dev *pdev;
/* structs defined in ixgbe_vf.h */
@@ -443,7 +446,8 @@ void ixgbevf_down(struct ixgbevf_adapter *adapter);
void ixgbevf_reinit_locked(struct ixgbevf_adapter *adapter);
void ixgbevf_reset(struct ixgbevf_adapter *adapter);
void ixgbevf_set_ethtool_ops(struct net_device *netdev);
-int ixgbevf_setup_rx_resources(struct ixgbevf_ring *);
+int ixgbevf_setup_rx_resources(struct ixgbevf_adapter *adapter,
+ struct ixgbevf_ring *rx_ring);
int ixgbevf_setup_tx_resources(struct ixgbevf_ring *);
void ixgbevf_free_rx_resources(struct ixgbevf_ring *);
void ixgbevf_free_tx_resources(struct ixgbevf_ring *);
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
index 4da449e0a4ba..2696b5a6806f 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
@@ -1,7 +1,7 @@
/*******************************************************************************
Intel 82599 Virtual Function driver
- Copyright(c) 1999 - 2015 Intel Corporation.
+ Copyright(c) 1999 - 2018 Intel Corporation.
This program is free software; you can redistribute it and/or modify it
under the terms and conditions of the GNU General Public License,
@@ -50,6 +50,9 @@
#include <linux/if_vlan.h>
#include <linux/prefetch.h>
#include <net/mpls.h>
+#include <linux/bpf.h>
+#include <linux/bpf_trace.h>
+#include <linux/atomic.h>
#include "ixgbevf.h"
@@ -552,19 +555,21 @@ struct ixgbevf_rx_buffer *ixgbevf_get_rx_buffer(struct ixgbevf_ring *rx_ring,
}
static void ixgbevf_put_rx_buffer(struct ixgbevf_ring *rx_ring,
- struct ixgbevf_rx_buffer *rx_buffer)
+ struct ixgbevf_rx_buffer *rx_buffer,
+ struct sk_buff *skb)
{
if (ixgbevf_can_reuse_rx_page(rx_buffer)) {
/* hand second half of page back to the ring */
ixgbevf_reuse_rx_page(rx_ring, rx_buffer);
} else {
- /* We are not reusing the buffer so unmap it and free
- * any references we are holding to it
- */
- dma_unmap_page_attrs(rx_ring->dev, rx_buffer->dma,
- ixgbevf_rx_pg_size(rx_ring),
- DMA_FROM_DEVICE,
- IXGBEVF_RX_DMA_ATTR);
+ if (IS_ERR(skb))
+ /* We are not reusing the buffer so unmap it and free
+ * any references we are holding to it
+ */
+ dma_unmap_page_attrs(rx_ring->dev, rx_buffer->dma,
+ ixgbevf_rx_pg_size(rx_ring),
+ DMA_FROM_DEVICE,
+ IXGBEVF_RX_DMA_ATTR);
__page_frag_cache_drain(rx_buffer->page,
rx_buffer->pagecnt_bias);
}
@@ -737,6 +742,10 @@ static bool ixgbevf_cleanup_headers(struct ixgbevf_ring *rx_ring,
union ixgbe_adv_rx_desc *rx_desc,
struct sk_buff *skb)
{
+ /* XDP packets use error pointer so abort at this point */
+ if (IS_ERR(skb))
+ return true;
+
/* verify that the packet does not have any known errors */
if (unlikely(ixgbevf_test_staterr(rx_desc,
IXGBE_RXDADV_ERR_FRAME_ERR_MASK))) {
@@ -853,22 +862,23 @@ static void ixgbevf_add_rx_frag(struct ixgbevf_ring *rx_ring,
static
struct sk_buff *ixgbevf_construct_skb(struct ixgbevf_ring *rx_ring,
struct ixgbevf_rx_buffer *rx_buffer,
- union ixgbe_adv_rx_desc *rx_desc,
- unsigned int size)
+ struct xdp_buff *xdp,
+ union ixgbe_adv_rx_desc *rx_desc)
{
- void *va = page_address(rx_buffer->page) + rx_buffer->page_offset;
+ unsigned int size = xdp->data_end - xdp->data;
#if (PAGE_SIZE < 8192)
unsigned int truesize = ixgbevf_rx_pg_size(rx_ring) / 2;
#else
- unsigned int truesize = SKB_DATA_ALIGN(size);
+ unsigned int truesize = SKB_DATA_ALIGN(xdp->data_end -
+ xdp->data_hard_start);
#endif
unsigned int headlen;
struct sk_buff *skb;
/* prefetch first cache line of first page */
- prefetch(va);
+ prefetch(xdp->data);
#if L1_CACHE_BYTES < 128
- prefetch(va + L1_CACHE_BYTES);
+ prefetch(xdp->data + L1_CACHE_BYTES);
#endif
/* allocate a skb to store the frags */
@@ -879,16 +889,18 @@ struct sk_buff *ixgbevf_construct_skb(struct ixgbevf_ring *rx_ring,
/* Determine available headroom for copy */
headlen = size;
if (headlen > IXGBEVF_RX_HDR_SIZE)
- headlen = eth_get_headlen(va, IXGBEVF_RX_HDR_SIZE);
+ headlen = eth_get_headlen(xdp->data, IXGBEVF_RX_HDR_SIZE);
/* align pull length to size of long to optimize memcpy performance */
- memcpy(__skb_put(skb, headlen), va, ALIGN(headlen, sizeof(long)));
+ memcpy(__skb_put(skb, headlen), xdp->data,
+ ALIGN(headlen, sizeof(long)));
/* update all of the pointers */
size -= headlen;
if (size) {
skb_add_rx_frag(skb, 0, rx_buffer->page,
- (va + headlen) - page_address(rx_buffer->page),
+ (xdp->data + headlen) -
+ page_address(rx_buffer->page),
size, truesize);
#if (PAGE_SIZE < 8192)
rx_buffer->page_offset ^= truesize;
@@ -912,32 +924,32 @@ static inline void ixgbevf_irq_enable_queues(struct ixgbevf_adapter *adapter,
static struct sk_buff *ixgbevf_build_skb(struct ixgbevf_ring *rx_ring,
struct ixgbevf_rx_buffer *rx_buffer,
- union ixgbe_adv_rx_desc *rx_desc,
- unsigned int size)
+ struct xdp_buff *xdp,
+ union ixgbe_adv_rx_desc *rx_desc)
{
- void *va = page_address(rx_buffer->page) + rx_buffer->page_offset;
#if (PAGE_SIZE < 8192)
unsigned int truesize = ixgbevf_rx_pg_size(rx_ring) / 2;
#else
unsigned int truesize = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) +
- SKB_DATA_ALIGN(IXGBEVF_SKB_PAD + size);
+ SKB_DATA_ALIGN(xdp->data_end -
+ xdp->data_hard_start);
#endif
struct sk_buff *skb;
/* prefetch first cache line of first page */
- prefetch(va);
+ prefetch(xdp->data);
#if L1_CACHE_BYTES < 128
- prefetch(va + L1_CACHE_BYTES);
+ prefetch(xdp->data + L1_CACHE_BYTES);
#endif
- /* build an skb to around the page buffer */
- skb = build_skb(va - IXGBEVF_SKB_PAD, truesize);
+ /* build an skb around the page buffer */
+ skb = build_skb(xdp->data_hard_start, truesize);
if (unlikely(!skb))
return NULL;
/* update pointers within the skb to store the data */
- skb_reserve(skb, IXGBEVF_SKB_PAD);
- __skb_put(skb, size);
+ skb_reserve(skb, xdp->data - xdp->data_hard_start);
+ __skb_put(skb, xdp->data_end - xdp->data);
/* update buffer offset */
#if (PAGE_SIZE < 8192)
@@ -948,6 +960,43 @@ static struct sk_buff *ixgbevf_build_skb(struct ixgbevf_ring *rx_ring,
return skb;
}
+
+#define IXGBEVF_XDP_PASS 0
+#define IXGBEVF_XDP_CONSUMED 1
+
+static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_ring *rx_ring,
+ struct xdp_buff *xdp)
+{
+ int result = IXGBEVF_XDP_PASS;
+ struct bpf_prog *xdp_prog;
+ u32 act;
+
+ rcu_read_lock();
+ xdp_prog = READ_ONCE(rx_ring->xdp_prog);
+
+ if (!xdp_prog)
+ goto xdp_out;
+
+ act = bpf_prog_run_xdp(xdp_prog, xdp);
+ switch (act) {
+ case XDP_PASS:
+ break;
+ default:
+ bpf_warn_invalid_xdp_action(act);
+ /* fallthrough */
+ case XDP_TX:
+ case XDP_ABORTED:
+ trace_xdp_exception(rx_ring->netdev, xdp_prog, act);
+ /* fallthrough -- handle aborts by dropping packet */
+ case XDP_DROP:
+ result = IXGBEVF_XDP_CONSUMED;
+ break;
+ }
+xdp_out:
+ rcu_read_unlock();
+ return ERR_PTR(-result);
+}
+
static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
struct ixgbevf_ring *rx_ring,
int budget)
@@ -955,10 +1004,13 @@ static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
unsigned int total_rx_bytes = 0, total_rx_packets = 0;
u16 cleaned_count = ixgbevf_desc_unused(rx_ring);
struct sk_buff *skb = rx_ring->skb;
+ struct xdp_buff xdp;
+
+ xdp.rxq = &rx_ring->xdp_rxq;
while (likely(total_rx_packets < budget)) {
- union ixgbe_adv_rx_desc *rx_desc;
struct ixgbevf_rx_buffer *rx_buffer;
+ union ixgbe_adv_rx_desc *rx_desc;
unsigned int size;
/* return some buffers to hardware, one at a time is too slow */
@@ -981,14 +1033,30 @@ static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
rx_buffer = ixgbevf_get_rx_buffer(rx_ring, size);
/* retrieve a buffer from the ring */
- if (skb)
+ if (!skb) {
+ xdp.data = page_address(rx_buffer->page) +
+ rx_buffer->page_offset;
+ xdp_set_data_meta_invalid(&xdp);
+ xdp.data_hard_start = xdp.data -
+ ixgbevf_rx_offset(rx_ring);
+ xdp.data_end = xdp.data + size;
+
+ skb = ixgbevf_run_xdp(rx_ring, &xdp);
+ }
+
+ if (IS_ERR(skb)) {
+ total_rx_packets++;
+ total_rx_bytes += size;
+ rx_buffer->pagecnt_bias++;
+ } else if (skb) {
ixgbevf_add_rx_frag(rx_ring, rx_buffer, skb, size);
- else if (ring_uses_build_skb(rx_ring))
+ } else if (ring_uses_build_skb(rx_ring)) {
skb = ixgbevf_build_skb(rx_ring, rx_buffer,
- rx_desc, size);
- else
+ &xdp, rx_desc);
+ } else {
skb = ixgbevf_construct_skb(rx_ring, rx_buffer,
- rx_desc, size);
+ &xdp, rx_desc);
+ }
/* exit if we failed to retrieve a buffer */
if (!skb) {
@@ -997,7 +1065,7 @@ static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
break;
}
- ixgbevf_put_rx_buffer(rx_ring, rx_buffer);
+ ixgbevf_put_rx_buffer(rx_ring, rx_buffer, skb);
cleaned_count++;
/* fetch next buffer in frame if non-eop */
@@ -3159,11 +3227,13 @@ static int ixgbevf_setup_all_tx_resources(struct ixgbevf_adapter *adapter)
/**
* ixgbevf_setup_rx_resources - allocate Rx resources (Descriptors)
+ * @adapter: board private structure
* @rx_ring: Rx descriptor ring (for a specific queue) to setup
*
* Returns 0 on success, negative on failure
**/
-int ixgbevf_setup_rx_resources(struct ixgbevf_ring *rx_ring)
+int ixgbevf_setup_rx_resources(struct ixgbevf_adapter *adapter,
+ struct ixgbevf_ring *rx_ring)
{
int size;
@@ -3184,6 +3254,13 @@ int ixgbevf_setup_rx_resources(struct ixgbevf_ring *rx_ring)
if (!rx_ring->desc)
goto err;
+ /* XDP RX-queue info */
+ if (xdp_rxq_info_reg(&rx_ring->xdp_rxq, adapter->netdev,
+ rx_ring->queue_index) < 0)
+ goto err;
+
+ rx_ring->xdp_prog = adapter->xdp_prog;
+
return 0;
err:
vfree(rx_ring->rx_buffer_info);
@@ -3207,7 +3284,7 @@ static int ixgbevf_setup_all_rx_resources(struct ixgbevf_adapter *adapter)
int i, err = 0;
for (i = 0; i < adapter->num_rx_queues; i++) {
- err = ixgbevf_setup_rx_resources(adapter->rx_ring[i]);
+ err = ixgbevf_setup_rx_resources(adapter, adapter->rx_ring[i]);
if (!err)
continue;
hw_dbg(&adapter->hw, "Allocation for Rx Queue %u failed\n", i);
@@ -3232,6 +3309,8 @@ void ixgbevf_free_rx_resources(struct ixgbevf_ring *rx_ring)
{
ixgbevf_clean_rx_ring(rx_ring);
+ rx_ring->xdp_prog = NULL;
+ xdp_rxq_info_unreg(&rx_ring->xdp_rxq);
vfree(rx_ring->rx_buffer_info);
rx_ring->rx_buffer_info = NULL;
@@ -3918,6 +3997,12 @@ static int ixgbevf_change_mtu(struct net_device *netdev, int new_mtu)
int max_frame = new_mtu + ETH_HLEN + ETH_FCS_LEN;
int ret;
+ /* prevent MTU being changed to a size unsupported by XDP */
+ if (adapter->xdp_prog) {
+ dev_warn(&adapter->pdev->dev, "MTU cannot be changed while XDP program is loaded\n");
+ return -EPERM;
+ }
+
spin_lock_bh(&adapter->mbx_lock);
/* notify the PF of our intent to use this size of frame */
ret = hw->mac.ops.set_rlpml(hw, max_frame);
@@ -4101,6 +4186,47 @@ ixgbevf_features_check(struct sk_buff *skb, struct net_device *dev,
return features;
}
+static int ixgbevf_xdp_setup(struct net_device *dev, struct bpf_prog *prog)
+{
+ int i, frame_size = dev->mtu + ETH_HLEN + ETH_FCS_LEN + VLAN_HLEN;
+ struct ixgbevf_adapter *adapter = netdev_priv(dev);
+ struct bpf_prog *old_prog;
+
+ /* verify ixgbevf ring attributes are sufficient for XDP */
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ struct ixgbevf_ring *ring = adapter->rx_ring[i];
+
+ if (frame_size > ixgbevf_rx_bufsz(ring))
+ return -EINVAL;
+ }
+
+ old_prog = xchg(&adapter->xdp_prog, prog);
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ xchg(&adapter->rx_ring[i]->xdp_prog, adapter->xdp_prog);
+
+ if (old_prog)
+ bpf_prog_put(old_prog);
+
+ return 0;
+}
+
+static int ixgbevf_xdp(struct net_device *dev, struct netdev_bpf *xdp)
+{
+ struct ixgbevf_adapter *adapter = netdev_priv(dev);
+
+ switch (xdp->command) {
+ case XDP_SETUP_PROG:
+ return ixgbevf_xdp_setup(dev, xdp->prog);
+ case XDP_QUERY_PROG:
+ xdp->prog_attached = !!(adapter->xdp_prog);
+ xdp->prog_id = adapter->xdp_prog ?
+ adapter->xdp_prog->aux->id : 0;
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
static const struct net_device_ops ixgbevf_netdev_ops = {
.ndo_open = ixgbevf_open,
.ndo_stop = ixgbevf_close,
@@ -4117,6 +4243,7 @@ static const struct net_device_ops ixgbevf_netdev_ops = {
.ndo_poll_controller = ixgbevf_netpoll,
#endif
.ndo_features_check = ixgbevf_features_check,
+ .ndo_bpf = ixgbevf_xdp,
};
static void ixgbevf_assign_netdev_ops(struct net_device *dev)
--
2.14.3
^ permalink raw reply related
* [net-next 08/12] ixgbevf: Add support for XDP_TX action
From: Jeff Kirsher @ 2018-03-23 23:16 UTC (permalink / raw)
To: davem; +Cc: Tony Nguyen, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180323231631.32502-1-jeffrey.t.kirsher@intel.com>
From: Tony Nguyen <anthony.l.nguyen@intel.com>
This implements the XDP_TX action which is modeled on the ixgbe
implementation. However instead of using CPU id to determine which XDP
queue to use, this uses the received RX queue index, which is similar
to i40e. Doing this eliminates the restriction that number of CPUs not
exceed number of XDP queues that ixgbe has.
Also, based on the number of queues available, the number of TX queues
may be reduced when an XDP program is loaded in order to accommodate the
XDP queues.
Based largely on
commit 33fdc82f0883 ("ixgbe: add support for XDP_TX action")
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
Acked-by: John Fastabend <john.fastabend@gmail.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbevf/ethtool.c | 35 ++-
drivers/net/ethernet/intel/ixgbevf/ixgbevf.h | 20 +-
drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 271 +++++++++++++++++++---
3 files changed, 294 insertions(+), 32 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbevf/ethtool.c b/drivers/net/ethernet/intel/ixgbevf/ethtool.c
index 4946a62c70a4..da8c0e299a37 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ethtool.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ethtool.c
@@ -269,7 +269,7 @@ static int ixgbevf_set_ringparam(struct net_device *netdev,
struct ixgbevf_adapter *adapter = netdev_priv(netdev);
struct ixgbevf_ring *tx_ring = NULL, *rx_ring = NULL;
u32 new_rx_count, new_tx_count;
- int i, err = 0;
+ int i, j, err = 0;
if ((ring->rx_mini_pending) || (ring->rx_jumbo_pending))
return -EINVAL;
@@ -293,15 +293,19 @@ static int ixgbevf_set_ringparam(struct net_device *netdev,
if (!netif_running(adapter->netdev)) {
for (i = 0; i < adapter->num_tx_queues; i++)
adapter->tx_ring[i]->count = new_tx_count;
+ for (i = 0; i < adapter->num_xdp_queues; i++)
+ adapter->xdp_ring[i]->count = new_tx_count;
for (i = 0; i < adapter->num_rx_queues; i++)
adapter->rx_ring[i]->count = new_rx_count;
adapter->tx_ring_count = new_tx_count;
+ adapter->xdp_ring_count = new_tx_count;
adapter->rx_ring_count = new_rx_count;
goto clear_reset;
}
if (new_tx_count != adapter->tx_ring_count) {
- tx_ring = vmalloc(adapter->num_tx_queues * sizeof(*tx_ring));
+ tx_ring = vmalloc((adapter->num_tx_queues +
+ adapter->num_xdp_queues) * sizeof(*tx_ring));
if (!tx_ring) {
err = -ENOMEM;
goto clear_reset;
@@ -324,6 +328,24 @@ static int ixgbevf_set_ringparam(struct net_device *netdev,
goto clear_reset;
}
}
+
+ for (j = 0; j < adapter->num_xdp_queues; i++, j++) {
+ /* clone ring and setup updated count */
+ tx_ring[i] = *adapter->xdp_ring[j];
+ tx_ring[i].count = new_tx_count;
+ err = ixgbevf_setup_tx_resources(&tx_ring[i]);
+ if (err) {
+ while (i) {
+ i--;
+ ixgbevf_free_tx_resources(&tx_ring[i]);
+ }
+
+ vfree(tx_ring);
+ tx_ring = NULL;
+
+ goto clear_reset;
+ }
+ }
}
if (new_rx_count != adapter->rx_ring_count) {
@@ -368,6 +390,12 @@ static int ixgbevf_set_ringparam(struct net_device *netdev,
}
adapter->tx_ring_count = new_tx_count;
+ for (j = 0; j < adapter->num_xdp_queues; i++, j++) {
+ ixgbevf_free_tx_resources(adapter->xdp_ring[j]);
+ *adapter->xdp_ring[j] = tx_ring[i];
+ }
+ adapter->xdp_ring_count = new_tx_count;
+
vfree(tx_ring);
tx_ring = NULL;
}
@@ -390,7 +418,8 @@ static int ixgbevf_set_ringparam(struct net_device *netdev,
clear_reset:
/* free Tx resources if Rx error is encountered */
if (tx_ring) {
- for (i = 0; i < adapter->num_tx_queues; i++)
+ for (i = 0;
+ i < adapter->num_tx_queues + adapter->num_xdp_queues; i++)
ixgbevf_free_tx_resources(&tx_ring[i]);
vfree(tx_ring);
}
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h b/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h
index d41365b1b674..447ce1d5e0e3 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf.h
@@ -52,7 +52,11 @@
struct ixgbevf_tx_buffer {
union ixgbe_adv_tx_desc *next_to_watch;
unsigned long time_stamp;
- struct sk_buff *skb;
+ union {
+ struct sk_buff *skb;
+ /* XDP uses address ptr on irq_clean */
+ void *data;
+ };
unsigned int bytecount;
unsigned short gso_segs;
__be16 protocol;
@@ -95,8 +99,16 @@ enum ixgbevf_ring_state_t {
__IXGBEVF_RX_BUILD_SKB_ENABLED,
__IXGBEVF_TX_DETECT_HANG,
__IXGBEVF_HANG_CHECK_ARMED,
+ __IXGBEVF_TX_XDP_RING,
};
+#define ring_is_xdp(ring) \
+ test_bit(__IXGBEVF_TX_XDP_RING, &(ring)->state)
+#define set_ring_xdp(ring) \
+ set_bit(__IXGBEVF_TX_XDP_RING, &(ring)->state)
+#define clear_ring_xdp(ring) \
+ clear_bit(__IXGBEVF_TX_XDP_RING, &(ring)->state)
+
struct ixgbevf_ring {
struct ixgbevf_ring *next;
struct ixgbevf_q_vector *q_vector; /* backpointer to q_vector */
@@ -139,6 +151,7 @@ struct ixgbevf_ring {
#define MAX_RX_QUEUES IXGBE_VF_MAX_RX_QUEUES
#define MAX_TX_QUEUES IXGBE_VF_MAX_TX_QUEUES
+#define MAX_XDP_QUEUES IXGBE_VF_MAX_TX_QUEUES
#define IXGBEVF_MAX_RSS_QUEUES 2
#define IXGBEVF_82599_RETA_SIZE 128 /* 128 entries */
#define IXGBEVF_X550_VFRETA_SIZE 64 /* 64 entries */
@@ -339,6 +352,10 @@ struct ixgbevf_adapter {
u32 eims_enable_mask;
u32 eims_other;
+ /* XDP */
+ int num_xdp_queues;
+ struct ixgbevf_ring *xdp_ring[MAX_XDP_QUEUES];
+
/* TX */
int num_tx_queues;
struct ixgbevf_ring *tx_ring[MAX_TX_QUEUES]; /* One per active queue */
@@ -373,6 +390,7 @@ struct ixgbevf_adapter {
unsigned long state;
u64 tx_busy;
unsigned int tx_ring_count;
+ unsigned int xdp_ring_count;
unsigned int rx_ring_count;
u8 __iomem *io_addr; /* Mainly for iounmap use */
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
index 2696b5a6806f..309f549808e4 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
@@ -324,7 +324,10 @@ static bool ixgbevf_clean_tx_irq(struct ixgbevf_q_vector *q_vector,
total_packets += tx_buffer->gso_segs;
/* free the skb */
- napi_consume_skb(tx_buffer->skb, napi_budget);
+ if (ring_is_xdp(tx_ring))
+ page_frag_free(tx_buffer->data);
+ else
+ napi_consume_skb(tx_buffer->skb, napi_budget);
/* unmap skb header data */
dma_unmap_single(tx_ring->dev,
@@ -388,7 +391,7 @@ static bool ixgbevf_clean_tx_irq(struct ixgbevf_q_vector *q_vector,
eop_desc = tx_ring->tx_buffer_info[i].next_to_watch;
- pr_err("Detected Tx Unit Hang\n"
+ pr_err("Detected Tx Unit Hang%s\n"
" Tx Queue <%d>\n"
" TDH, TDT <%x>, <%x>\n"
" next_to_use <%x>\n"
@@ -398,6 +401,7 @@ static bool ixgbevf_clean_tx_irq(struct ixgbevf_q_vector *q_vector,
" eop_desc->wb.status <%x>\n"
" time_stamp <%lx>\n"
" jiffies <%lx>\n",
+ ring_is_xdp(tx_ring) ? " XDP" : "",
tx_ring->queue_index,
IXGBE_READ_REG(hw, IXGBE_VFTDH(tx_ring->reg_idx)),
IXGBE_READ_REG(hw, IXGBE_VFTDT(tx_ring->reg_idx)),
@@ -405,7 +409,9 @@ static bool ixgbevf_clean_tx_irq(struct ixgbevf_q_vector *q_vector,
eop_desc, (eop_desc ? eop_desc->wb.status : 0),
tx_ring->tx_buffer_info[i].time_stamp, jiffies);
- netif_stop_subqueue(tx_ring->netdev, tx_ring->queue_index);
+ if (!ring_is_xdp(tx_ring))
+ netif_stop_subqueue(tx_ring->netdev,
+ tx_ring->queue_index);
/* schedule immediate reset if we believe we hung */
ixgbevf_tx_timeout_reset(adapter);
@@ -413,6 +419,9 @@ static bool ixgbevf_clean_tx_irq(struct ixgbevf_q_vector *q_vector,
return true;
}
+ if (ring_is_xdp(tx_ring))
+ return !!budget;
+
#define TX_WAKE_THRESHOLD (DESC_NEEDED * 2)
if (unlikely(total_packets && netif_carrier_ok(tx_ring->netdev) &&
(ixgbevf_desc_unused(tx_ring) >= TX_WAKE_THRESHOLD))) {
@@ -963,11 +972,78 @@ static struct sk_buff *ixgbevf_build_skb(struct ixgbevf_ring *rx_ring,
#define IXGBEVF_XDP_PASS 0
#define IXGBEVF_XDP_CONSUMED 1
+#define IXGBEVF_XDP_TX 2
+
+static int ixgbevf_xmit_xdp_ring(struct ixgbevf_ring *ring,
+ struct xdp_buff *xdp)
+{
+ struct ixgbevf_tx_buffer *tx_buffer;
+ union ixgbe_adv_tx_desc *tx_desc;
+ u32 len, cmd_type;
+ dma_addr_t dma;
+ u16 i;
+
+ len = xdp->data_end - xdp->data;
+
+ if (unlikely(!ixgbevf_desc_unused(ring)))
+ return IXGBEVF_XDP_CONSUMED;
+
+ dma = dma_map_single(ring->dev, xdp->data, len, DMA_TO_DEVICE);
+ if (dma_mapping_error(ring->dev, dma))
+ return IXGBEVF_XDP_CONSUMED;
+
+ /* record the location of the first descriptor for this packet */
+ tx_buffer = &ring->tx_buffer_info[ring->next_to_use];
+ tx_buffer->bytecount = len;
+ tx_buffer->gso_segs = 1;
+ tx_buffer->protocol = 0;
+
+ i = ring->next_to_use;
+ tx_desc = IXGBEVF_TX_DESC(ring, i);
+
+ dma_unmap_len_set(tx_buffer, len, len);
+ dma_unmap_addr_set(tx_buffer, dma, dma);
+ tx_buffer->data = xdp->data;
+ tx_desc->read.buffer_addr = cpu_to_le64(dma);
+
+ /* put descriptor type bits */
+ cmd_type = IXGBE_ADVTXD_DTYP_DATA |
+ IXGBE_ADVTXD_DCMD_DEXT |
+ IXGBE_ADVTXD_DCMD_IFCS;
+ cmd_type |= len | IXGBE_TXD_CMD;
+ tx_desc->read.cmd_type_len = cpu_to_le32(cmd_type);
+ tx_desc->read.olinfo_status =
+ cpu_to_le32((len << IXGBE_ADVTXD_PAYLEN_SHIFT) |
+ IXGBE_ADVTXD_CC);
+
+ /* Force memory writes to complete before letting h/w know there
+ * are new descriptors to fetch. (Only applicable for weak-ordered
+ * memory model archs, such as IA-64).
+ *
+ * We also need this memory barrier to make certain all of the
+ * status bits have been updated before next_to_watch is written.
+ */
+ wmb();
-static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_ring *rx_ring,
+ /* set next_to_watch value indicating a packet is present */
+ i++;
+ if (i == ring->count)
+ i = 0;
+
+ tx_buffer->next_to_watch = tx_desc;
+ ring->next_to_use = i;
+
+ /* notify HW of packet */
+ ixgbevf_write_tail(ring, i);
+ return IXGBEVF_XDP_TX;
+}
+
+static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_adapter *adapter,
+ struct ixgbevf_ring *rx_ring,
struct xdp_buff *xdp)
{
int result = IXGBEVF_XDP_PASS;
+ struct ixgbevf_ring *xdp_ring;
struct bpf_prog *xdp_prog;
u32 act;
@@ -981,10 +1057,13 @@ static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_ring *rx_ring,
switch (act) {
case XDP_PASS:
break;
+ case XDP_TX:
+ xdp_ring = adapter->xdp_ring[rx_ring->queue_index];
+ result = ixgbevf_xmit_xdp_ring(xdp_ring, xdp);
+ break;
default:
bpf_warn_invalid_xdp_action(act);
/* fallthrough */
- case XDP_TX:
case XDP_ABORTED:
trace_xdp_exception(rx_ring->netdev, xdp_prog, act);
/* fallthrough -- handle aborts by dropping packet */
@@ -997,11 +1076,29 @@ static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_ring *rx_ring,
return ERR_PTR(-result);
}
+static void ixgbevf_rx_buffer_flip(struct ixgbevf_ring *rx_ring,
+ struct ixgbevf_rx_buffer *rx_buffer,
+ unsigned int size)
+{
+#if (PAGE_SIZE < 8192)
+ unsigned int truesize = ixgbevf_rx_pg_size(rx_ring) / 2;
+
+ rx_buffer->page_offset ^= truesize;
+#else
+ unsigned int truesize = ring_uses_build_skb(rx_ring) ?
+ SKB_DATA_ALIGN(IXGBEVF_SKB_PAD + size) :
+ SKB_DATA_ALIGN(size);
+
+ rx_buffer->page_offset += truesize;
+#endif
+}
+
static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
struct ixgbevf_ring *rx_ring,
int budget)
{
unsigned int total_rx_bytes = 0, total_rx_packets = 0;
+ struct ixgbevf_adapter *adapter = q_vector->adapter;
u16 cleaned_count = ixgbevf_desc_unused(rx_ring);
struct sk_buff *skb = rx_ring->skb;
struct xdp_buff xdp;
@@ -1041,13 +1138,17 @@ static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
ixgbevf_rx_offset(rx_ring);
xdp.data_end = xdp.data + size;
- skb = ixgbevf_run_xdp(rx_ring, &xdp);
+ skb = ixgbevf_run_xdp(adapter, rx_ring, &xdp);
}
if (IS_ERR(skb)) {
+ if (PTR_ERR(skb) == -IXGBEVF_XDP_TX)
+ ixgbevf_rx_buffer_flip(rx_ring, rx_buffer,
+ size);
+ else
+ rx_buffer->pagecnt_bias++;
total_rx_packets++;
total_rx_bytes += size;
- rx_buffer->pagecnt_bias++;
} else if (skb) {
ixgbevf_add_rx_frag(rx_ring, rx_buffer, skb, size);
} else if (ring_uses_build_skb(rx_ring)) {
@@ -1608,6 +1709,8 @@ static void ixgbevf_configure_tx(struct ixgbevf_adapter *adapter)
/* Setup the HW Tx Head and Tail descriptor pointers */
for (i = 0; i < adapter->num_tx_queues; i++)
ixgbevf_configure_tx_ring(adapter, adapter->tx_ring[i]);
+ for (i = 0; i < adapter->num_xdp_queues; i++)
+ ixgbevf_configure_tx_ring(adapter, adapter->xdp_ring[i]);
}
#define IXGBE_SRRCTL_BSIZEHDRSIZE_SHIFT 2
@@ -2239,7 +2342,10 @@ static void ixgbevf_clean_tx_ring(struct ixgbevf_ring *tx_ring)
union ixgbe_adv_tx_desc *eop_desc, *tx_desc;
/* Free all the Tx ring sk_buffs */
- dev_kfree_skb_any(tx_buffer->skb);
+ if (ring_is_xdp(tx_ring))
+ page_frag_free(tx_buffer->data);
+ else
+ dev_kfree_skb_any(tx_buffer->skb);
/* unmap skb header data */
dma_unmap_single(tx_ring->dev,
@@ -2307,6 +2413,8 @@ static void ixgbevf_clean_all_tx_rings(struct ixgbevf_adapter *adapter)
for (i = 0; i < adapter->num_tx_queues; i++)
ixgbevf_clean_tx_ring(adapter->tx_ring[i]);
+ for (i = 0; i < adapter->num_xdp_queues; i++)
+ ixgbevf_clean_tx_ring(adapter->xdp_ring[i]);
}
void ixgbevf_down(struct ixgbevf_adapter *adapter)
@@ -2345,6 +2453,13 @@ void ixgbevf_down(struct ixgbevf_adapter *adapter)
IXGBE_TXDCTL_SWFLSH);
}
+ for (i = 0; i < adapter->num_xdp_queues; i++) {
+ u8 reg_idx = adapter->xdp_ring[i]->reg_idx;
+
+ IXGBE_WRITE_REG(hw, IXGBE_VFTXDCTL(reg_idx),
+ IXGBE_TXDCTL_SWFLSH);
+ }
+
if (!pci_channel_offline(adapter->pdev))
ixgbevf_reset(adapter);
@@ -2442,6 +2557,7 @@ static void ixgbevf_set_num_queues(struct ixgbevf_adapter *adapter)
/* Start with base case */
adapter->num_rx_queues = 1;
adapter->num_tx_queues = 1;
+ adapter->num_xdp_queues = 0;
spin_lock_bh(&adapter->mbx_lock);
@@ -2463,8 +2579,13 @@ static void ixgbevf_set_num_queues(struct ixgbevf_adapter *adapter)
case ixgbe_mbox_api_11:
case ixgbe_mbox_api_12:
case ixgbe_mbox_api_13:
+ if (adapter->xdp_prog &&
+ hw->mac.max_tx_queues == rss)
+ rss = rss > 3 ? 2 : 1;
+
adapter->num_rx_queues = rss;
adapter->num_tx_queues = rss;
+ adapter->num_xdp_queues = adapter->xdp_prog ? rss : 0;
default:
break;
}
@@ -2521,6 +2642,8 @@ static void ixgbevf_add_ring(struct ixgbevf_ring *ring,
* @v_idx: index of vector in adapter struct
* @txr_count: number of Tx rings for q vector
* @txr_idx: index of first Tx ring to assign
+ * @xdp_count: total number of XDP rings to allocate
+ * @xdp_idx: index of first XDP ring to allocate
* @rxr_count: number of Rx rings for q vector
* @rxr_idx: index of first Rx ring to assign
*
@@ -2528,13 +2651,15 @@ static void ixgbevf_add_ring(struct ixgbevf_ring *ring,
**/
static int ixgbevf_alloc_q_vector(struct ixgbevf_adapter *adapter, int v_idx,
int txr_count, int txr_idx,
+ int xdp_count, int xdp_idx,
int rxr_count, int rxr_idx)
{
struct ixgbevf_q_vector *q_vector;
+ int reg_idx = txr_idx + xdp_idx;
struct ixgbevf_ring *ring;
int ring_count, size;
- ring_count = txr_count + rxr_count;
+ ring_count = txr_count + xdp_count + rxr_count;
size = sizeof(*q_vector) + (sizeof(*ring) * ring_count);
/* allocate q_vector and rings */
@@ -2567,7 +2692,7 @@ static int ixgbevf_alloc_q_vector(struct ixgbevf_adapter *adapter, int v_idx,
/* apply Tx specific ring traits */
ring->count = adapter->tx_ring_count;
ring->queue_index = txr_idx;
- ring->reg_idx = txr_idx;
+ ring->reg_idx = reg_idx;
/* assign ring to adapter */
adapter->tx_ring[txr_idx] = ring;
@@ -2575,6 +2700,36 @@ static int ixgbevf_alloc_q_vector(struct ixgbevf_adapter *adapter, int v_idx,
/* update count and index */
txr_count--;
txr_idx++;
+ reg_idx++;
+
+ /* push pointer to next ring */
+ ring++;
+ }
+
+ while (xdp_count) {
+ /* assign generic ring traits */
+ ring->dev = &adapter->pdev->dev;
+ ring->netdev = adapter->netdev;
+
+ /* configure backlink on ring */
+ ring->q_vector = q_vector;
+
+ /* update q_vector Tx values */
+ ixgbevf_add_ring(ring, &q_vector->tx);
+
+ /* apply Tx specific ring traits */
+ ring->count = adapter->tx_ring_count;
+ ring->queue_index = xdp_idx;
+ ring->reg_idx = reg_idx;
+ set_ring_xdp(ring);
+
+ /* assign ring to adapter */
+ adapter->xdp_ring[xdp_idx] = ring;
+
+ /* update count and index */
+ xdp_count--;
+ xdp_idx++;
+ reg_idx++;
/* push pointer to next ring */
ring++;
@@ -2624,8 +2779,12 @@ static void ixgbevf_free_q_vector(struct ixgbevf_adapter *adapter, int v_idx)
struct ixgbevf_q_vector *q_vector = adapter->q_vector[v_idx];
struct ixgbevf_ring *ring;
- ixgbevf_for_each_ring(ring, q_vector->tx)
- adapter->tx_ring[ring->queue_index] = NULL;
+ ixgbevf_for_each_ring(ring, q_vector->tx) {
+ if (ring_is_xdp(ring))
+ adapter->xdp_ring[ring->queue_index] = NULL;
+ else
+ adapter->tx_ring[ring->queue_index] = NULL;
+ }
ixgbevf_for_each_ring(ring, q_vector->rx)
adapter->rx_ring[ring->queue_index] = NULL;
@@ -2651,15 +2810,16 @@ static int ixgbevf_alloc_q_vectors(struct ixgbevf_adapter *adapter)
int q_vectors = adapter->num_msix_vectors - NON_Q_VECTORS;
int rxr_remaining = adapter->num_rx_queues;
int txr_remaining = adapter->num_tx_queues;
- int rxr_idx = 0, txr_idx = 0, v_idx = 0;
+ int xdp_remaining = adapter->num_xdp_queues;
+ int rxr_idx = 0, txr_idx = 0, xdp_idx = 0, v_idx = 0;
int err;
- if (q_vectors >= (rxr_remaining + txr_remaining)) {
+ if (q_vectors >= (rxr_remaining + txr_remaining + xdp_remaining)) {
for (; rxr_remaining; v_idx++, q_vectors--) {
int rqpv = DIV_ROUND_UP(rxr_remaining, q_vectors);
err = ixgbevf_alloc_q_vector(adapter, v_idx,
- 0, 0, rqpv, rxr_idx);
+ 0, 0, 0, 0, rqpv, rxr_idx);
if (err)
goto err_out;
@@ -2672,9 +2832,11 @@ static int ixgbevf_alloc_q_vectors(struct ixgbevf_adapter *adapter)
for (; q_vectors; v_idx++, q_vectors--) {
int rqpv = DIV_ROUND_UP(rxr_remaining, q_vectors);
int tqpv = DIV_ROUND_UP(txr_remaining, q_vectors);
+ int xqpv = DIV_ROUND_UP(xdp_remaining, q_vectors);
err = ixgbevf_alloc_q_vector(adapter, v_idx,
tqpv, txr_idx,
+ xqpv, xdp_idx,
rqpv, rxr_idx);
if (err)
@@ -2685,6 +2847,8 @@ static int ixgbevf_alloc_q_vectors(struct ixgbevf_adapter *adapter)
rxr_idx += rqpv;
txr_remaining -= tqpv;
txr_idx += tqpv;
+ xdp_remaining -= xqpv;
+ xdp_idx += xqpv;
}
return 0;
@@ -2756,9 +2920,10 @@ static int ixgbevf_init_interrupt_scheme(struct ixgbevf_adapter *adapter)
goto err_alloc_q_vectors;
}
- hw_dbg(&adapter->hw, "Multiqueue %s: Rx Queue count = %u, Tx Queue count = %u\n",
- (adapter->num_rx_queues > 1) ? "Enabled" :
- "Disabled", adapter->num_rx_queues, adapter->num_tx_queues);
+ hw_dbg(&adapter->hw, "Multiqueue %s: Rx Queue count = %u, Tx Queue count = %u XDP Queue count %u\n",
+ (adapter->num_rx_queues > 1) ? "Enabled" : "Disabled",
+ adapter->num_rx_queues, adapter->num_tx_queues,
+ adapter->num_xdp_queues);
set_bit(__IXGBEVF_DOWN, &adapter->state);
@@ -2779,6 +2944,7 @@ static int ixgbevf_init_interrupt_scheme(struct ixgbevf_adapter *adapter)
static void ixgbevf_clear_interrupt_scheme(struct ixgbevf_adapter *adapter)
{
adapter->num_tx_queues = 0;
+ adapter->num_xdp_queues = 0;
adapter->num_rx_queues = 0;
ixgbevf_free_q_vectors(adapter);
@@ -2986,6 +3152,8 @@ static void ixgbevf_check_hang_subtask(struct ixgbevf_adapter *adapter)
if (netif_carrier_ok(adapter->netdev)) {
for (i = 0; i < adapter->num_tx_queues; i++)
set_check_for_tx_hang(adapter->tx_ring[i]);
+ for (i = 0; i < adapter->num_xdp_queues; i++)
+ set_check_for_tx_hang(adapter->xdp_ring[i]);
}
/* get one bit for every active Tx/Rx interrupt vector */
@@ -3157,6 +3325,9 @@ static void ixgbevf_free_all_tx_resources(struct ixgbevf_adapter *adapter)
for (i = 0; i < adapter->num_tx_queues; i++)
if (adapter->tx_ring[i]->desc)
ixgbevf_free_tx_resources(adapter->tx_ring[i]);
+ for (i = 0; i < adapter->num_xdp_queues; i++)
+ if (adapter->xdp_ring[i]->desc)
+ ixgbevf_free_tx_resources(adapter->xdp_ring[i]);
}
/**
@@ -3207,7 +3378,7 @@ int ixgbevf_setup_tx_resources(struct ixgbevf_ring *tx_ring)
**/
static int ixgbevf_setup_all_tx_resources(struct ixgbevf_adapter *adapter)
{
- int i, err = 0;
+ int i, j = 0, err = 0;
for (i = 0; i < adapter->num_tx_queues; i++) {
err = ixgbevf_setup_tx_resources(adapter->tx_ring[i]);
@@ -3217,11 +3388,22 @@ static int ixgbevf_setup_all_tx_resources(struct ixgbevf_adapter *adapter)
goto err_setup_tx;
}
+ for (j = 0; j < adapter->num_xdp_queues; j++) {
+ err = ixgbevf_setup_tx_resources(adapter->xdp_ring[j]);
+ if (!err)
+ continue;
+ hw_dbg(&adapter->hw, "Allocation for XDP Queue %u failed\n", j);
+ break;
+ }
+
return 0;
err_setup_tx:
/* rewind the index freeing the rings as we go */
+ while (j--)
+ ixgbevf_free_tx_resources(adapter->xdp_ring[j]);
while (i--)
ixgbevf_free_tx_resources(adapter->tx_ring[i]);
+
return err;
}
@@ -4114,6 +4296,23 @@ static void ixgbevf_shutdown(struct pci_dev *pdev)
ixgbevf_suspend(pdev, PMSG_SUSPEND);
}
+static void ixgbevf_get_tx_ring_stats(struct rtnl_link_stats64 *stats,
+ const struct ixgbevf_ring *ring)
+{
+ u64 bytes, packets;
+ unsigned int start;
+
+ if (ring) {
+ do {
+ start = u64_stats_fetch_begin_irq(&ring->syncp);
+ bytes = ring->stats.bytes;
+ packets = ring->stats.packets;
+ } while (u64_stats_fetch_retry_irq(&ring->syncp, start));
+ stats->tx_bytes += bytes;
+ stats->tx_packets += packets;
+ }
+}
+
static void ixgbevf_get_stats(struct net_device *netdev,
struct rtnl_link_stats64 *stats)
{
@@ -4141,13 +4340,12 @@ static void ixgbevf_get_stats(struct net_device *netdev,
for (i = 0; i < adapter->num_tx_queues; i++) {
ring = adapter->tx_ring[i];
- do {
- start = u64_stats_fetch_begin_irq(&ring->syncp);
- bytes = ring->stats.bytes;
- packets = ring->stats.packets;
- } while (u64_stats_fetch_retry_irq(&ring->syncp, start));
- stats->tx_bytes += bytes;
- stats->tx_packets += packets;
+ ixgbevf_get_tx_ring_stats(stats, ring);
+ }
+
+ for (i = 0; i < adapter->num_xdp_queues; i++) {
+ ring = adapter->xdp_ring[i];
+ ixgbevf_get_tx_ring_stats(stats, ring);
}
rcu_read_unlock();
}
@@ -4201,8 +4399,25 @@ static int ixgbevf_xdp_setup(struct net_device *dev, struct bpf_prog *prog)
}
old_prog = xchg(&adapter->xdp_prog, prog);
- for (i = 0; i < adapter->num_rx_queues; i++)
- xchg(&adapter->rx_ring[i]->xdp_prog, adapter->xdp_prog);
+
+ /* If transitioning XDP modes reconfigure rings */
+ if (!!prog != !!old_prog) {
+ /* Hardware has to reinitialize queues and interrupts to
+ * match packet buffer alignment. Unfortunately, the
+ * hardware is not flexible enough to do this dynamically.
+ */
+ if (netif_running(dev))
+ ixgbevf_close(dev);
+
+ ixgbevf_clear_interrupt_scheme(adapter);
+ ixgbevf_init_interrupt_scheme(adapter);
+
+ if (netif_running(dev))
+ ixgbevf_open(dev);
+ } else {
+ for (i = 0; i < adapter->num_rx_queues; i++)
+ xchg(&adapter->rx_ring[i]->xdp_prog, adapter->xdp_prog);
+ }
if (old_prog)
bpf_prog_put(old_prog);
--
2.14.3
^ permalink raw reply related
* [net-next 09/12] ixgbevf: Delay tail write for XDP packets
From: Jeff Kirsher @ 2018-03-23 23:16 UTC (permalink / raw)
To: davem; +Cc: Tony Nguyen, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180323231631.32502-1-jeffrey.t.kirsher@intel.com>
From: Tony Nguyen <anthony.l.nguyen@intel.com>
Current XDP implementation hits the tail on every XDP_TX; change the
driver to only hit the tail after packet processing is complete.
Based on
commit 7379f97a4fce ("ixgbe: delay tail write to every 'n' packets")
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
Acked-by: John Fastabend <john.fastabend@gmail.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 30 ++++++++++++++---------
1 file changed, 18 insertions(+), 12 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
index 309f549808e4..5167e81e0cf1 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
@@ -1016,14 +1016,8 @@ static int ixgbevf_xmit_xdp_ring(struct ixgbevf_ring *ring,
cpu_to_le32((len << IXGBE_ADVTXD_PAYLEN_SHIFT) |
IXGBE_ADVTXD_CC);
- /* Force memory writes to complete before letting h/w know there
- * are new descriptors to fetch. (Only applicable for weak-ordered
- * memory model archs, such as IA-64).
- *
- * We also need this memory barrier to make certain all of the
- * status bits have been updated before next_to_watch is written.
- */
- wmb();
+ /* Avoid any potential race with cleanup */
+ smp_wmb();
/* set next_to_watch value indicating a packet is present */
i++;
@@ -1033,8 +1027,6 @@ static int ixgbevf_xmit_xdp_ring(struct ixgbevf_ring *ring,
tx_buffer->next_to_watch = tx_desc;
ring->next_to_use = i;
- /* notify HW of packet */
- ixgbevf_write_tail(ring, i);
return IXGBEVF_XDP_TX;
}
@@ -1101,6 +1093,7 @@ static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
struct ixgbevf_adapter *adapter = q_vector->adapter;
u16 cleaned_count = ixgbevf_desc_unused(rx_ring);
struct sk_buff *skb = rx_ring->skb;
+ bool xdp_xmit = false;
struct xdp_buff xdp;
xdp.rxq = &rx_ring->xdp_rxq;
@@ -1142,11 +1135,13 @@ static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
}
if (IS_ERR(skb)) {
- if (PTR_ERR(skb) == -IXGBEVF_XDP_TX)
+ if (PTR_ERR(skb) == -IXGBEVF_XDP_TX) {
+ xdp_xmit = true;
ixgbevf_rx_buffer_flip(rx_ring, rx_buffer,
size);
- else
+ } else {
rx_buffer->pagecnt_bias++;
+ }
total_rx_packets++;
total_rx_bytes += size;
} else if (skb) {
@@ -1208,6 +1203,17 @@ static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
/* place incomplete frames back on ring for completion */
rx_ring->skb = skb;
+ if (xdp_xmit) {
+ struct ixgbevf_ring *xdp_ring =
+ adapter->xdp_ring[rx_ring->queue_index];
+
+ /* Force memory writes to complete before letting h/w
+ * know there are new descriptors to fetch.
+ */
+ wmb();
+ ixgbevf_write_tail(xdp_ring, xdp_ring->next_to_use);
+ }
+
u64_stats_update_begin(&rx_ring->syncp);
rx_ring->stats.packets += total_rx_packets;
rx_ring->stats.bytes += total_rx_bytes;
--
2.14.3
^ permalink raw reply related
* [net-next 11/12] ixgbevf: Add XDP queue stats reporting
From: Jeff Kirsher @ 2018-03-23 23:16 UTC (permalink / raw)
To: davem; +Cc: Tony Nguyen, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180323231631.32502-1-jeffrey.t.kirsher@intel.com>
From: Tony Nguyen <anthony.l.nguyen@intel.com>
XDP stats are included in TX stats, however, they are not
reported in TX queue stats since they are setup on different
queues. Add reporting for XDP queue stats to provide
consistency between the total stats and per queue stats.
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
Acked-by: John Fastabend <john.fastabend@gmail.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbevf/ethtool.c | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/drivers/net/ethernet/intel/ixgbevf/ethtool.c b/drivers/net/ethernet/intel/ixgbevf/ethtool.c
index da8c0e299a37..8e7d6c6f5c92 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ethtool.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ethtool.c
@@ -82,6 +82,7 @@ static struct ixgbe_stats ixgbevf_gstrings_stats[] = {
#define IXGBEVF_QUEUE_STATS_LEN ( \
(((struct ixgbevf_adapter *)netdev_priv(netdev))->num_tx_queues + \
+ ((struct ixgbevf_adapter *)netdev_priv(netdev))->num_xdp_queues + \
((struct ixgbevf_adapter *)netdev_priv(netdev))->num_rx_queues) * \
(sizeof(struct ixgbevf_stats) / sizeof(u64)))
#define IXGBEVF_GLOBAL_STATS_LEN ARRAY_SIZE(ixgbevf_gstrings_stats)
@@ -491,6 +492,23 @@ static void ixgbevf_get_ethtool_stats(struct net_device *netdev,
i += 2;
}
+ /* populate XDP queue data */
+ for (j = 0; j < adapter->num_xdp_queues; j++) {
+ ring = adapter->xdp_ring[j];
+ if (!ring) {
+ data[i++] = 0;
+ data[i++] = 0;
+ continue;
+ }
+
+ do {
+ start = u64_stats_fetch_begin_irq(&ring->syncp);
+ data[i] = ring->stats.packets;
+ data[i + 1] = ring->stats.bytes;
+ } while (u64_stats_fetch_retry_irq(&ring->syncp, start));
+ i += 2;
+ }
+
/* populate Rx queue data */
for (j = 0; j < adapter->num_rx_queues; j++) {
ring = adapter->rx_ring[j];
@@ -534,6 +552,12 @@ static void ixgbevf_get_strings(struct net_device *netdev, u32 stringset,
sprintf(p, "tx_queue_%u_bytes", i);
p += ETH_GSTRING_LEN;
}
+ for (i = 0; i < adapter->num_xdp_queues; i++) {
+ sprintf(p, "xdp_queue_%u_packets", i);
+ p += ETH_GSTRING_LEN;
+ sprintf(p, "xdp_queue_%u_bytes", i);
+ p += ETH_GSTRING_LEN;
+ }
for (i = 0; i < adapter->num_rx_queues; i++) {
sprintf(p, "rx_queue_%u_packets", i);
p += ETH_GSTRING_LEN;
--
2.14.3
^ permalink raw reply related
* [net-next 12/12] ixgbe: tweak page counting for XDP_REDIRECT
From: Jeff Kirsher @ 2018-03-23 23:16 UTC (permalink / raw)
To: davem
Cc: Björn Töpel, netdev, nhorman, sassmann, jogreene,
Jeff Kirsher
In-Reply-To: <20180323231631.32502-1-jeffrey.t.kirsher@intel.com>
From: Björn Töpel <bjorn.topel@intel.com>
The current page counting scheme assumes that the reference count
cannot decrease until the received frame is sent to the upper layers
of the networking stack. This assumption does not hold for the
XDP_REDIRECT action, since a page (pointed out by xdp_buff) can have
its reference count decreased via the xdp_do_redirect call.
To work around that, we now start off by a large page count and then
don't allow a refcount less than two.
Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index c0d8d7238f71..afadba99f7b8 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -1629,7 +1629,8 @@ static bool ixgbe_alloc_mapped_page(struct ixgbe_ring *rx_ring,
bi->dma = dma;
bi->page = page;
bi->page_offset = ixgbe_rx_offset(rx_ring);
- bi->pagecnt_bias = 1;
+ page_ref_add(page, USHRT_MAX - 1);
+ bi->pagecnt_bias = USHRT_MAX;
rx_ring->rx_stats.alloc_rx_page++;
return true;
@@ -2039,8 +2040,8 @@ static bool ixgbe_can_reuse_rx_page(struct ixgbe_rx_buffer *rx_buffer)
* the pagecnt_bias and page count so that we fully restock the
* number of references the driver holds.
*/
- if (unlikely(!pagecnt_bias)) {
- page_ref_add(page, USHRT_MAX);
+ if (unlikely(pagecnt_bias == 1)) {
+ page_ref_add(page, USHRT_MAX - 1);
rx_buffer->pagecnt_bias = USHRT_MAX;
}
--
2.14.3
^ permalink raw reply related
* [net-next 10/12] ixgbevf: Add support for meta data
From: Jeff Kirsher @ 2018-03-23 23:16 UTC (permalink / raw)
To: davem; +Cc: Tony Nguyen, netdev, nhorman, sassmann, jogreene, Jeff Kirsher
In-Reply-To: <20180323231631.32502-1-jeffrey.t.kirsher@intel.com>
From: Tony Nguyen <anthony.l.nguyen@intel.com>
Add support for XDP meta data when using build skb.
Based on commit 366a88fe2f40 ("bpf, ixgbe: add meta data support")
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
Acked-by: John Fastabend <john.fastabend@gmail.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 29 +++++++++++++++++++----
1 file changed, 25 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
index 5167e81e0cf1..3d9033f26eff 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
@@ -889,6 +889,20 @@ struct sk_buff *ixgbevf_construct_skb(struct ixgbevf_ring *rx_ring,
#if L1_CACHE_BYTES < 128
prefetch(xdp->data + L1_CACHE_BYTES);
#endif
+ /* Note, we get here by enabling legacy-rx via:
+ *
+ * ethtool --set-priv-flags <dev> legacy-rx on
+ *
+ * In this mode, we currently get 0 extra XDP headroom as
+ * opposed to having legacy-rx off, where we process XDP
+ * packets going to stack via ixgbevf_build_skb().
+ *
+ * For ixgbevf_construct_skb() mode it means that the
+ * xdp->data_meta will always point to xdp->data, since
+ * the helper cannot expand the head. Should this ever
+ * changed in future for legacy-rx mode on, then lets also
+ * add xdp->data_meta handling here.
+ */
/* allocate a skb to store the frags */
skb = napi_alloc_skb(&rx_ring->q_vector->napi, IXGBEVF_RX_HDR_SIZE);
@@ -936,6 +950,7 @@ static struct sk_buff *ixgbevf_build_skb(struct ixgbevf_ring *rx_ring,
struct xdp_buff *xdp,
union ixgbe_adv_rx_desc *rx_desc)
{
+ unsigned int metasize = xdp->data - xdp->data_meta;
#if (PAGE_SIZE < 8192)
unsigned int truesize = ixgbevf_rx_pg_size(rx_ring) / 2;
#else
@@ -945,10 +960,14 @@ static struct sk_buff *ixgbevf_build_skb(struct ixgbevf_ring *rx_ring,
#endif
struct sk_buff *skb;
- /* prefetch first cache line of first page */
- prefetch(xdp->data);
+ /* Prefetch first cache line of first page. If xdp->data_meta
+ * is unused, this points to xdp->data, otherwise, we likely
+ * have a consumer accessing first few bytes of meta data,
+ * and then actual data.
+ */
+ prefetch(xdp->data_meta);
#if L1_CACHE_BYTES < 128
- prefetch(xdp->data + L1_CACHE_BYTES);
+ prefetch(xdp->data_meta + L1_CACHE_BYTES);
#endif
/* build an skb around the page buffer */
@@ -959,6 +978,8 @@ static struct sk_buff *ixgbevf_build_skb(struct ixgbevf_ring *rx_ring,
/* update pointers within the skb to store the data */
skb_reserve(skb, xdp->data - xdp->data_hard_start);
__skb_put(skb, xdp->data_end - xdp->data);
+ if (metasize)
+ skb_metadata_set(skb, metasize);
/* update buffer offset */
#if (PAGE_SIZE < 8192)
@@ -1126,7 +1147,7 @@ static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
if (!skb) {
xdp.data = page_address(rx_buffer->page) +
rx_buffer->page_offset;
- xdp_set_data_meta_invalid(&xdp);
+ xdp.data_meta = xdp.data;
xdp.data_hard_start = xdp.data -
ixgbevf_rx_offset(rx_ring);
xdp.data_end = xdp.data + size;
--
2.14.3
^ permalink raw reply related
* Re: [RFC v3 net-next 13/18] net/sched: Introduce the TBS Qdisc
From: Jesus Sanchez-Palencia @ 2018-03-23 23:34 UTC (permalink / raw)
To: Thomas Gleixner
Cc: netdev, jhs, xiyou.wangcong, jiri, vinicius.gomes, richardcochran,
anna-maria, henrik, john.stultz, levi.pearson, edumazet, willemb,
mlichvar
In-Reply-To: <alpine.DEB.2.21.1803230948210.4625@nanos.tec.linutronix.de>
Hi Thomas,
On 03/23/2018 01:49 AM, Thomas Gleixner wrote:
> On Thu, 22 Mar 2018, Jesus Sanchez-Palencia wrote:
>> On 03/22/2018 03:11 PM, Thomas Gleixner wrote:
>> So, are you just opposing to the case where sorting off + offload off is used?
>> (i.e. the scheduled FIFO case)
>
> FIFO does not make any sense if your packets have a fixed transmission
> time. I yet have to see a reasonable explanation why FIFO in the context of
> time ordered would be a good thing.
On context of tbs, the scheduled FIFO was developed just so consistency was kept
between all 4 variants, basically (sw best-effort or hw offload vs sorting
enabled or sorting disabled).
I don't have any strong argument in favor of this mode at the moment, so I will
just remove it on a next version - unless someone else brings up a valid use
case for it, of course.
Thanks for the feedback,
Jesus
^ permalink raw reply
* [PATCH] net: qualcomm: rmnet: check for null ep to avoid null pointer dereference
From: Colin King @ 2018-03-23 23:51 UTC (permalink / raw)
To: David S . Miller, Subash Abhinov Kasiviswanathan, netdev
Cc: kernel-janitors, linux-kernel
From: Colin Ian King <colin.king@canonical.com>
The call to rmnet_get_endpoint can potentially return NULL so check
for this to avoid any subsequent null pointer dereferences on a NULL
ep.
Detected by CoverityScan, CID#1465385 ("Dereference null return value")
Fixes: 23790ef12082 ("net: qualcomm: rmnet: Allow to configure flags for existing devices")
Signed-off-by: Colin Ian King <colin.king@canonical.com>
---
drivers/net/ethernet/qualcomm/rmnet/rmnet_config.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.c b/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.c
index 38d9356ebcc4..d33988570217 100644
--- a/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.c
+++ b/drivers/net/ethernet/qualcomm/rmnet/rmnet_config.c
@@ -312,6 +312,8 @@ static int rmnet_changelink(struct net_device *dev, struct nlattr *tb[],
if (data[IFLA_RMNET_MUX_ID]) {
mux_id = nla_get_u16(data[IFLA_RMNET_MUX_ID]);
ep = rmnet_get_endpoint(port, priv->mux_id);
+ if (!ep)
+ return -ENODEV;
hlist_del_init_rcu(&ep->hlnode);
hlist_add_head_rcu(&ep->hlnode, &port->muxed_ep[mux_id]);
--
2.15.1
^ permalink raw reply related
* Re: [PATCH v7 0/7] netdev: intel: Eliminate duplicate barriers on weakly-ordered archs
From: Jeff Kirsher @ 2018-03-23 23:58 UTC (permalink / raw)
To: Alexander Duyck, Sinan Kaya, intel-wired-lan
Cc: Netdev, Timur Tabi, sulrich, linux-arm-kernel, linux-arm-msm
In-Reply-To: <CAKgT0UcaxM8Mj9Y9fR0B+aafTm1SzTqh+gHT5ozgvyaoHvrzRQ@mail.gmail.com>
[-- Attachment #1.1: Type: text/plain, Size: 3094 bytes --]
On Fri, 2018-03-23 at 14:53 -0700, Alexander Duyck wrote:
> On Fri, Mar 23, 2018 at 11:52 AM, Sinan Kaya <okaya@codeaurora.org>
> wrote:
> > Code includes wmb() followed by writel() in multiple places. writel()
> > already has a barrier on some architectures like arm64.
> >
> > This ends up CPU observing two barriers back to back before executing
> > the
> > register write.
> >
> > Since code already has an explicit barrier call, changing writel() to
> > writel_relaxed().
> >
> > I did a regex search for wmb() followed by writel() in each drivers
> > directory.
> > I scrubbed the ones I care about in this series.
> >
> > I considered "ease of change", "popular usage" and "performance
> > critical
> > path" as the determining criteria for my filtering.
> >
> > We used relaxed API heavily on ARM for a long time but
> > it did not exist on other architectures. For this reason, relaxed
> > architectures have been paying double penalty in order to use the
> > common
> > drivers.
> >
> > Now that relaxed API is present on all architectures, we can go and
> > scrub
> > all drivers to see what needs to change and what can remain.
> >
> > We start with mostly used ones and hope to increase the coverage over
> > time.
> > It will take a while to cover all drivers.
> >
> > Feel free to apply patches individually.
>
> I looked over the set and they seem good.
>
> Reviewed-by: Alexander Duyck <alexander.h.duyck@intel.com>
Grrr, patch 1 does not apply cleanly to my next-queue tree (dev-queue
branch). I will deal with this series in a day or two, after I have dealt
with my driver pull requests.
> >
> > Changes since v6:
> > clean up between 2..6 and then make your Alex's changes on 1 and 7
> > The mmiowb shouldn't be needed for Rx. Only one CPU will be running
> > NAPI for the queue and we will synchronize this with a full writel
> > anyway when we re-enable the interrupts.
> >
> > Sinan Kaya (7):
> > i40e/i40evf: Eliminate duplicate barriers on weakly-ordered archs
> > ixgbe: eliminate duplicate barriers on weakly-ordered archs
> > igbvf: eliminate duplicate barriers on weakly-ordered archs
> > igb: eliminate duplicate barriers on weakly-ordered archs
> > fm10k: Eliminate duplicate barriers on weakly-ordered archs
> > ixgbevf: keep writel() closer to wmb()
> > ixgbevf: eliminate duplicate barriers on weakly-ordered archs
> >
> > drivers/net/ethernet/intel/fm10k/fm10k_main.c | 4 ++--
> > drivers/net/ethernet/intel/i40e/i40e_txrx.c | 14 ++++++++++----
> > drivers/net/ethernet/intel/i40evf/i40e_txrx.c | 4 ++--
> > drivers/net/ethernet/intel/igb/igb_main.c | 4 ++--
> > drivers/net/ethernet/intel/igbvf/netdev.c | 4 ++--
> > drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 8 ++++----
> > drivers/net/ethernet/intel/ixgbevf/ixgbevf.h | 5 -----
> > drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 11 ++++++++---
> > 8 files changed, 30 insertions(+), 24 deletions(-)
> >
> > --
> > 2.7.4
> >
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 176 bytes --]
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply
* [PATCH net-next 00/13] liquidio: Tx queue cleanup
From: Felix Manlunas @ 2018-03-24 0:36 UTC (permalink / raw)
To: davem
Cc: netdev, raghu.vatsavayi, derek.chickles, satananda.burla,
intiyaz.basha
From: Intiyaz Basha <intiyaz.basha@cavium.com>
Moved some common function to octeon_network.h
Removed some unwanted functions and checks.
Intiyaz Basha (13):
liquidio: Moved common function txqs_stop to octeon_network.h
liquidio: Moved common function txqs_wake to octeon_network.h
liquidio: Moved common function txqs_start to octeon_network.h
liquidio: Moved common function skb_iq to to octeon_network.h
liquidio: Removed one line function stop_txq
liquidio: Removed start_txq function
liquidio: Removed netif_is_multiqueue check
liquidio: Removed one line function stop_q
liquidio: Removed one line function wake_q
liquidio: Function call skb_iq for deriving queue from skb
liquidio: Renamed txqs_wake to wake_txqs
liquidio: Renamed txqs_stop to stop_txqs
liquidio: Renamed txqs_start to start_txqs
drivers/net/ethernet/cavium/liquidio/lio_core.c | 18 +-
drivers/net/ethernet/cavium/liquidio/lio_main.c | 213 ++++-----------------
drivers/net/ethernet/cavium/liquidio/lio_vf_main.c | 167 ++--------------
.../net/ethernet/cavium/liquidio/octeon_network.h | 52 +++++
4 files changed, 109 insertions(+), 341 deletions(-)
--
1.8.3.1
^ permalink raw reply
* [PATCH net-next 01/13] liquidio: Moved common function txqs_stop to octeon_network.h
From: Felix Manlunas @ 2018-03-24 0:36 UTC (permalink / raw)
To: davem
Cc: netdev, raghu.vatsavayi, derek.chickles, satananda.burla,
intiyaz.basha
In-Reply-To: <20180324003618.GA6457@felix-thinkpad.cavium.com>
From: Intiyaz Basha <intiyaz.basha@cavium.com>
Moving common function txqs_stop to octeon_network.h
Signed-off-by: Intiyaz Basha <intiyaz.basha@cavium.com>
Acked-by: Derek Chickles <derek.chickles@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
---
drivers/net/ethernet/cavium/liquidio/lio_main.c | 16 ----------------
drivers/net/ethernet/cavium/liquidio/lio_vf_main.c | 16 ----------------
drivers/net/ethernet/cavium/liquidio/octeon_network.h | 16 ++++++++++++++++
3 files changed, 16 insertions(+), 32 deletions(-)
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c
index 140085b..05c5162f 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c
@@ -509,22 +509,6 @@ static void liquidio_deinit_pci(void)
}
/**
- * \brief Stop Tx queues
- * @param netdev network device
- */
-static inline void txqs_stop(struct net_device *netdev)
-{
- if (netif_is_multiqueue(netdev)) {
- int i;
-
- for (i = 0; i < netdev->num_tx_queues; i++)
- netif_stop_subqueue(netdev, i);
- } else {
- netif_stop_queue(netdev);
- }
-}
-
-/**
* \brief Start Tx queues
* @param netdev network device
*/
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
index 3342d64..2e31446 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
@@ -285,22 +285,6 @@ static pci_ers_result_t liquidio_pcie_error_detected(struct pci_dev *pdev,
};
/**
- * \brief Stop Tx queues
- * @param netdev network device
- */
-static void txqs_stop(struct net_device *netdev)
-{
- if (netif_is_multiqueue(netdev)) {
- int i;
-
- for (i = 0; i < netdev->num_tx_queues; i++)
- netif_stop_subqueue(netdev, i);
- } else {
- netif_stop_queue(netdev);
- }
-}
-
-/**
* \brief Start Tx queues
* @param netdev network device
*/
diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_network.h b/drivers/net/ethernet/cavium/liquidio/octeon_network.h
index 76803a5..d8079e35 100644
--- a/drivers/net/ethernet/cavium/liquidio/octeon_network.h
+++ b/drivers/net/ethernet/cavium/liquidio/octeon_network.h
@@ -506,4 +506,20 @@ static inline int wait_for_pending_requests(struct octeon_device *oct)
return 0;
}
+/**
+ * \brief Stop Tx queues
+ * @param netdev network device
+ */
+static inline void txqs_stop(struct net_device *netdev)
+{
+ if (netif_is_multiqueue(netdev)) {
+ int i;
+
+ for (i = 0; i < netdev->num_tx_queues; i++)
+ netif_stop_subqueue(netdev, i);
+ } else {
+ netif_stop_queue(netdev);
+ }
+}
+
#endif
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next 02/13] liquidio: Moved common function txqs_wake to octeon_network.h
From: Felix Manlunas @ 2018-03-24 0:36 UTC (permalink / raw)
To: davem
Cc: netdev, raghu.vatsavayi, derek.chickles, satananda.burla,
intiyaz.basha
In-Reply-To: <20180324003618.GA6457@felix-thinkpad.cavium.com>
From: Intiyaz Basha <intiyaz.basha@cavium.com>
Moving common function txqs_wake to octeon_network.h
Signed-off-by: Intiyaz Basha <intiyaz.basha@cavium.com>
Acked-by: Derek Chickles <derek.chickles@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
---
drivers/net/ethernet/cavium/liquidio/lio_main.c | 28 ----------------------
drivers/net/ethernet/cavium/liquidio/lio_vf_main.c | 27 ---------------------
.../net/ethernet/cavium/liquidio/octeon_network.h | 27 +++++++++++++++++++++
3 files changed, 27 insertions(+), 55 deletions(-)
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c
index 05c5162f..089f494 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c
@@ -525,34 +525,6 @@ static inline void txqs_start(struct net_device *netdev)
}
/**
- * \brief Wake Tx queues
- * @param netdev network device
- */
-static inline void txqs_wake(struct net_device *netdev)
-{
- struct lio *lio = GET_LIO(netdev);
-
- if (netif_is_multiqueue(netdev)) {
- int i;
-
- for (i = 0; i < netdev->num_tx_queues; i++) {
- int qno = lio->linfo.txpciq[i %
- lio->oct_dev->num_iqs].s.q_no;
-
- if (__netif_subqueue_stopped(netdev, i)) {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, qno,
- tx_restart, 1);
- netif_wake_subqueue(netdev, i);
- }
- }
- } else {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, lio->txq,
- tx_restart, 1);
- netif_wake_queue(netdev);
- }
-}
-
-/**
* \brief Stop Tx queue
* @param netdev network device
*/
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
index 2e31446..60743c3 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
@@ -301,33 +301,6 @@ static void txqs_start(struct net_device *netdev)
}
/**
- * \brief Wake Tx queues
- * @param netdev network device
- */
-static void txqs_wake(struct net_device *netdev)
-{
- struct lio *lio = GET_LIO(netdev);
-
- if (netif_is_multiqueue(netdev)) {
- int i;
-
- for (i = 0; i < netdev->num_tx_queues; i++) {
- int qno = lio->linfo.txpciq[i % lio->oct_dev->num_iqs]
- .s.q_no;
- if (__netif_subqueue_stopped(netdev, i)) {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, qno,
- tx_restart, 1);
- netif_wake_subqueue(netdev, i);
- }
- }
- } else {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, lio->txq,
- tx_restart, 1);
- netif_wake_queue(netdev);
- }
-}
-
-/**
* \brief Start Tx queue
* @param netdev network device
*/
diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_network.h b/drivers/net/ethernet/cavium/liquidio/octeon_network.h
index d8079e35..2d2f49b 100644
--- a/drivers/net/ethernet/cavium/liquidio/octeon_network.h
+++ b/drivers/net/ethernet/cavium/liquidio/octeon_network.h
@@ -522,4 +522,31 @@ static inline void txqs_stop(struct net_device *netdev)
}
}
+/**
+ * \brief Wake Tx queues
+ * @param netdev network device
+ */
+static inline void txqs_wake(struct net_device *netdev)
+{
+ struct lio *lio = GET_LIO(netdev);
+
+ if (netif_is_multiqueue(netdev)) {
+ int i;
+
+ for (i = 0; i < netdev->num_tx_queues; i++) {
+ int qno = lio->linfo.txpciq[i %
+ lio->oct_dev->num_iqs].s.q_no;
+
+ if (__netif_subqueue_stopped(netdev, i)) {
+ INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, qno,
+ tx_restart, 1);
+ netif_wake_subqueue(netdev, i);
+ }
+ }
+ } else {
+ INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, lio->txq,
+ tx_restart, 1);
+ netif_wake_queue(netdev);
+ }
+}
#endif
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next 03/13] liquidio: Moved common function txqs_start to octeon_network.h
From: Felix Manlunas @ 2018-03-24 0:37 UTC (permalink / raw)
To: davem
Cc: netdev, raghu.vatsavayi, derek.chickles, satananda.burla,
intiyaz.basha
In-Reply-To: <20180324003618.GA6457@felix-thinkpad.cavium.com>
From: Intiyaz Basha <intiyaz.basha@cavium.com>
Moving common function txqs_start to octeon_network.h
Signed-off-by: Intiyaz Basha <intiyaz.basha@cavium.com>
Acked-by: Derek Chickles <derek.chickles@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
---
drivers/net/ethernet/cavium/liquidio/lio_main.c | 16 ----------------
drivers/net/ethernet/cavium/liquidio/lio_vf_main.c | 16 ----------------
drivers/net/ethernet/cavium/liquidio/octeon_network.h | 17 +++++++++++++++++
3 files changed, 17 insertions(+), 32 deletions(-)
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c
index 089f494..8e280c5 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c
@@ -509,22 +509,6 @@ static void liquidio_deinit_pci(void)
}
/**
- * \brief Start Tx queues
- * @param netdev network device
- */
-static inline void txqs_start(struct net_device *netdev)
-{
- if (netif_is_multiqueue(netdev)) {
- int i;
-
- for (i = 0; i < netdev->num_tx_queues; i++)
- netif_start_subqueue(netdev, i);
- } else {
- netif_start_queue(netdev);
- }
-}
-
-/**
* \brief Stop Tx queue
* @param netdev network device
*/
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
index 60743c3..82d70e4 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
@@ -285,22 +285,6 @@ static pci_ers_result_t liquidio_pcie_error_detected(struct pci_dev *pdev,
};
/**
- * \brief Start Tx queues
- * @param netdev network device
- */
-static void txqs_start(struct net_device *netdev)
-{
- if (netif_is_multiqueue(netdev)) {
- int i;
-
- for (i = 0; i < netdev->num_tx_queues; i++)
- netif_start_subqueue(netdev, i);
- } else {
- netif_start_queue(netdev);
- }
-}
-
-/**
* \brief Start Tx queue
* @param netdev network device
*/
diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_network.h b/drivers/net/ethernet/cavium/liquidio/octeon_network.h
index 2d2f49b..0a14d77 100644
--- a/drivers/net/ethernet/cavium/liquidio/octeon_network.h
+++ b/drivers/net/ethernet/cavium/liquidio/octeon_network.h
@@ -549,4 +549,21 @@ static inline void txqs_wake(struct net_device *netdev)
netif_wake_queue(netdev);
}
}
+
+/**
+ * \brief Start Tx queues
+ * @param netdev network device
+ */
+static inline void txqs_start(struct net_device *netdev)
+{
+ if (netif_is_multiqueue(netdev)) {
+ int i;
+
+ for (i = 0; i < netdev->num_tx_queues; i++)
+ netif_start_subqueue(netdev, i);
+ } else {
+ netif_start_queue(netdev);
+ }
+}
+
#endif
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next 04/13] liquidio: Moved common function skb_iq to to octeon_network.h
From: Felix Manlunas @ 2018-03-24 0:37 UTC (permalink / raw)
To: davem
Cc: netdev, raghu.vatsavayi, derek.chickles, satananda.burla,
intiyaz.basha
In-Reply-To: <20180324003618.GA6457@felix-thinkpad.cavium.com>
From: Intiyaz Basha <intiyaz.basha@cavium.com>
Moving common function skb_iq to to octeon_network.h
Signed-off-by: Intiyaz Basha <intiyaz.basha@cavium.com>
Acked-by: Derek Chickles <derek.chickles@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
---
drivers/net/ethernet/cavium/liquidio/lio_main.c | 10 ----------
drivers/net/ethernet/cavium/liquidio/lio_vf_main.c | 10 ----------
drivers/net/ethernet/cavium/liquidio/octeon_network.h | 10 ++++++++++
3 files changed, 10 insertions(+), 20 deletions(-)
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c
index 8e280c5..ec7c403 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c
@@ -1687,16 +1687,6 @@ static int octeon_pci_os_setup(struct octeon_device *oct)
return 0;
}
-static inline int skb_iq(struct lio *lio, struct sk_buff *skb)
-{
- int q = 0;
-
- if (netif_is_multiqueue(lio->netdev))
- q = skb->queue_mapping % lio->linfo.num_txpciq;
-
- return q;
-}
-
/**
* \brief Check Tx queue state for a given network buffer
* @param lio per-network private data
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
index 82d70e4..5f0114d 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
@@ -993,16 +993,6 @@ static int octeon_pci_os_setup(struct octeon_device *oct)
return 0;
}
-static int skb_iq(struct lio *lio, struct sk_buff *skb)
-{
- int q = 0;
-
- if (netif_is_multiqueue(lio->netdev))
- q = skb->queue_mapping % lio->linfo.num_txpciq;
-
- return q;
-}
-
/**
* \brief Check Tx queue state for a given network buffer
* @param lio per-network private data
diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_network.h b/drivers/net/ethernet/cavium/liquidio/octeon_network.h
index 0a14d77..cd334a7 100644
--- a/drivers/net/ethernet/cavium/liquidio/octeon_network.h
+++ b/drivers/net/ethernet/cavium/liquidio/octeon_network.h
@@ -566,4 +566,14 @@ static inline void txqs_start(struct net_device *netdev)
}
}
+static inline int skb_iq(struct lio *lio, struct sk_buff *skb)
+{
+ int q = 0;
+
+ if (netif_is_multiqueue(lio->netdev))
+ q = skb->queue_mapping % lio->linfo.num_txpciq;
+
+ return q;
+}
+
#endif
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next 05/13] liquidio: Removed one line function stop_txq
From: Felix Manlunas @ 2018-03-24 0:37 UTC (permalink / raw)
To: davem
Cc: netdev, raghu.vatsavayi, derek.chickles, satananda.burla,
intiyaz.basha
In-Reply-To: <20180324003618.GA6457@felix-thinkpad.cavium.com>
From: Intiyaz Basha <intiyaz.basha@cavium.com>
Removing one line function stop_txq
Signed-off-by: Intiyaz Basha <intiyaz.basha@cavium.com>
Acked-by: Derek Chickles <derek.chickles@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
---
drivers/net/ethernet/cavium/liquidio/lio_main.c | 11 +----------
1 file changed, 1 insertion(+), 10 deletions(-)
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c
index ec7c403..6b44d44 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c
@@ -509,15 +509,6 @@ static void liquidio_deinit_pci(void)
}
/**
- * \brief Stop Tx queue
- * @param netdev network device
- */
-static void stop_txq(struct net_device *netdev)
-{
- txqs_stop(netdev);
-}
-
-/**
* \brief Start Tx queue
* @param netdev network device
*/
@@ -839,7 +830,7 @@ static inline void update_link_status(struct net_device *netdev,
} else {
dev_dbg(&oct->pci_dev->dev, "%s: link_off", __func__);
netif_carrier_off(netdev);
- stop_txq(netdev);
+ txqs_stop(netdev);
}
if (lio->linfo.link.s.mtu != current_max_mtu) {
netif_info(lio, probe, lio->netdev, "Max MTU changed from %d to %d\n",
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next 06/13] liquidio: Removed start_txq function
From: Felix Manlunas @ 2018-03-24 0:37 UTC (permalink / raw)
To: davem
Cc: netdev, raghu.vatsavayi, derek.chickles, satananda.burla,
intiyaz.basha
In-Reply-To: <20180324003618.GA6457@felix-thinkpad.cavium.com>
From: Intiyaz Basha <intiyaz.basha@cavium.com>
Removing start_txq function from VF and PF files
Signed-off-by: Intiyaz Basha <intiyaz.basha@cavium.com>
Acked-by: Derek Chickles <derek.chickles@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
---
drivers/net/ethernet/cavium/liquidio/lio_main.c | 16 +---------------
drivers/net/ethernet/cavium/liquidio/lio_vf_main.c | 16 +---------------
drivers/net/ethernet/cavium/liquidio/octeon_network.h | 16 ++++++++++------
3 files changed, 12 insertions(+), 36 deletions(-)
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c
index 6b44d44..41e992c 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c
@@ -509,20 +509,6 @@ static void liquidio_deinit_pci(void)
}
/**
- * \brief Start Tx queue
- * @param netdev network device
- */
-static void start_txq(struct net_device *netdev)
-{
- struct lio *lio = GET_LIO(netdev);
-
- if (lio->linfo.link.s.link_up) {
- txqs_start(netdev);
- return;
- }
-}
-
-/**
* \brief Wake a queue
* @param netdev network device
* @param q which queue to wake
@@ -2140,7 +2126,7 @@ static int liquidio_open(struct net_device *netdev)
return -1;
}
- start_txq(netdev);
+ txqs_start(netdev);
/* tell Octeon to start forwarding packets to host */
send_rx_ctrl_cmd(lio, 1);
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
index 5f0114d..57b6ee5 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
@@ -285,20 +285,6 @@ static pci_ers_result_t liquidio_pcie_error_detected(struct pci_dev *pdev,
};
/**
- * \brief Start Tx queue
- * @param netdev network device
- */
-static void start_txq(struct net_device *netdev)
-{
- struct lio *lio = GET_LIO(netdev);
-
- if (lio->linfo.link.s.link_up) {
- txqs_start(netdev);
- return;
- }
-}
-
-/**
* \brief Wake a queue
* @param netdev network device
* @param q which queue to wake
@@ -1189,7 +1175,7 @@ static int liquidio_open(struct net_device *netdev)
lio->intf_open = 1;
netif_info(lio, ifup, lio->netdev, "Interface Open, ready for traffic\n");
- start_txq(netdev);
+ txqs_start(netdev);
/* tell Octeon to start forwarding packets to host */
send_rx_ctrl_cmd(lio, 1);
diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_network.h b/drivers/net/ethernet/cavium/liquidio/octeon_network.h
index cd334a7..72a581a 100644
--- a/drivers/net/ethernet/cavium/liquidio/octeon_network.h
+++ b/drivers/net/ethernet/cavium/liquidio/octeon_network.h
@@ -556,13 +556,17 @@ static inline void txqs_wake(struct net_device *netdev)
*/
static inline void txqs_start(struct net_device *netdev)
{
- if (netif_is_multiqueue(netdev)) {
- int i;
+ struct lio *lio = GET_LIO(netdev);
- for (i = 0; i < netdev->num_tx_queues; i++)
- netif_start_subqueue(netdev, i);
- } else {
- netif_start_queue(netdev);
+ if (lio->linfo.link.s.link_up) {
+ if (netif_is_multiqueue(netdev)) {
+ int i;
+
+ for (i = 0; i < netdev->num_tx_queues; i++)
+ netif_start_subqueue(netdev, i);
+ } else {
+ netif_start_queue(netdev);
+ }
}
}
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next 07/13] liquidio: Removed netif_is_multiqueue check
From: Felix Manlunas @ 2018-03-24 0:37 UTC (permalink / raw)
To: davem
Cc: netdev, raghu.vatsavayi, derek.chickles, satananda.burla,
intiyaz.basha
In-Reply-To: <20180324003618.GA6457@felix-thinkpad.cavium.com>
From: Intiyaz Basha <intiyaz.basha@cavium.com>
Removing checks for netif_is_multiqueue.
Configuring single queue will be a multiqueue netdev with one queues.
Signed-off-by: Intiyaz Basha <intiyaz.basha@cavium.com>
Acked-by: Derek Chickles <derek.chickles@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
---
drivers/net/ethernet/cavium/liquidio/lio_core.c | 18 ++---
drivers/net/ethernet/cavium/liquidio/lio_main.c | 93 +++++++---------------
drivers/net/ethernet/cavium/liquidio/lio_vf_main.c | 55 ++++---------
.../net/ethernet/cavium/liquidio/octeon_network.h | 50 ++++--------
4 files changed, 64 insertions(+), 152 deletions(-)
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_core.c b/drivers/net/ethernet/cavium/liquidio/lio_core.c
index 666cf7e..73e70e0 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_core.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_core.c
@@ -377,20 +377,12 @@ static void lio_update_txq_status(struct octeon_device *oct, int iq_num)
return;
lio = GET_LIO(netdev);
- if (netif_is_multiqueue(netdev)) {
- if (__netif_subqueue_stopped(netdev, iq->q_index) &&
- lio->linfo.link.s.link_up &&
- (!octnet_iq_is_full(oct, iq_num))) {
- netif_wake_subqueue(netdev, iq->q_index);
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, iq_num,
- tx_restart, 1);
- }
- } else if (netif_queue_stopped(netdev) &&
- lio->linfo.link.s.link_up &&
- (!octnet_iq_is_full(oct, lio->txq))) {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, lio->txq,
+ if (__netif_subqueue_stopped(netdev, iq->q_index) &&
+ lio->linfo.link.s.link_up &&
+ (!octnet_iq_is_full(oct, iq_num))) {
+ netif_wake_subqueue(netdev, iq->q_index);
+ INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, iq_num,
tx_restart, 1);
- netif_wake_queue(netdev);
}
}
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c
index 41e992c..5ef9aa0 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c
@@ -515,10 +515,7 @@ static void liquidio_deinit_pci(void)
*/
static inline void wake_q(struct net_device *netdev, int q)
{
- if (netif_is_multiqueue(netdev))
- netif_wake_subqueue(netdev, q);
- else
- netif_wake_queue(netdev);
+ netif_wake_subqueue(netdev, q);
}
/**
@@ -528,10 +525,7 @@ static inline void wake_q(struct net_device *netdev, int q)
*/
static inline void stop_q(struct net_device *netdev, int q)
{
- if (netif_is_multiqueue(netdev))
- netif_stop_subqueue(netdev, q);
- else
- netif_stop_queue(netdev);
+ netif_stop_subqueue(netdev, q);
}
/**
@@ -541,33 +535,24 @@ static inline void stop_q(struct net_device *netdev, int q)
*/
static inline int check_txq_status(struct lio *lio)
{
+ int numqs = lio->netdev->num_tx_queues;
int ret_val = 0;
+ int q, iq;
- if (netif_is_multiqueue(lio->netdev)) {
- int numqs = lio->netdev->num_tx_queues;
- int q, iq = 0;
-
- /* check each sub-queue state */
- for (q = 0; q < numqs; q++) {
- iq = lio->linfo.txpciq[q %
- lio->oct_dev->num_iqs].s.q_no;
- if (octnet_iq_is_full(lio->oct_dev, iq))
- continue;
- if (__netif_subqueue_stopped(lio->netdev, q)) {
- wake_q(lio->netdev, q);
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, iq,
- tx_restart, 1);
- ret_val++;
- }
+ /* check each sub-queue state */
+ for (q = 0; q < numqs; q++) {
+ iq = lio->linfo.txpciq[q %
+ lio->oct_dev->num_iqs].s.q_no;
+ if (octnet_iq_is_full(lio->oct_dev, iq))
+ continue;
+ if (__netif_subqueue_stopped(lio->netdev, q)) {
+ wake_q(lio->netdev, q);
+ INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, iq,
+ tx_restart, 1);
+ ret_val++;
}
- } else {
- if (octnet_iq_is_full(lio->oct_dev, lio->txq))
- return 0;
- wake_q(lio->netdev, lio->txq);
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, lio->txq,
- tx_restart, 1);
- ret_val = 1;
}
+
return ret_val;
}
@@ -1671,15 +1656,10 @@ static int octeon_pci_os_setup(struct octeon_device *oct)
*/
static inline int check_txq_state(struct lio *lio, struct sk_buff *skb)
{
- int q = 0, iq = 0;
+ int q, iq;
- if (netif_is_multiqueue(lio->netdev)) {
- q = skb->queue_mapping;
- iq = lio->linfo.txpciq[(q % lio->oct_dev->num_iqs)].s.q_no;
- } else {
- iq = lio->txq;
- q = iq;
- }
+ q = skb->queue_mapping;
+ iq = lio->linfo.txpciq[(q % lio->oct_dev->num_iqs)].s.q_no;
if (octnet_iq_is_full(lio->oct_dev, iq))
return 0;
@@ -2568,14 +2548,10 @@ static int liquidio_xmit(struct sk_buff *skb, struct net_device *netdev)
lio = GET_LIO(netdev);
oct = lio->oct_dev;
- if (netif_is_multiqueue(netdev)) {
- q_idx = skb->queue_mapping;
- q_idx = (q_idx % (lio->linfo.num_txpciq));
- tag = q_idx;
- iq_no = lio->linfo.txpciq[q_idx].s.q_no;
- } else {
- iq_no = lio->txq;
- }
+ q_idx = skb->queue_mapping;
+ q_idx = (q_idx % (lio->linfo.num_txpciq));
+ tag = q_idx;
+ iq_no = lio->linfo.txpciq[q_idx].s.q_no;
stats = &oct->instr_queue[iq_no]->stats;
@@ -2606,23 +2582,14 @@ static int liquidio_xmit(struct sk_buff *skb, struct net_device *netdev)
ndata.q_no = iq_no;
- if (netif_is_multiqueue(netdev)) {
- if (octnet_iq_is_full(oct, ndata.q_no)) {
- /* defer sending if queue is full */
- netif_info(lio, tx_err, lio->netdev, "Transmit failed iq:%d full\n",
- ndata.q_no);
- stats->tx_iq_busy++;
- return NETDEV_TX_BUSY;
- }
- } else {
- if (octnet_iq_is_full(oct, lio->txq)) {
- /* defer sending if queue is full */
- stats->tx_iq_busy++;
- netif_info(lio, tx_err, lio->netdev, "Transmit failed iq:%d full\n",
- lio->txq);
- return NETDEV_TX_BUSY;
- }
+ if (octnet_iq_is_full(oct, ndata.q_no)) {
+ /* defer sending if queue is full */
+ netif_info(lio, tx_err, lio->netdev, "Transmit failed iq:%d full\n",
+ ndata.q_no);
+ stats->tx_iq_busy++;
+ return NETDEV_TX_BUSY;
}
+
/* pr_info(" XMIT - valid Qs: %d, 1st Q no: %d, cpu: %d, q_no:%d\n",
* lio->linfo.num_txpciq, lio->txq, cpu, ndata.q_no);
*/
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
index 57b6ee5..f46289d 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
@@ -291,10 +291,7 @@ static pci_ers_result_t liquidio_pcie_error_detected(struct pci_dev *pdev,
*/
static void wake_q(struct net_device *netdev, int q)
{
- if (netif_is_multiqueue(netdev))
- netif_wake_subqueue(netdev, q);
- else
- netif_wake_queue(netdev);
+ netif_wake_subqueue(netdev, q);
}
/**
@@ -304,10 +301,7 @@ static void wake_q(struct net_device *netdev, int q)
*/
static void stop_q(struct net_device *netdev, int q)
{
- if (netif_is_multiqueue(netdev))
- netif_stop_subqueue(netdev, q);
- else
- netif_stop_queue(netdev);
+ netif_stop_subqueue(netdev, q);
}
/**
@@ -986,15 +980,10 @@ static int octeon_pci_os_setup(struct octeon_device *oct)
*/
static int check_txq_state(struct lio *lio, struct sk_buff *skb)
{
- int q = 0, iq = 0;
+ int q, iq;
- if (netif_is_multiqueue(lio->netdev)) {
- q = skb->queue_mapping;
- iq = lio->linfo.txpciq[q % lio->oct_dev->num_iqs].s.q_no;
- } else {
- iq = lio->txq;
- q = iq;
- }
+ q = skb->queue_mapping;
+ iq = lio->linfo.txpciq[q % lio->oct_dev->num_iqs].s.q_no;
if (octnet_iq_is_full(lio->oct_dev, iq))
return 0;
@@ -1635,14 +1624,10 @@ static int liquidio_xmit(struct sk_buff *skb, struct net_device *netdev)
lio = GET_LIO(netdev);
oct = lio->oct_dev;
- if (netif_is_multiqueue(netdev)) {
- q_idx = skb->queue_mapping;
- q_idx = (q_idx % (lio->linfo.num_txpciq));
- tag = q_idx;
- iq_no = lio->linfo.txpciq[q_idx].s.q_no;
- } else {
- iq_no = lio->txq;
- }
+ q_idx = skb->queue_mapping;
+ q_idx = (q_idx % (lio->linfo.num_txpciq));
+ tag = q_idx;
+ iq_no = lio->linfo.txpciq[q_idx].s.q_no;
stats = &oct->instr_queue[iq_no]->stats;
@@ -1671,22 +1656,12 @@ static int liquidio_xmit(struct sk_buff *skb, struct net_device *netdev)
ndata.q_no = iq_no;
- if (netif_is_multiqueue(netdev)) {
- if (octnet_iq_is_full(oct, ndata.q_no)) {
- /* defer sending if queue is full */
- netif_info(lio, tx_err, lio->netdev, "Transmit failed iq:%d full\n",
- ndata.q_no);
- stats->tx_iq_busy++;
- return NETDEV_TX_BUSY;
- }
- } else {
- if (octnet_iq_is_full(oct, lio->txq)) {
- /* defer sending if queue is full */
- stats->tx_iq_busy++;
- netif_info(lio, tx_err, lio->netdev, "Transmit failed iq:%d full\n",
- ndata.q_no);
- return NETDEV_TX_BUSY;
- }
+ if (octnet_iq_is_full(oct, ndata.q_no)) {
+ /* defer sending if queue is full */
+ netif_info(lio, tx_err, lio->netdev, "Transmit failed iq:%d full\n",
+ ndata.q_no);
+ stats->tx_iq_busy++;
+ return NETDEV_TX_BUSY;
}
ndata.datasize = skb->len;
diff --git a/drivers/net/ethernet/cavium/liquidio/octeon_network.h b/drivers/net/ethernet/cavium/liquidio/octeon_network.h
index 72a581a..7922a69 100644
--- a/drivers/net/ethernet/cavium/liquidio/octeon_network.h
+++ b/drivers/net/ethernet/cavium/liquidio/octeon_network.h
@@ -512,14 +512,10 @@ static inline int wait_for_pending_requests(struct octeon_device *oct)
*/
static inline void txqs_stop(struct net_device *netdev)
{
- if (netif_is_multiqueue(netdev)) {
- int i;
+ int i;
- for (i = 0; i < netdev->num_tx_queues; i++)
- netif_stop_subqueue(netdev, i);
- } else {
- netif_stop_queue(netdev);
- }
+ for (i = 0; i < netdev->num_tx_queues; i++)
+ netif_stop_subqueue(netdev, i);
}
/**
@@ -529,24 +525,16 @@ static inline void txqs_stop(struct net_device *netdev)
static inline void txqs_wake(struct net_device *netdev)
{
struct lio *lio = GET_LIO(netdev);
+ int i, qno;
- if (netif_is_multiqueue(netdev)) {
- int i;
+ for (i = 0; i < netdev->num_tx_queues; i++) {
+ qno = lio->linfo.txpciq[i % lio->oct_dev->num_iqs].s.q_no;
- for (i = 0; i < netdev->num_tx_queues; i++) {
- int qno = lio->linfo.txpciq[i %
- lio->oct_dev->num_iqs].s.q_no;
-
- if (__netif_subqueue_stopped(netdev, i)) {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, qno,
- tx_restart, 1);
- netif_wake_subqueue(netdev, i);
- }
+ if (__netif_subqueue_stopped(netdev, i)) {
+ INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, qno,
+ tx_restart, 1);
+ netif_wake_subqueue(netdev, i);
}
- } else {
- INCR_INSTRQUEUE_PKT_COUNT(lio->oct_dev, lio->txq,
- tx_restart, 1);
- netif_wake_queue(netdev);
}
}
@@ -557,27 +545,17 @@ static inline void txqs_wake(struct net_device *netdev)
static inline void txqs_start(struct net_device *netdev)
{
struct lio *lio = GET_LIO(netdev);
+ int i;
if (lio->linfo.link.s.link_up) {
- if (netif_is_multiqueue(netdev)) {
- int i;
-
- for (i = 0; i < netdev->num_tx_queues; i++)
- netif_start_subqueue(netdev, i);
- } else {
- netif_start_queue(netdev);
- }
+ for (i = 0; i < netdev->num_tx_queues; i++)
+ netif_start_subqueue(netdev, i);
}
}
static inline int skb_iq(struct lio *lio, struct sk_buff *skb)
{
- int q = 0;
-
- if (netif_is_multiqueue(lio->netdev))
- q = skb->queue_mapping % lio->linfo.num_txpciq;
-
- return q;
+ return skb->queue_mapping % lio->linfo.num_txpciq;
}
#endif
--
1.8.3.1
^ permalink raw reply related
* [PATCH net-next 08/13] liquidio: Removed one line function stop_q
From: Felix Manlunas @ 2018-03-24 0:37 UTC (permalink / raw)
To: davem
Cc: netdev, raghu.vatsavayi, derek.chickles, satananda.burla,
intiyaz.basha
In-Reply-To: <20180324003618.GA6457@felix-thinkpad.cavium.com>
From: Intiyaz Basha <intiyaz.basha@cavium.com>
Removing one line function stop_q
Signed-off-by: Intiyaz Basha <intiyaz.basha@cavium.com>
Acked-by: Derek Chickles <derek.chickles@cavium.com>
Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
---
drivers/net/ethernet/cavium/liquidio/lio_main.c | 12 +-----------
drivers/net/ethernet/cavium/liquidio/lio_vf_main.c | 12 +-----------
2 files changed, 2 insertions(+), 22 deletions(-)
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_main.c b/drivers/net/ethernet/cavium/liquidio/lio_main.c
index 5ef9aa0..be16a1c 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_main.c
@@ -519,16 +519,6 @@ static inline void wake_q(struct net_device *netdev, int q)
}
/**
- * \brief Stop a queue
- * @param netdev network device
- * @param q which queue to stop
- */
-static inline void stop_q(struct net_device *netdev, int q)
-{
- netif_stop_subqueue(netdev, q);
-}
-
-/**
* \brief Check Tx queue status, and take appropriate action
* @param lio per-network private data
* @returns 0 if full, number of queues woken up otherwise
@@ -2745,7 +2735,7 @@ static int liquidio_xmit(struct sk_buff *skb, struct net_device *netdev)
netif_info(lio, tx_queued, lio->netdev, "Transmit queued successfully\n");
if (status == IQ_SEND_STOP)
- stop_q(netdev, q_idx);
+ netif_stop_subqueue(netdev, q_idx);
netif_trans_update(netdev);
diff --git a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
index f46289d..3120aed 100644
--- a/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
+++ b/drivers/net/ethernet/cavium/liquidio/lio_vf_main.c
@@ -295,16 +295,6 @@ static void wake_q(struct net_device *netdev, int q)
}
/**
- * \brief Stop a queue
- * @param netdev network device
- * @param q which queue to stop
- */
-static void stop_q(struct net_device *netdev, int q)
-{
- netif_stop_subqueue(netdev, q);
-}
-
-/**
* Remove the node at the head of the list. The list would be empty at
* the end of this call if there are no more nodes in the list.
*/
@@ -1803,7 +1793,7 @@ static int liquidio_xmit(struct sk_buff *skb, struct net_device *netdev)
if (status == IQ_SEND_STOP) {
dev_err(&oct->pci_dev->dev, "Rcvd IQ_SEND_STOP signal; stopping IQ-%d\n",
iq_no);
- stop_q(netdev, q_idx);
+ netif_stop_subqueue(netdev, q_idx);
}
netif_trans_update(netdev);
--
1.8.3.1
^ permalink raw reply related
* Re: [RFC v3 net-next 13/18] net/sched: Introduce the TBS Qdisc
From: Jesus Sanchez-Palencia @ 2018-03-24 0:34 UTC (permalink / raw)
To: Thomas Gleixner
Cc: netdev, jhs, xiyou.wangcong, jiri, vinicius.gomes, richardcochran,
anna-maria, henrik, John Stultz, levi.pearson, edumazet, willemb,
mlichvar
In-Reply-To: <alpine.DEB.2.21.1803222312061.1489@nanos.tec.linutronix.de>
Hi,
On 03/22/2018 03:52 PM, Thomas Gleixner wrote:
> On Thu, 22 Mar 2018, Jesus Sanchez-Palencia wrote:
>> Our plan was to work directly with the Qbv-like scheduling (per-port) just after
>> the cbs qdisc (Qav), but the feedback here and offline was that there were use
>> cases for a more simplistic launchtime approach (per-queue) as well. We've
>> decided to invest on it first (and postpone the 'taprio' qdisc until there was
>> NIC available with HW support for it, basically).
>
> I missed that discussion due to other urgent stuff on my plate. Just
> skimmed through it. More below.
>
>> You are right, and we agree, that using tbs for a per-port schedule of any sort
>> will require a SW scheduler to be developed on top of it, but we've never said
>> the contrary either. Our vision has always been that these are separate
>> mechanisms with different use-cases, so we do see the value for the kernel to
>> provide both.
>>
>> In other words, tbs is not the final solution for Qbv, and we agree that a 'TAS'
>> qdisc is still necessary. And due to the wide range of applications and hw being
>> used for those out there, we need both specially given that one does not block
>> the other.
>
> So what's the plan for this? Having TAS as a separate entity or TAS feeding
> into the proposed 'basic' time transmission thing?
The second one, I guess. Elaborating, the plan is at some point having TAS as a
separate entity, but which can use tbs for one of its classes (and cbs for
another, and strict priority for everything else, etc).
Basically, the design would something along the lines of 'taprio'. A root qdisc
that is both time and priority aware, and capable of running a schedule for the
port. That schedule can run inside the kernel with hrtimers, or just be
offloaded into the controller if Qbv is supported on HW.
Because it would expose the inner traffic classes in a mq / mqprio / prio style,
then it would allow for other per-queue qdiscs to be attached to it. On a system
using the i210, for instance, we could then have tbs installed on traffic class
0 just dialing hw offload. The Qbv schedule would be running in SW on the TAS
entity (i.e. 'taprio') which would be setting the packets' txtime before
dequeueing packets on a fast path -> tbs -> NIC.
Similarly, other qdisc, like cbs, could be installed if all that traffic class
requires is traffic shaping once its 'gate' is allowed to execute the selected
tx algorithm attached to it.
>
> The general objection I have with the current approach is that it creates
> the playground for all flavours of misdesigned user space implementations
> and just replaces the home brewn and ugly user mode network adapter
> drivers.
>
> But that's not helping the cause at all. There is enough crappy stuff out
> there already and I rather see a proper designed slice management which can
> be utilized and improved by all involved parties.
>
> All variants which utilize the basic time driven packet transmission are
> based on periodic explicit plan scheduling with (local) network wide time
> slice assignment.
>
> It does not matter whether you feed VLAN traffic into a time slice, where
> the VLAN itself does not even have to know about it, or if you have aware
> applications feeding packets to a designated timeslot. The basic principle
> of this is always the same.
>
> So coming back to last years discussion. It totally went into the wrong
> direction because it turned from an approach (the patches) which came from
> the big picture to an single use case and application centric view. That's
> just wrong and I regret that I didn't have the time to pay attention back
> then.
>
> You always need to look at the big picture first and design from there, not
> the other way round. There will always be the argument:
>
> But my application is special and needs X
>
> It's easy to fall for that. From a long experience I know that none of
> these claims ever held. These arguments are made because the people making
> them have either never looked at the big picture or are simply refusing to
> do so because it would cause them work.
>
> If you start from the use case and application centric view and ignore the
> big picture then you end up in a gazillion of extra magic features over
> time which could have been completely avoided if you had put your foot down
> and made everyone to agree on a proper and versatile design in the first
> place.
>
> The more low level access you hand out in the beginning the less commonly
> used, improved and maintained infrastrucure you will get in the end. That
> has happened before in other areas and it will happen here as well. You
> create a user space ABI which you cant get rid off and before you come out
> with the proper interface after that a large number of involved parties
> have gone off and implemented on top of the low level ABI and they will
> never look back.
>
> In the (not so) long run this will create a lot more issues than it
> solves. A simple example is that you cannot run two applications which
> easily could share the network in parallel without major surgery because
> both require to be the management authority.
>
> I've not yet seen a convincing argument why this low level stuff with all
> of its weird flavours is superiour over something which reflects the basic
> operating principle of TSN.
As you know, not all TSN systems are designed the same. Take AVB systems, for
example. These not always are running on networks that are aware of any time
schedule, or at least not quite like what is described by Qbv.
On those systems there is usually a certain number of streams with different
priorities that care mostly about having their bandwidth reserved along the
network. The applications running on such systems are usually based on AVTP,
thus they already have to calculate and set the "avtp presentation time"
per-packet themselves. A Qbv scheduler would probably provide very little
benefits to this domain, IMHO. For "talkers" of these AVB systems, shaping
traffic using txtime (i.e. tbs) can provide a low-jitter alternative to cbs, for
instance.
Thanks,
Jesus
>
> Thanks,
>
> tglx
>
>
>
>
>
>
>
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox