* [PATCH v2 14/22] qede:add scatter gather support
From: Rasesh Mody @ 2016-09-30 7:06 UTC (permalink / raw)
To: dev; +Cc: Dept-EngDPDKDev, Sony Chacko
In-Reply-To: <1475219169-8774-1-git-send-email-rasesh.mody@qlogic.com>
From: Sony Chacko <sony.chacko@qlogic.com>
Add scatter gather support, to enable trasmit
and receive of packets larger than descriptor buffer sizes.
Signed-off-by: Sony Chacko <sony.chacko@qlogic.com>
---
doc/guides/nics/features/qede.ini | 1 +
doc/guides/nics/features/qede_vf.ini | 1 +
doc/guides/nics/qede.rst | 4 +-
drivers/net/qede/qede_ethdev.c | 19 ++--
drivers/net/qede/qede_rxtx.c | 204 ++++++++++++++++++++++++++++-------
drivers/net/qede/qede_rxtx.h | 3 -
6 files changed, 177 insertions(+), 55 deletions(-)
diff --git a/doc/guides/nics/features/qede.ini b/doc/guides/nics/features/qede.ini
index 1d28a23..7d75030 100644
--- a/doc/guides/nics/features/qede.ini
+++ b/doc/guides/nics/features/qede.ini
@@ -9,6 +9,7 @@ Link status = Y
Link status event = Y
MTU update = Y
Jumbo frame = Y
+Scattered Rx = Y
Promiscuous mode = Y
Allmulticast mode = Y
Unicast MAC filter = Y
diff --git a/doc/guides/nics/features/qede_vf.ini b/doc/guides/nics/features/qede_vf.ini
index b4eba0c..acb1b99 100644
--- a/doc/guides/nics/features/qede_vf.ini
+++ b/doc/guides/nics/features/qede_vf.ini
@@ -9,6 +9,7 @@ Link status = Y
Link status event = Y
MTU update = Y
Jumbo frame = Y
+Scattered Rx = Y
Promiscuous mode = Y
Allmulticast mode = Y
Unicast MAC filter = Y
diff --git a/doc/guides/nics/qede.rst b/doc/guides/nics/qede.rst
index 5e31c11..2a585e7 100644
--- a/doc/guides/nics/qede.rst
+++ b/doc/guides/nics/qede.rst
@@ -47,7 +47,7 @@ Supported Features
- Promiscuous mode
- Allmulti mode
- Port hardware statistics
-- Jumbo frames (using single buffer)
+- Jumbo frames
- VLAN offload - Filtering and stripping
- Stateless checksum offloads (IPv4/TCP/UDP)
- Multiple Rx/Tx queues
@@ -58,11 +58,11 @@ Supported Features
- SR-IOV VF
- MTU change
- Multiprocess aware
+- Scatter-Gather
Non-supported Features
----------------------
-- Scatter-Gather Rx/Tx frames
- SR-IOV PF
- Tunneling offloads
- Reload of the PMD after a non-graceful termination
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index c4e82d0..eb9b8aa 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -434,14 +434,14 @@ static int qede_vlan_filter_set(struct rte_eth_dev *eth_dev,
struct qede_vlan_entry *vlan;
int rc;
- if (qdev->configured_vlans == dev_info->num_vlan_filters) {
- DP_NOTICE(edev, false, "Reached max VLAN filter limit"
- " enabling accept_any_vlan\n");
- qede_config_accept_any_vlan(qdev, true);
- return 0;
- }
-
if (on) {
+ if (qdev->configured_vlans == dev_info->num_vlan_filters) {
+ DP_INFO(edev, "Reached max VLAN filter limit"
+ " enabling accept_any_vlan\n");
+ qede_config_accept_any_vlan(qdev, true);
+ return 0;
+ }
+
SLIST_FOREACH(tmp, &qdev->vlan_list_head, list) {
if (tmp->vid == vlan_id) {
DP_ERR(edev, "VLAN %u already configured\n",
@@ -559,11 +559,6 @@ static int qede_dev_configure(struct rte_eth_dev *eth_dev)
}
/* Sanity checks and throw warnings */
- if (rxmode->enable_scatter == 1) {
- DP_ERR(edev, "RX scatter packets is not supported\n");
- return -EINVAL;
- }
-
if (rxmode->enable_lro == 1) {
DP_INFO(edev, "LRO is not supported\n");
return -EINVAL;
diff --git a/drivers/net/qede/qede_rxtx.c b/drivers/net/qede/qede_rxtx.c
index ab16c04..fce6f4f 100644
--- a/drivers/net/qede/qede_rxtx.c
+++ b/drivers/net/qede/qede_rxtx.c
@@ -804,6 +804,58 @@ static inline uint32_t qede_rx_cqe_to_pkt_type(uint16_t flags)
return RTE_PTYPE_L2_ETHER | p_type;
}
+int qede_process_sg_pkts(void *p_rxq, struct rte_mbuf *rx_mb,
+ int num_frags, uint16_t pkt_len)
+{
+ struct qede_rx_queue *rxq = p_rxq;
+ struct qede_dev *qdev = rxq->qdev;
+ struct ecore_dev *edev = &qdev->edev;
+ uint16_t sw_rx_index, cur_size;
+
+ register struct rte_mbuf *seg1 = NULL;
+ register struct rte_mbuf *seg2 = NULL;
+
+ seg1 = rx_mb;
+ while (num_frags) {
+ cur_size = pkt_len > rxq->rx_buf_size ?
+ rxq->rx_buf_size : pkt_len;
+ if (!cur_size) {
+ PMD_RX_LOG(DEBUG, rxq,
+ "SG packet, len and num BD mismatch\n");
+ qede_recycle_rx_bd_ring(rxq, qdev, num_frags);
+ return -EINVAL;
+ }
+
+ if (qede_alloc_rx_buffer(rxq)) {
+ uint8_t index;
+
+ PMD_RX_LOG(DEBUG, rxq, "Buffer allocation failed\n");
+ index = rxq->port_id;
+ rte_eth_devices[index].data->rx_mbuf_alloc_failed++;
+ rxq->rx_alloc_errors++;
+ return -ENOMEM;
+ }
+
+ sw_rx_index = rxq->sw_rx_cons & NUM_RX_BDS(rxq);
+ seg2 = rxq->sw_rx_ring[sw_rx_index].mbuf;
+ qede_rx_bd_ring_consume(rxq);
+ pkt_len -= cur_size;
+ seg2->data_len = cur_size;
+ seg1->next = seg2;
+ seg1 = seg1->next;
+ num_frags--;
+ continue;
+ }
+ seg1 = NULL;
+
+ if (pkt_len)
+ PMD_RX_LOG(DEBUG, rxq,
+ "Mapped all BDs of jumbo, but still have %d bytes\n",
+ pkt_len);
+
+ return ECORE_SUCCESS;
+}
+
uint16_t
qede_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
{
@@ -816,12 +868,12 @@ qede_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
union eth_rx_cqe *cqe;
struct eth_fast_path_rx_reg_cqe *fp_cqe;
register struct rte_mbuf *rx_mb = NULL;
+ register struct rte_mbuf *seg1 = NULL;
enum eth_rx_cqe_type cqe_type;
- uint16_t len, pad;
- uint16_t preload_idx;
- uint8_t csum_flag;
- uint16_t parse_flag;
+ uint16_t len, pad, preload_idx, pkt_len, parse_flag;
+ uint8_t csum_flag, num_frags;
enum rss_hash_type htype;
+ int ret;
hw_comp_cons = rte_le_to_cpu_16(*rxq->hw_cons_ptr);
sw_comp_cons = ecore_chain_get_cons_idx(&rxq->rx_comp_ring);
@@ -891,20 +943,31 @@ qede_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
qede_rx_bd_ring_consume(rxq);
+ if (fp_cqe->bd_num > 1) {
+ pkt_len = rte_le_to_cpu_16(fp_cqe->pkt_len);
+ num_frags = fp_cqe->bd_num - 1;
+
+ pkt_len -= len;
+ seg1 = rx_mb;
+ ret = qede_process_sg_pkts(p_rxq, seg1, num_frags,
+ pkt_len);
+ if (ret != ECORE_SUCCESS) {
+ qede_recycle_rx_bd_ring(rxq, qdev,
+ fp_cqe->bd_num);
+ goto next_cqe;
+ }
+ }
+
/* Prefetch next mbuf while processing current one. */
preload_idx = rxq->sw_rx_cons & NUM_RX_BDS(rxq);
rte_prefetch0(rxq->sw_rx_ring[preload_idx].mbuf);
- if (fp_cqe->bd_num != 1)
- PMD_RX_LOG(DEBUG, rxq,
- "Jumbo-over-BD packet not supported\n");
-
/* Update MBUF fields */
rx_mb->ol_flags = 0;
rx_mb->data_off = pad + RTE_PKTMBUF_HEADROOM;
- rx_mb->nb_segs = 1;
+ rx_mb->nb_segs = fp_cqe->bd_num;
rx_mb->data_len = len;
- rx_mb->pkt_len = len;
+ rx_mb->pkt_len = fp_cqe->pkt_len;
rx_mb->port = rxq->port_id;
rx_mb->packet_type = qede_rx_cqe_to_pkt_type(parse_flag);
@@ -957,24 +1020,28 @@ next_cqe:
static inline int
qede_free_tx_pkt(struct ecore_dev *edev, struct qede_tx_queue *txq)
{
- uint16_t idx = TX_CONS(txq);
+ uint16_t nb_segs, idx = TX_CONS(txq);
struct eth_tx_bd *tx_data_bd;
struct rte_mbuf *mbuf = txq->sw_tx_ring[idx].mbuf;
if (unlikely(!mbuf)) {
+ PMD_TX_LOG(ERR, txq, "null mbuf\n");
PMD_TX_LOG(ERR, txq,
- "null mbuf nb_tx_desc %u nb_tx_avail %u "
- "sw_tx_cons %u sw_tx_prod %u\n",
+ "tx_desc %u tx_avail %u tx_cons %u tx_prod %u\n",
txq->nb_tx_desc, txq->nb_tx_avail, idx,
TX_PROD(txq));
return -1;
}
- /* Free now */
- rte_pktmbuf_free_seg(mbuf);
+ nb_segs = mbuf->nb_segs;
+ while (nb_segs) {
+ /* It's like consuming rxbuf in recv() */
+ ecore_chain_consume(&txq->tx_pbl);
+ txq->nb_tx_avail++;
+ nb_segs--;
+ }
+ rte_pktmbuf_free(mbuf);
txq->sw_tx_ring[idx].mbuf = NULL;
- ecore_chain_consume(&txq->tx_pbl);
- txq->nb_tx_avail++;
return 0;
}
@@ -984,18 +1051,16 @@ qede_process_tx_compl(struct ecore_dev *edev, struct qede_tx_queue *txq)
{
uint16_t tx_compl = 0;
uint16_t hw_bd_cons;
- int rc;
hw_bd_cons = rte_le_to_cpu_16(*txq->hw_cons_ptr);
rte_compiler_barrier();
while (hw_bd_cons != ecore_chain_get_cons_idx(&txq->tx_pbl)) {
- rc = qede_free_tx_pkt(edev, txq);
- if (rc) {
- DP_NOTICE(edev, false,
- "hw_bd_cons = %d, chain_cons=%d\n",
- hw_bd_cons,
- ecore_chain_get_cons_idx(&txq->tx_pbl));
+ if (qede_free_tx_pkt(edev, txq)) {
+ PMD_TX_LOG(ERR, txq,
+ "hw_bd_cons = %u, chain_cons = %u\n",
+ hw_bd_cons,
+ ecore_chain_get_cons_idx(&txq->tx_pbl));
break;
}
txq->sw_tx_cons++; /* Making TXD available */
@@ -1007,6 +1072,55 @@ qede_process_tx_compl(struct ecore_dev *edev, struct qede_tx_queue *txq)
return tx_compl;
}
+/* Populate scatter gather buffer descriptor fields */
+static inline uint16_t qede_encode_sg_bd(struct qede_tx_queue *p_txq,
+ struct rte_mbuf *m_seg,
+ uint16_t count,
+ struct eth_tx_1st_bd *bd1)
+{
+ struct qede_tx_queue *txq = p_txq;
+ struct eth_tx_2nd_bd *bd2 = NULL;
+ struct eth_tx_3rd_bd *bd3 = NULL;
+ struct eth_tx_bd *tx_bd = NULL;
+ uint16_t nb_segs = count;
+ dma_addr_t mapping;
+
+ /* Check for scattered buffers */
+ while (m_seg) {
+ if (nb_segs == 1) {
+ bd2 = (struct eth_tx_2nd_bd *)
+ ecore_chain_produce(&txq->tx_pbl);
+ memset(bd2, 0, sizeof(*bd2));
+ mapping = rte_mbuf_data_dma_addr(m_seg);
+ bd2->addr.hi = rte_cpu_to_le_32(U64_HI(mapping));
+ bd2->addr.lo = rte_cpu_to_le_32(U64_LO(mapping));
+ bd2->nbytes = rte_cpu_to_le_16(m_seg->data_len);
+ } else if (nb_segs == 2) {
+ bd3 = (struct eth_tx_3rd_bd *)
+ ecore_chain_produce(&txq->tx_pbl);
+ memset(bd3, 0, sizeof(*bd3));
+ mapping = rte_mbuf_data_dma_addr(m_seg);
+ bd3->addr.hi = rte_cpu_to_le_32(U64_HI(mapping));
+ bd3->addr.lo = rte_cpu_to_le_32(U64_LO(mapping));
+ bd3->nbytes = rte_cpu_to_le_16(m_seg->data_len);
+ } else {
+ tx_bd = (struct eth_tx_bd *)
+ ecore_chain_produce(&txq->tx_pbl);
+ memset(tx_bd, 0, sizeof(*tx_bd));
+ mapping = rte_mbuf_data_dma_addr(m_seg);
+ tx_bd->addr.hi = rte_cpu_to_le_32(U64_HI(mapping));
+ tx_bd->addr.lo = rte_cpu_to_le_32(U64_LO(mapping));
+ tx_bd->nbytes = rte_cpu_to_le_16(m_seg->data_len);
+ }
+ nb_segs++;
+ bd1->data.nbds = nb_segs;
+ m_seg = m_seg->next;
+ }
+
+ /* Return total scattered buffers */
+ return nb_segs;
+}
+
uint16_t
qede_xmit_pkts(void *p_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
{
@@ -1014,12 +1128,14 @@ qede_xmit_pkts(void *p_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
struct qede_dev *qdev = txq->qdev;
struct ecore_dev *edev = &qdev->edev;
struct qede_fastpath *fp;
- struct eth_tx_1st_bd *first_bd;
+ struct eth_tx_1st_bd *bd1;
+ struct rte_mbuf *m_seg = NULL;
uint16_t nb_tx_pkts;
uint16_t nb_pkt_sent = 0;
uint16_t bd_prod;
uint16_t idx;
uint16_t tx_count;
+ uint16_t nb_segs = 0;
fp = &qdev->fp_array[QEDE_RSS_COUNT(qdev) + txq->queue_id];
@@ -1029,7 +1145,8 @@ qede_xmit_pkts(void *p_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
(void)qede_process_tx_compl(edev, txq);
}
- nb_tx_pkts = RTE_MIN(nb_pkts, (txq->nb_tx_avail / MAX_NUM_TX_BDS));
+ nb_tx_pkts = RTE_MIN(nb_pkts, (txq->nb_tx_avail /
+ ETH_TX_MAX_BDS_PER_NON_LSO_PACKET));
if (unlikely(nb_tx_pkts == 0)) {
PMD_TX_LOG(DEBUG, txq, "Out of BDs nb_pkts=%u avail=%u\n",
nb_pkts, txq->nb_tx_avail);
@@ -1041,38 +1158,49 @@ qede_xmit_pkts(void *p_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
/* Fill the entry in the SW ring and the BDs in the FW ring */
idx = TX_PROD(txq);
struct rte_mbuf *mbuf = *tx_pkts++;
+
txq->sw_tx_ring[idx].mbuf = mbuf;
- first_bd = (struct eth_tx_1st_bd *)
- ecore_chain_produce(&txq->tx_pbl);
- first_bd->data.bd_flags.bitfields =
- 1 << ETH_TX_1ST_BD_FLAGS_START_BD_SHIFT;
+ bd1 = (struct eth_tx_1st_bd *)ecore_chain_produce(&txq->tx_pbl);
+ /* Zero init struct fields */
+ bd1->data.bd_flags.bitfields = 0;
+ bd1->data.bitfields = 0;
+
+ bd1->data.bd_flags.bitfields =
+ 1 << ETH_TX_1ST_BD_FLAGS_START_BD_SHIFT;
/* Map MBUF linear data for DMA and set in the first BD */
- QEDE_BD_SET_ADDR_LEN(first_bd, rte_mbuf_data_dma_addr(mbuf),
- mbuf->data_len);
+ QEDE_BD_SET_ADDR_LEN(bd1, rte_mbuf_data_dma_addr(mbuf),
+ mbuf->pkt_len);
/* Descriptor based VLAN insertion */
if (mbuf->ol_flags & (PKT_TX_VLAN_PKT | PKT_TX_QINQ_PKT)) {
- first_bd->data.vlan = rte_cpu_to_le_16(mbuf->vlan_tci);
- first_bd->data.bd_flags.bitfields |=
+ bd1->data.vlan = rte_cpu_to_le_16(mbuf->vlan_tci);
+ bd1->data.bd_flags.bitfields |=
1 << ETH_TX_1ST_BD_FLAGS_VLAN_INSERTION_SHIFT;
}
/* Offload the IP checksum in the hardware */
if (mbuf->ol_flags & PKT_TX_IP_CKSUM) {
- first_bd->data.bd_flags.bitfields |=
+ bd1->data.bd_flags.bitfields |=
1 << ETH_TX_1ST_BD_FLAGS_IP_CSUM_SHIFT;
}
/* L4 checksum offload (tcp or udp) */
if (mbuf->ol_flags & (PKT_TX_TCP_CKSUM | PKT_TX_UDP_CKSUM)) {
- first_bd->data.bd_flags.bitfields |=
+ bd1->data.bd_flags.bitfields |=
1 << ETH_TX_1ST_BD_FLAGS_L4_CSUM_SHIFT;
/* IPv6 + extn. -> later */
}
- first_bd->data.nbds = MAX_NUM_TX_BDS;
+
+ /* Handle fragmented MBUF */
+ m_seg = mbuf->next;
+ nb_segs++;
+ bd1->data.nbds = nb_segs;
+ /* Encode scatter gather buffer descriptors if required */
+ nb_segs = qede_encode_sg_bd(txq, m_seg, nb_segs, bd1);
+ txq->nb_tx_avail = txq->nb_tx_avail - nb_segs;
+ nb_segs = 0;
txq->sw_tx_prod++;
rte_prefetch0(txq->sw_tx_ring[TX_PROD(txq)].mbuf);
- txq->nb_tx_avail--;
bd_prod =
rte_cpu_to_le_16(ecore_chain_get_prod_idx(&txq->tx_pbl));
nb_pkt_sent++;
diff --git a/drivers/net/qede/qede_rxtx.h b/drivers/net/qede/qede_rxtx.h
index 9ee69ed..da47b21 100644
--- a/drivers/net/qede/qede_rxtx.h
+++ b/drivers/net/qede/qede_rxtx.h
@@ -30,9 +30,6 @@
#define TX_CONS(txq) (txq->sw_tx_cons & NUM_TX_BDS(txq))
#define TX_PROD(txq) (txq->sw_tx_prod & NUM_TX_BDS(txq))
-/* Number of TX BDs per packet used currently */
-#define MAX_NUM_TX_BDS 1
-
#define QEDE_DEFAULT_TX_FREE_THRESH 32
#define QEDE_CSUM_ERROR (1 << 0)
--
1.8.3.1
^ permalink raw reply related
* [PATCH v2 15/22] qede/base: change rx tx queue start APIs
From: Rasesh Mody @ 2016-09-30 7:06 UTC (permalink / raw)
To: dev; +Cc: Dept-EngDPDKDev, Rasesh Mody
In-Reply-To: <1475219169-8774-1-git-send-email-rasesh.mody@qlogic.com>
Changed q_{rx,tx}_start APIs to use common queue start parameters
Signed-off-by: Rasesh Mody <rasesh.mody@qlogic.com>
---
drivers/net/qede/base/ecore_l2.c | 131 +++++++++++++++--------------------
drivers/net/qede/base/ecore_l2.h | 26 ++-----
drivers/net/qede/base/ecore_l2_api.h | 69 +++++++++---------
drivers/net/qede/base/ecore_sriov.c | 28 +++++---
drivers/net/qede/qede_eth_if.c | 47 ++++++-------
drivers/net/qede/qede_eth_if.h | 11 ++-
drivers/net/qede/qede_rxtx.c | 27 +++++---
7 files changed, 155 insertions(+), 184 deletions(-)
diff --git a/drivers/net/qede/base/ecore_l2.c b/drivers/net/qede/base/ecore_l2.c
index 83a62e0..e460868 100644
--- a/drivers/net/qede/base/ecore_l2.c
+++ b/drivers/net/qede/base/ecore_l2.c
@@ -548,12 +548,7 @@ enum _ecore_status_t
ecore_sp_eth_rxq_start_ramrod(struct ecore_hwfn *p_hwfn,
u16 opaque_fid,
u32 cid,
- u16 rx_queue_id,
- u8 vf_rx_queue_id,
- u8 vport_id,
- u8 stats_id,
- u16 sb,
- u8 sb_index,
+ struct ecore_queue_start_common_params *p_params,
u16 bd_max_bytes,
dma_addr_t bd_chain_phys_addr,
dma_addr_t cqe_pbl_addr,
@@ -568,22 +563,23 @@ ecore_sp_eth_rxq_start_ramrod(struct ecore_hwfn *p_hwfn,
enum _ecore_status_t rc = ECORE_NOTIMPL;
/* Store information for the stop */
- p_rx_cid = &p_hwfn->p_rx_cids[rx_queue_id];
+ p_rx_cid = &p_hwfn->p_rx_cids[p_params->queue_id];
p_rx_cid->cid = cid;
p_rx_cid->opaque_fid = opaque_fid;
- p_rx_cid->vport_id = vport_id;
+ p_rx_cid->vport_id = p_params->vport_id;
- rc = ecore_fw_vport(p_hwfn, vport_id, &abs_vport_id);
+ rc = ecore_fw_vport(p_hwfn, p_params->vport_id, &abs_vport_id);
if (rc != ECORE_SUCCESS)
return rc;
- rc = ecore_fw_l2_queue(p_hwfn, rx_queue_id, &abs_rx_q_id);
+ rc = ecore_fw_l2_queue(p_hwfn, p_params->queue_id, &abs_rx_q_id);
if (rc != ECORE_SUCCESS)
return rc;
DP_VERBOSE(p_hwfn, ECORE_MSG_SP,
"opaque_fid=0x%x, cid=0x%x, rx_qid=0x%x, vport_id=0x%x, sb_id=0x%x\n",
- opaque_fid, cid, rx_queue_id, vport_id, sb);
+ opaque_fid, cid, p_params->queue_id,
+ p_params->vport_id, p_params->sb);
/* Get SPQ entry */
OSAL_MEMSET(&init_data, 0, sizeof(init_data));
@@ -599,10 +595,10 @@ ecore_sp_eth_rxq_start_ramrod(struct ecore_hwfn *p_hwfn,
p_ramrod = &p_ent->ramrod.rx_queue_start;
- p_ramrod->sb_id = OSAL_CPU_TO_LE16(sb);
- p_ramrod->sb_index = sb_index;
+ p_ramrod->sb_id = OSAL_CPU_TO_LE16(p_params->sb);
+ p_ramrod->sb_index = (u8)p_params->sb_idx;
p_ramrod->vport_id = abs_vport_id;
- p_ramrod->stats_counter_id = stats_id;
+ p_ramrod->stats_counter_id = p_params->stats_id;
p_ramrod->rx_queue_id = OSAL_CPU_TO_LE16(abs_rx_q_id);
p_ramrod->complete_cqe_flg = 0;
p_ramrod->complete_event_flg = 1;
@@ -613,30 +609,27 @@ ecore_sp_eth_rxq_start_ramrod(struct ecore_hwfn *p_hwfn,
p_ramrod->num_of_pbl_pages = OSAL_CPU_TO_LE16(cqe_pbl_size);
DMA_REGPAIR_LE(p_ramrod->cqe_pbl_addr, cqe_pbl_addr);
- if (vf_rx_queue_id || b_use_zone_a_prod) {
- p_ramrod->vf_rx_prod_index = vf_rx_queue_id;
+ if (p_params->vf_qid || b_use_zone_a_prod) {
+ p_ramrod->vf_rx_prod_index = (u8)p_params->vf_qid;
DP_VERBOSE(p_hwfn, ECORE_MSG_SP,
"Queue%s is meant for VF rxq[%02x]\n",
b_use_zone_a_prod ? " [legacy]" : "",
- vf_rx_queue_id);
+ p_params->vf_qid);
p_ramrod->vf_rx_prod_use_zone_a = b_use_zone_a_prod;
}
return ecore_spq_post(p_hwfn, p_ent, OSAL_NULL);
}
-enum _ecore_status_t ecore_sp_eth_rx_queue_start(struct ecore_hwfn *p_hwfn,
- u16 opaque_fid,
- u8 rx_queue_id,
- u8 vport_id,
- u8 stats_id,
- u16 sb,
- u8 sb_index,
- u16 bd_max_bytes,
- dma_addr_t bd_chain_phys_addr,
- dma_addr_t cqe_pbl_addr,
- u16 cqe_pbl_size,
- void OSAL_IOMEM **pp_prod)
+enum _ecore_status_t
+ecore_sp_eth_rx_queue_start(struct ecore_hwfn *p_hwfn,
+ u16 opaque_fid,
+ struct ecore_queue_start_common_params *p_params,
+ u16 bd_max_bytes,
+ dma_addr_t bd_chain_phys_addr,
+ dma_addr_t cqe_pbl_addr,
+ u16 cqe_pbl_size,
+ void OSAL_IOMEM **pp_prod)
{
struct ecore_hw_cid_data *p_rx_cid;
u32 init_prod_val = 0;
@@ -646,20 +639,20 @@ enum _ecore_status_t ecore_sp_eth_rx_queue_start(struct ecore_hwfn *p_hwfn,
if (IS_VF(p_hwfn->p_dev)) {
return ecore_vf_pf_rxq_start(p_hwfn,
- rx_queue_id,
- sb,
- sb_index,
+ p_params->queue_id,
+ p_params->sb,
+ (u8)p_params->sb_idx,
bd_max_bytes,
bd_chain_phys_addr,
cqe_pbl_addr,
cqe_pbl_size, pp_prod);
}
- rc = ecore_fw_l2_queue(p_hwfn, rx_queue_id, &abs_l2_queue);
+ rc = ecore_fw_l2_queue(p_hwfn, p_params->queue_id, &abs_l2_queue);
if (rc != ECORE_SUCCESS)
return rc;
- rc = ecore_fw_vport(p_hwfn, stats_id, &abs_stats_id);
+ rc = ecore_fw_vport(p_hwfn, p_params->stats_id, &abs_stats_id);
if (rc != ECORE_SUCCESS)
return rc;
@@ -672,7 +665,7 @@ enum _ecore_status_t ecore_sp_eth_rx_queue_start(struct ecore_hwfn *p_hwfn,
(u32 *)(&init_prod_val));
/* Allocate a CID for the queue */
- p_rx_cid = &p_hwfn->p_rx_cids[rx_queue_id];
+ p_rx_cid = &p_hwfn->p_rx_cids[p_params->queue_id];
rc = ecore_cxt_acquire_cid(p_hwfn, PROTOCOLID_ETH,
&p_rx_cid->cid);
if (rc != ECORE_SUCCESS) {
@@ -680,16 +673,13 @@ enum _ecore_status_t ecore_sp_eth_rx_queue_start(struct ecore_hwfn *p_hwfn,
return rc;
}
p_rx_cid->b_cid_allocated = true;
+ p_params->stats_id = abs_stats_id;
+ p_params->vf_qid = 0;
rc = ecore_sp_eth_rxq_start_ramrod(p_hwfn,
opaque_fid,
p_rx_cid->cid,
- rx_queue_id,
- 0,
- vport_id,
- abs_stats_id,
- sb,
- sb_index,
+ p_params,
bd_max_bytes,
bd_chain_phys_addr,
cqe_pbl_addr,
@@ -816,12 +806,8 @@ ecore_sp_eth_rx_queue_stop(struct ecore_hwfn *p_hwfn,
enum _ecore_status_t
ecore_sp_eth_txq_start_ramrod(struct ecore_hwfn *p_hwfn,
u16 opaque_fid,
- u16 tx_queue_id,
u32 cid,
- u8 vport_id,
- u8 stats_id,
- u16 sb,
- u8 sb_index,
+ struct ecore_queue_start_common_params *p_params,
dma_addr_t pbl_addr,
u16 pbl_size,
union ecore_qm_pq_params *p_pq_params)
@@ -835,15 +821,15 @@ ecore_sp_eth_txq_start_ramrod(struct ecore_hwfn *p_hwfn,
enum _ecore_status_t rc = ECORE_NOTIMPL;
/* Store information for the stop */
- p_tx_cid = &p_hwfn->p_tx_cids[tx_queue_id];
+ p_tx_cid = &p_hwfn->p_tx_cids[p_params->queue_id];
p_tx_cid->cid = cid;
p_tx_cid->opaque_fid = opaque_fid;
- rc = ecore_fw_vport(p_hwfn, vport_id, &abs_vport_id);
+ rc = ecore_fw_vport(p_hwfn, p_params->vport_id, &abs_vport_id);
if (rc != ECORE_SUCCESS)
return rc;
- rc = ecore_fw_l2_queue(p_hwfn, tx_queue_id, &abs_tx_q_id);
+ rc = ecore_fw_l2_queue(p_hwfn, p_params->queue_id, &abs_tx_q_id);
if (rc != ECORE_SUCCESS)
return rc;
@@ -862,9 +848,9 @@ ecore_sp_eth_txq_start_ramrod(struct ecore_hwfn *p_hwfn,
p_ramrod = &p_ent->ramrod.tx_queue_start;
p_ramrod->vport_id = abs_vport_id;
- p_ramrod->sb_id = OSAL_CPU_TO_LE16(sb);
- p_ramrod->sb_index = sb_index;
- p_ramrod->stats_counter_id = stats_id;
+ p_ramrod->sb_id = OSAL_CPU_TO_LE16(p_params->sb);
+ p_ramrod->sb_index = (u8)p_params->sb_idx;
+ p_ramrod->stats_counter_id = p_params->stats_id;
p_ramrod->queue_zone_id = OSAL_CPU_TO_LE16(abs_tx_q_id);
@@ -877,17 +863,14 @@ ecore_sp_eth_txq_start_ramrod(struct ecore_hwfn *p_hwfn,
return ecore_spq_post(p_hwfn, p_ent, OSAL_NULL);
}
-enum _ecore_status_t ecore_sp_eth_tx_queue_start(struct ecore_hwfn *p_hwfn,
- u16 opaque_fid,
- u16 tx_queue_id,
- u8 vport_id,
- u8 stats_id,
- u16 sb,
- u8 sb_index,
- u8 tc,
- dma_addr_t pbl_addr,
- u16 pbl_size,
- void OSAL_IOMEM **pp_doorbell)
+enum _ecore_status_t
+ecore_sp_eth_tx_queue_start(struct ecore_hwfn *p_hwfn,
+ u16 opaque_fid,
+ struct ecore_queue_start_common_params *p_params,
+ u8 tc,
+ dma_addr_t pbl_addr,
+ u16 pbl_size,
+ void OSAL_IOMEM **pp_doorbell)
{
struct ecore_hw_cid_data *p_tx_cid;
union ecore_qm_pq_params pq_params;
@@ -896,19 +879,19 @@ enum _ecore_status_t ecore_sp_eth_tx_queue_start(struct ecore_hwfn *p_hwfn,
if (IS_VF(p_hwfn->p_dev)) {
return ecore_vf_pf_txq_start(p_hwfn,
- tx_queue_id,
- sb,
- sb_index,
+ p_params->queue_id,
+ p_params->sb,
+ (u8)p_params->sb_idx,
pbl_addr,
pbl_size,
pp_doorbell);
}
- rc = ecore_fw_vport(p_hwfn, stats_id, &abs_stats_id);
+ rc = ecore_fw_vport(p_hwfn, p_params->stats_id, &abs_stats_id);
if (rc != ECORE_SUCCESS)
return rc;
- p_tx_cid = &p_hwfn->p_tx_cids[tx_queue_id];
+ p_tx_cid = &p_hwfn->p_tx_cids[p_params->queue_id];
OSAL_MEMSET(p_tx_cid, 0, sizeof(*p_tx_cid));
OSAL_MEMSET(&pq_params, 0, sizeof(pq_params));
@@ -924,18 +907,16 @@ enum _ecore_status_t ecore_sp_eth_tx_queue_start(struct ecore_hwfn *p_hwfn,
DP_VERBOSE(p_hwfn, ECORE_MSG_SP,
"opaque_fid=0x%x, cid=0x%x, tx_qid=0x%x, vport_id=0x%x, sb_id=0x%x\n",
- opaque_fid, p_tx_cid->cid, tx_queue_id,
- vport_id, sb);
+ opaque_fid, p_tx_cid->cid, p_params->queue_id,
+ p_params->vport_id, p_params->sb);
+
+ p_params->stats_id = abs_stats_id;
/* TODO - set tc in the pq_params for multi-cos */
rc = ecore_sp_eth_txq_start_ramrod(p_hwfn,
opaque_fid,
- tx_queue_id,
p_tx_cid->cid,
- vport_id,
- abs_stats_id,
- sb,
- sb_index,
+ p_params,
pbl_addr,
pbl_size,
&pq_params);
diff --git a/drivers/net/qede/base/ecore_l2.h b/drivers/net/qede/base/ecore_l2.h
index c8419a3..9c1bd38 100644
--- a/drivers/net/qede/base/ecore_l2.h
+++ b/drivers/net/qede/base/ecore_l2.h
@@ -40,11 +40,8 @@ ecore_sp_eth_vport_start(struct ecore_hwfn *p_hwfn,
* @param p_hwfn
* @param opaque_fid
* @param cid
- * @param rx_queue_id
- * @param vport_id
- * @param stats_id
- * @param sb
- * @param sb_index
+ * @param p_params [queue_id, vport_id, stats_id, sb, sb_idx, vf_qid]
+ stats_id is absolute packed in p_params.
* @param bd_max_bytes
* @param bd_chain_phys_addr
* @param cqe_pbl_addr
@@ -57,12 +54,7 @@ enum _ecore_status_t
ecore_sp_eth_rxq_start_ramrod(struct ecore_hwfn *p_hwfn,
u16 opaque_fid,
u32 cid,
- u16 rx_queue_id,
- u8 vf_rx_queue_id,
- u8 vport_id,
- u8 stats_id,
- u16 sb,
- u8 sb_index,
+ struct ecore_queue_start_common_params *p_params,
u16 bd_max_bytes,
dma_addr_t bd_chain_phys_addr,
dma_addr_t cqe_pbl_addr,
@@ -74,12 +66,8 @@ ecore_sp_eth_rxq_start_ramrod(struct ecore_hwfn *p_hwfn,
*
* @param p_hwfn
* @param opaque_fid
- * @param tx_queue_id
* @param cid
- * @param vport_id
- * @param stats_id
- * @param sb
- * @param sb_index
+ * @param p_params [queue_id, vport_id,stats_id, sb, sb_idx, vf_qid]
* @param pbl_addr
* @param pbl_size
* @param p_pq_params - parameters for choosing the PQ for this Tx queue
@@ -89,12 +77,8 @@ ecore_sp_eth_rxq_start_ramrod(struct ecore_hwfn *p_hwfn,
enum _ecore_status_t
ecore_sp_eth_txq_start_ramrod(struct ecore_hwfn *p_hwfn,
u16 opaque_fid,
- u16 tx_queue_id,
u32 cid,
- u8 vport_id,
- u8 stats_id,
- u16 sb,
- u8 sb_index,
+ struct ecore_queue_start_common_params *p_params,
dma_addr_t pbl_addr,
u16 pbl_size,
union ecore_qm_pq_params *p_pq_params);
diff --git a/drivers/net/qede/base/ecore_l2_api.h b/drivers/net/qede/base/ecore_l2_api.h
index 447d1fb..1f160d9 100644
--- a/drivers/net/qede/base/ecore_l2_api.h
+++ b/drivers/net/qede/base/ecore_l2_api.h
@@ -27,6 +27,18 @@ enum ecore_rss_caps {
#define ECORE_RSS_KEY_SIZE 10 /* size in 32b chunks */
#endif
+struct ecore_queue_start_common_params {
+ /* Rx/Tx queue id */
+ u8 queue_id;
+ u8 vport_id;
+
+ /* stats_id is relative or absolute depends on function */
+ u8 stats_id;
+ u16 sb;
+ u16 sb_idx;
+ u16 vf_qid;
+};
+
struct ecore_rss_params {
u8 update_rss_config;
u8 rss_enable;
@@ -154,14 +166,7 @@ ecore_filter_accept_cmd(
*
* @param p_hwfn
* @param opaque_fid
- * @param rx_queue_id RX Queue ID: Zero based, per VPort, allocated
- * by assignment (=rssId)
- * @param vport_id VPort ID
- * @param u8 stats_id VPort ID which the queue stats
- * will be added to
- * @param sb Status Block of the Function Event Ring
- * @param sb_index Index into the status block of the
- * Function Event Ring
+ * @p_params [stats_id is relative, packed in p_params]
* @param bd_max_bytes Maximum bytes that can be placed on a BD
* @param bd_chain_phys_addr Physical address of BDs for receive.
* @param cqe_pbl_addr Physical address of the CQE PBL Table.
@@ -172,18 +177,15 @@ ecore_filter_accept_cmd(
*
* @return enum _ecore_status_t
*/
-enum _ecore_status_t ecore_sp_eth_rx_queue_start(struct ecore_hwfn *p_hwfn,
- u16 opaque_fid,
- u8 rx_queue_id,
- u8 vport_id,
- u8 stats_id,
- u16 sb,
- u8 sb_index,
- u16 bd_max_bytes,
- dma_addr_t bd_chain_phys_addr,
- dma_addr_t cqe_pbl_addr,
- u16 cqe_pbl_size,
- void OSAL_IOMEM **pp_prod);
+enum _ecore_status_t
+ecore_sp_eth_rx_queue_start(struct ecore_hwfn *p_hwfn,
+ u16 opaque_fid,
+ struct ecore_queue_start_common_params *p_params,
+ u16 bd_max_bytes,
+ dma_addr_t bd_chain_phys_addr,
+ dma_addr_t cqe_pbl_addr,
+ u16 cqe_pbl_size,
+ void OSAL_IOMEM **pp_prod);
/**
* @brief ecore_sp_eth_rx_queue_stop -
@@ -216,13 +218,7 @@ ecore_sp_eth_rx_queue_stop(struct ecore_hwfn *p_hwfn,
*
* @param p_hwfn
* @param opaque_fid
- * @param tx_queue_id TX Queue ID
- * @param vport_id VPort ID
- * @param u8 stats_id VPort ID which the queue stats
- * will be added to
- * @param sb Status Block of the Function Event Ring
- * @param sb_index Index into the status block of the Function
- * Event Ring
+ * @p_params
* @param tc traffic class to use with this L2 txq
* @param pbl_addr address of the pbl array
* @param pbl_size number of entries in pbl
@@ -232,17 +228,14 @@ ecore_sp_eth_rx_queue_stop(struct ecore_hwfn *p_hwfn,
*
* @return enum _ecore_status_t
*/
-enum _ecore_status_t ecore_sp_eth_tx_queue_start(struct ecore_hwfn *p_hwfn,
- u16 opaque_fid,
- u16 tx_queue_id,
- u8 vport_id,
- u8 stats_id,
- u16 sb,
- u8 sb_index,
- u8 tc,
- dma_addr_t pbl_addr,
- u16 pbl_size,
- void OSAL_IOMEM **pp_doorbell);
+enum _ecore_status_t
+ecore_sp_eth_tx_queue_start(struct ecore_hwfn *p_hwfn,
+ u16 opaque_fid,
+ struct ecore_queue_start_common_params *p_params,
+ u8 tc,
+ dma_addr_t pbl_addr,
+ u16 pbl_size,
+ void OSAL_IOMEM **pp_doorbell);
/**
* @brief ecore_sp_eth_tx_queue_stop -
diff --git a/drivers/net/qede/base/ecore_sriov.c b/drivers/net/qede/base/ecore_sriov.c
index eb3a1e2..b28d728 100644
--- a/drivers/net/qede/base/ecore_sriov.c
+++ b/drivers/net/qede/base/ecore_sriov.c
@@ -1961,6 +1961,7 @@ static void ecore_iov_vf_mbx_start_rxq(struct ecore_hwfn *p_hwfn,
struct ecore_ptt *p_ptt,
struct ecore_vf_info *vf)
{
+ struct ecore_queue_start_common_params p_params;
struct ecore_iov_vf_mbx *mbx = &vf->vf_mbx;
u8 status = PFVF_STATUS_NO_RESOURCE;
struct vfpf_start_rxq_tlv *req;
@@ -1968,6 +1969,13 @@ static void ecore_iov_vf_mbx_start_rxq(struct ecore_hwfn *p_hwfn,
enum _ecore_status_t rc;
req = &mbx->req_virt->start_rxq;
+ OSAL_MEMSET(&p_params, 0, sizeof(p_params));
+ p_params.queue_id = (u8)vf->vf_queues[req->rx_qid].fw_rx_qid;
+ p_params.vf_qid = req->rx_qid;
+ p_params.vport_id = vf->vport_id;
+ p_params.stats_id = vf->abs_vf_id + 0x10,
+ p_params.sb = req->hw_sb;
+ p_params.sb_idx = req->sb_index;
if (!ecore_iov_validate_rxq(p_hwfn, vf, req->rx_qid) ||
!ecore_iov_validate_sb(p_hwfn, vf, req->hw_sb))
@@ -1987,12 +1995,7 @@ static void ecore_iov_vf_mbx_start_rxq(struct ecore_hwfn *p_hwfn,
rc = ecore_sp_eth_rxq_start_ramrod(p_hwfn, vf->opaque_fid,
vf->vf_queues[req->rx_qid].fw_cid,
- vf->vf_queues[req->rx_qid].fw_rx_qid,
- (u8)req->rx_qid,
- vf->vport_id,
- vf->abs_vf_id + 0x10,
- req->hw_sb,
- req->sb_index,
+ &p_params,
req->bd_max_bytes,
req->rxq_addr,
req->cqe_pbl_addr,
@@ -2057,6 +2060,7 @@ static void ecore_iov_vf_mbx_start_txq(struct ecore_hwfn *p_hwfn,
struct ecore_ptt *p_ptt,
struct ecore_vf_info *vf)
{
+ struct ecore_queue_start_common_params p_params;
struct ecore_iov_vf_mbx *mbx = &vf->vf_mbx;
u8 status = PFVF_STATUS_NO_RESOURCE;
union ecore_qm_pq_params pq_params;
@@ -2069,6 +2073,12 @@ static void ecore_iov_vf_mbx_start_txq(struct ecore_hwfn *p_hwfn,
pq_params.eth.vf_id = vf->relative_vf_id;
req = &mbx->req_virt->start_txq;
+ OSAL_MEMSET(&p_params, 0, sizeof(p_params));
+ p_params.queue_id = (u8)vf->vf_queues[req->tx_qid].fw_tx_qid;
+ p_params.vport_id = vf->vport_id;
+ p_params.stats_id = vf->abs_vf_id + 0x10,
+ p_params.sb = req->hw_sb;
+ p_params.sb_idx = req->sb_index;
if (!ecore_iov_validate_txq(p_hwfn, vf, req->tx_qid) ||
!ecore_iov_validate_sb(p_hwfn, vf, req->hw_sb))
@@ -2077,12 +2087,8 @@ static void ecore_iov_vf_mbx_start_txq(struct ecore_hwfn *p_hwfn,
rc = ecore_sp_eth_txq_start_ramrod(
p_hwfn,
vf->opaque_fid,
- vf->vf_queues[req->tx_qid].fw_tx_qid,
vf->vf_queues[req->tx_qid].fw_cid,
- vf->vport_id,
- vf->abs_vf_id + 0x10,
- req->hw_sb,
- req->sb_index,
+ &p_params,
req->pbl_addr,
req->pbl_size,
&pq_params);
diff --git a/drivers/net/qede/qede_eth_if.c b/drivers/net/qede/qede_eth_if.c
index a19b22e..1ae6127 100644
--- a/drivers/net/qede/qede_eth_if.c
+++ b/drivers/net/qede/qede_eth_if.c
@@ -168,9 +168,9 @@ qed_update_vport(struct ecore_dev *edev, struct qed_update_vport_params *params)
static int
qed_start_rxq(struct ecore_dev *edev,
- uint8_t rss_id, uint8_t rx_queue_id,
- uint8_t vport_id, uint16_t sb,
- uint8_t sb_index, uint16_t bd_max_bytes,
+ uint8_t rss_num,
+ struct ecore_queue_start_common_params *p_params,
+ uint16_t bd_max_bytes,
dma_addr_t bd_chain_phys_addr,
dma_addr_t cqe_pbl_addr,
uint16_t cqe_pbl_size, void OSAL_IOMEM * *pp_prod)
@@ -178,28 +178,28 @@ qed_start_rxq(struct ecore_dev *edev,
struct ecore_hwfn *p_hwfn;
int rc, hwfn_index;
- hwfn_index = rss_id % edev->num_hwfns;
+ hwfn_index = rss_num % edev->num_hwfns;
p_hwfn = &edev->hwfns[hwfn_index];
+ p_params->queue_id = p_params->queue_id / edev->num_hwfns;
+ p_params->stats_id = p_params->vport_id;
+
rc = ecore_sp_eth_rx_queue_start(p_hwfn,
p_hwfn->hw_info.opaque_fid,
- rx_queue_id / edev->num_hwfns,
- vport_id,
- vport_id,
- sb,
- sb_index,
+ p_params,
bd_max_bytes,
bd_chain_phys_addr,
cqe_pbl_addr, cqe_pbl_size, pp_prod);
if (rc) {
- DP_ERR(edev, "Failed to start RXQ#%d\n", rx_queue_id);
+ DP_ERR(edev, "Failed to start RXQ#%d\n", p_params->queue_id);
return rc;
}
DP_VERBOSE(edev, ECORE_MSG_SPQ,
- "Started RX-Q %d [rss %d] on V-PORT %d and SB %d\n",
- rx_queue_id, rss_id, vport_id, sb);
+ "Started RX-Q %d [rss_num %d] on V-PORT %d and SB %d\n",
+ p_params->queue_id, rss_num, p_params->vport_id,
+ p_params->sb);
return 0;
}
@@ -226,36 +226,35 @@ qed_stop_rxq(struct ecore_dev *edev, struct qed_stop_rxq_params *params)
static int
qed_start_txq(struct ecore_dev *edev,
- uint8_t rss_id, uint16_t tx_queue_id,
- uint8_t vport_id, uint16_t sb,
- uint8_t sb_index,
+ uint8_t rss_num,
+ struct ecore_queue_start_common_params *p_params,
dma_addr_t pbl_addr,
uint16_t pbl_size, void OSAL_IOMEM * *pp_doorbell)
{
struct ecore_hwfn *p_hwfn;
int rc, hwfn_index;
- hwfn_index = rss_id % edev->num_hwfns;
+ hwfn_index = rss_num % edev->num_hwfns;
p_hwfn = &edev->hwfns[hwfn_index];
+ p_params->queue_id = p_params->queue_id / edev->num_hwfns;
+ p_params->stats_id = p_params->vport_id;
+
rc = ecore_sp_eth_tx_queue_start(p_hwfn,
p_hwfn->hw_info.opaque_fid,
- tx_queue_id / edev->num_hwfns,
- vport_id,
- vport_id,
- sb,
- sb_index,
+ p_params,
0 /* tc */,
pbl_addr, pbl_size, pp_doorbell);
if (rc) {
- DP_ERR(edev, "Failed to start TXQ#%d\n", tx_queue_id);
+ DP_ERR(edev, "Failed to start TXQ#%d\n", p_params->queue_id);
return rc;
}
DP_VERBOSE(edev, ECORE_MSG_SPQ,
- "Started TX-Q %d [rss %d] on V-PORT %d and SB %d\n",
- tx_queue_id, rss_id, vport_id, sb);
+ "Started TX-Q %d [rss_num %d] on V-PORT %d and SB %d\n",
+ p_params->queue_id, rss_num, p_params->vport_id,
+ p_params->sb);
return 0;
}
diff --git a/drivers/net/qede/qede_eth_if.h b/drivers/net/qede/qede_eth_if.h
index 5a7fdc9..33655c3 100644
--- a/drivers/net/qede/qede_eth_if.h
+++ b/drivers/net/qede/qede_eth_if.h
@@ -133,9 +133,9 @@ struct qed_eth_ops {
struct qed_update_vport_params *params);
int (*q_rx_start)(struct ecore_dev *cdev,
- uint8_t rss_id, uint8_t rx_queue_id,
- uint8_t vport_id, uint16_t sb,
- uint8_t sb_index, uint16_t bd_max_bytes,
+ uint8_t rss_num,
+ struct ecore_queue_start_common_params *p_params,
+ uint16_t bd_max_bytes,
dma_addr_t bd_chain_phys_addr,
dma_addr_t cqe_pbl_addr,
uint16_t cqe_pbl_size, void OSAL_IOMEM * *pp_prod);
@@ -144,9 +144,8 @@ struct qed_eth_ops {
struct qed_stop_rxq_params *params);
int (*q_tx_start)(struct ecore_dev *edev,
- uint8_t rss_id, uint16_t tx_queue_id,
- uint8_t vport_id, uint16_t sb,
- uint8_t sb_index,
+ uint8_t rss_num,
+ struct ecore_queue_start_common_params *p_params,
dma_addr_t pbl_addr,
uint16_t pbl_size, void OSAL_IOMEM * *pp_doorbell);
diff --git a/drivers/net/qede/qede_rxtx.c b/drivers/net/qede/qede_rxtx.c
index fce6f4f..d903a84 100644
--- a/drivers/net/qede/qede_rxtx.c
+++ b/drivers/net/qede/qede_rxtx.c
@@ -561,6 +561,7 @@ static int qede_start_queues(struct rte_eth_dev *eth_dev, bool clear_stats)
{
struct qede_dev *qdev = eth_dev->data->dev_private;
struct ecore_dev *edev = &qdev->edev;
+ struct ecore_queue_start_common_params q_params;
struct qed_update_vport_rss_params *rss_params = &qdev->rss_params;
struct qed_dev_info *qed_info = &qdev->dev_info.common;
struct qed_update_vport_params vport_update_params;
@@ -580,12 +581,15 @@ static int qede_start_queues(struct rte_eth_dev *eth_dev, bool clear_stats)
page_cnt = ecore_chain_get_page_cnt(&fp->rxq->
rx_comp_ring);
+ memset(&q_params, 0, sizeof(q_params));
+ q_params.queue_id = i;
+ q_params.vport_id = 0;
+ q_params.sb = fp->sb_info->igu_sb_id;
+ q_params.sb_idx = RX_PI;
+
ecore_sb_ack(fp->sb_info, IGU_INT_DISABLE, 0);
- rc = qdev->ops->q_rx_start(edev, i, fp->rxq->queue_id,
- 0,
- fp->sb_info->igu_sb_id,
- RX_PI,
+ rc = qdev->ops->q_rx_start(edev, i, &q_params,
fp->rxq->rx_buf_size,
fp->rxq->rx_bd_ring.p_phys_addr,
p_phys_table,
@@ -611,11 +615,16 @@ static int qede_start_queues(struct rte_eth_dev *eth_dev, bool clear_stats)
p_phys_table = ecore_chain_get_pbl_phys(&txq->tx_pbl);
page_cnt = ecore_chain_get_page_cnt(&txq->tx_pbl);
- rc = qdev->ops->q_tx_start(edev, i, txq->queue_id,
- 0,
- fp->sb_info->igu_sb_id,
- TX_PI(tc),
- p_phys_table, page_cnt,
+
+ memset(&q_params, 0, sizeof(q_params));
+ q_params.queue_id = txq->queue_id;
+ q_params.vport_id = 0;
+ q_params.sb = fp->sb_info->igu_sb_id;
+ q_params.sb_idx = TX_PI(tc);
+
+ rc = qdev->ops->q_tx_start(edev, i, &q_params,
+ p_phys_table,
+ page_cnt, /* **pp_doorbell */
&txq->doorbell_addr);
if (rc) {
DP_ERR(edev, "Start txq %u failed %d\n",
--
1.8.3.1
^ permalink raw reply related
* [PATCH v2 16/22] qede/base: add support to initiate PF FLR
From: Rasesh Mody @ 2016-09-30 7:06 UTC (permalink / raw)
To: dev; +Cc: Dept-EngDPDKDev, Harish Patil
In-Reply-To: <1475219169-8774-1-git-send-email-rasesh.mody@qlogic.com>
From: Harish Patil <harish.patil@qlogic.com>
Add support to send PF FLR request to the management firmware to
bringup the device in clean slate. This cleanup is necessary
in some corner cases where the device would be left in a bad
state from its previous operations. The driver will send PF FLR
request before slowpath initialization.
Signed-off-by: Harish Patil <harish.patil@qlogic.com>
---
doc/guides/nics/qede.rst | 1 -
drivers/net/qede/base/ecore_dev.c | 15 +++++++++++----
drivers/net/qede/base/ecore_mcp.c | 9 +++++++++
drivers/net/qede/base/ecore_mcp.h | 11 +++++++++++
4 files changed, 31 insertions(+), 5 deletions(-)
diff --git a/doc/guides/nics/qede.rst b/doc/guides/nics/qede.rst
index 2a585e7..50e6f87 100644
--- a/doc/guides/nics/qede.rst
+++ b/doc/guides/nics/qede.rst
@@ -65,7 +65,6 @@ Non-supported Features
- SR-IOV PF
- Tunneling offloads
-- Reload of the PMD after a non-graceful termination
Supported QLogic Adapters
-------------------------
diff --git a/drivers/net/qede/base/ecore_dev.c b/drivers/net/qede/base/ecore_dev.c
index c530934..0f814f2 100644
--- a/drivers/net/qede/base/ecore_dev.c
+++ b/drivers/net/qede/base/ecore_dev.c
@@ -2954,13 +2954,14 @@ ecore_hw_prepare_single(struct ecore_hwfn *p_hwfn, void OSAL_IOMEM *p_regview,
void OSAL_IOMEM *p_doorbells,
struct ecore_hw_prepare_params *p_params)
{
+ struct ecore_dev *p_dev = p_hwfn->p_dev;
enum _ecore_status_t rc = ECORE_SUCCESS;
/* Split PCI bars evenly between hwfns */
p_hwfn->regview = p_regview;
p_hwfn->doorbells = p_doorbells;
- if (IS_VF(p_hwfn->p_dev))
+ if (IS_VF(p_dev))
return ecore_vf_hw_prepare(p_hwfn);
/* Validate that chip access is feasible */
@@ -2984,7 +2985,7 @@ ecore_hw_prepare_single(struct ecore_hwfn *p_hwfn, void OSAL_IOMEM *p_regview,
/* First hwfn learns basic information, e.g., number of hwfns */
if (!p_hwfn->my_id) {
- rc = ecore_get_dev_info(p_hwfn->p_dev);
+ rc = ecore_get_dev_info(p_dev);
if (rc != ECORE_SUCCESS)
goto err1;
}
@@ -2998,6 +2999,12 @@ ecore_hw_prepare_single(struct ecore_hwfn *p_hwfn, void OSAL_IOMEM *p_regview,
goto err1;
}
+ if (p_hwfn == ECORE_LEADING_HWFN(p_dev) && !p_dev->recov_in_prog) {
+ rc = ecore_mcp_initiate_pf_flr(p_hwfn, p_hwfn->p_main_ptt);
+ if (rc != ECORE_SUCCESS)
+ DP_NOTICE(p_hwfn, false, "Failed to initiate PF FLR\n");
+ }
+
/* Read the device configuration information from the HW and SHMEM */
rc = ecore_get_hw_info(p_hwfn, p_hwfn->p_main_ptt,
p_params->personality, p_params->drv_resc_alloc);
@@ -3013,7 +3020,7 @@ ecore_hw_prepare_single(struct ecore_hwfn *p_hwfn, void OSAL_IOMEM *p_regview,
goto err2;
}
#ifndef ASIC_ONLY
- if (CHIP_REV_IS_FPGA(p_hwfn->p_dev)) {
+ if (CHIP_REV_IS_FPGA(p_dev)) {
DP_NOTICE(p_hwfn, false,
"FPGA: workaround; Prevent DMAE parities\n");
ecore_wr(p_hwfn, p_hwfn->p_main_ptt, PCIE_REG_PRTY_MASK, 7);
@@ -3028,7 +3035,7 @@ ecore_hw_prepare_single(struct ecore_hwfn *p_hwfn, void OSAL_IOMEM *p_regview,
return rc;
err2:
if (IS_LEAD_HWFN(p_hwfn))
- ecore_iov_free_hw_info(p_hwfn->p_dev);
+ ecore_iov_free_hw_info(p_dev);
ecore_mcp_free(p_hwfn);
err1:
ecore_hw_hwfn_free(p_hwfn);
diff --git a/drivers/net/qede/base/ecore_mcp.c b/drivers/net/qede/base/ecore_mcp.c
index 500368e..2ff9715 100644
--- a/drivers/net/qede/base/ecore_mcp.c
+++ b/drivers/net/qede/base/ecore_mcp.c
@@ -2442,3 +2442,12 @@ enum _ecore_status_t ecore_mcp_get_resc_info(struct ecore_hwfn *p_hwfn,
return ECORE_SUCCESS;
}
+
+enum _ecore_status_t ecore_mcp_initiate_pf_flr(struct ecore_hwfn *p_hwfn,
+ struct ecore_ptt *p_ptt)
+{
+ u32 mcp_resp, mcp_param;
+
+ return ecore_mcp_cmd(p_hwfn, p_ptt, DRV_MSG_CODE_INITIATE_PF_FLR,
+ 0, &mcp_resp, &mcp_param);
+}
diff --git a/drivers/net/qede/base/ecore_mcp.h b/drivers/net/qede/base/ecore_mcp.h
index 2fc503a..1f84355 100644
--- a/drivers/net/qede/base/ecore_mcp.h
+++ b/drivers/net/qede/base/ecore_mcp.h
@@ -358,4 +358,15 @@ enum _ecore_status_t ecore_mcp_get_resc_info(struct ecore_hwfn *p_hwfn,
struct resource_info *p_resc_info,
u32 *p_mcp_resp, u32 *p_mcp_param);
+/**
+ * @brief - Initiates PF FLR
+ *
+ * @param p_hwfn
+ * @param p_ptt
+ *
+ * @param return ECORE_SUCCESS upon success.
+ */
+enum _ecore_status_t ecore_mcp_initiate_pf_flr(struct ecore_hwfn *p_hwfn,
+ struct ecore_ptt *p_ptt);
+
#endif /* __ECORE_MCP_H__ */
--
1.8.3.1
^ permalink raw reply related
* [PATCH v2 17/22] qede: skip slowpath polling for 100G VF device
From: Rasesh Mody @ 2016-09-30 7:06 UTC (permalink / raw)
To: dev; +Cc: Dept-EngDPDKDev, Harish Patil
In-Reply-To: <1475219169-8774-1-git-send-email-rasesh.mody@qlogic.com>
From: Harish Patil <harish.patil@qlogic.com>
There is no need to poll for slowpath events for VF
device since the ramrod responses are received over
PF-VF backchannel synchronously. So the fix is to
restrict the slowpath polling for PF device only.
Fixes 2af14ca ("net/qede: support 100G")
Signed-off-by: Harish Patil <harish.patil@qlogic.com>
---
drivers/net/qede/qede_ethdev.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index eb9b8aa..ae188ed 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -1369,7 +1369,7 @@ static int qede_common_dev_init(struct rte_eth_dev *eth_dev, bool is_vf)
* This is required since uio device uses only one MSI-x
* interrupt vector but we need one for each engine.
*/
- if (edev->num_hwfns > 1) {
+ if (edev->num_hwfns > 1 && IS_PF(edev)) {
rc = rte_eal_alarm_set(timer_period * US_PER_S,
qede_poll_sp_sb_cb,
(void *)eth_dev);
--
1.8.3.1
^ permalink raw reply related
* [PATCH v2 18/22] qede: fix driver version string
From: Rasesh Mody @ 2016-09-30 7:06 UTC (permalink / raw)
To: dev; +Cc: Dept-EngDPDKDev, Harish Patil
In-Reply-To: <1475219169-8774-1-git-send-email-rasesh.mody@qlogic.com>
From: Harish Patil <harish.patil@qlogic.com>
This patch fixes the base driver version display.
The driver version notation is:
<Base-Version_PMD-Version>
Fixes: 2ea6f76 ("qede: add core driver")
Signed-off-by: Harish Patil <harish.patil@qlogic.com>
---
drivers/net/qede/qede_ethdev.c | 43 +++++++++++++++++++++---------------------
drivers/net/qede/qede_ethdev.h | 17 ++++++++---------
drivers/net/qede/qede_if.h | 3 +--
drivers/net/qede/qede_main.c | 4 ++--
4 files changed, 32 insertions(+), 35 deletions(-)
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index ae188ed..73b3b54 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -8,6 +8,7 @@
#include "qede_ethdev.h"
#include <rte_alarm.h>
+#include <rte_version.h>
/* Globals */
static const struct qed_eth_ops *qed_ops;
@@ -188,31 +189,28 @@ static void qede_print_adapter_info(struct qede_dev *qdev)
{
struct ecore_dev *edev = &qdev->edev;
struct qed_dev_info *info = &qdev->dev_info.common;
- static char ver_str[QED_DRV_VER_STR_SIZE];
+ static char drv_ver[QEDE_PMD_DRV_VER_STR_SIZE];
+ static char ver_str[QEDE_PMD_DRV_VER_STR_SIZE];
DP_INFO(edev, "*********************************\n");
+ DP_INFO(edev, " DPDK version:%s\n", rte_version());
DP_INFO(edev, " Chip details : %s%d\n",
- ECORE_IS_BB(edev) ? "BB" : "AH",
- CHIP_REV_IS_A0(edev) ? 0 : 1);
-
- sprintf(ver_str, "%s %s_%d.%d.%d.%d", QEDE_PMD_VER_PREFIX,
- edev->ver_str, QEDE_PMD_VERSION_MAJOR, QEDE_PMD_VERSION_MINOR,
- QEDE_PMD_VERSION_REVISION, QEDE_PMD_VERSION_PATCH);
- strcpy(qdev->drv_ver, ver_str);
- DP_INFO(edev, " Driver version : %s\n", ver_str);
-
- sprintf(ver_str, "%d.%d.%d.%d", info->fw_major, info->fw_minor,
- info->fw_rev, info->fw_eng);
+ ECORE_IS_BB(edev) ? "BB" : "AH",
+ CHIP_REV_IS_A0(edev) ? 0 : 1);
+ snprintf(ver_str, QEDE_PMD_DRV_VER_STR_SIZE, "%d.%d.%d.%d",
+ info->fw_major, info->fw_minor, info->fw_rev, info->fw_eng);
+ snprintf(drv_ver, QEDE_PMD_DRV_VER_STR_SIZE, "%s_%s",
+ ver_str, QEDE_PMD_VERSION);
+ DP_INFO(edev, " Driver version : %s\n", drv_ver);
DP_INFO(edev, " Firmware version : %s\n", ver_str);
- sprintf(ver_str, "%d.%d.%d.%d",
+ snprintf(ver_str, MCP_DRV_VER_STR_SIZE,
+ "%d.%d.%d.%d",
(info->mfw_rev >> 24) & 0xff,
(info->mfw_rev >> 16) & 0xff,
(info->mfw_rev >> 8) & 0xff, (info->mfw_rev) & 0xff);
- DP_INFO(edev, " Management firmware version : %s\n", ver_str);
-
+ DP_INFO(edev, " Management Firmware version : %s\n", ver_str);
DP_INFO(edev, " Firmware file : %s\n", fw_file);
-
DP_INFO(edev, "*********************************\n");
}
@@ -1359,11 +1357,12 @@ static int qede_common_dev_init(struct rte_eth_dev *eth_dev, bool is_vf)
/* Start the Slowpath-process */
memset(¶ms, 0, sizeof(struct qed_slowpath_params));
params.int_mode = ECORE_INT_MODE_MSIX;
- params.drv_major = QEDE_MAJOR_VERSION;
- params.drv_minor = QEDE_MINOR_VERSION;
- params.drv_rev = QEDE_REVISION_VERSION;
- params.drv_eng = QEDE_ENGINEERING_VERSION;
- strncpy((char *)params.name, "qede LAN", QED_DRV_VER_STR_SIZE);
+ params.drv_major = QEDE_PMD_VERSION_MAJOR;
+ params.drv_minor = QEDE_PMD_VERSION_MINOR;
+ params.drv_rev = QEDE_PMD_VERSION_REVISION;
+ params.drv_eng = QEDE_PMD_VERSION_PATCH;
+ strncpy((char *)params.name, QEDE_PMD_VER_PREFIX,
+ QEDE_PMD_DRV_VER_STR_SIZE);
/* For CMT mode device do periodic polling for slowpath events.
* This is required since uio device uses only one MSI-x
@@ -1400,7 +1399,7 @@ static int qede_common_dev_init(struct rte_eth_dev *eth_dev, bool is_vf)
qede_alloc_etherdev(adapter, &dev_info);
- adapter->ops->common->set_id(edev, edev->name, QEDE_DRV_MODULE_VERSION);
+ adapter->ops->common->set_id(edev, edev->name, QEDE_PMD_VERSION);
if (!is_vf)
adapter->dev_info.num_mac_addrs =
diff --git a/drivers/net/qede/qede_ethdev.h b/drivers/net/qede/qede_ethdev.h
index 5838f33..c3b87e8 100644
--- a/drivers/net/qede/qede_ethdev.h
+++ b/drivers/net/qede/qede_ethdev.h
@@ -46,15 +46,14 @@
#define QEDE_PMD_VERSION_REVISION 0
#define QEDE_PMD_VERSION_PATCH 1
-#define QEDE_MAJOR_VERSION 8
-#define QEDE_MINOR_VERSION 7
-#define QEDE_REVISION_VERSION 9
-#define QEDE_ENGINEERING_VERSION 0
+#define QEDE_PMD_VERSION qede_stringify(QEDE_PMD_VERSION_MAJOR) "." \
+ qede_stringify(QEDE_PMD_VERSION_MINOR) "." \
+ qede_stringify(QEDE_PMD_VERSION_REVISION) "." \
+ qede_stringify(QEDE_PMD_VERSION_PATCH)
+
+#define QEDE_PMD_DRV_VER_STR_SIZE NAME_SIZE
+#define QEDE_PMD_VER_PREFIX "QEDE PMD"
-#define QEDE_DRV_MODULE_VERSION qede_stringify(QEDE_MAJOR_VERSION) "." \
- qede_stringify(QEDE_MINOR_VERSION) "." \
- qede_stringify(QEDE_REVISION_VERSION) "." \
- qede_stringify(QEDE_ENGINEERING_VERSION)
#define QEDE_RSS_INDIR_INITED (1 << 0)
#define QEDE_RSS_KEY_INITED (1 << 1)
@@ -144,7 +143,7 @@ struct qede_dev {
bool accept_any_vlan;
struct ether_addr primary_mac;
bool handle_hw_err;
- char drv_ver[QED_DRV_VER_STR_SIZE];
+ char drv_ver[QEDE_PMD_DRV_VER_STR_SIZE];
};
/* Static functions */
diff --git a/drivers/net/qede/qede_if.h b/drivers/net/qede/qede_if.h
index 935eed8..2d38b1b 100644
--- a/drivers/net/qede/qede_if.h
+++ b/drivers/net/qede/qede_if.h
@@ -76,14 +76,13 @@ struct qed_link_output {
uint32_t pause_config;
};
-#define QED_DRV_VER_STR_SIZE 80
struct qed_slowpath_params {
uint32_t int_mode;
uint8_t drv_major;
uint8_t drv_minor;
uint8_t drv_rev;
uint8_t drv_eng;
- uint8_t name[QED_DRV_VER_STR_SIZE];
+ uint8_t name[NAME_SIZE];
};
#define ILT_PAGE_SIZE_TCFC 0x8000 /* 32KB */
diff --git a/drivers/net/qede/qede_main.c b/drivers/net/qede/qede_main.c
index c83893d..0483116 100644
--- a/drivers/net/qede/qede_main.c
+++ b/drivers/net/qede/qede_main.c
@@ -426,7 +426,7 @@ qed_fill_eth_dev_info(struct ecore_dev *edev, struct qed_dev_eth_info *info)
static void
qed_set_id(struct ecore_dev *edev, char name[NAME_SIZE],
- const char ver_str[VER_SIZE])
+ const char ver_str[NAME_SIZE])
{
int i;
@@ -434,7 +434,7 @@ qed_set_id(struct ecore_dev *edev, char name[NAME_SIZE],
for_each_hwfn(edev, i) {
snprintf(edev->hwfns[i].name, NAME_SIZE, "%s-%d", name, i);
}
- rte_memcpy(edev->ver_str, ver_str, VER_SIZE);
+ memcpy(edev->ver_str, ver_str, NAME_SIZE);
edev->drv_type = DRV_ID_DRV_TYPE_LINUX;
}
--
1.8.3.1
^ permalink raw reply related
* [PATCH v2 19/22] qede: add support for queue statistics
From: Rasesh Mody @ 2016-09-30 7:06 UTC (permalink / raw)
To: dev; +Cc: Dept-EngDPDKDev, Rasesh Mody
In-Reply-To: <1475219169-8774-1-git-send-email-rasesh.mody@qlogic.com>
This patch adds support for pulling per queue statistics.
Signed-off-by: Rasesh Mody <rasesh.mody@qlogic.com>
---
drivers/net/qede/qede_ethdev.c | 98 +++++++++++++++++++++++++++++++++++++-----
drivers/net/qede/qede_ethdev.h | 3 ++
drivers/net/qede/qede_rxtx.c | 23 +++++-----
drivers/net/qede/qede_rxtx.h | 4 +-
4 files changed, 107 insertions(+), 21 deletions(-)
diff --git a/drivers/net/qede/qede_ethdev.c b/drivers/net/qede/qede_ethdev.c
index 73b3b54..d1b5edc 100644
--- a/drivers/net/qede/qede_ethdev.c
+++ b/drivers/net/qede/qede_ethdev.c
@@ -160,6 +160,15 @@ static const struct rte_qede_xstats_name_off qede_xstats_strings[] = {
offsetof(struct ecore_eth_stats, tpa_coalesced_bytes)},
};
+static const struct rte_qede_xstats_name_off qede_rxq_xstats_strings[] = {
+ {"rx_q_segments",
+ offsetof(struct qede_rx_queue, rx_segs)},
+ {"rx_q_hw_errors",
+ offsetof(struct qede_rx_queue, rx_hw_errors)},
+ {"rx_q_allocation_errors",
+ offsetof(struct qede_rx_queue, rx_alloc_errors)}
+};
+
static void qede_interrupt_action(struct ecore_hwfn *p_hwfn)
{
ecore_int_sp_dpc((osal_int_ptr_t)(p_hwfn));
@@ -825,6 +834,8 @@ qede_get_stats(struct rte_eth_dev *eth_dev, struct rte_eth_stats *eth_stats)
struct qede_dev *qdev = eth_dev->data->dev_private;
struct ecore_dev *edev = &qdev->edev;
struct ecore_eth_stats stats;
+ unsigned int i = 0, j = 0, qid;
+ struct qede_tx_queue *txq;
qdev->ops->get_vport_stats(edev, &stats);
@@ -855,20 +866,73 @@ qede_get_stats(struct rte_eth_dev *eth_dev, struct rte_eth_stats *eth_stats)
stats.tx_mcast_bytes + stats.tx_bcast_bytes;
eth_stats->oerrors = stats.tx_err_drop_pkts;
+
+ /* Queue stats */
+ for (qid = 0; qid < QEDE_QUEUE_CNT(qdev); qid++) {
+ if (qdev->fp_array[qid].type & QEDE_FASTPATH_RX) {
+ eth_stats->q_ipackets[i] =
+ *(uint64_t *)(
+ ((char *)(qdev->fp_array[(qid)].rxq)) +
+ offsetof(struct qede_rx_queue,
+ rcv_pkts));
+ eth_stats->q_errors[i] =
+ *(uint64_t *)(
+ ((char *)(qdev->fp_array[(qid)].rxq)) +
+ offsetof(struct qede_rx_queue,
+ rx_hw_errors)) +
+ *(uint64_t *)(
+ ((char *)(qdev->fp_array[(qid)].rxq)) +
+ offsetof(struct qede_rx_queue,
+ rx_alloc_errors));
+ i++;
+ }
+
+ if (qdev->fp_array[qid].type & QEDE_FASTPATH_TX) {
+ txq = qdev->fp_array[(qid)].txqs[0];
+ eth_stats->q_opackets[j] =
+ *((uint64_t *)(uintptr_t)
+ (((uint64_t)(uintptr_t)(txq)) +
+ offsetof(struct qede_tx_queue,
+ xmit_pkts)));
+ j++;
+ }
+ }
+}
+
+static unsigned
+qede_get_xstats_count(struct qede_dev *qdev) {
+ return RTE_DIM(qede_xstats_strings) +
+ (RTE_DIM(qede_rxq_xstats_strings) * QEDE_RSS_COUNT(qdev));
}
static int
qede_get_xstats_names(__rte_unused struct rte_eth_dev *dev,
struct rte_eth_xstat_name *xstats_names, unsigned limit)
{
- unsigned int i, stat_cnt = RTE_DIM(qede_xstats_strings);
+ struct qede_dev *qdev = dev->data->dev_private;
+ const unsigned int stat_cnt = qede_get_xstats_count(qdev);
+ unsigned int i, qid, stat_idx = 0;
- if (xstats_names != NULL)
- for (i = 0; i < stat_cnt; i++)
- snprintf(xstats_names[i].name,
- sizeof(xstats_names[i].name),
+ if (xstats_names != NULL) {
+ for (i = 0; i < RTE_DIM(qede_xstats_strings); i++) {
+ snprintf(xstats_names[stat_idx].name,
+ sizeof(xstats_names[stat_idx].name),
"%s",
qede_xstats_strings[i].name);
+ stat_idx++;
+ }
+
+ for (qid = 0; qid < QEDE_RSS_COUNT(qdev); qid++) {
+ for (i = 0; i < RTE_DIM(qede_rxq_xstats_strings); i++) {
+ snprintf(xstats_names[stat_idx].name,
+ sizeof(xstats_names[stat_idx].name),
+ "%.4s%d%s",
+ qede_rxq_xstats_strings[i].name, qid,
+ qede_rxq_xstats_strings[i].name + 4);
+ stat_idx++;
+ }
+ }
+ }
return stat_cnt;
}
@@ -880,18 +944,32 @@ qede_get_xstats(struct rte_eth_dev *dev, struct rte_eth_xstat *xstats,
struct qede_dev *qdev = dev->data->dev_private;
struct ecore_dev *edev = &qdev->edev;
struct ecore_eth_stats stats;
- unsigned int num = RTE_DIM(qede_xstats_strings);
+ const unsigned int num = qede_get_xstats_count(qdev);
+ unsigned int i, qid, stat_idx = 0;
if (n < num)
return num;
qdev->ops->get_vport_stats(edev, &stats);
- for (num = 0; num < n; num++)
- xstats[num].value = *(u64 *)(((char *)&stats) +
- qede_xstats_strings[num].offset);
+ for (i = 0; i < RTE_DIM(qede_xstats_strings); i++) {
+ xstats[stat_idx].value = *(uint64_t *)(((char *)&stats) +
+ qede_xstats_strings[i].offset);
+ stat_idx++;
+ }
+
+ for (qid = 0; qid < QEDE_QUEUE_CNT(qdev); qid++) {
+ if (qdev->fp_array[qid].type & QEDE_FASTPATH_RX) {
+ for (i = 0; i < RTE_DIM(qede_rxq_xstats_strings); i++) {
+ xstats[stat_idx].value = *(uint64_t *)(
+ ((char *)(qdev->fp_array[(qid)].rxq)) +
+ qede_rxq_xstats_strings[i].offset);
+ stat_idx++;
+ }
+ }
+ }
- return num;
+ return stat_idx;
}
static void
diff --git a/drivers/net/qede/qede_ethdev.h b/drivers/net/qede/qede_ethdev.h
index c3b87e8..dfbbcf4 100644
--- a/drivers/net/qede/qede_ethdev.h
+++ b/drivers/net/qede/qede_ethdev.h
@@ -68,6 +68,9 @@
#define QEDE_TSS_COUNT(qdev) (((qdev)->num_queues - (qdev)->fp_num_rx) * \
(qdev)->num_tc)
+#define QEDE_FASTPATH_TX (1 << 0)
+#define QEDE_FASTPATH_RX (1 << 1)
+
#define QEDE_DUPLEX_FULL 1
#define QEDE_DUPLEX_HALF 2
#define QEDE_DUPLEX_UNKNOWN 0xff
diff --git a/drivers/net/qede/qede_rxtx.c b/drivers/net/qede/qede_rxtx.c
index d903a84..9d7e704 100644
--- a/drivers/net/qede/qede_rxtx.c
+++ b/drivers/net/qede/qede_rxtx.c
@@ -10,9 +10,6 @@
static bool gro_disable = 1; /* mod_param */
-#define QEDE_FASTPATH_TX (1 << 0)
-#define QEDE_FASTPATH_RX (1 << 1)
-
static inline int qede_alloc_rx_buffer(struct qede_rx_queue *rxq)
{
struct rte_mbuf *new_mb = NULL;
@@ -814,7 +811,7 @@ static inline uint32_t qede_rx_cqe_to_pkt_type(uint16_t flags)
}
int qede_process_sg_pkts(void *p_rxq, struct rte_mbuf *rx_mb,
- int num_frags, uint16_t pkt_len)
+ int num_segs, uint16_t pkt_len)
{
struct qede_rx_queue *rxq = p_rxq;
struct qede_dev *qdev = rxq->qdev;
@@ -825,13 +822,13 @@ int qede_process_sg_pkts(void *p_rxq, struct rte_mbuf *rx_mb,
register struct rte_mbuf *seg2 = NULL;
seg1 = rx_mb;
- while (num_frags) {
+ while (num_segs) {
cur_size = pkt_len > rxq->rx_buf_size ?
rxq->rx_buf_size : pkt_len;
if (!cur_size) {
PMD_RX_LOG(DEBUG, rxq,
"SG packet, len and num BD mismatch\n");
- qede_recycle_rx_bd_ring(rxq, qdev, num_frags);
+ qede_recycle_rx_bd_ring(rxq, qdev, num_segs);
return -EINVAL;
}
@@ -852,7 +849,8 @@ int qede_process_sg_pkts(void *p_rxq, struct rte_mbuf *rx_mb,
seg2->data_len = cur_size;
seg1->next = seg2;
seg1 = seg1->next;
- num_frags--;
+ num_segs--;
+ rxq->rx_segs++;
continue;
}
seg1 = NULL;
@@ -880,7 +878,7 @@ qede_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
register struct rte_mbuf *seg1 = NULL;
enum eth_rx_cqe_type cqe_type;
uint16_t len, pad, preload_idx, pkt_len, parse_flag;
- uint8_t csum_flag, num_frags;
+ uint8_t csum_flag, num_segs;
enum rss_hash_type htype;
int ret;
@@ -954,11 +952,13 @@ qede_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
if (fp_cqe->bd_num > 1) {
pkt_len = rte_le_to_cpu_16(fp_cqe->pkt_len);
- num_frags = fp_cqe->bd_num - 1;
+ num_segs = fp_cqe->bd_num - 1;
+
+ rxq->rx_segs++;
pkt_len -= len;
seg1 = rx_mb;
- ret = qede_process_sg_pkts(p_rxq, seg1, num_frags,
+ ret = qede_process_sg_pkts(p_rxq, seg1, num_segs,
pkt_len);
if (ret != ECORE_SUCCESS) {
qede_recycle_rx_bd_ring(rxq, qdev,
@@ -1021,6 +1021,8 @@ next_cqe:
qede_update_rx_prod(qdev, rxq);
+ rxq->rcv_pkts += rx_pkt;
+
PMD_RX_LOG(DEBUG, rxq, "rx_pkts=%u core=%d\n", rx_pkt, rte_lcore_id());
return rx_pkt;
@@ -1213,6 +1215,7 @@ qede_xmit_pkts(void *p_txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
bd_prod =
rte_cpu_to_le_16(ecore_chain_get_prod_idx(&txq->tx_pbl));
nb_pkt_sent++;
+ txq->xmit_pkts++;
}
/* Write value of prod idx into bd_prod */
diff --git a/drivers/net/qede/qede_rxtx.h b/drivers/net/qede/qede_rxtx.h
index da47b21..ed9a529 100644
--- a/drivers/net/qede/qede_rxtx.h
+++ b/drivers/net/qede/qede_rxtx.h
@@ -99,6 +99,8 @@ struct qede_rx_queue {
uint16_t queue_id;
uint16_t port_id;
uint16_t rx_buf_size;
+ uint64_t rcv_pkts;
+ uint64_t rx_segs;
uint64_t rx_hw_errors;
uint64_t rx_alloc_errors;
struct qede_dev *qdev;
@@ -130,7 +132,7 @@ struct qede_tx_queue {
void OSAL_IOMEM *doorbell_addr;
volatile union db_prod tx_db;
uint16_t port_id;
- uint64_t txq_counter;
+ uint64_t xmit_pkts;
struct qede_dev *qdev;
};
--
1.8.3.1
^ permalink raw reply related
* [PATCH v2 20/22] qede: remove external dependency and enable by default
From: Rasesh Mody @ 2016-09-30 7:06 UTC (permalink / raw)
To: dev; +Cc: Dept-EngDPDKDev, Rasesh Mody
In-Reply-To: <1475219169-8774-1-git-send-email-rasesh.mody@qlogic.com>
The qede driver doesn't depend on libz anymore. Hence remove the LDLIBS
entry form the Makefile.
This patch enables the qede PMD by default.
Fixes: 6adac0bf ("qede: add missing external dependency and disable by default")
Signed-off-by: Rasesh Mody <rasesh.mody@qlogic.com>
---
config/common_base | 2 +-
doc/guides/nics/qede.rst | 5 +----
drivers/net/qede/Makefile | 2 --
drivers/net/qede/base/ecore.h | 2 +-
drivers/net/qede/qede_main.c | 2 +-
mk/rte.app.mk | 2 +-
6 files changed, 5 insertions(+), 10 deletions(-)
diff --git a/config/common_base b/config/common_base
index 7830535..2b03d61 100644
--- a/config/common_base
+++ b/config/common_base
@@ -317,7 +317,7 @@ CONFIG_RTE_LIBRTE_BOND_DEBUG_ALB_L1=n
# QLogic 25G/40G/100G PMD
#
-CONFIG_RTE_LIBRTE_QEDE_PMD=n
+CONFIG_RTE_LIBRTE_QEDE_PMD=y
CONFIG_RTE_LIBRTE_QEDE_DEBUG_INIT=n
CONFIG_RTE_LIBRTE_QEDE_DEBUG_INFO=n
CONFIG_RTE_LIBRTE_QEDE_DEBUG_DRIVER=n
diff --git a/doc/guides/nics/qede.rst b/doc/guides/nics/qede.rst
index 50e6f87..d32fba2 100644
--- a/doc/guides/nics/qede.rst
+++ b/doc/guides/nics/qede.rst
@@ -77,14 +77,11 @@ Prerequisites
- Requires firmware version **8.10.x.** and management firmware
version **8.10.x or higher**. Firmware may be available
inbox in certain newer Linux distros under the standard directory
- ``E.g. /lib/firmware/qed/qed_init_values_zipped-8.10.9.0.bin``
+ ``E.g. /lib/firmware/qed/qed_init_values-8.10.9.0.bin``
- If the required firmware files are not available then visit
`QLogic Driver Download Center <http://driverdownloads.qlogic.com>`_.
-- This driver relies on external zlib library (-lz) for uncompressing
- the firmware file.
-
Performance note
~~~~~~~~~~~~~~~~
diff --git a/drivers/net/qede/Makefile b/drivers/net/qede/Makefile
index 7965a83..39751e4 100644
--- a/drivers/net/qede/Makefile
+++ b/drivers/net/qede/Makefile
@@ -14,8 +14,6 @@ LIB = librte_pmd_qede.a
CFLAGS += -O3
CFLAGS += $(WERROR_FLAGS)
-LDLIBS += -lz
-
EXPORT_MAP := rte_pmd_qede_version.map
LIBABIVER := 1
diff --git a/drivers/net/qede/base/ecore.h b/drivers/net/qede/base/ecore.h
index 4359343..f99a045 100644
--- a/drivers/net/qede/base/ecore.h
+++ b/drivers/net/qede/base/ecore.h
@@ -15,7 +15,7 @@
#include <unistd.h>
#define CONFIG_ECORE_BINARY_FW
-#define CONFIG_ECORE_ZIPPED_FW
+#undef CONFIG_ECORE_ZIPPED_FW
#ifdef CONFIG_ECORE_ZIPPED_FW
#include <zlib.h>
diff --git a/drivers/net/qede/qede_main.c b/drivers/net/qede/qede_main.c
index 0483116..a0a146a 100644
--- a/drivers/net/qede/qede_main.c
+++ b/drivers/net/qede/qede_main.c
@@ -21,7 +21,7 @@ static uint8_t npar_tx_switching = 1;
char fw_file[PATH_MAX];
const char *QEDE_DEFAULT_FIRMWARE =
- "/lib/firmware/qed/qed_init_values_zipped-8.10.9.0.bin";
+ "/lib/firmware/qed/qed_init_values-8.10.9.0.bin";
static void
qed_update_pf_params(struct ecore_dev *edev, struct ecore_pf_params *params)
diff --git a/mk/rte.app.mk b/mk/rte.app.mk
index 1a0095b..9b12da3 100644
--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk
@@ -120,7 +120,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD) += -lrte_pmd_mpipe -lgxio
_LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD) += -lrte_pmd_nfp -lm
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += -lrte_pmd_null
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += -lrte_pmd_pcap -lpcap
-_LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD) += -lrte_pmd_qede -lz
+_LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD) += -lrte_pmd_qede
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING) += -lrte_pmd_ring
_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += -lrte_pmd_szedata2 -lsze2
_LDLIBS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += -lrte_pmd_thunderx_nicvf -lm
--
1.8.3.1
^ permalink raw reply related
* [PATCH v2 21/22] doc: update qede pmd documentation
From: Rasesh Mody @ 2016-09-30 7:06 UTC (permalink / raw)
To: dev; +Cc: Dept-EngDPDKDev, Rasesh Mody
In-Reply-To: <1475219169-8774-1-git-send-email-rasesh.mody@qlogic.com>
Signed-off-by: Rasesh Mody <rasesh.mody@qlogic.com>
---
doc/guides/nics/qede.rst | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/doc/guides/nics/qede.rst b/doc/guides/nics/qede.rst
index d32fba2..105ae37 100644
--- a/doc/guides/nics/qede.rst
+++ b/doc/guides/nics/qede.rst
@@ -65,6 +65,8 @@ Non-supported Features
- SR-IOV PF
- Tunneling offloads
+- LRO/TSO
+- NPAR
Supported QLogic Adapters
-------------------------
@@ -239,7 +241,7 @@ SR-IOV: Prerequisites and Sample Application Notes
This section provides instructions to configure SR-IOV with Linux OS.
-**Note**: librte_pmd_qede will be used to bind to SR-IOV VF device and Linux native kernel driver (QEDE) will function as SR-IOV PF driver.
+**Note**: librte_pmd_qede will be used to bind to SR-IOV VF device and Linux native kernel driver (QEDE) will function as SR-IOV PF driver. Requires PF driver to be 8.10.x.x or higher.
#. Verify SR-IOV and ARI capability is enabled on the adapter using ``lspci``:
--
1.8.3.1
^ permalink raw reply related
* [PATCH v2 22/22] qede: update driver version
From: Rasesh Mody @ 2016-09-30 7:06 UTC (permalink / raw)
To: dev; +Cc: Dept-EngDPDKDev, Rasesh Mody
In-Reply-To: <1475219169-8774-1-git-send-email-rasesh.mody@qlogic.com>
This patch updates the qede pmd version to 1.2.2.1.
Signed-off-by: Rasesh Mody <rasesh.mody@qlogic.com>
---
drivers/net/qede/qede_ethdev.h | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/qede/qede_ethdev.h b/drivers/net/qede/qede_ethdev.h
index dfbbcf4..91d3c90 100644
--- a/drivers/net/qede/qede_ethdev.h
+++ b/drivers/net/qede/qede_ethdev.h
@@ -42,8 +42,8 @@
/* Driver versions */
#define QEDE_PMD_VER_PREFIX "QEDE PMD"
#define QEDE_PMD_VERSION_MAJOR 1
-#define QEDE_PMD_VERSION_MINOR 1
-#define QEDE_PMD_VERSION_REVISION 0
+#define QEDE_PMD_VERSION_MINOR 2
+#define QEDE_PMD_VERSION_REVISION 2
#define QEDE_PMD_VERSION_PATCH 1
#define QEDE_PMD_VERSION qede_stringify(QEDE_PMD_VERSION_MAJOR) "." \
--
1.8.3.1
^ permalink raw reply related
* [PATCH v4 0/4] Cuckoo hash enhancements
From: Pablo de Lara @ 2016-09-30 7:38 UTC (permalink / raw)
To: dev; +Cc: bruce.richardson, Pablo de Lara
In-Reply-To: <1473190397-120741-1-git-send-email-pablo.de.lara.guarch@intel.com>
This patchset improves lookup performance on the current hash library
by changing the existing lookup bulk pipeline, with an improved pipeline,
based on a loop-and-jump model, instead of the current 4-stage 2-entry pipeline.
Also, x86 vectorized intrinsics are used to improve performance when comparing signatures.
First patch reorganizes the order of the hash structure.
The structure takes more than one 64-byte cache line, but not all
the fields are used in the lookup operation (the most common operation).
Therefore, all these fields have been moved to the first part of the structure,
so they all fit in one cache line, improving slightly the performance in some
scenarios.
Second patch modifies the order of the bucket structure.
Currently, the buckets store all the signatures together (current and alternative).
In order to be able to perform a vectorized signature comparison,
all current signatures have to be together, so the order of the bucket has been changed,
having separated all the current signatures from the alternative signatures.
Third patch introduces x86 vectorized intrinsics.
When performing a lookup bulk operation, all current signatures in a bucket
are compared against the signature of the key being looked up.
Now that they all are together, a vectorized comparison can be performed,
which takes less instructions to be carried out.
In case of having a machine with AVX2, number of entries per bucket are
increased from 4 to 8, as AVX2 allows comparing two 256-bit values, with 8x32-bit integers,
which are the 8 signatures on the bucket.
Fourth (and last) patch modifies the current pipeline of the lookup bulk function.
The new pipeline is based on a loop-and-jump model. The two key improvements are:
- Better prefetching: in this case, first 4 keys to be looked up are prefetched,
and after that, the rest of the keys are prefetched at the time the calculation
of the signatures are being performed. This gives more time for the CPU to
prefetch the data requesting before actually need it, which result in less
cache misses and therefore, higher throughput.
- Lower performance penalty when using fallback: the lookup bulk algorithm
assumes that most times there will not be a collision in a bucket, but it might
happen that two or more signatures are equal, which means that more than one
key comparison might be necessary. In that case, only the key of the first hit is prefetched,
like in the current implementation. The difference now is that if this comparison
results in a miss, the information of the other keys to be compared has been stored,
unlike the current implementation, which needs to perform an entire simple lookup again.
Changes in v4:
- Reordered hash structure, so alt signature is at the start
of the next cache line, and explain in the commit message
why it has been moved
- Reordered hash structure, so name field is on top of the structure,
leaving all the fields used in lookup in the next cache line
(instead of the first cache line)
Changes in v3:
- Corrected the cover letter (wrong number of patches)
Changes in v2:
- Increased entries per bucket from 4 to 8 for all cases,
so it is not architecture dependent any longer.
- Replaced compile-time signature comparison function election
with run-time election, so best optimization available
will be used from a single binary.
- Reordered the hash structure, so all the fields used by lookup
are in the same cache line (first).
Byron Marohn (3):
hash: reorganize bucket structure
hash: add vectorized comparison
hash: modify lookup bulk pipeline
Pablo de Lara (1):
hash: reorder hash structure
lib/librte_hash/rte_cuckoo_hash.c | 455 ++++++++++++++--------------------
lib/librte_hash/rte_cuckoo_hash.h | 56 +++--
lib/librte_hash/rte_cuckoo_hash_x86.h | 20 +-
3 files changed, 228 insertions(+), 303 deletions(-)
--
2.7.4
^ permalink raw reply
* [PATCH v4 1/4] hash: reorder hash structure
From: Pablo de Lara @ 2016-09-30 7:38 UTC (permalink / raw)
To: dev; +Cc: bruce.richardson, Pablo de Lara
In-Reply-To: <1475221136-213246-1-git-send-email-pablo.de.lara.guarch@intel.com>
In order to optimize lookup performance, hash structure
is reordered, so all fields used for lookup will be
in the first cache line.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
lib/librte_hash/rte_cuckoo_hash.h | 24 ++++++++++++++----------
1 file changed, 14 insertions(+), 10 deletions(-)
diff --git a/lib/librte_hash/rte_cuckoo_hash.h b/lib/librte_hash/rte_cuckoo_hash.h
index e290dab..5a32ea6 100644
--- a/lib/librte_hash/rte_cuckoo_hash.h
+++ b/lib/librte_hash/rte_cuckoo_hash.h
@@ -185,7 +185,20 @@ struct rte_hash {
char name[RTE_HASH_NAMESIZE]; /**< Name of the hash. */
uint32_t entries; /**< Total table entries. */
uint32_t num_buckets; /**< Number of buckets in table. */
- uint32_t key_len; /**< Length of hash key. */
+
+ struct rte_ring *free_slots; /**< Ring that stores all indexes
+ of the free slots in the key table */
+ uint8_t hw_trans_mem_support; /**< Hardware transactional
+ memory support */
+ struct lcore_cache *local_free_slots;
+ /**< Local cache per lcore, storing some indexes of the free slots */
+ enum add_key_case add_key; /**< Multi-writer hash add behavior */
+
+ rte_spinlock_t *multiwriter_lock; /**< Multi-writer spinlock for w/o TM */
+
+ /* Fields used in lookup */
+ uint32_t key_len __rte_cache_aligned;
+ /**< Length of hash key. */
rte_hash_function hash_func; /**< Function used to calculate hash. */
uint32_t hash_func_init_val; /**< Init value used by hash_func. */
rte_hash_cmp_eq_t rte_hash_custom_cmp_eq;
@@ -196,19 +209,10 @@ struct rte_hash {
from hash signature. */
uint32_t key_entry_size; /**< Size of each key entry. */
- struct rte_ring *free_slots; /**< Ring that stores all indexes
- of the free slots in the key table */
void *key_store; /**< Table storing all keys and data */
struct rte_hash_bucket *buckets; /**< Table with buckets storing all the
hash values and key indexes
to the key table*/
- uint8_t hw_trans_mem_support; /**< Hardware transactional
- memory support */
- struct lcore_cache *local_free_slots;
- /**< Local cache per lcore, storing some indexes of the free slots */
- enum add_key_case add_key; /**< Multi-writer hash add behavior */
-
- rte_spinlock_t *multiwriter_lock; /**< Multi-writer spinlock for w/o TM */
} __rte_cache_aligned;
struct queue_node {
--
2.7.4
^ permalink raw reply related
* [PATCH v4 2/4] hash: reorganize bucket structure
From: Pablo de Lara @ 2016-09-30 7:38 UTC (permalink / raw)
To: dev; +Cc: bruce.richardson, Byron Marohn, Saikrishna Edupuganti
In-Reply-To: <1475221136-213246-1-git-send-email-pablo.de.lara.guarch@intel.com>
From: Byron Marohn <byron.marohn@intel.com>
Move current signatures of all entries together in the bucket
and same with all alternative signatures, instead of having
current and alternative signatures together per entry in the bucket.
This will be benefitial in the next commits, where a vectorized
comparison will be performed, achieving better performance.
The alternative signatures have been moved away from
the current signatures, to make the key indices be consecutive
to the current signatures, as these two fields are used by lookup,
so they are in the same cache line.
Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
---
lib/librte_hash/rte_cuckoo_hash.c | 43 ++++++++++++++++++-----------------
lib/librte_hash/rte_cuckoo_hash.h | 17 ++++----------
lib/librte_hash/rte_cuckoo_hash_x86.h | 20 ++++++++--------
3 files changed, 37 insertions(+), 43 deletions(-)
diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c
index 4de4422..a7ee2b9 100644
--- a/lib/librte_hash/rte_cuckoo_hash.c
+++ b/lib/librte_hash/rte_cuckoo_hash.c
@@ -421,7 +421,7 @@ make_space_bucket(const struct rte_hash *h, struct rte_hash_bucket *bkt)
*/
for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
/* Search for space in alternative locations */
- next_bucket_idx = bkt->signatures[i].alt & h->bucket_bitmask;
+ next_bucket_idx = bkt->sig_alt[i] & h->bucket_bitmask;
next_bkt[i] = &h->buckets[next_bucket_idx];
for (j = 0; j < RTE_HASH_BUCKET_ENTRIES; j++) {
if (next_bkt[i]->key_idx[j] == EMPTY_SLOT)
@@ -434,8 +434,8 @@ make_space_bucket(const struct rte_hash *h, struct rte_hash_bucket *bkt)
/* Alternative location has spare room (end of recursive function) */
if (i != RTE_HASH_BUCKET_ENTRIES) {
- next_bkt[i]->signatures[j].alt = bkt->signatures[i].current;
- next_bkt[i]->signatures[j].current = bkt->signatures[i].alt;
+ next_bkt[i]->sig_alt[j] = bkt->sig_current[i];
+ next_bkt[i]->sig_current[j] = bkt->sig_alt[i];
next_bkt[i]->key_idx[j] = bkt->key_idx[i];
return i;
}
@@ -461,8 +461,8 @@ make_space_bucket(const struct rte_hash *h, struct rte_hash_bucket *bkt)
*/
bkt->flag[i] = 0;
if (ret >= 0) {
- next_bkt[i]->signatures[ret].alt = bkt->signatures[i].current;
- next_bkt[i]->signatures[ret].current = bkt->signatures[i].alt;
+ next_bkt[i]->sig_alt[ret] = bkt->sig_current[i];
+ next_bkt[i]->sig_current[ret] = bkt->sig_alt[i];
next_bkt[i]->key_idx[ret] = bkt->key_idx[i];
return i;
} else
@@ -544,8 +544,8 @@ __rte_hash_add_key_with_hash(const struct rte_hash *h, const void *key,
/* Check if key is already inserted in primary location */
for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
- if (prim_bkt->signatures[i].current == sig &&
- prim_bkt->signatures[i].alt == alt_hash) {
+ if (prim_bkt->sig_current[i] == sig &&
+ prim_bkt->sig_alt[i] == alt_hash) {
k = (struct rte_hash_key *) ((char *)keys +
prim_bkt->key_idx[i] * h->key_entry_size);
if (rte_hash_cmp_eq(key, k->key, h) == 0) {
@@ -564,8 +564,8 @@ __rte_hash_add_key_with_hash(const struct rte_hash *h, const void *key,
/* Check if key is already inserted in secondary location */
for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
- if (sec_bkt->signatures[i].alt == sig &&
- sec_bkt->signatures[i].current == alt_hash) {
+ if (sec_bkt->sig_alt[i] == sig &&
+ sec_bkt->sig_current[i] == alt_hash) {
k = (struct rte_hash_key *) ((char *)keys +
sec_bkt->key_idx[i] * h->key_entry_size);
if (rte_hash_cmp_eq(key, k->key, h) == 0) {
@@ -611,8 +611,8 @@ __rte_hash_add_key_with_hash(const struct rte_hash *h, const void *key,
for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
/* Check if slot is available */
if (likely(prim_bkt->key_idx[i] == EMPTY_SLOT)) {
- prim_bkt->signatures[i].current = sig;
- prim_bkt->signatures[i].alt = alt_hash;
+ prim_bkt->sig_current[i] = sig;
+ prim_bkt->sig_alt[i] = alt_hash;
prim_bkt->key_idx[i] = new_idx;
break;
}
@@ -632,8 +632,8 @@ __rte_hash_add_key_with_hash(const struct rte_hash *h, const void *key,
*/
ret = make_space_bucket(h, prim_bkt);
if (ret >= 0) {
- prim_bkt->signatures[ret].current = sig;
- prim_bkt->signatures[ret].alt = alt_hash;
+ prim_bkt->sig_current[ret] = sig;
+ prim_bkt->sig_alt[ret] = alt_hash;
prim_bkt->key_idx[ret] = new_idx;
if (h->add_key == ADD_KEY_MULTIWRITER)
rte_spinlock_unlock(h->multiwriter_lock);
@@ -707,7 +707,7 @@ __rte_hash_lookup_with_hash(const struct rte_hash *h, const void *key,
/* Check if key is in primary location */
for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
- if (bkt->signatures[i].current == sig &&
+ if (bkt->sig_current[i] == sig &&
bkt->key_idx[i] != EMPTY_SLOT) {
k = (struct rte_hash_key *) ((char *)keys +
bkt->key_idx[i] * h->key_entry_size);
@@ -730,8 +730,8 @@ __rte_hash_lookup_with_hash(const struct rte_hash *h, const void *key,
/* Check if key is in secondary location */
for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
- if (bkt->signatures[i].current == alt_hash &&
- bkt->signatures[i].alt == sig) {
+ if (bkt->sig_current[i] == alt_hash &&
+ bkt->sig_alt[i] == sig) {
k = (struct rte_hash_key *) ((char *)keys +
bkt->key_idx[i] * h->key_entry_size);
if (rte_hash_cmp_eq(key, k->key, h) == 0) {
@@ -785,7 +785,8 @@ remove_entry(const struct rte_hash *h, struct rte_hash_bucket *bkt, unsigned i)
unsigned lcore_id, n_slots;
struct lcore_cache *cached_free_slots;
- bkt->signatures[i].sig = NULL_SIGNATURE;
+ bkt->sig_current[i] = NULL_SIGNATURE;
+ bkt->sig_alt[i] = NULL_SIGNATURE;
if (h->hw_trans_mem_support) {
lcore_id = rte_lcore_id();
cached_free_slots = &h->local_free_slots[lcore_id];
@@ -823,7 +824,7 @@ __rte_hash_del_key_with_hash(const struct rte_hash *h, const void *key,
/* Check if key is in primary location */
for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
- if (bkt->signatures[i].current == sig &&
+ if (bkt->sig_current[i] == sig &&
bkt->key_idx[i] != EMPTY_SLOT) {
k = (struct rte_hash_key *) ((char *)keys +
bkt->key_idx[i] * h->key_entry_size);
@@ -848,7 +849,7 @@ __rte_hash_del_key_with_hash(const struct rte_hash *h, const void *key,
/* Check if key is in secondary location */
for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
- if (bkt->signatures[i].current == alt_hash &&
+ if (bkt->sig_current[i] == alt_hash &&
bkt->key_idx[i] != EMPTY_SLOT) {
k = (struct rte_hash_key *) ((char *)keys +
bkt->key_idx[i] * h->key_entry_size);
@@ -957,8 +958,8 @@ lookup_stage2(unsigned idx, hash_sig_t prim_hash, hash_sig_t sec_hash,
prim_hash_matches = 1 << RTE_HASH_BUCKET_ENTRIES;
sec_hash_matches = 1 << RTE_HASH_BUCKET_ENTRIES;
for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
- prim_hash_matches |= ((prim_hash == prim_bkt->signatures[i].current) << i);
- sec_hash_matches |= ((sec_hash == sec_bkt->signatures[i].current) << i);
+ prim_hash_matches |= ((prim_hash == prim_bkt->sig_current[i]) << i);
+ sec_hash_matches |= ((sec_hash == sec_bkt->sig_current[i]) << i);
}
key_idx = prim_bkt->key_idx[__builtin_ctzl(prim_hash_matches)];
diff --git a/lib/librte_hash/rte_cuckoo_hash.h b/lib/librte_hash/rte_cuckoo_hash.h
index 5a32ea6..24f8437 100644
--- a/lib/librte_hash/rte_cuckoo_hash.h
+++ b/lib/librte_hash/rte_cuckoo_hash.h
@@ -151,17 +151,6 @@ struct lcore_cache {
void *objs[LCORE_CACHE_SIZE]; /**< Cache objects */
} __rte_cache_aligned;
-/* Structure storing both primary and secondary hashes */
-struct rte_hash_signatures {
- union {
- struct {
- hash_sig_t current;
- hash_sig_t alt;
- };
- uint64_t sig;
- };
-};
-
/* Structure that stores key-value pair */
struct rte_hash_key {
union {
@@ -174,9 +163,13 @@ struct rte_hash_key {
/** Bucket structure */
struct rte_hash_bucket {
- struct rte_hash_signatures signatures[RTE_HASH_BUCKET_ENTRIES];
+ hash_sig_t sig_current[RTE_HASH_BUCKET_ENTRIES];
+
/* Includes dummy key index that always contains index 0 */
uint32_t key_idx[RTE_HASH_BUCKET_ENTRIES + 1];
+
+ hash_sig_t sig_alt[RTE_HASH_BUCKET_ENTRIES];
+
uint8_t flag[RTE_HASH_BUCKET_ENTRIES];
} __rte_cache_aligned;
diff --git a/lib/librte_hash/rte_cuckoo_hash_x86.h b/lib/librte_hash/rte_cuckoo_hash_x86.h
index e16d69c..494c160 100644
--- a/lib/librte_hash/rte_cuckoo_hash_x86.h
+++ b/lib/librte_hash/rte_cuckoo_hash_x86.h
@@ -54,8 +54,8 @@ rte_hash_cuckoo_insert_mw_tm(struct rte_hash_bucket *prim_bkt,
for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
/* Check if slot is available */
if (likely(prim_bkt->key_idx == EMPTY_SLOT)) {
- prim_bkt->signatures[i].current = sig;
- prim_bkt->signatures[i].alt = alt_hash;
+ prim_bkt->sig_current[i] = sig;
+ prim_bkt->sig_alt[i] = alt_hash;
prim_bkt->key_idx[i] = new_idx;
break;
}
@@ -101,7 +101,7 @@ rte_hash_cuckoo_move_insert_mw_tm(const struct rte_hash *h,
prev_slot = curr_node->prev_slot;
prev_alt_bkt_idx
- = prev_bkt->signatures[prev_slot].alt
+ = prev_bkt->sig_alt[prev_slot]
& h->bucket_bitmask;
if (unlikely(&h->buckets[prev_alt_bkt_idx]
@@ -113,10 +113,10 @@ rte_hash_cuckoo_move_insert_mw_tm(const struct rte_hash *h,
* Cuckoo insert to move elements back to its
* primary bucket if available
*/
- curr_bkt->signatures[curr_slot].alt =
- prev_bkt->signatures[prev_slot].current;
- curr_bkt->signatures[curr_slot].current =
- prev_bkt->signatures[prev_slot].alt;
+ curr_bkt->sig_alt[curr_slot] =
+ prev_bkt->sig_current[prev_slot];
+ curr_bkt->sig_current[curr_slot] =
+ prev_bkt->sig_alt[prev_slot];
curr_bkt->key_idx[curr_slot]
= prev_bkt->key_idx[prev_slot];
@@ -125,8 +125,8 @@ rte_hash_cuckoo_move_insert_mw_tm(const struct rte_hash *h,
curr_bkt = curr_node->bkt;
}
- curr_bkt->signatures[curr_slot].current = sig;
- curr_bkt->signatures[curr_slot].alt = alt_hash;
+ curr_bkt->sig_current[curr_slot] = sig;
+ curr_bkt->sig_alt[curr_slot] = alt_hash;
curr_bkt->key_idx[curr_slot] = new_idx;
rte_xend();
@@ -178,7 +178,7 @@ rte_hash_cuckoo_make_space_mw_tm(const struct rte_hash *h,
}
/* Enqueue new node and keep prev node info */
- alt_bkt = &(h->buckets[curr_bkt->signatures[i].alt
+ alt_bkt = &(h->buckets[curr_bkt->sig_alt[i]
& h->bucket_bitmask]);
head->bkt = alt_bkt;
head->prev = tail;
--
2.7.4
^ permalink raw reply related
* [PATCH v4 3/4] hash: add vectorized comparison
From: Pablo de Lara @ 2016-09-30 7:38 UTC (permalink / raw)
To: dev; +Cc: bruce.richardson, Byron Marohn, Saikrishna Edupuganti,
Pablo de Lara
In-Reply-To: <1475221136-213246-1-git-send-email-pablo.de.lara.guarch@intel.com>
From: Byron Marohn <byron.marohn@intel.com>
In lookup bulk function, the signatures of all entries
are compared against the signature of the key that is being looked up.
Now that all the signatures are together, they can be compared
with vector instructions (SSE, AVX2), achieving higher lookup performance.
Also, entries per bucket are increased to 8 when using processors
with AVX2, as 256 bits can be compared at once, which is the size of
8x32-bit signatures.
Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
lib/librte_hash/rte_cuckoo_hash.c | 73 ++++++++++++++++++++++++++++++++++++---
lib/librte_hash/rte_cuckoo_hash.h | 12 ++++++-
2 files changed, 79 insertions(+), 6 deletions(-)
diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c
index a7ee2b9..397aa8e 100644
--- a/lib/librte_hash/rte_cuckoo_hash.c
+++ b/lib/librte_hash/rte_cuckoo_hash.c
@@ -284,6 +284,15 @@ rte_hash_create(const struct rte_hash_parameters *params)
h->free_slots = r;
h->hw_trans_mem_support = hw_trans_mem_support;
+#if defined(RTE_ARCH_X86)
+ if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_AVX2))
+ h->sig_cmp_fn = RTE_HASH_COMPARE_AVX2;
+ else if (rte_cpu_get_flag_enabled(RTE_CPUFLAG_SSE2))
+ h->sig_cmp_fn = RTE_HASH_COMPARE_SSE;
+ else
+#endif
+ h->sig_cmp_fn = RTE_HASH_COMPARE_SCALAR;
+
/* Turn on multi-writer only with explicit flat from user and TM
* support.
*/
@@ -940,6 +949,61 @@ lookup_stage1(unsigned idx, hash_sig_t *prim_hash, hash_sig_t *sec_hash,
rte_prefetch0(*secondary_bkt);
}
+static inline void
+compare_signatures(unsigned *prim_hash_matches, unsigned *sec_hash_matches,
+ const struct rte_hash_bucket *prim_bkt,
+ const struct rte_hash_bucket *sec_bkt,
+ hash_sig_t prim_hash, hash_sig_t sec_hash,
+ enum rte_hash_sig_compare_function sig_cmp_fn)
+{
+ unsigned i;
+
+ switch (sig_cmp_fn) {
+#ifdef RTE_MACHINE_CPUFLAG_AVX2
+ case RTE_HASH_COMPARE_AVX2:
+ *prim_hash_matches |= _mm256_movemask_ps((__m256)_mm256_cmpeq_epi32(
+ _mm256_load_si256(
+ (__m256i const *)prim_bkt->sig_current),
+ _mm256_set1_epi32(prim_hash)));
+ *sec_hash_matches |= _mm256_movemask_ps((__m256)_mm256_cmpeq_epi32(
+ _mm256_load_si256(
+ (__m256i const *)sec_bkt->sig_current),
+ _mm256_set1_epi32(sec_hash)));
+ break;
+#endif
+#ifdef RTE_MACHINE_CPUFLAG_SSE2
+ case RTE_HASH_COMPARE_SSE:
+ /* Compare the first 4 signatures in the bucket */
+ *prim_hash_matches |= _mm_movemask_ps((__m128)_mm_cmpeq_epi16(
+ _mm_load_si128(
+ (__m128i const *)prim_bkt->sig_current),
+ _mm_set1_epi32(prim_hash)));
+ *prim_hash_matches |= (_mm_movemask_ps((__m128)_mm_cmpeq_epi16(
+ _mm_load_si128(
+ (__m128i const *)&prim_bkt->sig_current[4]),
+ _mm_set1_epi32(prim_hash)))) << 4;
+ /* Compare the first 4 signatures in the bucket */
+ *sec_hash_matches |= _mm_movemask_ps((__m128)_mm_cmpeq_epi16(
+ _mm_load_si128(
+ (__m128i const *)sec_bkt->sig_current),
+ _mm_set1_epi32(sec_hash)));
+ *sec_hash_matches |= (_mm_movemask_ps((__m128)_mm_cmpeq_epi16(
+ _mm_load_si128(
+ (__m128i const *)&sec_bkt->sig_current[4]),
+ _mm_set1_epi32(sec_hash)))) << 4;
+ break;
+#endif
+ default:
+ for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
+ *prim_hash_matches |=
+ ((prim_hash == prim_bkt->sig_current[i]) << i);
+ *sec_hash_matches |=
+ ((sec_hash == sec_bkt->sig_current[i]) << i);
+ }
+ }
+
+}
+
/*
* Lookup bulk stage 2: Search for match hashes in primary/secondary locations
* and prefetch first key slot
@@ -952,15 +1016,14 @@ lookup_stage2(unsigned idx, hash_sig_t prim_hash, hash_sig_t sec_hash,
uint64_t *extra_hits_mask, const void *keys,
const struct rte_hash *h)
{
- unsigned prim_hash_matches, sec_hash_matches, key_idx, i;
+ unsigned prim_hash_matches, sec_hash_matches, key_idx;
unsigned total_hash_matches;
prim_hash_matches = 1 << RTE_HASH_BUCKET_ENTRIES;
sec_hash_matches = 1 << RTE_HASH_BUCKET_ENTRIES;
- for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) {
- prim_hash_matches |= ((prim_hash == prim_bkt->sig_current[i]) << i);
- sec_hash_matches |= ((sec_hash == sec_bkt->sig_current[i]) << i);
- }
+
+ compare_signatures(&prim_hash_matches, &sec_hash_matches, prim_bkt,
+ sec_bkt, prim_hash, sec_hash, h->sig_cmp_fn);
key_idx = prim_bkt->key_idx[__builtin_ctzl(prim_hash_matches)];
if (key_idx == 0)
diff --git a/lib/librte_hash/rte_cuckoo_hash.h b/lib/librte_hash/rte_cuckoo_hash.h
index 24f8437..9ff79c0 100644
--- a/lib/librte_hash/rte_cuckoo_hash.h
+++ b/lib/librte_hash/rte_cuckoo_hash.h
@@ -130,7 +130,7 @@ enum add_key_case {
};
/** Number of items per bucket. */
-#define RTE_HASH_BUCKET_ENTRIES 4
+#define RTE_HASH_BUCKET_ENTRIES 8
#define NULL_SIGNATURE 0
@@ -161,6 +161,14 @@ struct rte_hash_key {
char key[0];
} __attribute__((aligned(KEY_ALIGNMENT)));
+/* All different signature compare functions */
+enum rte_hash_sig_compare_function {
+ RTE_HASH_COMPARE_SCALAR = 0,
+ RTE_HASH_COMPARE_SSE,
+ RTE_HASH_COMPARE_AVX2,
+ RTE_HASH_COMPARE_NUM
+};
+
/** Bucket structure */
struct rte_hash_bucket {
hash_sig_t sig_current[RTE_HASH_BUCKET_ENTRIES];
@@ -198,6 +206,8 @@ struct rte_hash {
/**< Custom function used to compare keys. */
enum cmp_jump_table_case cmp_jump_table_idx;
/**< Indicates which compare function to use. */
+ enum rte_hash_sig_compare_function sig_cmp_fn;
+ /**< Indicates which signature compare function to use. */
uint32_t bucket_bitmask; /**< Bitmask for getting bucket index
from hash signature. */
uint32_t key_entry_size; /**< Size of each key entry. */
--
2.7.4
^ permalink raw reply related
* [PATCH v4 4/4] hash: modify lookup bulk pipeline
From: Pablo de Lara @ 2016-09-30 7:38 UTC (permalink / raw)
To: dev; +Cc: bruce.richardson, Byron Marohn, Saikrishna Edupuganti,
Pablo de Lara
In-Reply-To: <1475221136-213246-1-git-send-email-pablo.de.lara.guarch@intel.com>
From: Byron Marohn <byron.marohn@intel.com>
This patch replaces the pipelined rte_hash lookup mechanism with a
loop-and-jump model, which performs significantly better,
especially for smaller table sizes and smaller table occupancies.
Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
---
lib/librte_hash/rte_cuckoo_hash.c | 377 ++++++++++++--------------------------
lib/librte_hash/rte_cuckoo_hash.h | 3 +-
2 files changed, 117 insertions(+), 263 deletions(-)
diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c
index 397aa8e..1443ee1 100644
--- a/lib/librte_hash/rte_cuckoo_hash.c
+++ b/lib/librte_hash/rte_cuckoo_hash.c
@@ -914,43 +914,8 @@ rte_hash_get_key_with_position(const struct rte_hash *h, const int32_t position,
return 0;
}
-/* Lookup bulk stage 0: Prefetch input key */
static inline void
-lookup_stage0(unsigned *idx, uint64_t *lookup_mask,
- const void * const *keys)
-{
- *idx = __builtin_ctzl(*lookup_mask);
- if (*lookup_mask == 0)
- *idx = 0;
-
- rte_prefetch0(keys[*idx]);
- *lookup_mask &= ~(1llu << *idx);
-}
-
-/*
- * Lookup bulk stage 1: Calculate primary/secondary hashes
- * and prefetch primary/secondary buckets
- */
-static inline void
-lookup_stage1(unsigned idx, hash_sig_t *prim_hash, hash_sig_t *sec_hash,
- const struct rte_hash_bucket **primary_bkt,
- const struct rte_hash_bucket **secondary_bkt,
- hash_sig_t *hash_vals, const void * const *keys,
- const struct rte_hash *h)
-{
- *prim_hash = rte_hash_hash(h, keys[idx]);
- hash_vals[idx] = *prim_hash;
- *sec_hash = rte_hash_secondary_hash(*prim_hash);
-
- *primary_bkt = &h->buckets[*prim_hash & h->bucket_bitmask];
- *secondary_bkt = &h->buckets[*sec_hash & h->bucket_bitmask];
-
- rte_prefetch0(*primary_bkt);
- rte_prefetch0(*secondary_bkt);
-}
-
-static inline void
-compare_signatures(unsigned *prim_hash_matches, unsigned *sec_hash_matches,
+compare_signatures(uint32_t *prim_hash_matches, uint32_t *sec_hash_matches,
const struct rte_hash_bucket *prim_bkt,
const struct rte_hash_bucket *sec_bkt,
hash_sig_t prim_hash, hash_sig_t sec_hash,
@@ -961,11 +926,11 @@ compare_signatures(unsigned *prim_hash_matches, unsigned *sec_hash_matches,
switch (sig_cmp_fn) {
#ifdef RTE_MACHINE_CPUFLAG_AVX2
case RTE_HASH_COMPARE_AVX2:
- *prim_hash_matches |= _mm256_movemask_ps((__m256)_mm256_cmpeq_epi32(
+ *prim_hash_matches = _mm256_movemask_ps((__m256)_mm256_cmpeq_epi32(
_mm256_load_si256(
(__m256i const *)prim_bkt->sig_current),
_mm256_set1_epi32(prim_hash)));
- *sec_hash_matches |= _mm256_movemask_ps((__m256)_mm256_cmpeq_epi32(
+ *sec_hash_matches = _mm256_movemask_ps((__m256)_mm256_cmpeq_epi32(
_mm256_load_si256(
(__m256i const *)sec_bkt->sig_current),
_mm256_set1_epi32(sec_hash)));
@@ -974,7 +939,7 @@ compare_signatures(unsigned *prim_hash_matches, unsigned *sec_hash_matches,
#ifdef RTE_MACHINE_CPUFLAG_SSE2
case RTE_HASH_COMPARE_SSE:
/* Compare the first 4 signatures in the bucket */
- *prim_hash_matches |= _mm_movemask_ps((__m128)_mm_cmpeq_epi16(
+ *prim_hash_matches = _mm_movemask_ps((__m128)_mm_cmpeq_epi16(
_mm_load_si128(
(__m128i const *)prim_bkt->sig_current),
_mm_set1_epi32(prim_hash)));
@@ -983,7 +948,7 @@ compare_signatures(unsigned *prim_hash_matches, unsigned *sec_hash_matches,
(__m128i const *)&prim_bkt->sig_current[4]),
_mm_set1_epi32(prim_hash)))) << 4;
/* Compare the first 4 signatures in the bucket */
- *sec_hash_matches |= _mm_movemask_ps((__m128)_mm_cmpeq_epi16(
+ *sec_hash_matches = _mm_movemask_ps((__m128)_mm_cmpeq_epi16(
_mm_load_si128(
(__m128i const *)sec_bkt->sig_current),
_mm_set1_epi32(sec_hash)));
@@ -1004,244 +969,134 @@ compare_signatures(unsigned *prim_hash_matches, unsigned *sec_hash_matches,
}
-/*
- * Lookup bulk stage 2: Search for match hashes in primary/secondary locations
- * and prefetch first key slot
- */
+#define PREFETCH_OFFSET 4
static inline void
-lookup_stage2(unsigned idx, hash_sig_t prim_hash, hash_sig_t sec_hash,
- const struct rte_hash_bucket *prim_bkt,
- const struct rte_hash_bucket *sec_bkt,
- const struct rte_hash_key **key_slot, int32_t *positions,
- uint64_t *extra_hits_mask, const void *keys,
- const struct rte_hash *h)
+__rte_hash_lookup_bulk(const struct rte_hash *h, const void **keys,
+ int32_t num_keys, int32_t *positions,
+ uint64_t *hit_mask, void *data[])
{
- unsigned prim_hash_matches, sec_hash_matches, key_idx;
- unsigned total_hash_matches;
+ uint64_t hits = 0;
+ int32_t i;
+ uint32_t prim_hash[RTE_HASH_LOOKUP_BULK_MAX];
+ uint32_t sec_hash[RTE_HASH_LOOKUP_BULK_MAX];
+ const struct rte_hash_bucket *primary_bkt[RTE_HASH_LOOKUP_BULK_MAX];
+ const struct rte_hash_bucket *secondary_bkt[RTE_HASH_LOOKUP_BULK_MAX];
+ uint32_t prim_hitmask[RTE_HASH_LOOKUP_BULK_MAX] = {0};
+ uint32_t sec_hitmask[RTE_HASH_LOOKUP_BULK_MAX] = {0};
+
+ /* Prefetch first keys */
+ for (i = 0; i < PREFETCH_OFFSET && i < num_keys; i++)
+ rte_prefetch0(keys[i]);
- prim_hash_matches = 1 << RTE_HASH_BUCKET_ENTRIES;
- sec_hash_matches = 1 << RTE_HASH_BUCKET_ENTRIES;
+ /*
+ * Prefetch rest of the keys, calculate primary and
+ * secondary bucket and prefetch them
+ */
+ for (i = 0; i < (num_keys - PREFETCH_OFFSET); i++) {
+ rte_prefetch0(keys[i + PREFETCH_OFFSET]);
- compare_signatures(&prim_hash_matches, &sec_hash_matches, prim_bkt,
- sec_bkt, prim_hash, sec_hash, h->sig_cmp_fn);
+ prim_hash[i] = rte_hash_hash(h, keys[i]);
+ sec_hash[i] = rte_hash_secondary_hash(prim_hash[i]);
- key_idx = prim_bkt->key_idx[__builtin_ctzl(prim_hash_matches)];
- if (key_idx == 0)
- key_idx = sec_bkt->key_idx[__builtin_ctzl(sec_hash_matches)];
+ primary_bkt[i] = &h->buckets[prim_hash[i] & h->bucket_bitmask];
+ secondary_bkt[i] = &h->buckets[sec_hash[i] & h->bucket_bitmask];
- total_hash_matches = (prim_hash_matches |
- (sec_hash_matches << (RTE_HASH_BUCKET_ENTRIES + 1)));
- *key_slot = (const struct rte_hash_key *) ((const char *)keys +
- key_idx * h->key_entry_size);
+ rte_prefetch0(primary_bkt[i]);
+ rte_prefetch0(secondary_bkt[i]);
+ }
- rte_prefetch0(*key_slot);
- /*
- * Return index where key is stored,
- * substracting the first dummy index
- */
- positions[idx] = (key_idx - 1);
+ /* Calculate and prefetch rest of the buckets */
+ for (; i < num_keys; i++) {
+ prim_hash[i] = rte_hash_hash(h, keys[i]);
+ sec_hash[i] = rte_hash_secondary_hash(prim_hash[i]);
- *extra_hits_mask |= (uint64_t)(__builtin_popcount(total_hash_matches) > 3) << idx;
+ primary_bkt[i] = &h->buckets[prim_hash[i] & h->bucket_bitmask];
+ secondary_bkt[i] = &h->buckets[sec_hash[i] & h->bucket_bitmask];
-}
+ rte_prefetch0(primary_bkt[i]);
+ rte_prefetch0(secondary_bkt[i]);
+ }
+ /* Compare signatures and prefetch key slot of first hit */
+ for (i = 0; i < num_keys; i++) {
+ compare_signatures(&prim_hitmask[i], &sec_hitmask[i],
+ primary_bkt[i], secondary_bkt[i],
+ prim_hash[i], sec_hash[i], h->sig_cmp_fn);
+
+ if (prim_hitmask[i]) {
+ uint32_t first_hit = __builtin_ctzl(prim_hitmask[i]);
+ uint32_t key_idx = primary_bkt[i]->key_idx[first_hit];
+ const struct rte_hash_key *key_slot =
+ (const struct rte_hash_key *)(
+ (const char *)h->key_store +
+ key_idx * h->key_entry_size);
+ rte_prefetch0(key_slot);
+ continue;
+ }
-/* Lookup bulk stage 3: Check if key matches, update hit mask and return data */
-static inline void
-lookup_stage3(unsigned idx, const struct rte_hash_key *key_slot, const void * const *keys,
- const int32_t *positions, void *data[], uint64_t *hits,
- const struct rte_hash *h)
-{
- unsigned hit;
- unsigned key_idx;
+ if (sec_hitmask[i]) {
+ uint32_t first_hit = __builtin_ctzl(sec_hitmask[i]);
+ uint32_t key_idx = secondary_bkt[i]->key_idx[first_hit];
+ const struct rte_hash_key *key_slot =
+ (const struct rte_hash_key *)(
+ (const char *)h->key_store +
+ key_idx * h->key_entry_size);
+ rte_prefetch0(key_slot);
+ }
+ }
- hit = !rte_hash_cmp_eq(key_slot->key, keys[idx], h);
- if (data != NULL)
- data[idx] = key_slot->pdata;
+ /* Compare keys, first hits in primary first */
+ for (i = 0; i < num_keys; i++) {
+ positions[i] = -ENOENT;
+ while (prim_hitmask[i]) {
+ uint32_t hit_index = __builtin_ctzl(prim_hitmask[i]);
+
+ uint32_t key_idx = primary_bkt[i]->key_idx[hit_index];
+ const struct rte_hash_key *key_slot =
+ (const struct rte_hash_key *)(
+ (const char *)h->key_store +
+ key_idx * h->key_entry_size);
+ /*
+ * If key index is 0, do not compare key,
+ * as it is checking the dummy slot
+ */
+ if (!!key_idx & !rte_hash_cmp_eq(key_slot->key, keys[i], h)) {
+ if (data != NULL)
+ data[i] = key_slot->pdata;
- key_idx = positions[idx] + 1;
- /*
- * If key index is 0, force hit to be 0, in case key to be looked up
- * is all zero (as in the dummy slot), which would result in a wrong hit
- */
- *hits |= (uint64_t)(hit && !!key_idx) << idx;
-}
+ hits |= 1ULL << i;
+ positions[i] = key_idx - 1;
+ goto next_key;
+ }
+ prim_hitmask[i] &= ~(1 << (hit_index));
+ }
-static inline void
-__rte_hash_lookup_bulk(const struct rte_hash *h, const void **keys,
- uint32_t num_keys, int32_t *positions,
- uint64_t *hit_mask, void *data[])
-{
- uint64_t hits = 0;
- uint64_t extra_hits_mask = 0;
- uint64_t lookup_mask, miss_mask;
- unsigned idx;
- const void *key_store = h->key_store;
- int ret;
- hash_sig_t hash_vals[RTE_HASH_LOOKUP_BULK_MAX];
-
- unsigned idx00, idx01, idx10, idx11, idx20, idx21, idx30, idx31;
- const struct rte_hash_bucket *primary_bkt10, *primary_bkt11;
- const struct rte_hash_bucket *secondary_bkt10, *secondary_bkt11;
- const struct rte_hash_bucket *primary_bkt20, *primary_bkt21;
- const struct rte_hash_bucket *secondary_bkt20, *secondary_bkt21;
- const struct rte_hash_key *k_slot20, *k_slot21, *k_slot30, *k_slot31;
- hash_sig_t primary_hash10, primary_hash11;
- hash_sig_t secondary_hash10, secondary_hash11;
- hash_sig_t primary_hash20, primary_hash21;
- hash_sig_t secondary_hash20, secondary_hash21;
-
- lookup_mask = (uint64_t) -1 >> (64 - num_keys);
- miss_mask = lookup_mask;
-
- lookup_stage0(&idx00, &lookup_mask, keys);
- lookup_stage0(&idx01, &lookup_mask, keys);
-
- idx10 = idx00, idx11 = idx01;
-
- lookup_stage0(&idx00, &lookup_mask, keys);
- lookup_stage0(&idx01, &lookup_mask, keys);
- lookup_stage1(idx10, &primary_hash10, &secondary_hash10,
- &primary_bkt10, &secondary_bkt10, hash_vals, keys, h);
- lookup_stage1(idx11, &primary_hash11, &secondary_hash11,
- &primary_bkt11, &secondary_bkt11, hash_vals, keys, h);
-
- primary_bkt20 = primary_bkt10;
- primary_bkt21 = primary_bkt11;
- secondary_bkt20 = secondary_bkt10;
- secondary_bkt21 = secondary_bkt11;
- primary_hash20 = primary_hash10;
- primary_hash21 = primary_hash11;
- secondary_hash20 = secondary_hash10;
- secondary_hash21 = secondary_hash11;
- idx20 = idx10, idx21 = idx11;
- idx10 = idx00, idx11 = idx01;
-
- lookup_stage0(&idx00, &lookup_mask, keys);
- lookup_stage0(&idx01, &lookup_mask, keys);
- lookup_stage1(idx10, &primary_hash10, &secondary_hash10,
- &primary_bkt10, &secondary_bkt10, hash_vals, keys, h);
- lookup_stage1(idx11, &primary_hash11, &secondary_hash11,
- &primary_bkt11, &secondary_bkt11, hash_vals, keys, h);
- lookup_stage2(idx20, primary_hash20, secondary_hash20, primary_bkt20,
- secondary_bkt20, &k_slot20, positions, &extra_hits_mask,
- key_store, h);
- lookup_stage2(idx21, primary_hash21, secondary_hash21, primary_bkt21,
- secondary_bkt21, &k_slot21, positions, &extra_hits_mask,
- key_store, h);
-
- while (lookup_mask) {
- k_slot30 = k_slot20, k_slot31 = k_slot21;
- idx30 = idx20, idx31 = idx21;
- primary_bkt20 = primary_bkt10;
- primary_bkt21 = primary_bkt11;
- secondary_bkt20 = secondary_bkt10;
- secondary_bkt21 = secondary_bkt11;
- primary_hash20 = primary_hash10;
- primary_hash21 = primary_hash11;
- secondary_hash20 = secondary_hash10;
- secondary_hash21 = secondary_hash11;
- idx20 = idx10, idx21 = idx11;
- idx10 = idx00, idx11 = idx01;
-
- lookup_stage0(&idx00, &lookup_mask, keys);
- lookup_stage0(&idx01, &lookup_mask, keys);
- lookup_stage1(idx10, &primary_hash10, &secondary_hash10,
- &primary_bkt10, &secondary_bkt10, hash_vals, keys, h);
- lookup_stage1(idx11, &primary_hash11, &secondary_hash11,
- &primary_bkt11, &secondary_bkt11, hash_vals, keys, h);
- lookup_stage2(idx20, primary_hash20, secondary_hash20,
- primary_bkt20, secondary_bkt20, &k_slot20, positions,
- &extra_hits_mask, key_store, h);
- lookup_stage2(idx21, primary_hash21, secondary_hash21,
- primary_bkt21, secondary_bkt21, &k_slot21, positions,
- &extra_hits_mask, key_store, h);
- lookup_stage3(idx30, k_slot30, keys, positions, data, &hits, h);
- lookup_stage3(idx31, k_slot31, keys, positions, data, &hits, h);
- }
+ while (sec_hitmask[i]) {
+ uint32_t hit_index = __builtin_ctzl(sec_hitmask[i]);
+
+ uint32_t key_idx = secondary_bkt[i]->key_idx[hit_index];
+ const struct rte_hash_key *key_slot =
+ (const struct rte_hash_key *)(
+ (const char *)h->key_store +
+ key_idx * h->key_entry_size);
+ /*
+ * If key index is 0, do not compare key,
+ * as it is checking the dummy slot
+ */
+
+ if (!!key_idx & !rte_hash_cmp_eq(key_slot->key, keys[i], h)) {
+ if (data != NULL)
+ data[i] = key_slot->pdata;
- k_slot30 = k_slot20, k_slot31 = k_slot21;
- idx30 = idx20, idx31 = idx21;
- primary_bkt20 = primary_bkt10;
- primary_bkt21 = primary_bkt11;
- secondary_bkt20 = secondary_bkt10;
- secondary_bkt21 = secondary_bkt11;
- primary_hash20 = primary_hash10;
- primary_hash21 = primary_hash11;
- secondary_hash20 = secondary_hash10;
- secondary_hash21 = secondary_hash11;
- idx20 = idx10, idx21 = idx11;
- idx10 = idx00, idx11 = idx01;
-
- lookup_stage1(idx10, &primary_hash10, &secondary_hash10,
- &primary_bkt10, &secondary_bkt10, hash_vals, keys, h);
- lookup_stage1(idx11, &primary_hash11, &secondary_hash11,
- &primary_bkt11, &secondary_bkt11, hash_vals, keys, h);
- lookup_stage2(idx20, primary_hash20, secondary_hash20, primary_bkt20,
- secondary_bkt20, &k_slot20, positions, &extra_hits_mask,
- key_store, h);
- lookup_stage2(idx21, primary_hash21, secondary_hash21, primary_bkt21,
- secondary_bkt21, &k_slot21, positions, &extra_hits_mask,
- key_store, h);
- lookup_stage3(idx30, k_slot30, keys, positions, data, &hits, h);
- lookup_stage3(idx31, k_slot31, keys, positions, data, &hits, h);
-
- k_slot30 = k_slot20, k_slot31 = k_slot21;
- idx30 = idx20, idx31 = idx21;
- primary_bkt20 = primary_bkt10;
- primary_bkt21 = primary_bkt11;
- secondary_bkt20 = secondary_bkt10;
- secondary_bkt21 = secondary_bkt11;
- primary_hash20 = primary_hash10;
- primary_hash21 = primary_hash11;
- secondary_hash20 = secondary_hash10;
- secondary_hash21 = secondary_hash11;
- idx20 = idx10, idx21 = idx11;
-
- lookup_stage2(idx20, primary_hash20, secondary_hash20, primary_bkt20,
- secondary_bkt20, &k_slot20, positions, &extra_hits_mask,
- key_store, h);
- lookup_stage2(idx21, primary_hash21, secondary_hash21, primary_bkt21,
- secondary_bkt21, &k_slot21, positions, &extra_hits_mask,
- key_store, h);
- lookup_stage3(idx30, k_slot30, keys, positions, data, &hits, h);
- lookup_stage3(idx31, k_slot31, keys, positions, data, &hits, h);
-
- k_slot30 = k_slot20, k_slot31 = k_slot21;
- idx30 = idx20, idx31 = idx21;
-
- lookup_stage3(idx30, k_slot30, keys, positions, data, &hits, h);
- lookup_stage3(idx31, k_slot31, keys, positions, data, &hits, h);
-
- /* ignore any items we have already found */
- extra_hits_mask &= ~hits;
-
- if (unlikely(extra_hits_mask)) {
- /* run a single search for each remaining item */
- do {
- idx = __builtin_ctzl(extra_hits_mask);
- if (data != NULL) {
- ret = rte_hash_lookup_with_hash_data(h,
- keys[idx], hash_vals[idx], &data[idx]);
- if (ret >= 0)
- hits |= 1ULL << idx;
- } else {
- positions[idx] = rte_hash_lookup_with_hash(h,
- keys[idx], hash_vals[idx]);
- if (positions[idx] >= 0)
- hits |= 1llu << idx;
+ hits |= 1ULL << i;
+ positions[i] = key_idx - 1;
+ goto next_key;
}
- extra_hits_mask &= ~(1llu << idx);
- } while (extra_hits_mask);
- }
+ sec_hitmask[i] &= ~(1 << (hit_index));
+ }
- miss_mask &= ~hits;
- if (unlikely(miss_mask)) {
- do {
- idx = __builtin_ctzl(miss_mask);
- positions[idx] = -ENOENT;
- miss_mask &= ~(1llu << idx);
- } while (miss_mask);
+next_key:
+ continue;
}
if (hit_mask != NULL)
diff --git a/lib/librte_hash/rte_cuckoo_hash.h b/lib/librte_hash/rte_cuckoo_hash.h
index 9ff79c0..2172b2c 100644
--- a/lib/librte_hash/rte_cuckoo_hash.h
+++ b/lib/librte_hash/rte_cuckoo_hash.h
@@ -173,8 +173,7 @@ enum rte_hash_sig_compare_function {
struct rte_hash_bucket {
hash_sig_t sig_current[RTE_HASH_BUCKET_ENTRIES];
- /* Includes dummy key index that always contains index 0 */
- uint32_t key_idx[RTE_HASH_BUCKET_ENTRIES + 1];
+ uint32_t key_idx[RTE_HASH_BUCKET_ENTRIES];
hash_sig_t sig_alt[RTE_HASH_BUCKET_ENTRIES];
--
2.7.4
^ permalink raw reply related
* Re: [PATCH] cryptodev: fix compilation error in SUSE 11 SP2
From: Adrien Mazarguil @ 2016-09-30 8:33 UTC (permalink / raw)
To: De Lara Guarch, Pablo; +Cc: dev@dpdk.org, Doherty, Declan
In-Reply-To: <E115CCD9D858EF4F90C690B0DCB4D8973CA02375@IRSMSX108.ger.corp.intel.com>
On Thu, Sep 29, 2016 at 07:30:31PM +0000, De Lara Guarch, Pablo wrote:
> > -----Original Message-----
> > From: Adrien Mazarguil [mailto:adrien.mazarguil@6wind.com]
> > Sent: Tuesday, September 27, 2016 12:45 AM
> > To: De Lara Guarch, Pablo
> > Cc: dev@dpdk.org; Doherty, Declan
> > Subject: Re: [PATCH] cryptodev: fix compilation error in SUSE 11 SP2
> >
> > On Mon, Sep 26, 2016 at 10:50:35PM +0100, Pablo de Lara wrote:
> > > This commit fixes following build error, which happens in SUSE 11 SP2,
> > > with gcc 4.5.1:
> > >
> > > In file included from lib/librte_cryptodev/rte_cryptodev.c:71:0:
> > > lib/librte_cryptodev/rte_cryptodev_pmd.h:76:7:
> > > error: flexible array member in otherwise empty struct
> > >
> > > Fixes: 347a1e037fd3 ("lib: use C99 syntax for zero-size arrays")
> > >
> > > Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
> >
> > Hmm, this error message does not seem related to your patch. Assuming a
> > similar error is caused by the original code, I think there is a more
> > important issue as the struct should not be empty. Can you check the
> > error?
>
> Well, I don't really understand what is the difference between array[] and array[0],
> I thought both were the same, but some compilers only accept the latter.
Before array[] got standardized by C99, a common trick was to use array[0],
in a sense they are similar except for this one case: a struct with a single
array[] field is explicitly not allowed in C99 since it causes the structure
to be empty (this syntax only provides an alignment constraint for what
follows in case padding is required), no such problem with array[0] which
although nonstandard, is an accepted behavior, sizeof(struct foo) may yield
0 without complaint.
> In any case, the struct will not be empty, as there are other fields, that are not variable sized.
>
> I saw that in your patch you made these two changes (among others):
>
> diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h
> index affbdec..1e30a19 100644
> --- a/lib/librte_cryptodev/rte_cryptodev.h
> +++ b/lib/librte_cryptodev/rte_cryptodev.h
> @@ -759,7 +759,7 @@ struct rte_cryptodev_sym_session {
> } __rte_aligned(8);
> /**< Public symmetric session details */
>
> - char _private[0];
> + char _private[];
> /**< Private session material */
> };
>
> diff --git a/lib/librte_cryptodev/rte_cryptodev_pmd.h b/lib/librte_cryptodev/rte_cryptodev_pmd.h
> index 7d049ea..42e7b79 100644
> --- a/lib/librte_cryptodev/rte_cryptodev_pmd.h
> +++ b/lib/librte_cryptodev/rte_cryptodev_pmd.h
> @@ -71,7 +71,7 @@ struct rte_cryptodev_session {
> struct rte_mempool *mp;
> } __rte_aligned(8);
>
> - char _private[0];
> + __extension__ char _private[0];
> };
>
> So I would expect the same change in both, as they are almost identical,
> but you took different approaches (do you know why? I would like to know :))
Yes, this was done to address the exact same error (probably with the same
old GCC version (4.4.7 perhaps?)), hence my surprise to see it fixed once
again according to your commit log, I think your only mistake was to paste
the error message for the wrong header in there (rte_cryptodev_pmd.h instead
of rte_cryptodev.h), nothing wrong with your patch besides this.
> Basically, I noticed that gcc 4.5 doesn't complain when using your second approach,
> that's why I changed it.
For the record GCC wrongly thinks the structure is empty because a unnamed
struct field is declared inside. Before C11 such declarations only created a
new type that did not occupy any space and not an actual field, hence why it
complains when faced with [] instead of the well-behaved [0].
In this particular case it's a parsing error fixed in subsequent GCC
versions, the unnamed struct actually uses some space otherwise it would
have crashed during non-regression testing (right?)
--
Adrien Mazarguil
6WIND
^ permalink raw reply
* [PATCH v4 0/6] add Tx preparation
From: Tomasz Kulasek @ 2016-09-30 9:00 UTC (permalink / raw)
To: dev; +Cc: konstantin.ananyev, Tomasz Kulasek
In-Reply-To: <20160928111052.9968-1-tomaszx.kulasek@intel.com>
As discussed in that thread:
http://dpdk.org/ml/archives/dev/2015-September/023603.html
Different NIC models depending on HW offload requested might impose
different requirements on packets to be TX-ed in terms of:
- Max number of fragments per packet allowed
- Max number of fragments per TSO segments
- The way pseudo-header checksum should be pre-calculated
- L3/L4 header fields filling
- etc.
MOTIVATION:
-----------
1) Some work cannot (and didn't should) be done in rte_eth_tx_burst.
However, this work is sometimes required, and now, it's an
application issue.
2) Different hardware may have different requirements for TX offloads,
other subset can be supported and so on.
3) Some parameters (e.g. number of segments in ixgbe driver) may hung
device. These parameters may be vary for different devices.
For example i40e HW allows 8 fragments per packet, but that is after
TSO segmentation. While ixgbe has a 38-fragment pre-TSO limit.
4) Fields in packet may require different initialization (like e.g. will
require pseudo-header checksum precalculation, sometimes in a
different way depending on packet type, and so on). Now application
needs to care about it.
5) Using additional API (rte_eth_tx_prep) before rte_eth_tx_burst let to
prepare packet burst in acceptable form for specific device.
6) Some additional checks may be done in debug mode keeping tx_burst
implementation clean.
PROPOSAL:
---------
To help user to deal with all these varieties we propose to:
1) Introduce rte_eth_tx_prep() function to do necessary preparations of
packet burst to be safely transmitted on device for desired HW
offloads (set/reset checksum field according to the hardware
requirements) and check HW constraints (number of segments per
packet, etc).
While the limitations and requirements may differ for devices, it
requires to extend rte_eth_dev structure with new function pointer
"tx_pkt_prep" which can be implemented in the driver to prepare and
verify packets, in devices specific way, before burst, what should to
prevent application to send malformed packets.
2) Also new fields will be introduced in rte_eth_desc_lim:
nb_seg_max and nb_mtu_seg_max, providing an information about max
segments in TSO and non-TSO packets acceptable by device.
This information is useful for application to not create/limit
malicious packet.
APPLICATION (CASE OF USE):
--------------------------
1) Application should to initialize burst of packets to send, set
required tx offload flags and required fields, like l2_len, l3_len,
l4_len, and tso_segsz
2) Application passes burst to the rte_eth_tx_prep to check conditions
required to send packets through the NIC.
3) The result of rte_eth_tx_prep can be used to send valid packets
and/or restore invalid if function fails.
e.g.
for (i = 0; i < nb_pkts; i++) {
/* initialize or process packet */
bufs[i]->tso_segsz = 800;
bufs[i]->ol_flags = PKT_TX_TCP_SEG | PKT_TX_IPV4
| PKT_TX_IP_CKSUM;
bufs[i]->l2_len = sizeof(struct ether_hdr);
bufs[i]->l3_len = sizeof(struct ipv4_hdr);
bufs[i]->l4_len = sizeof(struct tcp_hdr);
}
/* Prepare burst of TX packets */
nb_prep = rte_eth_tx_prep(port, 0, bufs, nb_pkts);
if (nb_prep < nb_pkts) {
printf("tx_prep failed\n");
/* nb_prep indicates here first invalid packet. rte_eth_tx_prep
* can be used on remaining packets to find another ones.
*/
}
/* Send burst of TX packets */
nb_tx = rte_eth_tx_burst(port, 0, bufs, nb_prep);
/* Free any unsent packets. */
v4 changes:
- tx_prep is now set to default behavior (NULL) for simple/vector path
in fm10k, i40e and ixgbe drivers to increase performance, when
Tx offloads are not intentionally available
v3 changes:
- reworked csum testpmd engine instead adding new one,
- fixed checksum initialization procedure to include also outer
checksum offloads,
- some minor formattings and optimalizations
v2 changes:
- rte_eth_tx_prep() returns number of packets when device doesn't
support tx_prep functionality,
- introduced CONFIG_RTE_ETHDEV_TX_PREP allowing to turn off tx_prep
Tomasz Kulasek (6):
ethdev: add Tx preparation
e1000: add Tx preparation
fm10k: add Tx preparation
i40e: add Tx preparation
ixgbe: add Tx preparation
testpmd: use Tx preparation in csum engine
app/test-pmd/csumonly.c | 97 +++++++++++++++------------
config/common_base | 1 +
drivers/net/e1000/e1000_ethdev.h | 11 ++++
drivers/net/e1000/em_ethdev.c | 5 +-
drivers/net/e1000/em_rxtx.c | 48 +++++++++++++-
drivers/net/e1000/igb_ethdev.c | 4 ++
drivers/net/e1000/igb_rxtx.c | 52 ++++++++++++++-
drivers/net/fm10k/fm10k.h | 6 ++
drivers/net/fm10k/fm10k_ethdev.c | 5 ++
drivers/net/fm10k/fm10k_rxtx.c | 50 +++++++++++++-
drivers/net/i40e/i40e_ethdev.c | 3 +
drivers/net/i40e/i40e_rxtx.c | 72 ++++++++++++++++++++-
drivers/net/i40e/i40e_rxtx.h | 8 +++
drivers/net/ixgbe/ixgbe_ethdev.c | 3 +
drivers/net/ixgbe/ixgbe_ethdev.h | 5 +-
drivers/net/ixgbe/ixgbe_rxtx.c | 56 +++++++++++++++-
drivers/net/ixgbe/ixgbe_rxtx.h | 2 +
lib/librte_ether/rte_ethdev.h | 85 ++++++++++++++++++++++++
lib/librte_mbuf/rte_mbuf.h | 8 +++
lib/librte_net/Makefile | 2 +-
lib/librte_net/rte_pkt.h | 133 ++++++++++++++++++++++++++++++++++++++
21 files changed, 605 insertions(+), 51 deletions(-)
create mode 100644 lib/librte_net/rte_pkt.h
--
1.7.9.5
^ permalink raw reply
* [PATCH v4 1/6] ethdev: add Tx preparation
From: Tomasz Kulasek @ 2016-09-30 9:00 UTC (permalink / raw)
To: dev; +Cc: konstantin.ananyev, Tomasz Kulasek
In-Reply-To: <20160930090039.10164-1-tomaszx.kulasek@intel.com>
Added API for `rte_eth_tx_prep`
uint16_t rte_eth_tx_prep(uint8_t port_id, uint16_t queue_id,
struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
Added fields to the `struct rte_eth_desc_lim`:
uint16_t nb_seg_max;
/**< Max number of segments per whole packet. */
uint16_t nb_mtu_seg_max;
/**< Max number of segments per one MTU */
Created `rte_pkt.h` header with common used functions:
int rte_validate_tx_offload(struct rte_mbuf *m)
to validate general requirements for tx offload in packet such a
flag completness. In current implementation this function is called
optionaly when RTE_LIBRTE_ETHDEV_DEBUG is enabled.
int rte_phdr_cksum_fix(struct rte_mbuf *m)
to fix pseudo header checksum for TSO and non-TSO tcp/udp packets
before hardware tx checksum offload.
- for non-TSO tcp/udp packets full pseudo-header checksum is
counted and set.
- for TSO the IP payload length is not included.
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
---
config/common_base | 1 +
lib/librte_ether/rte_ethdev.h | 85 ++++++++++++++++++++++++++
lib/librte_mbuf/rte_mbuf.h | 8 +++
lib/librte_net/Makefile | 2 +-
lib/librte_net/rte_pkt.h | 133 +++++++++++++++++++++++++++++++++++++++++
5 files changed, 228 insertions(+), 1 deletion(-)
create mode 100644 lib/librte_net/rte_pkt.h
diff --git a/config/common_base b/config/common_base
index 7830535..7ada9e0 100644
--- a/config/common_base
+++ b/config/common_base
@@ -120,6 +120,7 @@ CONFIG_RTE_MAX_QUEUES_PER_PORT=1024
CONFIG_RTE_LIBRTE_IEEE1588=n
CONFIG_RTE_ETHDEV_QUEUE_STAT_CNTRS=16
CONFIG_RTE_ETHDEV_RXTX_CALLBACKS=y
+CONFIG_RTE_ETHDEV_TX_PREP=y
#
# Support NIC bypass logic
diff --git a/lib/librte_ether/rte_ethdev.h b/lib/librte_ether/rte_ethdev.h
index 96575e8..6594544 100644
--- a/lib/librte_ether/rte_ethdev.h
+++ b/lib/librte_ether/rte_ethdev.h
@@ -182,6 +182,7 @@ extern "C" {
#include <rte_pci.h>
#include <rte_dev.h>
#include <rte_devargs.h>
+#include <rte_errno.h>
#include "rte_ether.h"
#include "rte_eth_ctrl.h"
#include "rte_dev_info.h"
@@ -699,6 +700,8 @@ struct rte_eth_desc_lim {
uint16_t nb_max; /**< Max allowed number of descriptors. */
uint16_t nb_min; /**< Min allowed number of descriptors. */
uint16_t nb_align; /**< Number of descriptors should be aligned to. */
+ uint16_t nb_seg_max; /**< Max number of segments per whole packet. */
+ uint16_t nb_mtu_seg_max; /**< Max number of segments per one MTU */
};
/**
@@ -1184,6 +1187,11 @@ typedef uint16_t (*eth_tx_burst_t)(void *txq,
uint16_t nb_pkts);
/**< @internal Send output packets on a transmit queue of an Ethernet device. */
+typedef uint16_t (*eth_tx_prep_t)(void *txq,
+ struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+/**< @internal Prepare output packets on a transmit queue of an Ethernet device. */
+
typedef int (*flow_ctrl_get_t)(struct rte_eth_dev *dev,
struct rte_eth_fc_conf *fc_conf);
/**< @internal Get current flow control parameter on an Ethernet device */
@@ -1629,6 +1637,7 @@ enum rte_eth_dev_type {
struct rte_eth_dev {
eth_rx_burst_t rx_pkt_burst; /**< Pointer to PMD receive function. */
eth_tx_burst_t tx_pkt_burst; /**< Pointer to PMD transmit function. */
+ eth_tx_prep_t tx_pkt_prep; /**< Pointer to PMD transmit prepare function. */
struct rte_eth_dev_data *data; /**< Pointer to device data */
const struct eth_driver *driver;/**< Driver for this device */
const struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */
@@ -2837,6 +2846,82 @@ rte_eth_tx_burst(uint8_t port_id, uint16_t queue_id,
return (*dev->tx_pkt_burst)(dev->data->tx_queues[queue_id], tx_pkts, nb_pkts);
}
+/**
+ * Process a burst of output packets on a transmit queue of an Ethernet device.
+ *
+ * The rte_eth_tx_prep() function is invoked to prepare output packets to be
+ * transmitted on the output queue *queue_id* of the Ethernet device designated
+ * by its *port_id*.
+ * The *nb_pkts* parameter is the number of packets to be prepared which are
+ * supplied in the *tx_pkts* array of *rte_mbuf* structures, each of them
+ * allocated from a pool created with rte_pktmbuf_pool_create().
+ * For each packet to send, the rte_eth_tx_prep() function performs
+ * the following operations:
+ *
+ * - Check if packet meets devices requirements for tx offloads.
+ *
+ * - Check limitations about number of segments.
+ *
+ * - Check additional requirements when debug is enabled.
+ *
+ * - Update and/or reset required checksums when tx offload is set for packet.
+ *
+ * The rte_eth_tx_prep() function returns the number of packets ready to be
+ * sent. A return value equal to *nb_pkts* means that all packets are valid and
+ * ready to be sent.
+ *
+ * @param port_id
+ * The port identifier of the Ethernet device.
+ * @param queue_id
+ * The index of the transmit queue through which output packets must be
+ * sent.
+ * The value must be in the range [0, nb_tx_queue - 1] previously supplied
+ * to rte_eth_dev_configure().
+ * @param tx_pkts
+ * The address of an array of *nb_pkts* pointers to *rte_mbuf* structures
+ * which contain the output packets.
+ * @param nb_pkts
+ * The maximum number of packets to process.
+ * @return
+ * The number of packets correct and ready to be sent. The return value can be
+ * less than the value of the *tx_pkts* parameter when some packet doesn't
+ * meet devices requirements with rte_errno set appropriately.
+ */
+
+#ifdef RTE_ETHDEV_TX_PREP
+
+static inline uint16_t
+rte_eth_tx_prep(uint8_t port_id, uint16_t queue_id, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ struct rte_eth_dev *dev = &rte_eth_devices[port_id];
+
+ if (!dev->tx_pkt_prep)
+ return nb_pkts;
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ if (queue_id >= dev->data->nb_tx_queues) {
+ RTE_PMD_DEBUG_TRACE("Invalid TX queue_id=%d\n", queue_id);
+ rte_errno = -EINVAL;
+ return 0;
+ }
+#endif
+
+ return (*dev->tx_pkt_prep)(dev->data->tx_queues[queue_id],
+ tx_pkts, nb_pkts);
+}
+
+#else
+
+static inline uint16_t
+rte_eth_tx_prep(uint8_t port_id __rte_unused, uint16_t queue_id __rte_unused,
+ struct rte_mbuf **tx_pkts __rte_unused, uint16_t nb_pkts)
+{
+ return nb_pkts;
+}
+
+#endif
+
typedef void (*buffer_tx_error_fn)(struct rte_mbuf **unsent, uint16_t count,
void *userdata);
diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
index 23b7bf8..8b73261 100644
--- a/lib/librte_mbuf/rte_mbuf.h
+++ b/lib/librte_mbuf/rte_mbuf.h
@@ -211,6 +211,14 @@ extern "C" {
*/
#define PKT_TX_OUTER_IPV4 (1ULL << 59)
+#define PKT_TX_OFFLOAD_MASK ( \
+ PKT_TX_IP_CKSUM | \
+ PKT_TX_L4_MASK | \
+ PKT_TX_OUTER_IP_CKSUM | \
+ PKT_TX_TCP_SEG | \
+ PKT_TX_QINQ_PKT | \
+ PKT_TX_VLAN_PKT)
+
/**
* Packet outer header is IPv6. This flag must be set when using any
* outer offload feature (L4 checksum) to tell the NIC that the outer
diff --git a/lib/librte_net/Makefile b/lib/librte_net/Makefile
index ad2e482..b5abe84 100644
--- a/lib/librte_net/Makefile
+++ b/lib/librte_net/Makefile
@@ -34,7 +34,7 @@ include $(RTE_SDK)/mk/rte.vars.mk
CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3
# install includes
-SYMLINK-$(CONFIG_RTE_LIBRTE_NET)-include := rte_ip.h rte_tcp.h rte_udp.h rte_sctp.h rte_icmp.h rte_arp.h
+SYMLINK-$(CONFIG_RTE_LIBRTE_NET)-include := rte_ip.h rte_tcp.h rte_udp.h rte_sctp.h rte_icmp.h rte_arp.h rte_pkt.h
include $(RTE_SDK)/mk/rte.install.mk
diff --git a/lib/librte_net/rte_pkt.h b/lib/librte_net/rte_pkt.h
new file mode 100644
index 0000000..72903ac
--- /dev/null
+++ b/lib/librte_net/rte_pkt.h
@@ -0,0 +1,133 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2016 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef _RTE_PKT_H_
+#define _RTE_PKT_H_
+
+#include <rte_ip.h>
+#include <rte_udp.h>
+#include <rte_tcp.h>
+#include <rte_sctp.h>
+
+/**
+ * Validate general requirements for tx offload in packet.
+ */
+static inline int
+rte_validate_tx_offload(struct rte_mbuf *m)
+{
+ uint64_t ol_flags = m->ol_flags;
+
+ /* Does packet set any of available offloads? */
+ if (!(ol_flags & PKT_TX_OFFLOAD_MASK))
+ return 0;
+
+ /* IP checksum can be counted only for IPv4 packet */
+ if ((ol_flags & PKT_TX_IP_CKSUM) && (ol_flags & PKT_TX_IPV6))
+ return -EINVAL;
+
+ if (ol_flags & (PKT_TX_L4_MASK | PKT_TX_TCP_SEG))
+ /* IP type not set */
+ if (!(ol_flags & (PKT_TX_IPV4 | PKT_TX_IPV6)))
+ return -EINVAL;
+
+ if (ol_flags & PKT_TX_TCP_SEG)
+ /* PKT_TX_IP_CKSUM offload not set for IPv4 TSO packet */
+ if ((m->tso_segsz == 0) ||
+ ((ol_flags & PKT_TX_IPV4) && !(ol_flags & PKT_TX_IP_CKSUM)))
+ return -EINVAL;
+
+ /* PKT_TX_OUTER_IP_CKSUM set for non outer IPv4 packet. */
+ if ((ol_flags & PKT_TX_OUTER_IP_CKSUM) && !(ol_flags & PKT_TX_OUTER_IPV4))
+ return -EINVAL;
+
+ return 0;
+}
+
+/**
+ * Fix pseudo header checksum for TSO and non-TSO tcp/udp packets before
+ * hardware tx checksum.
+ * For non-TSO tcp/udp packets full pseudo-header checksum is counted and set.
+ * For TSO the IP payload length is not included.
+ */
+static inline int
+rte_phdr_cksum_fix(struct rte_mbuf *m)
+{
+ struct ipv4_hdr *ipv4_hdr;
+ struct ipv6_hdr *ipv6_hdr;
+ struct tcp_hdr *tcp_hdr;
+ struct udp_hdr *udp_hdr;
+ uint64_t inner_l3_offset = m->l2_len;
+
+ if (m->ol_flags & PKT_TX_OUTER_IP_CKSUM)
+ inner_l3_offset += m->outer_l2_len + m->outer_l3_len;
+
+ if (m->ol_flags & PKT_TX_IPV4) {
+ ipv4_hdr = rte_pktmbuf_mtod_offset(m, struct ipv4_hdr *,
+ inner_l3_offset);
+
+ if (m->ol_flags & PKT_TX_IP_CKSUM)
+ ipv4_hdr->hdr_checksum = 0;
+
+ if ((m->ol_flags & PKT_TX_UDP_CKSUM) == PKT_TX_UDP_CKSUM) {
+ /* non-TSO udp */
+ udp_hdr = rte_pktmbuf_mtod_offset(m, struct udp_hdr *,
+ inner_l3_offset + m->l3_len);
+ udp_hdr->dgram_cksum = rte_ipv4_phdr_cksum(ipv4_hdr, m->ol_flags);
+ } else if ((m->ol_flags & PKT_TX_TCP_CKSUM) ||
+ (m->ol_flags & PKT_TX_TCP_SEG)) {
+ /* non-TSO tcp or TSO */
+ tcp_hdr = rte_pktmbuf_mtod_offset(m, struct tcp_hdr *,
+ inner_l3_offset + m->l3_len);
+ tcp_hdr->cksum = rte_ipv4_phdr_cksum(ipv4_hdr, m->ol_flags);
+ }
+ } else if (m->ol_flags & PKT_TX_IPV6) {
+ ipv6_hdr = rte_pktmbuf_mtod_offset(m, struct ipv6_hdr *,
+ inner_l3_offset);
+
+ if ((m->ol_flags & PKT_TX_UDP_CKSUM) == PKT_TX_UDP_CKSUM) {
+ /* non-TSO udp */
+ udp_hdr = rte_pktmbuf_mtod_offset(m, struct udp_hdr *,
+ inner_l3_offset + m->l3_len);
+ udp_hdr->dgram_cksum = rte_ipv6_phdr_cksum(ipv6_hdr, m->ol_flags);
+ } else if ((m->ol_flags & PKT_TX_TCP_CKSUM) ||
+ (m->ol_flags & PKT_TX_TCP_SEG)) {
+ /* non-TSO tcp or TSO */
+ tcp_hdr = rte_pktmbuf_mtod_offset(m, struct tcp_hdr *,
+ inner_l3_offset + m->l3_len);
+ tcp_hdr->cksum = rte_ipv6_phdr_cksum(ipv6_hdr, m->ol_flags);
+ }
+ }
+ return 0;
+}
+
+#endif /* _RTE_PKT_H_ */
--
1.7.9.5
^ permalink raw reply related
* [PATCH v4 2/6] e1000: add Tx preparation
From: Tomasz Kulasek @ 2016-09-30 9:00 UTC (permalink / raw)
To: dev; +Cc: konstantin.ananyev, Tomasz Kulasek
In-Reply-To: <20160930090039.10164-1-tomaszx.kulasek@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
---
drivers/net/e1000/e1000_ethdev.h | 11 ++++++++
drivers/net/e1000/em_ethdev.c | 5 +++-
drivers/net/e1000/em_rxtx.c | 48 ++++++++++++++++++++++++++++++++++-
drivers/net/e1000/igb_ethdev.c | 4 +++
drivers/net/e1000/igb_rxtx.c | 52 +++++++++++++++++++++++++++++++++++++-
5 files changed, 117 insertions(+), 3 deletions(-)
diff --git a/drivers/net/e1000/e1000_ethdev.h b/drivers/net/e1000/e1000_ethdev.h
index 6c25c8d..bd0f277 100644
--- a/drivers/net/e1000/e1000_ethdev.h
+++ b/drivers/net/e1000/e1000_ethdev.h
@@ -138,6 +138,11 @@
#define E1000_MISC_VEC_ID RTE_INTR_VEC_ZERO_OFFSET
#define E1000_RX_VEC_START RTE_INTR_VEC_RXTX_OFFSET
+#define IGB_TX_MAX_SEG UINT8_MAX
+#define IGB_TX_MAX_MTU_SEG UINT8_MAX
+#define EM_TX_MAX_SEG UINT8_MAX
+#define EM_TX_MAX_MTU_SEG UINT8_MAX
+
/* structure for interrupt relative data */
struct e1000_interrupt {
uint32_t flags;
@@ -315,6 +320,9 @@ void eth_igb_tx_init(struct rte_eth_dev *dev);
uint16_t eth_igb_xmit_pkts(void *txq, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t eth_igb_prep_pkts(void *txq, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
uint16_t eth_igb_recv_pkts(void *rxq, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
@@ -376,6 +384,9 @@ void eth_em_tx_init(struct rte_eth_dev *dev);
uint16_t eth_em_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t eth_em_prep_pkts(void *txq, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
uint16_t eth_em_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
uint16_t nb_pkts);
diff --git a/drivers/net/e1000/em_ethdev.c b/drivers/net/e1000/em_ethdev.c
index c5bf294..46515d4 100644
--- a/drivers/net/e1000/em_ethdev.c
+++ b/drivers/net/e1000/em_ethdev.c
@@ -1,7 +1,7 @@
/*-
* BSD LICENSE
*
- * Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+ * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -300,6 +300,7 @@ eth_em_dev_init(struct rte_eth_dev *eth_dev)
eth_dev->dev_ops = ð_em_ops;
eth_dev->rx_pkt_burst = (eth_rx_burst_t)ð_em_recv_pkts;
eth_dev->tx_pkt_burst = (eth_tx_burst_t)ð_em_xmit_pkts;
+ eth_dev->tx_pkt_prep = (eth_tx_prep_t)ð_em_prep_pkts;
/* for secondary processes, we don't initialise any further as primary
* has already done this work. Only check we don't need a different
@@ -1073,6 +1074,8 @@ eth_em_infos_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.nb_max = E1000_MAX_RING_DESC,
.nb_min = E1000_MIN_RING_DESC,
.nb_align = EM_TXD_ALIGN,
+ .nb_seg_max = EM_TX_MAX_SEG,
+ .nb_mtu_seg_max = EM_TX_MAX_MTU_SEG,
};
dev_info->speed_capa = ETH_LINK_SPEED_10M_HD | ETH_LINK_SPEED_10M |
diff --git a/drivers/net/e1000/em_rxtx.c b/drivers/net/e1000/em_rxtx.c
index 41f51c0..44009d6 100644
--- a/drivers/net/e1000/em_rxtx.c
+++ b/drivers/net/e1000/em_rxtx.c
@@ -1,7 +1,7 @@
/*-
* BSD LICENSE
*
- * Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+ * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -66,6 +66,7 @@
#include <rte_udp.h>
#include <rte_tcp.h>
#include <rte_sctp.h>
+#include <rte_pkt.h>
#include <rte_string_fns.h>
#include "e1000_logs.h"
@@ -77,6 +78,14 @@
#define E1000_RXDCTL_GRAN 0x01000000 /* RXDCTL Granularity */
+#define E1000_TX_OFFLOAD_MASK ( \
+ PKT_TX_IP_CKSUM | \
+ PKT_TX_L4_MASK | \
+ PKT_TX_VLAN_PKT)
+
+#define E1000_TX_OFFLOAD_NOTSUP_MASK \
+ (PKT_TX_OFFLOAD_MASK ^ E1000_TX_OFFLOAD_MASK)
+
/**
* Structure associated with each descriptor of the RX ring of a RX queue.
*/
@@ -618,6 +627,43 @@ end_of_tx:
/*********************************************************************
*
+ * TX prep functions
+ *
+ **********************************************************************/
+uint16_t
+eth_em_prep_pkts(void *tx_queue __rte_unused, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+
+ if (m->ol_flags & E1000_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = -EINVAL;
+ return i;
+ }
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = ret;
+ return i;
+ }
+#endif
+ ret = rte_phdr_cksum_fix(m);
+ if (ret != 0) {
+ rte_errno = ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
+/*********************************************************************
+ *
* RX functions
*
**********************************************************************/
diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethdev.c
index f7cfa18..5baacd0 100644
--- a/drivers/net/e1000/igb_ethdev.c
+++ b/drivers/net/e1000/igb_ethdev.c
@@ -369,6 +369,8 @@ static const struct rte_eth_desc_lim tx_desc_lim = {
.nb_max = E1000_MAX_RING_DESC,
.nb_min = E1000_MIN_RING_DESC,
.nb_align = IGB_RXD_ALIGN,
+ .nb_seg_max = IGB_TX_MAX_SEG,
+ .nb_mtu_seg_max = IGB_TX_MAX_MTU_SEG,
};
static const struct eth_dev_ops eth_igb_ops = {
@@ -760,6 +762,7 @@ eth_igb_dev_init(struct rte_eth_dev *eth_dev)
eth_dev->dev_ops = ð_igb_ops;
eth_dev->rx_pkt_burst = ð_igb_recv_pkts;
eth_dev->tx_pkt_burst = ð_igb_xmit_pkts;
+ eth_dev->tx_pkt_prep = ð_igb_prep_pkts;
/* for secondary processes, we don't initialise any further as primary
* has already done this work. Only check we don't need a different
@@ -963,6 +966,7 @@ eth_igbvf_dev_init(struct rte_eth_dev *eth_dev)
eth_dev->dev_ops = &igbvf_eth_dev_ops;
eth_dev->rx_pkt_burst = ð_igb_recv_pkts;
eth_dev->tx_pkt_burst = ð_igb_xmit_pkts;
+ eth_dev->tx_pkt_prep = ð_igb_prep_pkts;
/* for secondary processes, we don't initialise any further as primary
* has already done this work. Only check we don't need a different
diff --git a/drivers/net/e1000/igb_rxtx.c b/drivers/net/e1000/igb_rxtx.c
index a47c640..d466888 100644
--- a/drivers/net/e1000/igb_rxtx.c
+++ b/drivers/net/e1000/igb_rxtx.c
@@ -1,7 +1,7 @@
/*-
* BSD LICENSE
*
- * Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+ * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -65,6 +65,7 @@
#include <rte_udp.h>
#include <rte_tcp.h>
#include <rte_sctp.h>
+#include <rte_pkt.h>
#include <rte_string_fns.h>
#include "e1000_logs.h"
@@ -78,6 +79,9 @@
PKT_TX_L4_MASK | \
PKT_TX_TCP_SEG)
+#define IGB_TX_OFFLOAD_NOTSUP_MASK \
+ (PKT_TX_OFFLOAD_MASK ^ IGB_TX_OFFLOAD_MASK)
+
/**
* Structure associated with each descriptor of the RX ring of a RX queue.
*/
@@ -616,6 +620,51 @@ eth_igb_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
/*********************************************************************
*
+ * TX prep functions
+ *
+ **********************************************************************/
+uint16_t
+eth_igb_prep_pkts(void *tx_queue __rte_unused, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+
+ /* Check some limitations for TSO in hardware */
+ if (m->ol_flags & PKT_TX_TCP_SEG)
+ if ((m->tso_segsz > IGB_TSO_MAX_MSS) || (m->l2_len + m->l3_len +
+ m->l4_len > IGB_TSO_MAX_HDRLEN)) {
+ rte_errno = -EINVAL;
+ return i;
+ }
+
+ if (m->ol_flags & IGB_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = -EINVAL;
+ return i;
+ }
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = ret;
+ return i;
+ }
+#endif
+ ret = rte_phdr_cksum_fix(m);
+ if (ret != 0) {
+ rte_errno = ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
+/*********************************************************************
+ *
* RX functions
*
**********************************************************************/
@@ -1362,6 +1411,7 @@ eth_igb_tx_queue_setup(struct rte_eth_dev *dev,
igb_reset_tx_queue(txq, dev);
dev->tx_pkt_burst = eth_igb_xmit_pkts;
+ dev->tx_pkt_prep = ð_igb_prep_pkts;
dev->data->tx_queues[queue_idx] = txq;
return 0;
--
1.7.9.5
^ permalink raw reply related
* [PATCH v4 3/6] fm10k: add Tx preparation
From: Tomasz Kulasek @ 2016-09-30 9:00 UTC (permalink / raw)
To: dev; +Cc: konstantin.ananyev, Tomasz Kulasek
In-Reply-To: <20160930090039.10164-1-tomaszx.kulasek@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
---
drivers/net/fm10k/fm10k.h | 6 +++++
drivers/net/fm10k/fm10k_ethdev.c | 5 ++++
drivers/net/fm10k/fm10k_rxtx.c | 50 +++++++++++++++++++++++++++++++++++++-
3 files changed, 60 insertions(+), 1 deletion(-)
diff --git a/drivers/net/fm10k/fm10k.h b/drivers/net/fm10k/fm10k.h
index 05aa1a2..c6fed21 100644
--- a/drivers/net/fm10k/fm10k.h
+++ b/drivers/net/fm10k/fm10k.h
@@ -69,6 +69,9 @@
#define FM10K_MAX_RX_DESC (FM10K_MAX_RX_RING_SZ / sizeof(union fm10k_rx_desc))
#define FM10K_MAX_TX_DESC (FM10K_MAX_TX_RING_SZ / sizeof(struct fm10k_tx_desc))
+#define FM10K_TX_MAX_SEG UINT8_MAX
+#define FM10K_TX_MAX_MTU_SEG UINT8_MAX
+
/*
* byte aligment for HW RX data buffer
* Datasheet requires RX buffer addresses shall either be 512-byte aligned or
@@ -356,6 +359,9 @@ fm10k_dev_rx_descriptor_done(void *rx_queue, uint16_t offset);
uint16_t fm10k_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t fm10k_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
int fm10k_rxq_vec_setup(struct fm10k_rx_queue *rxq);
int fm10k_rx_vec_condition_check(struct rte_eth_dev *);
void fm10k_rx_queue_release_mbufs_vec(struct fm10k_rx_queue *rxq);
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 0ecc167..8dacba7 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -1441,6 +1441,8 @@ fm10k_dev_infos_get(struct rte_eth_dev *dev,
.nb_max = FM10K_MAX_TX_DESC,
.nb_min = FM10K_MIN_TX_DESC,
.nb_align = FM10K_MULT_TX_DESC,
+ .nb_seg_max = FM10K_TX_MAX_SEG,
+ .nb_mtu_seg_max = FM10K_TX_MAX_MTU_SEG,
};
dev_info->speed_capa = ETH_LINK_SPEED_1G | ETH_LINK_SPEED_2_5G |
@@ -2749,8 +2751,10 @@ fm10k_set_tx_function(struct rte_eth_dev *dev)
fm10k_txq_vec_setup(txq);
}
dev->tx_pkt_burst = fm10k_xmit_pkts_vec;
+ dev->tx_pkt_prep = NULL;
} else {
dev->tx_pkt_burst = fm10k_xmit_pkts;
+ dev->tx_pkt_prep = fm10k_prep_pkts;
PMD_INIT_LOG(DEBUG, "Use regular Tx func");
}
}
@@ -2829,6 +2833,7 @@ eth_fm10k_dev_init(struct rte_eth_dev *dev)
dev->dev_ops = &fm10k_eth_dev_ops;
dev->rx_pkt_burst = &fm10k_recv_pkts;
dev->tx_pkt_burst = &fm10k_xmit_pkts;
+ dev->tx_pkt_prep = &fm10k_prep_pkts;
/* only initialize in the primary process */
if (rte_eal_process_type() != RTE_PROC_PRIMARY)
diff --git a/drivers/net/fm10k/fm10k_rxtx.c b/drivers/net/fm10k/fm10k_rxtx.c
index 5b2d04b..fa5bf9c 100644
--- a/drivers/net/fm10k/fm10k_rxtx.c
+++ b/drivers/net/fm10k/fm10k_rxtx.c
@@ -1,7 +1,7 @@
/*-
* BSD LICENSE
*
- * Copyright(c) 2013-2015 Intel Corporation. All rights reserved.
+ * Copyright(c) 2013-2016 Intel Corporation. All rights reserved.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -35,6 +35,7 @@
#include <rte_ethdev.h>
#include <rte_common.h>
+#include <rte_pkt.h>
#include "fm10k.h"
#include "base/fm10k_type.h"
@@ -65,6 +66,15 @@ static inline void dump_rxd(union fm10k_rx_desc *rxd)
}
#endif
+#define FM10K_TX_OFFLOAD_MASK ( \
+ PKT_TX_VLAN_PKT | \
+ PKT_TX_IP_CKSUM | \
+ PKT_TX_L4_MASK | \
+ PKT_TX_TCP_SEG)
+
+#define FM10K_TX_OFFLOAD_NOTSUP_MASK \
+ (PKT_TX_OFFLOAD_MASK ^ FM10K_TX_OFFLOAD_MASK)
+
/* @note: When this function is changed, make corresponding change to
* fm10k_dev_supported_ptypes_get()
*/
@@ -583,3 +593,41 @@ fm10k_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
return count;
}
+
+uint16_t
+fm10k_prep_pkts(void *tx_queue __rte_unused, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+
+ if ((m->ol_flags & PKT_TX_TCP_SEG) &&
+ (m->tso_segsz < FM10K_TSO_MINMSS)) {
+ rte_errno = -EINVAL;
+ return i;
+ }
+
+ if (m->ol_flags & FM10K_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = -EINVAL;
+ return i;
+ }
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = ret;
+ return i;
+ }
+#endif
+ ret = rte_phdr_cksum_fix(m);
+ if (ret != 0) {
+ rte_errno = ret;
+ return i;
+ }
+ }
+
+ return i;
+}
--
1.7.9.5
^ permalink raw reply related
* [PATCH v4 4/6] i40e: add Tx preparation
From: Tomasz Kulasek @ 2016-09-30 9:00 UTC (permalink / raw)
To: dev; +Cc: konstantin.ananyev, Tomasz Kulasek
In-Reply-To: <20160930090039.10164-1-tomaszx.kulasek@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
---
drivers/net/i40e/i40e_ethdev.c | 3 ++
drivers/net/i40e/i40e_rxtx.c | 72 +++++++++++++++++++++++++++++++++++++++-
drivers/net/i40e/i40e_rxtx.h | 8 +++++
3 files changed, 82 insertions(+), 1 deletion(-)
diff --git a/drivers/net/i40e/i40e_ethdev.c b/drivers/net/i40e/i40e_ethdev.c
index b04c833..c1ee7e6 100644
--- a/drivers/net/i40e/i40e_ethdev.c
+++ b/drivers/net/i40e/i40e_ethdev.c
@@ -948,6 +948,7 @@ eth_i40e_dev_init(struct rte_eth_dev *dev)
dev->dev_ops = &i40e_eth_dev_ops;
dev->rx_pkt_burst = i40e_recv_pkts;
dev->tx_pkt_burst = i40e_xmit_pkts;
+ dev->tx_pkt_prep = i40e_prep_pkts;
/* for secondary processes, we don't initialise any further as primary
* has already done this work. Only check we don't need a different
@@ -2614,6 +2615,8 @@ i40e_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
.nb_max = I40E_MAX_RING_DESC,
.nb_min = I40E_MIN_RING_DESC,
.nb_align = I40E_ALIGN_RING_DESC,
+ .nb_seg_max = I40E_TX_MAX_SEG,
+ .nb_mtu_seg_max = I40E_TX_MAX_MTU_SEG,
};
if (pf->flags & I40E_FLAG_VMDQ) {
diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c
index 554d167..bb69175 100644
--- a/drivers/net/i40e/i40e_rxtx.c
+++ b/drivers/net/i40e/i40e_rxtx.c
@@ -1,7 +1,7 @@
/*-
* BSD LICENSE
*
- * Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+ * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -50,6 +50,8 @@
#include <rte_tcp.h>
#include <rte_sctp.h>
#include <rte_udp.h>
+#include <rte_ip.h>
+#include <rte_pkt.h>
#include "i40e_logs.h"
#include "base/i40e_prototype.h"
@@ -79,6 +81,17 @@
PKT_TX_TCP_SEG | \
PKT_TX_OUTER_IP_CKSUM)
+#define I40E_TX_OFFLOAD_MASK ( \
+ PKT_TX_IP_CKSUM | \
+ PKT_TX_L4_MASK | \
+ PKT_TX_OUTER_IP_CKSUM | \
+ PKT_TX_TCP_SEG | \
+ PKT_TX_QINQ_PKT | \
+ PKT_TX_VLAN_PKT)
+
+#define I40E_TX_OFFLOAD_NOTSUP_MASK \
+ (PKT_TX_OFFLOAD_MASK ^ I40E_TX_OFFLOAD_MASK)
+
static uint16_t i40e_xmit_pkts_simple(void *tx_queue,
struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
@@ -1930,6 +1943,61 @@ i40e_xmit_pkts_simple(void *tx_queue,
return nb_tx;
}
+/*********************************************************************
+ *
+ * TX prep functions
+ *
+ **********************************************************************/
+uint16_t
+i40e_prep_pkts(void *tx_queue __rte_unused, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts)
+{
+ int i, ret;
+ uint64_t ol_flags;
+ struct rte_mbuf *m;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+ ol_flags = m->ol_flags;
+
+ /**
+ * m->nb_segs is uint8_t, so m->nb_segs is always less than
+ * I40E_TX_MAX_SEG.
+ * We check only a condition for m->nb_segs > I40E_TX_MAX_MTU_SEG.
+ */
+ if (!(ol_flags & PKT_TX_TCP_SEG)) {
+ if (m->nb_segs > I40E_TX_MAX_MTU_SEG) {
+ rte_errno = -1;
+ return i;
+ }
+ } else if ((m->tso_segsz < I40E_MIN_TSO_MSS) ||
+ (m->tso_segsz > I40E_MAX_TSO_MSS)) {
+ /* MSS outside the range (256B - 9674B) are considered malicious */
+ rte_errno = -EINVAL;
+ return i;
+ }
+
+ if (ol_flags & I40E_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = -EINVAL;
+ return i;
+ }
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = ret;
+ return i;
+ }
+#endif
+ ret = rte_phdr_cksum_fix(m);
+ if (ret != 0) {
+ rte_errno = ret;
+ return i;
+ }
+ }
+ return i;
+}
+
/*
* Find the VSI the queue belongs to. 'queue_idx' is the queue index
* application used, which assume having sequential ones. But from driver's
@@ -3271,9 +3339,11 @@ i40e_set_tx_function(struct rte_eth_dev *dev)
PMD_INIT_LOG(DEBUG, "Simple tx finally be used.");
dev->tx_pkt_burst = i40e_xmit_pkts_simple;
}
+ dev->tx_pkt_prep = NULL;
} else {
PMD_INIT_LOG(DEBUG, "Xmit tx finally be used.");
dev->tx_pkt_burst = i40e_xmit_pkts;
+ dev->tx_pkt_prep = i40e_prep_pkts;
}
}
diff --git a/drivers/net/i40e/i40e_rxtx.h b/drivers/net/i40e/i40e_rxtx.h
index 98179f0..e7eb89c 100644
--- a/drivers/net/i40e/i40e_rxtx.h
+++ b/drivers/net/i40e/i40e_rxtx.h
@@ -63,6 +63,12 @@
#define I40E_MIN_RING_DESC 64
#define I40E_MAX_RING_DESC 4096
+#define I40E_MIN_TSO_MSS 256
+#define I40E_MAX_TSO_MSS 9674
+
+#define I40E_TX_MAX_SEG UINT8_MAX
+#define I40E_TX_MAX_MTU_SEG 8
+
#undef container_of
#define container_of(ptr, type, member) ({ \
typeof(((type *)0)->member)(*__mptr) = (ptr); \
@@ -223,6 +229,8 @@ uint16_t i40e_recv_scattered_pkts(void *rx_queue,
uint16_t i40e_xmit_pkts(void *tx_queue,
struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t i40e_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
int i40e_tx_queue_init(struct i40e_tx_queue *txq);
int i40e_rx_queue_init(struct i40e_rx_queue *rxq);
void i40e_free_tx_resources(struct i40e_tx_queue *txq);
--
1.7.9.5
^ permalink raw reply related
* [PATCH v4 5/6] ixgbe: add Tx preparation
From: Tomasz Kulasek @ 2016-09-30 9:00 UTC (permalink / raw)
To: dev; +Cc: konstantin.ananyev, Tomasz Kulasek
In-Reply-To: <20160930090039.10164-1-tomaszx.kulasek@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
---
drivers/net/ixgbe/ixgbe_ethdev.c | 3 ++
drivers/net/ixgbe/ixgbe_ethdev.h | 5 +++-
drivers/net/ixgbe/ixgbe_rxtx.c | 56 +++++++++++++++++++++++++++++++++++++-
drivers/net/ixgbe/ixgbe_rxtx.h | 2 ++
4 files changed, 64 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 73a406b..fa6f045 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -515,6 +515,8 @@ static const struct rte_eth_desc_lim tx_desc_lim = {
.nb_max = IXGBE_MAX_RING_DESC,
.nb_min = IXGBE_MIN_RING_DESC,
.nb_align = IXGBE_TXD_ALIGN,
+ .nb_seg_max = IXGBE_TX_MAX_SEG,
+ .nb_mtu_seg_max = IXGBE_TX_MAX_SEG,
};
static const struct eth_dev_ops ixgbe_eth_dev_ops = {
@@ -1101,6 +1103,7 @@ eth_ixgbe_dev_init(struct rte_eth_dev *eth_dev)
eth_dev->dev_ops = &ixgbe_eth_dev_ops;
eth_dev->rx_pkt_burst = &ixgbe_recv_pkts;
eth_dev->tx_pkt_burst = &ixgbe_xmit_pkts;
+ eth_dev->tx_pkt_prep = &ixgbe_prep_pkts;
/*
* For secondary processes, we don't initialise any further as primary
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.h b/drivers/net/ixgbe/ixgbe_ethdev.h
index 4ff6338..e229cf5 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.h
+++ b/drivers/net/ixgbe/ixgbe_ethdev.h
@@ -1,7 +1,7 @@
/*-
* BSD LICENSE
*
- * Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+ * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -396,6 +396,9 @@ uint16_t ixgbe_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t ixgbe_xmit_pkts_simple(void *tx_queue, struct rte_mbuf **tx_pkts,
uint16_t nb_pkts);
+uint16_t ixgbe_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts,
+ uint16_t nb_pkts);
+
int ixgbe_dev_rss_hash_update(struct rte_eth_dev *dev,
struct rte_eth_rss_conf *rss_conf);
diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c
index 8b99282..a0caa74 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx.c
+++ b/drivers/net/ixgbe/ixgbe_rxtx.c
@@ -1,7 +1,7 @@
/*-
* BSD LICENSE
*
- * Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+ * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
* Copyright 2014 6WIND S.A.
* All rights reserved.
*
@@ -70,6 +70,7 @@
#include <rte_string_fns.h>
#include <rte_errno.h>
#include <rte_ip.h>
+#include <rte_pkt.h>
#include "ixgbe_logs.h"
#include "base/ixgbe_api.h"
@@ -87,6 +88,9 @@
PKT_TX_TCP_SEG | \
PKT_TX_OUTER_IP_CKSUM)
+#define IXGBE_TX_OFFLOAD_NOTSUP_MASK \
+ (PKT_TX_OFFLOAD_MASK ^ IXGBE_TX_OFFLOAD_MASK)
+
#if 1
#define RTE_PMD_USE_PREFETCH
#endif
@@ -905,6 +909,54 @@ end_of_tx:
/*********************************************************************
*
+ * TX prep functions
+ *
+ **********************************************************************/
+uint16_t
+ixgbe_prep_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
+{
+ int i, ret;
+ struct rte_mbuf *m;
+ struct ixgbe_tx_queue *txq = (struct ixgbe_tx_queue *)tx_queue;
+
+ for (i = 0; i < nb_pkts; i++) {
+ m = tx_pkts[i];
+
+ /**
+ * Check if packet meets requirements for number of segments
+ *
+ * NOTE: for ixgbe it's always (40 - WTHRESH) for both TSO and non-TSO
+ */
+
+ if (m->nb_segs > IXGBE_TX_MAX_SEG - txq->wthresh) {
+ rte_errno = -EINVAL;
+ return i;
+ }
+
+ if (m->ol_flags & IXGBE_TX_OFFLOAD_NOTSUP_MASK) {
+ rte_errno = -EINVAL;
+ return i;
+ }
+
+#ifdef RTE_LIBRTE_ETHDEV_DEBUG
+ ret = rte_validate_tx_offload(m);
+ if (ret != 0) {
+ rte_errno = ret;
+ return i;
+ }
+#endif
+ ret = rte_phdr_cksum_fix(m);
+ if (ret != 0) {
+ rte_errno = ret;
+ return i;
+ }
+ }
+
+ return i;
+}
+
+/*********************************************************************
+ *
* RX functions
*
**********************************************************************/
@@ -2280,6 +2332,7 @@ ixgbe_set_tx_function(struct rte_eth_dev *dev, struct ixgbe_tx_queue *txq)
if (((txq->txq_flags & IXGBE_SIMPLE_FLAGS) == IXGBE_SIMPLE_FLAGS)
&& (txq->tx_rs_thresh >= RTE_PMD_IXGBE_TX_MAX_BURST)) {
PMD_INIT_LOG(DEBUG, "Using simple tx code path");
+ dev->tx_pkt_prep = NULL;
#ifdef RTE_IXGBE_INC_VECTOR
if (txq->tx_rs_thresh <= RTE_IXGBE_TX_MAX_FREE_BUF_SZ &&
(rte_eal_process_type() != RTE_PROC_PRIMARY ||
@@ -2300,6 +2353,7 @@ ixgbe_set_tx_function(struct rte_eth_dev *dev, struct ixgbe_tx_queue *txq)
(unsigned long)txq->tx_rs_thresh,
(unsigned long)RTE_PMD_IXGBE_TX_MAX_BURST);
dev->tx_pkt_burst = ixgbe_xmit_pkts;
+ dev->tx_pkt_prep = ixgbe_prep_pkts;
}
}
diff --git a/drivers/net/ixgbe/ixgbe_rxtx.h b/drivers/net/ixgbe/ixgbe_rxtx.h
index 2608b36..7bbd9b8 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx.h
+++ b/drivers/net/ixgbe/ixgbe_rxtx.h
@@ -80,6 +80,8 @@
#define RTE_IXGBE_WAIT_100_US 100
#define RTE_IXGBE_VMTXSW_REGISTER_COUNT 2
+#define IXGBE_TX_MAX_SEG 40
+
#define IXGBE_PACKET_TYPE_MASK_82599 0X7F
#define IXGBE_PACKET_TYPE_MASK_X550 0X10FF
#define IXGBE_PACKET_TYPE_MASK_TUNNEL 0XFF
--
1.7.9.5
^ permalink raw reply related
* [PATCH v4 6/6] testpmd: use Tx preparation in csum engine
From: Tomasz Kulasek @ 2016-09-30 9:00 UTC (permalink / raw)
To: dev; +Cc: konstantin.ananyev, Tomasz Kulasek
In-Reply-To: <20160930090039.10164-1-tomaszx.kulasek@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
---
app/test-pmd/csumonly.c | 97 ++++++++++++++++++++++++++---------------------
1 file changed, 54 insertions(+), 43 deletions(-)
diff --git a/app/test-pmd/csumonly.c b/app/test-pmd/csumonly.c
index 21cb78f..8fcf814 100644
--- a/app/test-pmd/csumonly.c
+++ b/app/test-pmd/csumonly.c
@@ -1,7 +1,7 @@
/*-
* BSD LICENSE
*
- * Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+ * Copyright(c) 2010-2016 Intel Corporation. All rights reserved.
* Copyright 2014 6WIND S.A.
* All rights reserved.
*
@@ -110,15 +110,6 @@ struct simple_gre_hdr {
} __attribute__((__packed__));
static uint16_t
-get_psd_sum(void *l3_hdr, uint16_t ethertype, uint64_t ol_flags)
-{
- if (ethertype == _htons(ETHER_TYPE_IPv4))
- return rte_ipv4_phdr_cksum(l3_hdr, ol_flags);
- else /* assume ethertype == ETHER_TYPE_IPv6 */
- return rte_ipv6_phdr_cksum(l3_hdr, ol_flags);
-}
-
-static uint16_t
get_udptcp_checksum(void *l3_hdr, void *l4_hdr, uint16_t ethertype)
{
if (ethertype == _htons(ETHER_TYPE_IPv4))
@@ -368,11 +359,9 @@ process_inner_cksums(void *l3_hdr, const struct testpmd_offload_info *info,
/* do not recalculate udp cksum if it was 0 */
if (udp_hdr->dgram_cksum != 0) {
udp_hdr->dgram_cksum = 0;
- if (testpmd_ol_flags & TESTPMD_TX_OFFLOAD_UDP_CKSUM) {
+ if (testpmd_ol_flags & TESTPMD_TX_OFFLOAD_UDP_CKSUM)
ol_flags |= PKT_TX_UDP_CKSUM;
- udp_hdr->dgram_cksum = get_psd_sum(l3_hdr,
- info->ethertype, ol_flags);
- } else {
+ else {
udp_hdr->dgram_cksum =
get_udptcp_checksum(l3_hdr, udp_hdr,
info->ethertype);
@@ -381,15 +370,11 @@ process_inner_cksums(void *l3_hdr, const struct testpmd_offload_info *info,
} else if (info->l4_proto == IPPROTO_TCP) {
tcp_hdr = (struct tcp_hdr *)((char *)l3_hdr + info->l3_len);
tcp_hdr->cksum = 0;
- if (info->tso_segsz != 0) {
+ if (info->tso_segsz != 0)
ol_flags |= PKT_TX_TCP_SEG;
- tcp_hdr->cksum = get_psd_sum(l3_hdr, info->ethertype,
- ol_flags);
- } else if (testpmd_ol_flags & TESTPMD_TX_OFFLOAD_TCP_CKSUM) {
+ else if (testpmd_ol_flags & TESTPMD_TX_OFFLOAD_TCP_CKSUM)
ol_flags |= PKT_TX_TCP_CKSUM;
- tcp_hdr->cksum = get_psd_sum(l3_hdr, info->ethertype,
- ol_flags);
- } else {
+ else {
tcp_hdr->cksum =
get_udptcp_checksum(l3_hdr, tcp_hdr,
info->ethertype);
@@ -639,7 +624,8 @@ pkt_burst_checksum_forward(struct fwd_stream *fs)
void *l3_hdr = NULL, *outer_l3_hdr = NULL; /* can be IPv4 or IPv6 */
uint16_t nb_rx;
uint16_t nb_tx;
- uint16_t i;
+ uint16_t nb_prep;
+ uint16_t i, n;
uint64_t ol_flags;
uint16_t testpmd_ol_flags;
uint32_t retry;
@@ -847,31 +833,56 @@ pkt_burst_checksum_forward(struct fwd_stream *fs)
printf("\n");
}
}
- nb_tx = rte_eth_tx_burst(fs->tx_port, fs->tx_queue, pkts_burst, nb_rx);
- /*
- * Retry if necessary
- */
- if (unlikely(nb_tx < nb_rx) && fs->retry_enabled) {
- retry = 0;
- while (nb_tx < nb_rx && retry++ < burst_tx_retry_num) {
- rte_delay_us(burst_tx_delay_time);
- nb_tx += rte_eth_tx_burst(fs->tx_port, fs->tx_queue,
- &pkts_burst[nb_tx], nb_rx - nb_tx);
+
+ n = 0;
+
+ do {
+ nb_prep = rte_eth_tx_prep(fs->tx_port, fs->tx_queue, &pkts_burst[n],
+ nb_rx - n);
+
+ if (nb_prep != nb_rx - n) {
+ printf("Preparing packet burst to transmit failed: %s\n",
+ rte_strerror(rte_errno));
+ /* Drop malicious packet */
+ rte_pktmbuf_free(pkts_burst[n + nb_prep]);
+ fs->fwd_dropped++;
+ }
+
+ nb_tx = rte_eth_tx_burst(fs->tx_port, fs->tx_queue, &pkts_burst[n],
+ nb_prep);
+
+ /*
+ * Retry if necessary
+ */
+ if (unlikely(nb_tx < nb_prep) && fs->retry_enabled) {
+ retry = 0;
+ while ((nb_tx < nb_prep) && (retry++ < burst_tx_retry_num)) {
+ rte_delay_us(burst_tx_delay_time);
+ nb_tx += rte_eth_tx_burst(fs->tx_port, fs->tx_queue,
+ &pkts_burst[nb_tx + n], nb_prep - nb_tx);
+ }
}
- }
- fs->tx_packets += nb_tx;
- fs->rx_bad_ip_csum += rx_bad_ip_csum;
- fs->rx_bad_l4_csum += rx_bad_l4_csum;
+
+ fs->tx_packets += nb_tx;
#ifdef RTE_TEST_PMD_RECORD_BURST_STATS
- fs->tx_burst_stats.pkt_burst_spread[nb_tx]++;
+ fs->tx_burst_stats.pkt_burst_spread[nb_tx]++;
#endif
- if (unlikely(nb_tx < nb_rx)) {
- fs->fwd_dropped += (nb_rx - nb_tx);
- do {
- rte_pktmbuf_free(pkts_burst[nb_tx]);
- } while (++nb_tx < nb_rx);
- }
+ if (unlikely(nb_tx < nb_prep)) {
+ fs->fwd_dropped += (nb_prep - nb_tx);
+ do {
+ rte_pktmbuf_free(pkts_burst[nb_tx]);
+ } while (++nb_tx < nb_prep);
+ }
+
+ /* If tx_prep failed, skip malicious packet */
+ n += (nb_prep + 1);
+
+ } while (n < nb_rx);
+
+ fs->rx_bad_ip_csum += rx_bad_ip_csum;
+ fs->rx_bad_l4_csum += rx_bad_l4_csum;
+
#ifdef RTE_TEST_PMD_RECORD_CORE_CYCLES
end_tsc = rte_rdtsc();
core_cycles = (end_tsc - start_tsc);
--
1.7.9.5
^ permalink raw reply related
* Re: [PATCH] drivers/i40e: fix the hash filter invalid calculation in X722
From: Ferruh Yigit @ 2016-09-30 9:09 UTC (permalink / raw)
To: Wu, Jingjing, Guo, Jia, Zhang, Helin; +Cc: dev@dpdk.org
In-Reply-To: <9BB6961774997848B5B42BEC655768F80E274D11@SHSMSX103.ccr.corp.intel.com>
Hi Jingjing,
On 9/30/2016 7:05 AM, Wu, Jingjing wrote:
>
>
>> -----Original Message-----
>> From: Yigit, Ferruh
>> Sent: Friday, September 30, 2016 2:16 AM
>> To: Guo, Jia; Zhang, Helin; Wu, Jingjing
>> Cc: dev@dpdk.org
>> Subject: Re: [dpdk-dev] [PATCH] drivers/i40e: fix the hash filter invalid
>> calculation in X722
>>
>> On 9/26/2016 11:51 AM, Jeff Guo wrote:
>>> As X722 extracts IPv4 header to Field Vector different with
>>> XL710/X710, need to corresponding to modify the fields of IPv4 header
>>> in input set to map different default Field Vector Table of different nics.
>>>
>>> Signed-off-by: Jeff Guo <jia.guo@intel.com>
<...>
>>> +#ifdef X722_SUPPORT
>>> +/* Source IPv4 address for X722 */
>>> +#define I40E_X722_REG_INSET_L3_SRC_IP4 0x0006000000000000ULL
>>
>> These macros defined here within "#ifdef X722_SUPPORT" and later used
>> unconditionally, this will cause a compile error when "X722_SUPPORT" not
>> defined.
> These macros defined are used in condition later.
No they are not used in condition, please check that macros have been
used in inset_map1[] without an ifdef.
> Maybe the compile error
> Already exist in current driver. We need to check that and fix this.
And yes compile error already exist in driver! but lets not add a new ones.
>>
>>> +/* Destination IPv4 address for X722 */
>>> +#define I40E_X722_REG_INSET_L3_DST_IP4 0x0000060000000000ULL
>>> +/* IPv4 Type of Service (TOS) */
>>> +#define I40E_X722_REG_INSET_L3_IP4_TOS 0x0040000000000000ULL
>>
>> This value seems same as I40E_REG_INSET_L3_IP4_TOS, why creating a X722
>> version of this?
>>
>>> +/* IPv4 Protocol */
>>> +#define I40E_X722_REG_INSET_L3_IP4_PROTO
>> 0x0010000000000000ULL
>>> +/* IPv4 Time to Live */
>>> +#define I40E_X722_REG_INSET_L3_IP4_TTL 0x0010000000000000ULL
>>> +#endif
Overall, these new values seems like not conflicting with existing
values, can you please confirm that, if this is the case why not define
these values without ifdef?
<...>
>>
>> ---->
>>> + {I40E_INSET_IPV4_SRC, I40E_X722_REG_INSET_L3_SRC_IP4},
>>> + {I40E_INSET_IPV4_DST, I40E_X722_REG_INSET_L3_DST_IP4},
>>> + {I40E_INSET_IPV4_TOS, I40E_X722_REG_INSET_L3_IP4_TOS},
>>> + {I40E_INSET_IPV4_PROTO,
>> I40E_X722_REG_INSET_L3_IP4_PROTO},
>>> + {I40E_INSET_IPV4_TTL, I40E_X722_REG_INSET_L3_IP4_TTL},
>> <----
>> Since limited number of values are different from inset_map[], and most of
>> them are duplication, is it possible to prevent duplication?
> Didn't find and proposal on that. Because we need to support
> I40E_X722_REG_INSET_XX and I40E_REG_INSET_XX at the same time. So it
> Cannot be achieved by #ifdef and #endif.
It doesn't have to be with #ifdef #endif, whole array duplicated for 5
changes, there must be a way to remove duplication. First thing I can
think of is extracting common part into a new array?
<...>
>>> /* Translate input set to register aware inset */
>>> +#ifdef X722_SUPPORT
>>> + if (type == I40E_MAC_X722) {
>>> + for (i = 0; i < RTE_DIM(inset_map1); i++) {
>>> + if (input & inset_map1[i].inset)
>>> + val |= inset_map1[i].inset_reg;
>>> + }
>>> + } else {
>>> + for (i = 0; i < RTE_DIM(inset_map); i++) {
>>> + if (input & inset_map[i].inset)
>>> + val |= inset_map[i].inset_reg;
>>> + }
>>> + }
>>> +#else
>>> for (i = 0; i < RTE_DIM(inset_map); i++) {
>>> if (input & inset_map[i].inset)
>>> val |= inset_map[i].inset_reg;
>>> }
>>> +#endif
>>
>> What about something like this, to prevent duplication:
>>
>> inset_map_x = inset_map;
>>
>> #ifdef X722_SUPPORT
>> if (type == I40E_MAC_X722)
>> inset_map_x = inset_map1;
>> #endif
>>
>> for (i = 0; i < RTE_DIM(inset_map_x); i++) {
>> if (input & inset_map_x[i].inset)
>> val |= inset_map_x[i].inset_reg;
>> }
>>
> Also thought about that way, but if X722_SUPPORT is not set
> Compile error will report because of unused parameter.
I believe there won't be a unused parameter warning for above suggestion
but the intention here is to simplify the logic, which I think can be
done, it doesn't have to be like suggested way.
^ permalink raw reply
* Re: [RFC PATCH v2 3/5] librte_ether: add API's for VF management
From: Bruce Richardson @ 2016-09-30 9:21 UTC (permalink / raw)
To: Thomas Monjalon
Cc: Ananyev, Konstantin, Iremonger, Bernard, dev, Jerin Jacob,
Shah, Rahul R, Lu, Wenzhuo, azelezniak
In-Reply-To: <5655161.kOlSb9MaAr@xps13>
On Wed, Sep 28, 2016 at 08:02:21PM +0200, Thomas Monjalon wrote:
> 2016-09-28 16:52, Ananyev, Konstantin:
> >
> > >
> > > 2016-09-28 14:30, Ananyev, Konstantin:
> > > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> > > > > 2016-09-28 13:26, Ananyev, Konstantin:
> > > > > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> > > > > > > 2016-09-28 11:23, Ananyev, Konstantin:
> > > > > > > > If we this way (force user to include driver specific headers
> > > > > > > > and call driver specific functions), how you guys plan to make this functionality available for multiple driver types.
> > > > > > >
> > > > > > > Multiple drivers won't have exactly the same specific features.
> > > > > > > But yes, there are some things common to several Intel NICs.
> > > > > > >
> > > > > > > > From discussion with Bernard understand that customers would need similar functionality for i40e.
> > > > > > > > Does it mean that they'll have to re-implement this part of their code again?
> > > > > > > > Or would have to create (and maintain) their own shim layer that would provide some s of abstraction?
> > > > > > > > Basically their own version of rte_ethdev?
> > > > > > >
> > > > > > > No definitive answer.
> > > > > > > But we can argue the contrary: how to handle a generic API which
> > > > > > > is implemented only in 1 or 2 drivers? If the application tries to use it, we can imagine that a specific range of hardware is
> > > expected.
> > > > > >
> > > > > > Yes, as I understand, it is a specific subset of supported HW (just Inel NICs for now, but different models/drivers).
> > > > > > Obviously users would like to have an ability to run their app on all HW from this subset without rebuilding/implementing the
> > > app.
> > > > > >
> > > > > > >
> > > > > > > I think it is an important question.
> > > > > > > Previously we had the issue of having some API which are too
> > > > > > > specific and need a rework to be used with other NICs. In order
> > > > > > > to avoid such rework and API break, we can try to make them
> > > > > > > available in a driver-specific or vendor-specific staging area,
> > > > > > > waiting for
> > > > > a later generalization.
> > > > > >
> > > > > > Could you remind me why you guys were that opposed to ioctl style approach?
> > > > > > It is not my favorite thing either, but it seems pretty generic way to handle such situations.
> > > > >
> > > > > We prefer having well-defined functions instead of opaque ioctl-style encoding.
> > > > > And it was not clear what is the benefit of ioctl.
> > > > > Now I think I understand you would like to have a common ioctl service for features available on 2 drivers. Right?
> > > >
> > > > Yes.
> > > >
> > > > > Example (trying to read your mind):
> > > > > rte_ethdev_ioctl(port_id, <TLV encoding VF_PING service and VF id>); instead of
> > > > > rte_pmd_ixgbe_vf_ping(port_id, vf_id);
> > > > > rte_pmd_i40e_vf_ping(port_id, vf_id); Please confirm I understand
> > > > > what you are thinking about.
> > > >
> > > > Yep, you read my mind correctly :)
> > >
> > > Both could coexist (if ioctl was accepted by community).
> >
> > True.
> >
> > > What about starting to implement the PMD functions and postpone ioctl to later with a dedicated thread?
> >
> > You mean something like:
> > - 16.11: implement rte_pmd_ixgbe_vf_ping()
> > - 17.02:
> > a) implement rte_pmd_i40e_vf_ping()
> > b) introduce ioctl PMD API
> > c) make possible to vf_ping via ioctl API
> > ?
> > If so, then it sounds like reasonable approach to me.
> > Though would be inserting to hear what other guys think.
>
> Yes.
> I would just add that we have to start a discussion thread to decide
> wether we'll add an ioctl call in 17.02 or not.
I still don't like IOCTLs as an option, I still think that they are awkward to
use and hinder code readability. Here is an addition suggestion to get people
thinking.
How about allowing a set of "vendor-specific" operations for each device. We
could do this using Bernard's suggestion of adding a set of additional
function pointers to the device struct. The reason for doing this is that it
would allow us to group functionality into families of functionality, rather
than having to things either fully generically or specifically to each NIC. If
we look at the capabilities of most NICs, the other NICs which support similar
functions are generally other NICs from the same vendor. We could therefore
have ops common across ixgbe and i40e and other ops mlx4 and mlx5 drivers.
Could such a scheme work? Would it be worth doing?
/Bruce
^ permalink raw reply
* Re: [PATCH] log: do not drop debug logs at compile time
From: Thomas Monjalon @ 2016-09-30 9:33 UTC (permalink / raw)
To: dev; +Cc: Olivier Matz, david.marchand
In-Reply-To: <1474011832-29987-1-git-send-email-olivier.matz@6wind.com>
2016-09-16 09:43, Olivier Matz:
> Today, all logs whose level is lower than INFO are dropped at
> compile-time. This prevents from enabling debug logs at runtime using
> --log-level=8.
>
> The rationale was to remove debug logs from the data path at
> compile-time, avoiding a test at run-time.
>
> This patch changes the behavior of RTE_LOG() to avoid the compile-time
> optimization, and introduces the RTE_LOG_DP() macro that has the same
> behavior than the previous RTE_LOG(), for the rare cases where debug
> logs are in the data path.
>
> So it is now possible to enable debug logs at run-time by just
> specifying --log-level=8. Some drivers still have special compile-time
> options to enable more debug log. Maintainers may consider to
> remove/reduce them.
>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
I think it is a good change.
However I'm not sure we should take it for 16.11 as it was sent late and
there is no review comment.
It is neither really a fix nor really a feature.
If there are some +1, and no opinions against, it will go in 16.11.
Note that some drivers would need some changes to fully benefit of
debug logs enabled at run-time.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox