* [net-next PATCH v2 1/3] octeontx2-pf: Simplify the receive buffer size calculation
2021-10-06 7:18 [net-next PATCH v2 0/3] Add devlink params to vary cqe and rbuf Subbaraya Sundeep
@ 2021-10-06 7:18 ` Subbaraya Sundeep
2021-10-06 7:18 ` [net-next PATCH v2 2/3] octeontx2-pf: Add devlink param to vary cqe size Subbaraya Sundeep
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Subbaraya Sundeep @ 2021-10-06 7:18 UTC (permalink / raw)
To: davem, kuba, netdev; +Cc: sgoutham, hkelam, gakula, Subbaraya Sundeep
This patch separates the logic of configuring hardware
maximum transmit frame size and receive frame size.
This simplifies the logic to calculate receive buffer
size and using cqe descriptor of different size.
Also additional size of skb_shared_info structure is
allocated for each receive buffer pointer given to
hardware which is not necessary. Hence change the
size calculation to remove the size of
skb_shared_info. Add a check for array out of
bounds while adding fragments to the network stack.
Signed-off-by: Subbaraya Sundeep <sbhatta@marvell.com>
Signed-off-by: Hariprasad Kelam <hkelam@marvell.com>
Signed-off-by: Sunil Goutham <sgoutham@marvell.com>
---
drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c | 2 +-
.../ethernet/marvell/octeontx2/nic/otx2_common.c | 10 +++-----
.../ethernet/marvell/octeontx2/nic/otx2_common.h | 2 +-
.../net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 15 +++++------
.../net/ethernet/marvell/octeontx2/nic/otx2_txrx.c | 30 +++++++++++++++-------
.../net/ethernet/marvell/octeontx2/nic/otx2_txrx.h | 4 +--
6 files changed, 35 insertions(+), 28 deletions(-)
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c
index 95f21df..fd4f083 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/cn10k.c
@@ -88,7 +88,7 @@ int cn10k_sq_aq_init(void *dev, u16 qidx, u16 sqb_aura)
aq->sq.ena = 1;
/* Only one SMQ is allocated, map all SQ's to that SMQ */
aq->sq.smq = pfvf->hw.txschq_list[NIX_TXSCH_LVL_SMQ][0];
- aq->sq.smq_rr_weight = mtu_to_dwrr_weight(pfvf, pfvf->max_frs);
+ aq->sq.smq_rr_weight = mtu_to_dwrr_weight(pfvf, pfvf->tx_max_pktlen);
aq->sq.default_chan = pfvf->hw.tx_chan_base;
aq->sq.sqe_stype = NIX_STYPE_STF; /* Cache SQB */
aq->sq.sqb_aura = sqb_aura;
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c
index 0c89eb8..66da31f 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c
@@ -231,7 +231,7 @@ int otx2_hw_set_mtu(struct otx2_nic *pfvf, int mtu)
return -ENOMEM;
}
- req->maxlen = pfvf->max_frs;
+ req->maxlen = pfvf->netdev->mtu + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN;
err = otx2_sync_mbox_msg(&pfvf->mbox);
mutex_unlock(&pfvf->mbox.lock);
@@ -590,7 +590,7 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl)
u64 schq, parent;
u64 dwrr_val;
- dwrr_val = mtu_to_dwrr_weight(pfvf, pfvf->max_frs);
+ dwrr_val = mtu_to_dwrr_weight(pfvf, pfvf->tx_max_pktlen);
req = otx2_mbox_alloc_msg_nix_txschq_cfg(&pfvf->mbox);
if (!req)
@@ -603,9 +603,7 @@ int otx2_txschq_config(struct otx2_nic *pfvf, int lvl)
/* Set topology e.t.c configuration */
if (lvl == NIX_TXSCH_LVL_SMQ) {
req->reg[0] = NIX_AF_SMQX_CFG(schq);
- req->regval[0] = ((pfvf->netdev->max_mtu + OTX2_ETH_HLEN) << 8)
- | OTX2_MIN_MTU;
-
+ req->regval[0] = ((u64)pfvf->tx_max_pktlen << 8) | OTX2_MIN_MTU;
req->regval[0] |= (0x20ULL << 51) | (0x80ULL << 39) |
(0x2ULL << 36);
req->num_regs++;
@@ -800,7 +798,7 @@ int otx2_sq_aq_init(void *dev, u16 qidx, u16 sqb_aura)
aq->sq.ena = 1;
/* Only one SMQ is allocated, map all SQ's to that SMQ */
aq->sq.smq = pfvf->hw.txschq_list[NIX_TXSCH_LVL_SMQ][0];
- aq->sq.smq_rr_quantum = mtu_to_dwrr_weight(pfvf, pfvf->max_frs);
+ aq->sq.smq_rr_quantum = mtu_to_dwrr_weight(pfvf, pfvf->tx_max_pktlen);
aq->sq.default_chan = pfvf->hw.tx_chan_base;
aq->sq.sqe_stype = NIX_STYPE_STF; /* Cache SQB */
aq->sq.sqb_aura = sqb_aura;
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
index a05f6bd..61e5281 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
@@ -326,7 +326,7 @@ struct otx2_nic {
struct net_device *netdev;
struct dev_hw_ops *hw_ops;
void *iommu_domain;
- u16 max_frs;
+ u16 tx_max_pktlen;
u16 rbsize; /* Receive buffer size */
#define OTX2_FLAG_RX_TSTAMP_ENABLED BIT_ULL(0)
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
index f24e920..1e0d0c9c 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
@@ -1312,16 +1312,14 @@ static int otx2_get_rbuf_size(struct otx2_nic *pf, int mtu)
* NIX transfers entire data using 6 segments/buffers and writes
* a CQE_RX descriptor with those segment addresses. First segment
* has additional data prepended to packet. Also software omits a
- * headroom of 128 bytes and sizeof(struct skb_shared_info) in
- * each segment. Hence the total size of memory needed
- * to receive a packet with 'mtu' is:
+ * headroom of 128 bytes in each segment. Hence the total size of
+ * memory needed to receive a packet with 'mtu' is:
* frame size = mtu + additional data;
- * memory = frame_size + (headroom + struct skb_shared_info size) * 6;
+ * memory = frame_size + headroom * 6;
* each receive buffer size = memory / 6;
*/
frame_size = mtu + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN;
- total_size = frame_size + (OTX2_HEAD_ROOM +
- OTX2_DATA_ALIGN(sizeof(struct skb_shared_info))) * 6;
+ total_size = frame_size + OTX2_HEAD_ROOM * 6;
rbuf_size = total_size / 6;
return ALIGN(rbuf_size, 2048);
@@ -1343,7 +1341,8 @@ static int otx2_init_hw_resources(struct otx2_nic *pf)
hw->sqpool_cnt = hw->tot_tx_queues;
hw->pool_cnt = hw->rqpool_cnt + hw->sqpool_cnt;
- pf->max_frs = pf->netdev->mtu + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN;
+ /* Maximum hardware supported transmit length */
+ pf->tx_max_pktlen = pf->netdev->max_mtu + OTX2_ETH_HLEN;
pf->rbsize = otx2_get_rbuf_size(pf, pf->netdev->mtu);
@@ -1807,7 +1806,7 @@ static netdev_tx_t otx2_xmit(struct sk_buff *skb, struct net_device *netdev)
/* Check for minimum and maximum packet length */
if (skb->len <= ETH_HLEN ||
- (!skb_shinfo(skb)->gso_size && skb->len > pf->max_frs)) {
+ (!skb_shinfo(skb)->gso_size && skb->len > pf->tx_max_pktlen)) {
dev_kfree_skb(skb);
return NETDEV_TX_OK;
}
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
index 2d2b673..0cc6353 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c
@@ -181,8 +181,9 @@ static void otx2_set_rxtstamp(struct otx2_nic *pfvf,
skb_hwtstamps(skb)->hwtstamp = ns_to_ktime(tsns);
}
-static void otx2_skb_add_frag(struct otx2_nic *pfvf, struct sk_buff *skb,
- u64 iova, int len, struct nix_rx_parse_s *parse)
+static bool otx2_skb_add_frag(struct otx2_nic *pfvf, struct sk_buff *skb,
+ u64 iova, int len, struct nix_rx_parse_s *parse,
+ int qidx)
{
struct page *page;
int off = 0;
@@ -203,11 +204,22 @@ static void otx2_skb_add_frag(struct otx2_nic *pfvf, struct sk_buff *skb,
}
page = virt_to_page(va);
- skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page,
- va - page_address(page) + off, len - off, pfvf->rbsize);
+ if (likely(skb_shinfo(skb)->nr_frags < MAX_SKB_FRAGS)) {
+ skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, page,
+ va - page_address(page) + off,
+ len - off, pfvf->rbsize);
+
+ otx2_dma_unmap_page(pfvf, iova - OTX2_HEAD_ROOM,
+ pfvf->rbsize, DMA_FROM_DEVICE);
+ return true;
+ }
- otx2_dma_unmap_page(pfvf, iova - OTX2_HEAD_ROOM,
- pfvf->rbsize, DMA_FROM_DEVICE);
+ /* If more than MAX_SKB_FRAGS fragments are received then
+ * give back those buffer pointers to hardware for reuse.
+ */
+ pfvf->hw_ops->aura_freeptr(pfvf, qidx, iova & ~0x07ULL);
+
+ return false;
}
static void otx2_set_rxhash(struct otx2_nic *pfvf,
@@ -349,9 +361,9 @@ static void otx2_rcv_pkt_handler(struct otx2_nic *pfvf,
seg_addr = &sg->seg_addr;
seg_size = (void *)sg;
for (seg = 0; seg < sg->segs; seg++, seg_addr++) {
- otx2_skb_add_frag(pfvf, skb, *seg_addr, seg_size[seg],
- parse);
- cq->pool_ptrs++;
+ if (otx2_skb_add_frag(pfvf, skb, *seg_addr,
+ seg_size[seg], parse, cq->cq_idx))
+ cq->pool_ptrs++;
}
start += sizeof(*sg);
}
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h
index 5c05774..f1a04cf 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.h
@@ -39,9 +39,7 @@
#define RCV_FRAG_LEN(x) \
((RCV_FRAG_LEN1(x) < 2048) ? 2048 : RCV_FRAG_LEN1(x))
-#define DMA_BUFFER_LEN(x) \
- ((x) - OTX2_HEAD_ROOM - \
- OTX2_DATA_ALIGN(sizeof(struct skb_shared_info)))
+#define DMA_BUFFER_LEN(x) ((x) - OTX2_HEAD_ROOM)
/* IRQ triggered when NIX_LF_CINTX_CNT[ECOUNT]
* is equal to this value.
--
2.7.4
^ permalink raw reply related [flat|nested] 6+ messages in thread* [net-next PATCH v2 2/3] octeontx2-pf: Add devlink param to vary cqe size
2021-10-06 7:18 [net-next PATCH v2 0/3] Add devlink params to vary cqe and rbuf Subbaraya Sundeep
2021-10-06 7:18 ` [net-next PATCH v2 1/3] octeontx2-pf: Simplify the receive buffer size calculation Subbaraya Sundeep
@ 2021-10-06 7:18 ` Subbaraya Sundeep
2021-10-06 7:18 ` [net-next PATCH v2 3/3] octeontx2-pf: Add devlink param to vary rbuf size Subbaraya Sundeep
2021-10-06 13:43 ` [net-next PATCH v2 0/3] Add devlink params to vary cqe and rbuf Jakub Kicinski
3 siblings, 0 replies; 6+ messages in thread
From: Subbaraya Sundeep @ 2021-10-06 7:18 UTC (permalink / raw)
To: davem, kuba, netdev; +Cc: sgoutham, hkelam, gakula, Subbaraya Sundeep
Completion Queue Entry(CQE) is a descriptor written
by hardware to notify software about the send and
receive completion status. The CQE can be of size
128 or 512 bytes. A 512 bytes CQE can hold more receive
fragments pointers compared to 128 bytes CQE. This
patch adds devlink param to change CQE descriptor
size.
Signed-off-by: Subbaraya Sundeep <sbhatta@marvell.com>
Signed-off-by: Sunil Goutham <sgoutham@marvell.com>
---
.../ethernet/marvell/octeontx2/nic/otx2_common.c | 10 +++-
.../ethernet/marvell/octeontx2/nic/otx2_common.h | 1 +
.../ethernet/marvell/octeontx2/nic/otx2_devlink.c | 56 ++++++++++++++++++++++
.../net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 2 +
.../net/ethernet/marvell/octeontx2/nic/otx2_vf.c | 2 +
5 files changed, 69 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c
index 66da31f..3777f41 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.c
@@ -222,8 +222,11 @@ EXPORT_SYMBOL(otx2_set_mac_address);
int otx2_hw_set_mtu(struct otx2_nic *pfvf, int mtu)
{
struct nix_frs_cfg *req;
+ u16 maxlen;
int err;
+ maxlen = otx2_get_max_mtu(pfvf) + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN;
+
mutex_lock(&pfvf->mbox.lock);
req = otx2_mbox_alloc_msg_nix_set_hw_frs(&pfvf->mbox);
if (!req) {
@@ -233,6 +236,9 @@ int otx2_hw_set_mtu(struct otx2_nic *pfvf, int mtu)
req->maxlen = pfvf->netdev->mtu + OTX2_ETH_HLEN + OTX2_HW_TIMESTAMP_LEN;
+ if (is_otx2_lbkvf(pfvf->pdev))
+ req->maxlen = maxlen;
+
err = otx2_sync_mbox_msg(&pfvf->mbox);
mutex_unlock(&pfvf->mbox.lock);
return err;
@@ -1036,7 +1042,7 @@ int otx2_config_nix(struct otx2_nic *pfvf)
struct nix_lf_alloc_rsp *rsp;
int err;
- pfvf->qset.xqe_size = NIX_XQESZ_W16 ? 128 : 512;
+ pfvf->qset.xqe_size = pfvf->hw.xqe_size;
/* Get memory to put this msg */
nixlf = otx2_mbox_alloc_msg_nix_lf_alloc(&pfvf->mbox);
@@ -1049,7 +1055,7 @@ int otx2_config_nix(struct otx2_nic *pfvf)
nixlf->cq_cnt = pfvf->qset.cq_cnt;
nixlf->rss_sz = MAX_RSS_INDIR_TBL_SIZE;
nixlf->rss_grps = MAX_RSS_GROUPS;
- nixlf->xqe_sz = NIX_XQESZ_W16;
+ nixlf->xqe_sz = pfvf->hw.xqe_size == 128 ? NIX_XQESZ_W16 : NIX_XQESZ_W64;
/* We don't know absolute NPA LF idx attached.
* AF will replace 'RVU_DEFAULT_PF_FUNC' with
* NPA LF attached to this RVU PF/VF.
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
index 61e5281..6e0d1ac 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
@@ -177,6 +177,7 @@ struct otx2_hw {
u16 pool_cnt;
u16 rqpool_cnt;
u16 sqpool_cnt;
+ u16 xqe_size;
/* NPA */
u32 stack_pg_ptrs; /* No of ptrs per stack page */
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c
index 777a270..98450e1 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c
@@ -64,9 +64,60 @@ static int otx2_dl_mcam_count_get(struct devlink *devlink, u32 id,
return 0;
}
+static int otx2_dl_cqe_size_validate(struct devlink *devlink, u32 id,
+ union devlink_param_value val,
+ struct netlink_ext_ack *extack)
+{
+ if (val.vu16 != 128 && val.vu16 != 512) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Only 128 or 512 byte descriptor allowed");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int otx2_dl_cqe_size_set(struct devlink *devlink, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ struct otx2_devlink *otx2_dl = devlink_priv(devlink);
+ struct otx2_nic *pfvf = otx2_dl->pfvf;
+ struct net_device *netdev;
+ int err = 0;
+ bool if_up;
+
+ rtnl_lock();
+
+ netdev = pfvf->netdev;
+ if_up = netif_running(netdev);
+ if (if_up)
+ netdev->netdev_ops->ndo_stop(netdev);
+
+ pfvf->hw.xqe_size = ctx->val.vu16;
+
+ if (if_up)
+ err = netdev->netdev_ops->ndo_open(netdev);
+
+ rtnl_unlock();
+
+ return err;
+}
+
+static int otx2_dl_cqe_size_get(struct devlink *devlink, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ struct otx2_devlink *otx2_dl = devlink_priv(devlink);
+ struct otx2_nic *pfvf = otx2_dl->pfvf;
+
+ ctx->val.vu16 = pfvf->hw.xqe_size;
+
+ return 0;
+}
+
enum otx2_dl_param_id {
OTX2_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX,
OTX2_DEVLINK_PARAM_ID_MCAM_COUNT,
+ OTX2_DEVLINK_PARAM_ID_CQE_SIZE,
};
static const struct devlink_param otx2_dl_params[] = {
@@ -75,6 +126,11 @@ static const struct devlink_param otx2_dl_params[] = {
BIT(DEVLINK_PARAM_CMODE_RUNTIME),
otx2_dl_mcam_count_get, otx2_dl_mcam_count_set,
otx2_dl_mcam_count_validate),
+ DEVLINK_PARAM_DRIVER(OTX2_DEVLINK_PARAM_ID_CQE_SIZE,
+ "completion_descriptor_size", DEVLINK_PARAM_TYPE_U16,
+ BIT(DEVLINK_PARAM_CMODE_RUNTIME),
+ otx2_dl_cqe_size_get, otx2_dl_cqe_size_set,
+ otx2_dl_cqe_size_validate),
};
/* Devlink OPs */
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
index 1e0d0c9c..8618cf7 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
@@ -2624,6 +2624,8 @@ static int otx2_probe(struct pci_dev *pdev, const struct pci_device_id *id)
hw->tx_queues = qcount;
hw->tot_tx_queues = qcount;
hw->max_queues = qcount;
+ /* Use CQE of 128 byte descriptor size by default */
+ hw->xqe_size = 128;
num_vec = pci_msix_vec_count(pdev);
hw->irq_name = devm_kmalloc_array(&hw->pdev->dev, num_vec, NAME_SIZE,
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
index 980219a..672be05 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
@@ -587,6 +587,8 @@ static int otx2vf_probe(struct pci_dev *pdev, const struct pci_device_id *id)
hw->tx_queues = qcount;
hw->max_queues = qcount;
hw->tot_tx_queues = qcount;
+ /* Use CQE of 128 byte descriptor size by default */
+ hw->xqe_size = 128;
hw->irq_name = devm_kmalloc_array(&hw->pdev->dev, num_vec, NAME_SIZE,
GFP_KERNEL);
--
2.7.4
^ permalink raw reply related [flat|nested] 6+ messages in thread* [net-next PATCH v2 3/3] octeontx2-pf: Add devlink param to vary rbuf size
2021-10-06 7:18 [net-next PATCH v2 0/3] Add devlink params to vary cqe and rbuf Subbaraya Sundeep
2021-10-06 7:18 ` [net-next PATCH v2 1/3] octeontx2-pf: Simplify the receive buffer size calculation Subbaraya Sundeep
2021-10-06 7:18 ` [net-next PATCH v2 2/3] octeontx2-pf: Add devlink param to vary cqe size Subbaraya Sundeep
@ 2021-10-06 7:18 ` Subbaraya Sundeep
2021-10-06 13:43 ` [net-next PATCH v2 0/3] Add devlink params to vary cqe and rbuf Jakub Kicinski
3 siblings, 0 replies; 6+ messages in thread
From: Subbaraya Sundeep @ 2021-10-06 7:18 UTC (permalink / raw)
To: davem, kuba, netdev; +Cc: sgoutham, hkelam, gakula, Subbaraya Sundeep
The size of receive buffers for receiving packets
is calculated based on the mtu of interface. This
patch adds devlink parameter to utilize user given
size for receive buffers instead. By changing CQE
descriptor size and receive buffer sizes the number
of buffer pointers used by hardware for a large packet
can be configured.
Signed-off-by: Subbaraya Sundeep <sbhatta@marvell.com>
Signed-off-by: Sunil Goutham <sgoutham@marvell.com>
---
.../ethernet/marvell/octeontx2/nic/otx2_common.h | 1 +
.../ethernet/marvell/octeontx2/nic/otx2_devlink.c | 60 ++++++++++++++++++++++
.../net/ethernet/marvell/octeontx2/nic/otx2_pf.c | 7 +++
.../net/ethernet/marvell/octeontx2/nic/otx2_vf.c | 5 ++
4 files changed, 73 insertions(+)
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
index 6e0d1ac..f885abe 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h
@@ -178,6 +178,7 @@ struct otx2_hw {
u16 rqpool_cnt;
u16 sqpool_cnt;
u16 xqe_size;
+ u16 rbuf_fixed_size;
/* NPA */
u32 stack_pg_ptrs; /* No of ptrs per stack page */
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c
index 98450e1..0fc7e32 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_devlink.c
@@ -64,6 +64,60 @@ static int otx2_dl_mcam_count_get(struct devlink *devlink, u32 id,
return 0;
}
+static int otx2_dl_rbuf_size_validate(struct devlink *devlink, u32 id,
+ union devlink_param_value val,
+ struct netlink_ext_ack *extack)
+{
+ /* Hardware supports max size of 32k for a receive buffer
+ * and 1536 is typical ethernet frame size.
+ */
+ if (val.vu16 < 1536 || val.vu16 > 32768) {
+ NL_SET_ERR_MSG_MOD(extack,
+ "Receive buffer range is 1536 - 32768");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int otx2_dl_rbuf_size_set(struct devlink *devlink, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ struct otx2_devlink *otx2_dl = devlink_priv(devlink);
+ struct otx2_nic *pfvf = otx2_dl->pfvf;
+ struct net_device *netdev;
+ int err = 0;
+ bool if_up;
+
+ rtnl_lock();
+
+ netdev = pfvf->netdev;
+ if_up = netif_running(netdev);
+ if (if_up)
+ netdev->netdev_ops->ndo_stop(netdev);
+
+ pfvf->hw.rbuf_fixed_size = ALIGN(ctx->val.vu16, OTX2_ALIGN) +
+ OTX2_HEAD_ROOM;
+
+ if (if_up)
+ err = netdev->netdev_ops->ndo_open(netdev);
+
+ rtnl_unlock();
+
+ return err;
+}
+
+static int otx2_dl_rbuf_size_get(struct devlink *devlink, u32 id,
+ struct devlink_param_gset_ctx *ctx)
+{
+ struct otx2_devlink *otx2_dl = devlink_priv(devlink);
+ struct otx2_nic *pfvf = otx2_dl->pfvf;
+
+ ctx->val.vu16 = pfvf->hw.rbuf_fixed_size;
+
+ return 0;
+}
+
static int otx2_dl_cqe_size_validate(struct devlink *devlink, u32 id,
union devlink_param_value val,
struct netlink_ext_ack *extack)
@@ -118,6 +172,7 @@ enum otx2_dl_param_id {
OTX2_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX,
OTX2_DEVLINK_PARAM_ID_MCAM_COUNT,
OTX2_DEVLINK_PARAM_ID_CQE_SIZE,
+ OTX2_DEVLINK_PARAM_ID_RBUF_SIZE,
};
static const struct devlink_param otx2_dl_params[] = {
@@ -131,6 +186,11 @@ static const struct devlink_param otx2_dl_params[] = {
BIT(DEVLINK_PARAM_CMODE_RUNTIME),
otx2_dl_cqe_size_get, otx2_dl_cqe_size_set,
otx2_dl_cqe_size_validate),
+ DEVLINK_PARAM_DRIVER(OTX2_DEVLINK_PARAM_ID_RBUF_SIZE,
+ "receive_buffer_size", DEVLINK_PARAM_TYPE_U16,
+ BIT(DEVLINK_PARAM_CMODE_RUNTIME),
+ otx2_dl_rbuf_size_get, otx2_dl_rbuf_size_set,
+ otx2_dl_rbuf_size_validate),
};
/* Devlink OPs */
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
index 8618cf7..1be524d3 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c
@@ -65,6 +65,10 @@ static int otx2_change_mtu(struct net_device *netdev, int new_mtu)
netdev_info(netdev, "Changing MTU from %d to %d\n",
netdev->mtu, new_mtu);
netdev->mtu = new_mtu;
+ /* Modify receive buffer size based on MTU and do not
+ * use the fixed size set.
+ */
+ pf->hw.rbuf_fixed_size = 0;
if (if_up)
err = otx2_open(netdev);
@@ -1306,6 +1310,9 @@ static int otx2_get_rbuf_size(struct otx2_nic *pf, int mtu)
int total_size;
int rbuf_size;
+ if (pf->hw.rbuf_fixed_size)
+ return pf->hw.rbuf_fixed_size;
+
/* The data transferred by NIX to memory consists of actual packet
* plus additional data which has timestamp and/or EDSA/HIGIG2
* headers if interface is configured in corresponding modes.
diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
index 672be05..229d051 100644
--- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
+++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_vf.c
@@ -439,6 +439,7 @@ static void otx2vf_do_set_rx_mode(struct work_struct *work)
static int otx2vf_change_mtu(struct net_device *netdev, int new_mtu)
{
+ struct otx2_nic *vf = netdev_priv(netdev);
bool if_up = netif_running(netdev);
int err = 0;
@@ -448,6 +449,10 @@ static int otx2vf_change_mtu(struct net_device *netdev, int new_mtu)
netdev_info(netdev, "Changing MTU from %d to %d\n",
netdev->mtu, new_mtu);
netdev->mtu = new_mtu;
+ /* Modify receive buffer size based on MTU and do not
+ * use the fixed size set.
+ */
+ vf->hw.rbuf_fixed_size = 0;
if (if_up)
err = otx2vf_open(netdev);
--
2.7.4
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [net-next PATCH v2 0/3] Add devlink params to vary cqe and rbuf
2021-10-06 7:18 [net-next PATCH v2 0/3] Add devlink params to vary cqe and rbuf Subbaraya Sundeep
` (2 preceding siblings ...)
2021-10-06 7:18 ` [net-next PATCH v2 3/3] octeontx2-pf: Add devlink param to vary rbuf size Subbaraya Sundeep
@ 2021-10-06 13:43 ` Jakub Kicinski
2021-10-06 13:48 ` sundeep subbaraya
3 siblings, 1 reply; 6+ messages in thread
From: Jakub Kicinski @ 2021-10-06 13:43 UTC (permalink / raw)
To: Subbaraya Sundeep; +Cc: davem, netdev, sgoutham, hkelam, gakula
On Wed, 6 Oct 2021 12:48:43 +0530 Subbaraya Sundeep wrote:
> Octeontx2 hardware writes a Completion Queue Entry(CQE) in the
> memory provided by software when a packet is received or
> transmitted. CQE has the buffer pointers (IOVAs) where the
> packet data fragments are written by hardware. One 128 byte
> CQE can hold 6 buffer pointers and a 512 byte CQE can hold
> 42 buffer pointers. Hence large packets can be received either
> by using 512 byte CQEs or by increasing size of receive buffers.
> Current driver only supports 128 byte CQEs.
> This patchset adds devlink params to change CQE and receive
> buffer sizes which inturn helps to tune whether many small size
> buffers or less big size buffers are needed to receive larger
> packets. Below is the patches description:
nak. Stop ignoring feedback and reposting your patches.
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [net-next PATCH v2 0/3] Add devlink params to vary cqe and rbuf
2021-10-06 13:43 ` [net-next PATCH v2 0/3] Add devlink params to vary cqe and rbuf Jakub Kicinski
@ 2021-10-06 13:48 ` sundeep subbaraya
0 siblings, 0 replies; 6+ messages in thread
From: sundeep subbaraya @ 2021-10-06 13:48 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Subbaraya Sundeep, David Miller, netdev, Sunil Kovvuri Goutham,
hariprasad, Geetha sowjanya
Hi Jakub,
On Wed, Oct 6, 2021 at 7:14 PM Jakub Kicinski <kuba@kernel.org> wrote:
>
> On Wed, 6 Oct 2021 12:48:43 +0530 Subbaraya Sundeep wrote:
> > Octeontx2 hardware writes a Completion Queue Entry(CQE) in the
> > memory provided by software when a packet is received or
> > transmitted. CQE has the buffer pointers (IOVAs) where the
> > packet data fragments are written by hardware. One 128 byte
> > CQE can hold 6 buffer pointers and a 512 byte CQE can hold
> > 42 buffer pointers. Hence large packets can be received either
> > by using 512 byte CQEs or by increasing size of receive buffers.
> > Current driver only supports 128 byte CQEs.
> > This patchset adds devlink params to change CQE and receive
> > buffer sizes which inturn helps to tune whether many small size
> > buffers or less big size buffers are needed to receive larger
> > packets. Below is the patches description:
>
> nak. Stop ignoring feedback and reposting your patches.
Sure and noted.
Thanks,
Sundeep
^ permalink raw reply [flat|nested] 6+ messages in thread