From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-177.mta1.migadu.com (out-177.mta1.migadu.com [95.215.58.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5698E3B27C4 for ; Fri, 10 Apr 2026 22:37:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.177 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775860644; cv=none; b=HJT1mKdSJ4MbUpSiHt44dgechvIRiKoXxOm4GdVsRCsH7JWAos3Wj+fzE39oiGJzY84/FX//X6KK9U0kjxuRiHIIQuRtVhig5J2Mz6uEVPvGFDiBd3ha4j1bsX+7V2l2fWezrhZwx5eLcfdzUEfJBqGe/gN6GMJU71XEmWsouGI= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775860644; c=relaxed/simple; bh=qgsrtqUov5rsHtiTXc3lRid9ByZ5Ht45ocdPgaxU0f8=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=iIer2tPowqi/RTo18KiFeDevZwiL+8myY1ZTiLXqpSR1abdnlLM0GYgdW4Uw7tC//eV58MeVWydoI9/EbBAKvUKFJBGWj2Ven1wxhVNQ8w8O4I+O92FBtJ9DSjzorPfdmEWqhvN+1Lam4Ne+o4UBMEwAicKF9dM+J2o27NcHe5o= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=wi/p5+3W; arc=none smtp.client-ip=95.215.58.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="wi/p5+3W" Message-ID: <090caa77-f1ca-4854-9975-87b9e4f2bf74@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1775860639; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hAwfcYXNXSZsXqiGuiWdDazTY4DTbiaWIUs7rzwgVs8=; b=wi/p5+3W3UzX+5OhxKv1b3skKz9mfXyXBTLsBBLddc4OKXm3C2FKa+n0Uj7+SxV3HtK9EE m45C07k3m8fxDpOKURLrDdHdQiK4cN+V1CQBQoL5VBwkhJ7HGFTGfhVFqcvOykSW8W3hiL C5U0UwxtxusfPHw0USpK9Q6Ndv+Qjxg= Date: Fri, 10 Apr 2026 15:37:09 -0700 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Subject: Re: [PATCH v5 2/3] RDMA/rxe: add SENT/RCVD bytes To: zhenwei pi , linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, Zhu Yanjun Cc: zyjzyj2000@gmail.com, jgg@ziepe.ca, leon@kernel.org References: <20260408000956.486522-1-zhenwei.pi@linux.dev> <20260408000956.486522-3-zhenwei.pi@linux.dev> Content-Language: en-US X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: "yanjun.zhu" In-Reply-To: <20260408000956.486522-3-zhenwei.pi@linux.dev> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_OUT On 4/7/26 5:09 PM, zhenwei pi wrote: > There is a lack of sent/received counter in bytes. > > Signed-off-by: zhenwei pi > --- > drivers/infiniband/sw/rxe/rxe_hw_counters.c | 2 ++ > drivers/infiniband/sw/rxe/rxe_hw_counters.h | 2 ++ > drivers/infiniband/sw/rxe/rxe_net.c | 2 ++ > drivers/infiniband/sw/rxe/rxe_recv.c | 2 ++ > drivers/infiniband/sw/rxe/rxe_verbs.h | 6 ++++++ > 5 files changed, 14 insertions(+) > > diff --git a/drivers/infiniband/sw/rxe/rxe_hw_counters.c b/drivers/infiniband/sw/rxe/rxe_hw_counters.c > index 437917a7d8f2..17edaa9a9b9b 100644 > --- a/drivers/infiniband/sw/rxe/rxe_hw_counters.c > +++ b/drivers/infiniband/sw/rxe/rxe_hw_counters.c > @@ -22,6 +22,8 @@ static const struct rdma_stat_desc rxe_counter_descs[] = { > [RXE_CNT_LINK_DOWNED].name = "link_downed", > [RXE_CNT_RDMA_SEND].name = "rdma_sends", > [RXE_CNT_RDMA_RECV].name = "rdma_recvs", > + [RXE_CNT_SENT_BYTES].name = "sent_bytes", > + [RXE_CNT_RCVD_BYTES].name = "rcvd_bytes", > }; > > int rxe_ib_get_hw_stats(struct ib_device *ibdev, > diff --git a/drivers/infiniband/sw/rxe/rxe_hw_counters.h b/drivers/infiniband/sw/rxe/rxe_hw_counters.h > index 051f9e1c3852..01b355103cbc 100644 > --- a/drivers/infiniband/sw/rxe/rxe_hw_counters.h > +++ b/drivers/infiniband/sw/rxe/rxe_hw_counters.h > @@ -26,6 +26,8 @@ enum rxe_counters { > RXE_CNT_LINK_DOWNED, > RXE_CNT_RDMA_SEND, > RXE_CNT_RDMA_RECV, > + RXE_CNT_SENT_BYTES, > + RXE_CNT_RCVD_BYTES, > RXE_NUM_OF_COUNTERS > }; > > diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c > index 6621d01ac32d..86660031ffa2 100644 > --- a/drivers/infiniband/sw/rxe/rxe_net.c > +++ b/drivers/infiniband/sw/rxe/rxe_net.c > @@ -503,6 +503,7 @@ int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt, > int err; > int is_request = pkt->mask & RXE_REQ_MASK; > struct rxe_dev *rxe = to_rdev(qp->ibqp.device); > + unsigned int skblen = skb->len; > unsigned long flags; > > spin_lock_irqsave(&qp->state_lock, flags); > @@ -526,6 +527,7 @@ int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt, > } > > rxe_counter_inc(rxe, RXE_CNT_SENT_PKTS); > + rxe_counter_add(rxe, RXE_CNT_SENT_BYTES, skblen); > goto done; > > drop: > diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c > index 5861e4244049..e7bab89e7d8d 100644 > --- a/drivers/infiniband/sw/rxe/rxe_recv.c > +++ b/drivers/infiniband/sw/rxe/rxe_recv.c > @@ -318,6 +318,7 @@ void rxe_rcv(struct sk_buff *skb) > int err; > struct rxe_pkt_info *pkt = SKB_TO_PKT(skb); > struct rxe_dev *rxe = pkt->rxe; > + unsigned int skblen = skb->len - skb_network_offset(skb); > > if (unlikely(skb->len < RXE_BTH_BYTES)) > goto drop; > @@ -341,6 +342,7 @@ void rxe_rcv(struct sk_buff *skb) > if (unlikely(err)) > goto drop; > > + rxe_counter_add(rxe, RXE_CNT_RCVD_BYTES, skblen); > rxe_counter_inc(rxe, RXE_CNT_RCVD_PKTS); > > if (unlikely(bth_qpn(pkt) == IB_MULTICAST_QPN)) int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt, struct sk_buff *skb) { int err; int is_request = pkt->mask & RXE_REQ_MASK; struct rxe_dev *rxe = to_rdev(qp->ibqp.device); unsigned long flags; skb->len is printed here, that is len1 ... if (pkt->mask & RXE_LOOPBACK_MASK) err = rxe_loopback(skb, pkt); else err = rxe_send(skb, pkt); ... } In the following function static int rxe_loopback(struct sk_buff *skb, struct rxe_pkt_info *pkt) { ... if (skb->protocol == htons(ETH_P_IP)) skb_pull(skb, sizeof(struct iphdr)); else skb_pull(skb, sizeof(struct ipv6hdr)); ... /* remove udp header */ skb_pull(skb, sizeof(struct udphdr)); print skb->len here, that is len2 rxe_rcv(skb); ... } Does len1 equal to len2? If not, the transmitted length appears to differ from the received length when using loopback. I am not sure whether this is expected behavior. The same observation also applies to the non-loopback case. Zhu Yanjun > diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.h b/drivers/infiniband/sw/rxe/rxe_verbs.h > index e800545d1046..0f5ffd94643f 100644 > --- a/drivers/infiniband/sw/rxe/rxe_verbs.h > +++ b/drivers/infiniband/sw/rxe/rxe_verbs.h > @@ -455,6 +455,12 @@ static inline void rxe_counter_inc(struct rxe_dev *rxe, enum rxe_counters index) > atomic64_inc(&rxe->stats_counters[index]); > } > > +static inline void rxe_counter_add(struct rxe_dev *rxe, enum rxe_counters index, > + s64 val) > +{ > + atomic64_add(val, &rxe->stats_counters[index]); > +} > + > static inline struct rxe_dev *to_rdev(struct ib_device *dev) > { > return dev ? container_of(dev, struct rxe_dev, ib_dev) : NULL;