From: Zhu Yanjun <yanjun.zhu@linux.dev>
To: Bob Pearson <rpearsonhpe@gmail.com>,
jgg@nvidia.com, linux-rdma@vger.kernel.org, dsahern@kernel.org,
"davem@davemloft.net" <davem@davemloft.net>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: Re: [PATCH for-next v5 2/7] RDMA/rxe: Fix sending of mcast packets
Date: Tue, 5 Dec 2023 13:53:35 +0800 [thread overview]
Message-ID: <0b103b8d-3462-42d3-bc06-803c0b6ca153@linux.dev> (raw)
In-Reply-To: <20231205002322.10143-3-rpearsonhpe@gmail.com>
Add David S. Miller and David Ahern.
They are the maintainers in netdev and very familiar with mcast.
Zhu Yanjun
在 2023/12/5 8:23, Bob Pearson 写道:
> Currently the rdma_rxe driver does not send mcast packets correctly.
> It uses the wrong qp number for the packets.
>
> Add a mask bit to indicate that a multicast packet has been locally
> sent and use to set the correct qpn for multicast packets and
> identify mcast packets when sending.
>
> Fixes: 8700e3e7c485 ("Soft RoCE driver")
> Signed-off-by: Bob Pearson <rpearsonhpe@gmail.com>
> ---
> drivers/infiniband/sw/rxe/rxe_av.c | 7 +++++++
> drivers/infiniband/sw/rxe/rxe_loc.h | 1 +
> drivers/infiniband/sw/rxe/rxe_net.c | 4 +++-
> drivers/infiniband/sw/rxe/rxe_opcode.h | 2 +-
> drivers/infiniband/sw/rxe/rxe_recv.c | 4 ++++
> drivers/infiniband/sw/rxe/rxe_req.c | 11 +++++++++--
> 6 files changed, 25 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/infiniband/sw/rxe/rxe_av.c b/drivers/infiniband/sw/rxe/rxe_av.c
> index 4ac17b8def28..022173eb5d75 100644
> --- a/drivers/infiniband/sw/rxe/rxe_av.c
> +++ b/drivers/infiniband/sw/rxe/rxe_av.c
> @@ -7,6 +7,13 @@
> #include "rxe.h"
> #include "rxe_loc.h"
>
> +bool rxe_is_mcast_av(struct rxe_av *av)
> +{
> + struct in6_addr *daddr = (struct in6_addr *)av->grh.dgid.raw;
> +
> + return rdma_is_multicast_addr(daddr);
> +}
> +
> void rxe_init_av(struct rdma_ah_attr *attr, struct rxe_av *av)
> {
> rxe_av_from_attr(rdma_ah_get_port_num(attr), av, attr);
> diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h
> index 3d2504a0ae56..62b2b25903fc 100644
> --- a/drivers/infiniband/sw/rxe/rxe_loc.h
> +++ b/drivers/infiniband/sw/rxe/rxe_loc.h
> @@ -8,6 +8,7 @@
> #define RXE_LOC_H
>
> /* rxe_av.c */
> +bool rxe_is_mcast_av(struct rxe_av *av);
> void rxe_init_av(struct rdma_ah_attr *attr, struct rxe_av *av);
> int rxe_chk_ah_attr(struct rxe_dev *rxe, struct rdma_ah_attr *attr);
> void rxe_av_from_attr(u8 port_num, struct rxe_av *av,
> diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c
> index cd59666158b1..58c3f3759bf0 100644
> --- a/drivers/infiniband/sw/rxe/rxe_net.c
> +++ b/drivers/infiniband/sw/rxe/rxe_net.c
> @@ -431,7 +431,9 @@ int rxe_xmit_packet(struct rxe_qp *qp, struct rxe_pkt_info *pkt,
>
> rxe_icrc_generate(skb, pkt);
>
> - if (pkt->mask & RXE_LOOPBACK_MASK)
> + if (pkt->mask & RXE_MCAST_MASK)
> + err = rxe_send(skb, pkt);
> + else if (pkt->mask & RXE_LOOPBACK_MASK)
> err = rxe_loopback(skb, pkt);
> else
> err = rxe_send(skb, pkt);
> diff --git a/drivers/infiniband/sw/rxe/rxe_opcode.h b/drivers/infiniband/sw/rxe/rxe_opcode.h
> index 5686b691d6b8..c4cf672ea26d 100644
> --- a/drivers/infiniband/sw/rxe/rxe_opcode.h
> +++ b/drivers/infiniband/sw/rxe/rxe_opcode.h
> @@ -85,7 +85,7 @@ enum rxe_hdr_mask {
> RXE_END_MASK = BIT(NUM_HDR_TYPES + 11),
>
> RXE_LOOPBACK_MASK = BIT(NUM_HDR_TYPES + 12),
> -
> + RXE_MCAST_MASK = BIT(NUM_HDR_TYPES + 13),
> RXE_ATOMIC_WRITE_MASK = BIT(NUM_HDR_TYPES + 14),
>
> RXE_READ_OR_ATOMIC_MASK = (RXE_READ_MASK | RXE_ATOMIC_MASK),
> diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c
> index 5861e4244049..7153de0799fc 100644
> --- a/drivers/infiniband/sw/rxe/rxe_recv.c
> +++ b/drivers/infiniband/sw/rxe/rxe_recv.c
> @@ -217,6 +217,10 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb)
> list_for_each_entry(mca, &mcg->qp_list, qp_list) {
> qp = mca->qp;
>
> + /* don't reply packet to sender if locally sent */
> + if (pkt->mask & RXE_MCAST_MASK && qp_num(qp) == deth_sqp(pkt))
> + continue;
> +
> /* validate qp for incoming packet */
> err = check_type_state(rxe, pkt, qp);
> if (err)
> diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c
> index d8c41fd626a9..599bec88cb54 100644
> --- a/drivers/infiniband/sw/rxe/rxe_req.c
> +++ b/drivers/infiniband/sw/rxe/rxe_req.c
> @@ -442,8 +442,12 @@ static struct sk_buff *init_req_packet(struct rxe_qp *qp,
> (pkt->mask & (RXE_WRITE_MASK | RXE_IMMDT_MASK)) ==
> (RXE_WRITE_MASK | RXE_IMMDT_MASK));
>
> - qp_num = (pkt->mask & RXE_DETH_MASK) ? ibwr->wr.ud.remote_qpn :
> - qp->attr.dest_qp_num;
> + if (pkt->mask & RXE_MCAST_MASK)
> + qp_num = IB_MULTICAST_QPN;
> + else if (pkt->mask & RXE_DETH_MASK)
> + qp_num = ibwr->wr.ud.remote_qpn;
> + else
> + qp_num = qp->attr.dest_qp_num;
>
> ack_req = ((pkt->mask & RXE_END_MASK) ||
> (qp->req.noack_pkts++ > RXE_MAX_PKT_PER_ACK));
> @@ -809,6 +813,9 @@ int rxe_requester(struct rxe_qp *qp)
> goto err;
> }
>
> + if (rxe_is_mcast_av(av))
> + pkt.mask |= RXE_MCAST_MASK;
> +
> skb = init_req_packet(qp, av, wqe, opcode, payload, &pkt);
> if (unlikely(!skb)) {
> rxe_dbg_qp(qp, "Failed allocating skb\n");
prev parent reply other threads:[~2023-12-05 5:53 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20231205002322.10143-1-rpearsonhpe@gmail.com>
2023-12-05 5:50 ` [PATCH for-next v5 0/7] RDMA/rxe: Make multicast work Zhu Yanjun
2023-12-07 16:15 ` David Ahern
[not found] ` <20231205002322.10143-2-rpearsonhpe@gmail.com>
2023-12-05 5:52 ` [PATCH for-next v5 1/7] RDMA/rxe: Cleanup rxe_ah/av_chk_attr Zhu Yanjun
[not found] ` <20231205002322.10143-3-rpearsonhpe@gmail.com>
2023-12-05 5:53 ` Zhu Yanjun [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0b103b8d-3462-42d3-bc06-803c0b6ca153@linux.dev \
--to=yanjun.zhu@linux.dev \
--cc=davem@davemloft.net \
--cc=dsahern@kernel.org \
--cc=jgg@nvidia.com \
--cc=linux-rdma@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=rpearsonhpe@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).