Re: Seeking Guidance: Creating an IBV Multicast Group?

[Zhu Yanjun <yanjun.zhu@linux.dev>]

在 2024/7/30 1:10, Andrew Sheinberg 写道:
> Hello all,
> 
> I’m not sure if this the right place to ask, but I will give it a try.
> 
> I have a system with many initialized UD queue pairs (info for address handle creations and qp numbers exchanged out-of-band). I am only using libibverbs for establishment (purposefully not using librdmacm, to allow for more flexible environment configuration) — everything is working smoothly for unicast.  Now I would like to create a multicast group and attach some of these queue pairs (ibv_mcast_attach); however I am struggling to find any details on how to create such a group (and obtain a proper MGID and MLID).
> 
> I found a few examples online but am left with questions:
> 	- There is code within perftest's "multicast_resources.c", but this seems a bit hacky and oddly verbose
> 	- There is code within Nvidia Docs’  "Programming Examples using IBV” showcasing joining an already created multicast group at a given IP address using rdma_cm, but It is unclear how to create the group in the first place
> 
> 
> Questions (please correct me if these do not make sense):
> 
> 1. What is the role of the OpenSM — is there a C API?
> 	- Are there any examples using opensm programmatically and not with CLI?
> 	- Does the API differ on InfiniBand vs. RoCEv2 fabric?

I have made tests with Infiniband vs. RoCEv2. From my perspective, I 
think, there are some differences between the 2 fabrics.

To the basic functionalities, the difference between 2 fabrics is small. 
But to the extended functionalities, the difference is big.

The difference is based on the different features.

I just made some simple tests. Perhaps some engineers who made a lot of 
tests can give more suggestions about this.

Zhu Yanjun

> 
> 2. Is there any high-level documentation to describe the role of libibumad? (Looking at the man pages on a per-function basis is a bit too fine-grained for my understanding as of now).
> 	- I also see libibmad — what is the responsibility breakdown between these two?
> 	- How do they relate to OpenSM?
> 
> Any guidance is greatly appreciated.
> 
> Thanks,
> Andrew Sheinberg