Re: [PATCH for-next v3] RDMA/efa: Add EFA query MR support

["Margolin, Michael" <mrgolin@amazon.com>]

On 1/3/2024 1:47 AM, Jason Gunthorpe wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> On Wed, Dec 13, 2023 at 07:05:24PM +0200, Margolin, Michael wrote:
>>> On Mon, Dec 11, 2023 at 05:47:15PM +0000, Michael Margolin wrote:
>>>> diff --git a/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h b/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h
>>>> index 9c65bd27bae0..597f7ca6f31d 100644
>>>> --- a/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h
>>>> +++ b/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h
>>>> @@ -415,6 +415,32 @@ struct efa_admin_reg_mr_resp {
>>>>         * memory region
>>>>         */
>>>>        u32 r_key;
>>>> +
>>>> +     /*
>>>> +      * Mask indicating which fields have valid values
>>>> +      * 0 : recv_pci_bus_id
>>>> +      * 1 : rdma_read_pci_bus_id
>>>> +      * 2 : rdma_recv_pci_bus_id
>>>> +      */
>>>> +     u8 validity;
>>>> +
>>>> +     /*
>>>> +      * Physical PCIe bus used by the device to reach the MR for receive
>>>> +      * operation
>>>> +      */
>>>> +     u8 recv_pci_bus_id;
>>>> +
>>>> +     /*
>>>> +      * Physical PCIe bus used by the device to reach the MR for RDMA read
>>>> +      * operation
>>>> +      */
>>>> +     u8 rdma_read_pci_bus_id;
>>>> +
>>>> +     /*
>>>> +      * Physical PCIe bus used by the device to reach the MR for RDMA write
>>>> +      * receive
>>>> +      */
>>>> +     u8 rdma_recv_pci_bus_id;
>>> What driver is bound to this other PCIe bus and how did the iommu get
>>> setup for it?
>> It's internal bus that is not directly exposed to the host. Addresses
>> mapping is acquired from accelerator's driver as for any MR residing in
>> accelerator memory, and the translation is owned by devices on that bus.
> So if it isn't visible to the host, or connectable to anything Linux
> would call a PCI RID, why are you giving it such specific names? Just
> call it 'interconnect path id' or something and make it opaque?
>
> Jason

I would like it to be as opaque as possible but on the other hand to 
express that the mentioned 'interconnect path' relates specifically to 
PCI path.

I think that except of this, since 'bus_id' is quite general term that 
cannot be used without proper context, it isn't likely to cause any 
unintended use of those fields.

Michael