Re: [PATCH for-next v3] RDMA/efa: Add EFA query MR support

["Margolin, Michael" <mrgolin@amazon.com>]

On 12/11/2023 7:50 PM, Jason Gunthorpe wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.
>
>
>
> On Mon, Dec 11, 2023 at 05:47:15PM +0000, Michael Margolin wrote:
>> diff --git a/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h b/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h
>> index 9c65bd27bae0..597f7ca6f31d 100644
>> --- a/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h
>> +++ b/drivers/infiniband/hw/efa/efa_admin_cmds_defs.h
>> @@ -415,6 +415,32 @@ struct efa_admin_reg_mr_resp {
>>        * memory region
>>        */
>>       u32 r_key;
>> +
>> +     /*
>> +      * Mask indicating which fields have valid values
>> +      * 0 : recv_pci_bus_id
>> +      * 1 : rdma_read_pci_bus_id
>> +      * 2 : rdma_recv_pci_bus_id
>> +      */
>> +     u8 validity;
>> +
>> +     /*
>> +      * Physical PCIe bus used by the device to reach the MR for receive
>> +      * operation
>> +      */
>> +     u8 recv_pci_bus_id;
>> +
>> +     /*
>> +      * Physical PCIe bus used by the device to reach the MR for RDMA read
>> +      * operation
>> +      */
>> +     u8 rdma_read_pci_bus_id;
>> +
>> +     /*
>> +      * Physical PCIe bus used by the device to reach the MR for RDMA write
>> +      * receive
>> +      */
>> +     u8 rdma_recv_pci_bus_id;
> What driver is bound to this other PCIe bus and how did the iommu get
> setup for it?
>
> Jason

It's internal bus that is not directly exposed to the host. Addresses
mapping is acquired from accelerator's driver as for any MR residing in
accelerator memory, and the translation is owned by devices on that bus.


Michael