Re: Question for srpt in kernel-4.14

[Bart Van Assche <bvanassche@acm.org>]

On 7/19/19 11:54 PM, oulijun wrote:
> Hi, Bart Van Assche & Doug Ledford
>    I am targeting a problem about RoCE and SCSI over RDMA from srpt in kernel-4.14. When insmod srpt.ko and insmod hns-roce-hw-v2.ko, it will
> report a warning in srpt_add_one:
>    ib_srpt srpt_add_one(hns_0) failed.
> 
>    I am tracking the error from ib_cm_listen in srpt_add_one.I found it returned an error when doing server_id validation
> the error code as follows:
>    static int __ib_cm_listen(struct ib_cm_id *cm_id, __be64 service_id,
> 			  __be64 service_mask)
> {
> 	struct cm_id_private *cm_id_priv, *cur_cm_id_priv;
> 	int ret = 0;
> 
> 	service_mask = service_mask ? service_mask : ~cpu_to_be64(0);
> 	service_id &= service_mask;
> 	if ((service_id & IB_SERVICE_ID_AGN_MASK) == IB_CM_ASSIGN_SERVICE_ID &&
> 	    (service_id != IB_CM_ASSIGN_SERVICE_ID))
> 		return -EINVAL;
>   	......
>   }
> 
> static void srpt_add_one(struct ib_device *device)
> {
> 	struct srpt_device *sdev;
> 	struct srpt_port *sport;
> 	int i;
> 
> 	pr_debug("device = %p\n", device);
> 
> 	sdev = kzalloc(sizeof(*sdev), GFP_KERNEL);
> 	if (!sdev)
> 		goto err;
> 
> 	sdev->device = device;
> 	mutex_init(&sdev->sdev_mutex);
> 
> 	sdev->pd = ib_alloc_pd(device, 0);
> 	if (IS_ERR(sdev->pd))
> 		goto free_dev;
> 
> 	sdev->lkey = sdev->pd->local_dma_lkey;
> 
> 	sdev->srq_size = min(srpt_srq_size, sdev->device->attrs.max_srq_wr);
> 
> 	srpt_use_srq(sdev, sdev->port[0].port_attrib.use_srq);
> 
> 	if (!srpt_service_guid)
> 		srpt_service_guid = be64_to_cpu(device->node_guid);
> 
> 	sdev->cm_id = ib_create_cm_id(device, srpt_cm_handler, sdev);
> 	if (IS_ERR(sdev->cm_id))
> 		goto err_ring;
> 
> 	/* print out target login information */
> 	pr_debug("Target login info: id_ext=%016llx,ioc_guid=%016llx,"
> 		 "pkey=ffff,service_id=%016llx\n", srpt_service_guid,
> 		 srpt_service_guid, srpt_service_guid);
> 
> 	/*
> 	 * We do not have a consistent service_id (ie. also id_ext of target_id)
> 	 * to identify this target. We currently use the guid of the first HCA
> 	 * in the system as service_id; therefore, the target_id will change
> 	 * if this HCA is gone bad and replaced by different HCA
> 	 */
> 	if (ib_cm_listen(sdev->cm_id, cpu_to_be64(srpt_service_guid), 0))
> 		goto err_cm;
> 
> 	......
> }
> 
> However, I check the srpt_service_guid is obtained by device->node_guid. I think that the compute algorithm is ok for device->node_guid.
> In addition, I analyzed a patch in kernel-4.17(IB/srpt: Add RDMA/CM support). As a result, I can understand that the previous srpt is not supported by RDMA/CM?
> So, all RoCE will failed when use kernel-4.14 version to run srpt.ko?

Before commit 63cf1a902c9d ("IB/srpt: Add RDMA/CM support"; v4.17) 
ib_cm_listen() was called with srpt_service_guid as argument for all 
RDMA adapters. Since that commit ib_cm_listen() is only called for ports 
that have IB as link layer. In other words, I think the failure that you 
reported only occurs with kernel before 4.17 and not with kernel v4.17 
or any later kernel.

Bart.