Re: [PATCH] powerpc/pseries/dlpar: Search DRC index from ibm,drc-indexes for IO add

[Amit Machhiwal <amachhiw@linux.ibm.com>]

Hi Haren,

Thanks for the patch.

On 2025/04/30 09:10 PM, Haren Myneni wrote:
> IO hotplug add event is handled in the user space with drmgr tool.
> After the device is enabled, the user space uses /sys/kerne/dlpar
> interface with “dt add index <drc_index>” to update the device tree.
> The kernel interface (dlpar_hp_dt_add()) finds the parent node for
> the specified ‘drc_index’ from ibm,drc-info property. But old FW
> provides ibm,drc-indexes property instead of ibm,drc-info.
> 
> If the ibm,drc-info is not available, this patch adds changes to
> search ‘drc_index’ from the indexes array in ibm,drc-indexes
> property to support old FW.
> 
> Fixes: 02b98ff44a57 ("powerpc/pseries/dlpar: Add device tree nodes for DLPAR IO add")
> Signed-off-by: Haren Myneni <haren@linux.ibm.com>
> ---
>  arch/powerpc/platforms/pseries/dlpar.c | 52 +++++++++++++++++++++++++-
>  1 file changed, 50 insertions(+), 2 deletions(-)
> 

I tested this patch on fc42 pSeries KVM guest (L2) with PCI device passhthrough.
The device hotplug operation without this patch fails with the below errors.
While with the patch, PCI device hotplug/unplug operations work fine. Hence,

Tested-by: Amit Machhiwal <amachhiw@linux.ibm.com>

 Installed pkgs
 --------------
  [root@fedora42 ~]# rpm -qa | grep -e 'ppc64-diag\|powerpc-utils'
  ppc64-diag-rtas-2.7.10-4.fc42.ppc64le
  ppc64-diag-2.7.10-4.fc42.ppc64le
  powerpc-utils-1.3.13-3.fc42.ppc64le
  powerpc-utils-core-1.3.13-3.fc42.ppc64le

 Without Patch
 -------------
  dmesg
  -----
  (on device attach)
   [ 1451.166029] RTAS: event: 1, Type: Hotplug Event (229), Severity: 1
   [ 1451.393168] dlpar: Could not handle DLPAR request "dt add index 0x40010008"

  drmgr log
  ---------
  (on device attach)
   ########## May 05 07:36:25 2025 ##########
   drmgr: -c pci -a -s 0x40010008 -n -d4 -V
   Retrieving hotplug nodes
   [...]
   Found drc_name C65544
   is calling rtas_set_indicator(ISOLATE index 0x40010008)
   is calling set_power(POWER_OFF index 0x40010008, power_domain 0xffffffff)
   is calling card_present
   Waiting for the card to appear...
   get-sensor for 40010008: 0, 1
   is calling set_power(POWER_ON index 0x40010008, power_domain 0xffffffff
   calling rtas_set_indicator(UNISOLATE index 0x40010008)
   Initiating kernel DLPAR "dt add index 0x40010008"
   Failed to write to /sys/kernel/dlpar: Input/output error
   add_device_tree_nodes failed at /proc/device-tree/pci@800000020000001
   Internal software error. Contact your service representative.
   ########## May 05 07:36:25 2025 ##########

 With Patch
 ----------
  dmesg
  -----
  (on device attach)
   [  129.503732] RTAS: event: 1, Type: Hotplug Event (229), Severity: 1
   [...]
   [  129.819677] tg3 0001:00:01.0: ibm,query-pe-dma-windows(2026) 800 8000000 20000001 returned 0, lb=2000000000 ps=107 wn=1
   [  129.822331] tg3 0001:00:01.0: ibm,create-pe-dma-window(2027) 800 8000000 20000001 18 0 returned -3 (liobn = 0x0 starting addr = 0 0)
   [...]
   [  133.095657] tg3 0001:00:01.0 enP1p0s1: Link is up at 1000 Mbps, full duplex

  (on device detach)
   [  525.609818] RTAS: event: 2, Type: Hotplug Event (229), Severity: 1

  drmgr log
  ---------
  (on device attach)
   ########## May 05 07:49:17 2025 ##########
   drmgr: -c pci -a -s 0x40010008 -n -d4 -V
   Retrieving hotplug nodes
   [...]
   Found drc_name C65544
   is calling rtas_set_indicator(ISOLATE index 0x40010008)
   is calling set_power(POWER_OFF index 0x40010008, power_domain 0xffffffff)
   is calling card_present
   Waiting for the card to appear...
   get-sensor for 40010008: 0, 1
   is calling set_power(POWER_ON index 0x40010008, power_domain 0xffffffff
   calling rtas_set_indicator(UNISOLATE index 0x40010008)
   Initiating kernel DLPAR "dt add index 0x40010008"
   Success
   is calling enable_slot to config adapter
   ########## May 05 07:49:17 2025 ##########

  (on device detach)
   ########## May 05 07:55:53 2025 ##########
   drmgr: -c pci -r -s 0x40010008 -n -d4 -V
   Retrieving hotplug nodes
   [...]
   Found drc_name C65544
   found node: drc name=C65544, index=0x40010008, path=/proc/device-tree/pci@800000020000001
   Initiating kernel DLPAR "dt remove index 0x40010008"
   Success
   is calling rtas_set_indicator(ISOLATE index 0x40010008)
   is calling set_power(POWER_OFF index 0x40010008, power_domain 0xffffffff)
   ########## May 05 07:55:57 2025 ##########

Thanks,
Amit

> diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c
> index 213aa26dc8b3..979487da6522 100644
> --- a/arch/powerpc/platforms/pseries/dlpar.c
> +++ b/arch/powerpc/platforms/pseries/dlpar.c
> @@ -404,6 +404,45 @@ get_device_node_with_drc_info(u32 index)
>  	return NULL;
>  }
>  
> +static struct device_node *
> +get_device_node_with_drc_indexes(u32 drc_index)
> +{
> +	struct device_node *np = NULL;
> +	u32 nr_indexes, index;
> +	int i, rc;
> +
> +	for_each_node_with_property(np, "ibm,drc-indexes") {
> +		/*
> +		 * First element in the array is the total number of
> +		 * DRC indexes returned.
> +		 */
> +		rc = of_property_read_u32_index(np, "ibm,drc-indexes",
> +				0, &nr_indexes);
> +		if (rc)
> +			goto out_put_np;
> +
> +		/*
> +		 * Retrieve DRC index from the list and return the
> +		 * device node if matched with the specified index.
> +		 */
> +		for (i = 0; i < nr_indexes; i++) {
> +			rc = of_property_read_u32_index(np, "ibm,drc-indexes",
> +							i+1, &index);
> +			if (rc)
> +				goto out_put_np;
> +
> +			if (drc_index == index)
> +				return np;
> +		}
> +	}
> +
> +	return NULL;
> +
> +out_put_np:
> +	of_node_put(np);
> +	return NULL;
> +}
> +
>  static int dlpar_hp_dt_add(u32 index)
>  {
>  	struct device_node *np, *nodes;
> @@ -423,10 +462,19 @@ static int dlpar_hp_dt_add(u32 index)
>  		goto out;
>  	}
>  
> +	/*
> +	 * Recent FW provides ibm,drc-info property. So search
> +	 * for the user specified DRC index from ibm,drc-info
> +	 * property. If this property is not available, search
> +	 * in the indexes array from ibm,drc-indexes property.
> +	 */
>  	np = get_device_node_with_drc_info(index);
>  
> -	if (!np)
> -		return -EIO;
> +	if (!np) {
> +		np = get_device_node_with_drc_indexes(index);
> +		if (!np)
> +			return -EIO;
> +	}
>  
>  	/* Next, configure the connector. */
>  	nodes = dlpar_configure_connector(cpu_to_be32(index), np);
> -- 
> 2.43.5
> 
>