[PATCH] powerpc/pseries/dlpar: Search DRC index from ibm,drc-indexes for IO add

[PATCH] powerpc/pseries/dlpar: Search DRC index from ibm,drc-indexes for IO add

[Haren Myneni]

IO hotplug add event is handled in the user space with drmgr tool.
After the device is enabled, the user space uses /sys/kerne/dlpar
interface with “dt add index <drc_index>” to update the device tree.
The kernel interface (dlpar_hp_dt_add()) finds the parent node for
the specified ‘drc_index’ from ibm,drc-info property. But old FW
provides ibm,drc-indexes property instead of ibm,drc-info.

If the ibm,drc-info is not available, this patch adds changes to
search ‘drc_index’ from the indexes array in ibm,drc-indexes
property to support old FW.

Fixes: 02b98ff44a57 ("powerpc/pseries/dlpar: Add device tree nodes for DLPAR IO add")
Signed-off-by: Haren Myneni <haren@linux.ibm.com>
---
 arch/powerpc/platforms/pseries/dlpar.c | 52 +++++++++++++++++++++++++-
 1 file changed, 50 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c
index 213aa26dc8b3..979487da6522 100644
--- a/arch/powerpc/platforms/pseries/dlpar.c
+++ b/arch/powerpc/platforms/pseries/dlpar.c
@@ -404,6 +404,45 @@ get_device_node_with_drc_info(u32 index)
 	return NULL;
 }
 
+static struct device_node *
+get_device_node_with_drc_indexes(u32 drc_index)
+{
+	struct device_node *np = NULL;
+	u32 nr_indexes, index;
+	int i, rc;
+
+	for_each_node_with_property(np, "ibm,drc-indexes") {
+		/*
+		 * First element in the array is the total number of
+		 * DRC indexes returned.
+		 */
+		rc = of_property_read_u32_index(np, "ibm,drc-indexes",
+				0, &nr_indexes);
+		if (rc)
+			goto out_put_np;
+
+		/*
+		 * Retrieve DRC index from the list and return the
+		 * device node if matched with the specified index.
+		 */
+		for (i = 0; i < nr_indexes; i++) {
+			rc = of_property_read_u32_index(np, "ibm,drc-indexes",
+							i+1, &index);
+			if (rc)
+				goto out_put_np;
+
+			if (drc_index == index)
+				return np;
+		}
+	}
+
+	return NULL;
+
+out_put_np:
+	of_node_put(np);
+	return NULL;
+}
+
 static int dlpar_hp_dt_add(u32 index)
 {
 	struct device_node *np, *nodes;
@@ -423,10 +462,19 @@ static int dlpar_hp_dt_add(u32 index)
 		goto out;
 	}
 
+	/*
+	 * Recent FW provides ibm,drc-info property. So search
+	 * for the user specified DRC index from ibm,drc-info
+	 * property. If this property is not available, search
+	 * in the indexes array from ibm,drc-indexes property.
+	 */
 	np = get_device_node_with_drc_info(index);
 
-	if (!np)
-		return -EIO;
+	if (!np) {
+		np = get_device_node_with_drc_indexes(index);
+		if (!np)
+			return -EIO;
+	}
 
 	/* Next, configure the connector. */
 	nodes = dlpar_configure_connector(cpu_to_be32(index), np);
-- 
2.43.5

Re: [PATCH] powerpc/pseries/dlpar: Search DRC index from ibm,drc-indexes for IO add

[Amit Machhiwal]

Hi Haren,

Thanks for the patch.

On 2025/04/30 09:10 PM, Haren Myneni wrote:
> IO hotplug add event is handled in the user space with drmgr tool.
> After the device is enabled, the user space uses /sys/kerne/dlpar
> interface with “dt add index <drc_index>” to update the device tree.
> The kernel interface (dlpar_hp_dt_add()) finds the parent node for
> the specified ‘drc_index’ from ibm,drc-info property. But old FW
> provides ibm,drc-indexes property instead of ibm,drc-info.
> 
> If the ibm,drc-info is not available, this patch adds changes to
> search ‘drc_index’ from the indexes array in ibm,drc-indexes
> property to support old FW.
> 
> Fixes: 02b98ff44a57 ("powerpc/pseries/dlpar: Add device tree nodes for DLPAR IO add")
> Signed-off-by: Haren Myneni <haren@linux.ibm.com>
> ---
>  arch/powerpc/platforms/pseries/dlpar.c | 52 +++++++++++++++++++++++++-
>  1 file changed, 50 insertions(+), 2 deletions(-)
> 

I tested this patch on fc42 pSeries KVM guest (L2) with PCI device passhthrough.
The device hotplug operation without this patch fails with the below errors.
While with the patch, PCI device hotplug/unplug operations work fine. Hence,

Tested-by: Amit Machhiwal <amachhiw@linux.ibm.com>

 Installed pkgs
 --------------
  [root@fedora42 ~]# rpm -qa | grep -e 'ppc64-diag\|powerpc-utils'
  ppc64-diag-rtas-2.7.10-4.fc42.ppc64le
  ppc64-diag-2.7.10-4.fc42.ppc64le
  powerpc-utils-1.3.13-3.fc42.ppc64le
  powerpc-utils-core-1.3.13-3.fc42.ppc64le

 Without Patch
 -------------
  dmesg
  -----
  (on device attach)
   [ 1451.166029] RTAS: event: 1, Type: Hotplug Event (229), Severity: 1
   [ 1451.393168] dlpar: Could not handle DLPAR request "dt add index 0x40010008"

  drmgr log
  ---------
  (on device attach)
   ########## May 05 07:36:25 2025 ##########
   drmgr: -c pci -a -s 0x40010008 -n -d4 -V
   Retrieving hotplug nodes
   [...]
   Found drc_name C65544
   is calling rtas_set_indicator(ISOLATE index 0x40010008)
   is calling set_power(POWER_OFF index 0x40010008, power_domain 0xffffffff)
   is calling card_present
   Waiting for the card to appear...
   get-sensor for 40010008: 0, 1
   is calling set_power(POWER_ON index 0x40010008, power_domain 0xffffffff
   calling rtas_set_indicator(UNISOLATE index 0x40010008)
   Initiating kernel DLPAR "dt add index 0x40010008"
   Failed to write to /sys/kernel/dlpar: Input/output error
   add_device_tree_nodes failed at /proc/device-tree/pci@800000020000001
   Internal software error. Contact your service representative.
   ########## May 05 07:36:25 2025 ##########

 With Patch
 ----------
  dmesg
  -----
  (on device attach)
   [  129.503732] RTAS: event: 1, Type: Hotplug Event (229), Severity: 1
   [...]
   [  129.819677] tg3 0001:00:01.0: ibm,query-pe-dma-windows(2026) 800 8000000 20000001 returned 0, lb=2000000000 ps=107 wn=1
   [  129.822331] tg3 0001:00:01.0: ibm,create-pe-dma-window(2027) 800 8000000 20000001 18 0 returned -3 (liobn = 0x0 starting addr = 0 0)
   [...]
   [  133.095657] tg3 0001:00:01.0 enP1p0s1: Link is up at 1000 Mbps, full duplex

  (on device detach)
   [  525.609818] RTAS: event: 2, Type: Hotplug Event (229), Severity: 1

  drmgr log
  ---------
  (on device attach)
   ########## May 05 07:49:17 2025 ##########
   drmgr: -c pci -a -s 0x40010008 -n -d4 -V
   Retrieving hotplug nodes
   [...]
   Found drc_name C65544
   is calling rtas_set_indicator(ISOLATE index 0x40010008)
   is calling set_power(POWER_OFF index 0x40010008, power_domain 0xffffffff)
   is calling card_present
   Waiting for the card to appear...
   get-sensor for 40010008: 0, 1
   is calling set_power(POWER_ON index 0x40010008, power_domain 0xffffffff
   calling rtas_set_indicator(UNISOLATE index 0x40010008)
   Initiating kernel DLPAR "dt add index 0x40010008"
   Success
   is calling enable_slot to config adapter
   ########## May 05 07:49:17 2025 ##########

  (on device detach)
   ########## May 05 07:55:53 2025 ##########
   drmgr: -c pci -r -s 0x40010008 -n -d4 -V
   Retrieving hotplug nodes
   [...]
   Found drc_name C65544
   found node: drc name=C65544, index=0x40010008, path=/proc/device-tree/pci@800000020000001
   Initiating kernel DLPAR "dt remove index 0x40010008"
   Success
   is calling rtas_set_indicator(ISOLATE index 0x40010008)
   is calling set_power(POWER_OFF index 0x40010008, power_domain 0xffffffff)
   ########## May 05 07:55:57 2025 ##########

Thanks,
Amit

> diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c
> index 213aa26dc8b3..979487da6522 100644
> --- a/arch/powerpc/platforms/pseries/dlpar.c
> +++ b/arch/powerpc/platforms/pseries/dlpar.c
> @@ -404,6 +404,45 @@ get_device_node_with_drc_info(u32 index)
>  	return NULL;
>  }
>  
> +static struct device_node *
> +get_device_node_with_drc_indexes(u32 drc_index)
> +{
> +	struct device_node *np = NULL;
> +	u32 nr_indexes, index;
> +	int i, rc;
> +
> +	for_each_node_with_property(np, "ibm,drc-indexes") {
> +		/*
> +		 * First element in the array is the total number of
> +		 * DRC indexes returned.
> +		 */
> +		rc = of_property_read_u32_index(np, "ibm,drc-indexes",
> +				0, &nr_indexes);
> +		if (rc)
> +			goto out_put_np;
> +
> +		/*
> +		 * Retrieve DRC index from the list and return the
> +		 * device node if matched with the specified index.
> +		 */
> +		for (i = 0; i < nr_indexes; i++) {
> +			rc = of_property_read_u32_index(np, "ibm,drc-indexes",
> +							i+1, &index);
> +			if (rc)
> +				goto out_put_np;
> +
> +			if (drc_index == index)
> +				return np;
> +		}
> +	}
> +
> +	return NULL;
> +
> +out_put_np:
> +	of_node_put(np);
> +	return NULL;
> +}
> +
>  static int dlpar_hp_dt_add(u32 index)
>  {
>  	struct device_node *np, *nodes;
> @@ -423,10 +462,19 @@ static int dlpar_hp_dt_add(u32 index)
>  		goto out;
>  	}
>  
> +	/*
> +	 * Recent FW provides ibm,drc-info property. So search
> +	 * for the user specified DRC index from ibm,drc-info
> +	 * property. If this property is not available, search
> +	 * in the indexes array from ibm,drc-indexes property.
> +	 */
>  	np = get_device_node_with_drc_info(index);
>  
> -	if (!np)
> -		return -EIO;
> +	if (!np) {
> +		np = get_device_node_with_drc_indexes(index);
> +		if (!np)
> +			return -EIO;
> +	}
>  
>  	/* Next, configure the connector. */
>  	nodes = dlpar_configure_connector(cpu_to_be32(index), np);
> -- 
> 2.43.5
> 
> 

Re: [PATCH] powerpc/pseries/dlpar: Search DRC index from ibm,drc-indexes for IO add

[Kowshik Jois B S]

[-- Attachment #1: Type: text/plain, Size: 2966 bytes --]

Haren, could you please add the below tag:


Reported-by:Kowshik Jois<kowsjois@linux.ibm.com


Regards,

Kowshik Jois B S


On 01/05/25 09:40, Haren Myneni wrote:
> IO hotplug add event is handled in the user space with drmgr tool.
> After the device is enabled, the user space uses /sys/kerne/dlpar
> interface with “dt add index <drc_index>” to update the device tree.
> The kernel interface (dlpar_hp_dt_add()) finds the parent node for
> the specified ‘drc_index’ from ibm,drc-info property. But old FW
> provides ibm,drc-indexes property instead of ibm,drc-info.
>
> If the ibm,drc-info is not available, this patch adds changes to
> search ‘drc_index’ from the indexes array in ibm,drc-indexes
> property to support old FW.
>
> Fixes: 02b98ff44a57 ("powerpc/pseries/dlpar: Add device tree nodes for DLPAR IO add")
> Signed-off-by: Haren Myneni<haren@linux.ibm.com>
> ---
>   arch/powerpc/platforms/pseries/dlpar.c | 52 +++++++++++++++++++++++++-
>   1 file changed, 50 insertions(+), 2 deletions(-)
>
> diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c
> index 213aa26dc8b3..979487da6522 100644
> --- a/arch/powerpc/platforms/pseries/dlpar.c
> +++ b/arch/powerpc/platforms/pseries/dlpar.c
> @@ -404,6 +404,45 @@ get_device_node_with_drc_info(u32 index)
>   	return NULL;
>   }
>   
> +static struct device_node *
> +get_device_node_with_drc_indexes(u32 drc_index)
> +{
> +	struct device_node *np = NULL;
> +	u32 nr_indexes, index;
> +	int i, rc;
> +
> +	for_each_node_with_property(np, "ibm,drc-indexes") {
> +		/*
> +		 * First element in the array is the total number of
> +		 * DRC indexes returned.
> +		 */
> +		rc = of_property_read_u32_index(np, "ibm,drc-indexes",
> +				0, &nr_indexes);
> +		if (rc)
> +			goto out_put_np;
> +
> +		/*
> +		 * Retrieve DRC index from the list and return the
> +		 * device node if matched with the specified index.
> +		 */
> +		for (i = 0; i < nr_indexes; i++) {
> +			rc = of_property_read_u32_index(np, "ibm,drc-indexes",
> +							i+1, &index);
> +			if (rc)
> +				goto out_put_np;
> +
> +			if (drc_index == index)
> +				return np;
> +		}
> +	}
> +
> +	return NULL;
> +
> +out_put_np:
> +	of_node_put(np);
> +	return NULL;
> +}
> +
>   static int dlpar_hp_dt_add(u32 index)
>   {
>   	struct device_node *np, *nodes;
> @@ -423,10 +462,19 @@ static int dlpar_hp_dt_add(u32 index)
>   		goto out;
>   	}
>   
> +	/*
> +	 * Recent FW provides ibm,drc-info property. So search
> +	 * for the user specified DRC index from ibm,drc-info
> +	 * property. If this property is not available, search
> +	 * in the indexes array from ibm,drc-indexes property.
> +	 */
>   	np = get_device_node_with_drc_info(index);
>   
> -	if (!np)
> -		return -EIO;
> +	if (!np) {
> +		np = get_device_node_with_drc_indexes(index);
> +		if (!np)
> +			return -EIO;
> +	}
>   
>   	/* Next, configure the connector. */
>   	nodes = dlpar_configure_connector(cpu_to_be32(index), np);

[-- Attachment #2: Type: text/html, Size: 8913 bytes --]

Re: [PATCH] powerpc/pseries/dlpar: Search DRC index from ibm,drc-indexes for IO add

[Kowshik Jois B S]

[-- Attachment #1: Type: text/plain, Size: 2975 bytes --]


On 01/05/25 09:40, Haren Myneni wrote:
> IO hotplug add event is handled in the user space with drmgr tool.
> After the device is enabled, the user space uses /sys/kerne/dlpar
> interface with “dt add index <drc_index>” to update the device tree.
> The kernel interface (dlpar_hp_dt_add()) finds the parent node for
> the specified ‘drc_index’ from ibm,drc-info property. But old FW
> provides ibm,drc-indexes property instead of ibm,drc-info.
>
> If the ibm,drc-info is not available, this patch adds changes to
> search ‘drc_index’ from the indexes array in ibm,drc-indexes
> property to support old FW.
>
> Fixes: 02b98ff44a57 ("powerpc/pseries/dlpar: Add device tree nodes for DLPAR IO add")
> Signed-off-by: Haren Myneni<haren@linux.ibm.com>


Haren, Could you please add the below tag:

Reported-by: Kowshik Jois <kowsjois@linux.ibm.com>

Regards,
Kowshik Jois

> ---
>   arch/powerpc/platforms/pseries/dlpar.c | 52 +++++++++++++++++++++++++-
>   1 file changed, 50 insertions(+), 2 deletions(-)
>
> diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c
> index 213aa26dc8b3..979487da6522 100644
> --- a/arch/powerpc/platforms/pseries/dlpar.c
> +++ b/arch/powerpc/platforms/pseries/dlpar.c
> @@ -404,6 +404,45 @@ get_device_node_with_drc_info(u32 index)
>   	return NULL;
>   }
>   
> +static struct device_node *
> +get_device_node_with_drc_indexes(u32 drc_index)
> +{
> +	struct device_node *np = NULL;
> +	u32 nr_indexes, index;
> +	int i, rc;
> +
> +	for_each_node_with_property(np, "ibm,drc-indexes") {
> +		/*
> +		 * First element in the array is the total number of
> +		 * DRC indexes returned.
> +		 */
> +		rc = of_property_read_u32_index(np, "ibm,drc-indexes",
> +				0, &nr_indexes);
> +		if (rc)
> +			goto out_put_np;
> +
> +		/*
> +		 * Retrieve DRC index from the list and return the
> +		 * device node if matched with the specified index.
> +		 */
> +		for (i = 0; i < nr_indexes; i++) {
> +			rc = of_property_read_u32_index(np, "ibm,drc-indexes",
> +							i+1, &index);
> +			if (rc)
> +				goto out_put_np;
> +
> +			if (drc_index == index)
> +				return np;
> +		}
> +	}
> +
> +	return NULL;
> +
> +out_put_np:
> +	of_node_put(np);
> +	return NULL;
> +}
> +
>   static int dlpar_hp_dt_add(u32 index)
>   {
>   	struct device_node *np, *nodes;
> @@ -423,10 +462,19 @@ static int dlpar_hp_dt_add(u32 index)
>   		goto out;
>   	}
>   
> +	/*
> +	 * Recent FW provides ibm,drc-info property. So search
> +	 * for the user specified DRC index from ibm,drc-info
> +	 * property. If this property is not available, search
> +	 * in the indexes array from ibm,drc-indexes property.
> +	 */
>   	np = get_device_node_with_drc_info(index);
>   
> -	if (!np)
> -		return -EIO;
> +	if (!np) {
> +		np = get_device_node_with_drc_indexes(index);
> +		if (!np)
> +			return -EIO;
> +	}
>   
>   	/* Next, configure the connector. */
>   	nodes = dlpar_configure_connector(cpu_to_be32(index), np);

[-- Attachment #2: Type: text/html, Size: 3732 bytes --]

Re: [PATCH] powerpc/pseries/dlpar: Search DRC index from ibm,drc-indexes for IO add

[Tyrel Datwyler]

On 4/30/25 9:10 PM, Haren Myneni wrote:
> IO hotplug add event is handled in the user space with drmgr tool.
> After the device is enabled, the user space uses /sys/kerne/dlpar
> interface with “dt add index <drc_index>” to update the device tree.
> The kernel interface (dlpar_hp_dt_add()) finds the parent node for
> the specified ‘drc_index’ from ibm,drc-info property. But old FW
> provides ibm,drc-indexes property instead of ibm,drc-info.
> 
> If the ibm,drc-info is not available, this patch adds changes to
> search ‘drc_index’ from the indexes array in ibm,drc-indexes
> property to support old FW.
> 
> Fixes: 02b98ff44a57 ("powerpc/pseries/dlpar: Add device tree nodes for DLPAR IO add")
> Signed-off-by: Haren Myneni <haren@linux.ibm.com>
> ---

My only comment would be maybe mentioning that this issue was discovered with a
KVM guest which would be using some version of SLOF firmware as opposed to
PowerVM's proprietary Open Firmware. Older Open Firmware using ibm,drc-indexes
would present the same issue, but is probably less likely to be encountered in
the wild as we got drc-info support showing up in the kernel as far back as 2017
which aligns with Power 7 that has since hit EOL.

Otherwise...

Reviewed-by: Tyrel Datwyler <tyreld@linux.ibm.com>