Re: vio.c:__vio_register_driver && LPAR Migration issue

[Michael Bringmann <mwb@linux.vnet.ibm.com>]

On 08/04/2018 06:01 PM, Tyrel Datwyler wrote:
> On 08/03/2018 02:23 PM, Tyrel Datwyler wrote:
>> On 08/02/2018 11:15 AM, Michael Bringmann wrote:
>>> Hello:
>>>     I have been observing an anomaly during LPAR migrations between
>>> a couple of P8 systems.
>>>
>>> This is the problem.  After migrating an LPAR, the PPC mobility code
>>> receives RTAS requests to delete nodes with platform-/hardware-specific
>>> attributes when restarting the kernel after a migration.  My example is
>>> for migration between a P8 Alpine and a P8 Brazos.  Among the nodes
>>> that I see being deleted are 'ibm,random-v1', 'ibm,compression-v1',
>>> 'ibm,platform-facilities', and 'ibm,sym-encryption-v1'.  Of these
>>> nodes, the following are created during initial boot by calls to
>>> vio_register_driver:
>>>
>>> drivers/char/hw_random/pseries-rng.c
>>>         ibm,random-v1
>>>
>>> drivers/crypto/nx/nx-842-pseries.c
>>>         ibm,compression-v1
>>>
>>> drivers/crypto/nx/nx.c
>>>         ibm,sym-encryption-v1
>>>
>>> After the migration, these nodes are deleted, but nothing recreates
>>> them.  If I boot the LPAR on the target system, the nodes are added
>>> again.
>>>
>>> My question is how do we recreate these nodes after migration?
>>
>> Hmm, I'd have to see the scenario in action, but these should be added back by ibm,update-nodes RTAS call. There is some debug code in driver/of/dynamic.c that can be enabled that will log node/property dynamic reconfiguration events.
>>
> 
> I took a quick look by turning of phandle caching and turning on the OF reconfig debug on one of my lpars. I noticed that it looks like this could be a firmware issue. Looks to me like we aren't being notified to re-add those child nodes of ibm,platform-facilities.
> 
> [ 1671.638041] OF: notify DETACH_NODE     /cpus/l2-cache@2022
> [ 1671.638094] OF: notify DETACH_NODE     /cpus/l3-cache@3122
> [ 1671.638119] OF: notify DETACH_NODE     /cpus/l2-cache@2023
> [ 1671.638136] OF: notify DETACH_NODE     /cpus/l3-cache@3123
> [ 1671.638164] OF: notify DETACH_NODE     /ibm,platform-facilities/ibm,random-v1
> [ 1671.638182] OF: notify DETACH_NODE     /ibm,platform-facilities/ibm,compression-v1
> [ 1671.638198] OF: notify DETACH_NODE     /ibm,platform-facilities/ibm,sym-encryption-v1
> [ 1671.638219] OF: notify DETACH_NODE     /ibm,platform-facilities
> [ 1671.776419] OF: notify UPDATE_PROPERTY /:ibm,model-class
> 
> ... snipped property updates
> 
> [ 1672.129941] OF: notify UPDATE_PROPERTY /cpus/PowerPC,POWER8@8:ibm,dfp
> [ 1672.147551] OF: notify ATTACH_NODE     /cpus/l2-cache@202e
> [ 1672.166321] OF: notify ATTACH_NODE     /cpus/l3-cache@312e
> [ 1672.183971] OF: notify ATTACH_NODE     /cpus/l2-cache@202f
> [ 1672.202752] OF: notify ATTACH_NODE     /cpus/l3-cache@312f
> [ 1672.230760] OF: notify ATTACH_NODE     /ibm,platform-facilities
> 
> Need to verify this by tracing the RTAS calls to ibm,update-nodes. I'll try and look at that tomorrow. The loop that processes the nodes to update in pseries_devicetree_update() blindly ignores the return codes from delete_dt_node(), update_dt_node(), and add_dt_node(). So, it is also possible that we are notified, but are silently failing the add.

I am seeing the same set of detach/attach events for migration between an Alpine and Brazos.
I did the logging up in delete_dt_node, update_dt_node, and add_dt_node, and did not see any
notification of additional nodes.

> 
> -Tyrel
> 

Michael

-- 
Michael W. Bringmann
Linux Technology Center
IBM Corporation
Tie-Line  363-5196
External: (512) 286-5196
Cell:       (512) 466-0650
mwb@linux.vnet.ibm.com