Re: [FIX PATCH v0] powerpc: Fix memory unplug failure on radix guest

[Bharata B Rao <bharata@linux.vnet.ibm.com>]

On Fri, Aug 11, 2017 at 02:12:04PM +0530, Aneesh Kumar K.V wrote:
> Bharata B Rao <bharata@linux.vnet.ibm.com> writes:
> 
> > For a PowerKVM guest, it is possible to specify a DIMM device in
> > addition to the system RAM at boot time. When such a cold plugged DIMM
> > device is removed from a radix guest, we hit the following warning in the
> > guest kernel resulting in the eventual failure of memory unplug:
> >
> > remove_pud_table: unaligned range
> > WARNING: CPU: 3 PID: 164 at arch/powerpc/mm/pgtable-radix.c:597 remove_pagetable+0x468/0xca0
> > Call Trace:
> > remove_pagetable+0x464/0xca0 (unreliable)
> > radix__remove_section_mapping+0x24/0x40
> > remove_section_mapping+0x28/0x60
> > arch_remove_memory+0xcc/0x120
> > remove_memory+0x1ac/0x270
> > dlpar_remove_lmb+0x1ac/0x210
> > dlpar_memory+0xbc4/0xeb0
> > pseries_hp_work_fn+0x1a4/0x230
> > process_one_work+0x1cc/0x660
> > worker_thread+0xac/0x6d0
> > kthread+0x16c/0x1b0
> > ret_from_kernel_thread+0x5c/0x74
> >
> > The DIMM memory that is cold plugged gets merged to the same memblock
> > region as RAM and hence gets mapped at 1G alignment. However since the
> > removal is done for one LMB (lmb size 256MB) at a time, the address
> > of the LMB (which is 256MB aligned) would get flagged as unaligned
> > in remove_pud_table() resulting in the above failure.
> >
> > This problem is not seen for hot plugged memory because for the
> > hot plugged memory, the mappings are created separately for each
> > LMB and hence they all get aligned at 256MB.
> >
> > To fix this problem for the cold plugged memory, let us mark the
> > cold plugged memblock region explicitly as HOTPLUGGED so that the
> > region doesn't get merged with RAM. All the memory that is discovered
> > via ibm,dynamic-memory-configuration is marked so(1). Next identify
> > such regions in radix_init_pgtable() and create separate mappings
> > within that region for each LMB so that they get don't get aligned
> > like RAM region at 1G (2).
> >
> > (1) For PowerKVM guests, all boot time memory is represented via
> > memory@XXXX nodes and hot plugged/pluggable memory is represented via
> > ibm,dynamic-memory-reconfiguration property. We are marking all
> > hotplugged memory that is in ASSIGNED state during boot as HOTPLUGGED.
> > With this only cold plugged memory gets marked for PowerKVM but
> > need to check how this will affect PowerVM guests.
> 
> Can you verify this on PowerVM too ? ie we should in most case not find
> anything under ibm,dynamic-memory-reconfiguration ?

Checked with a couple of PowerVM systems. Look like except for RMA which
is represented by memory@0, rest of the memory is coming under
ibm,dynamic-memory-reconfiguration. So the approach I have taken in this
fix wouldn't be optimal on PowerVM ?

Regards,
Bharata.