Re: [PATCH 2/3] hotplug-memory.c: enhance dlpar_memory_remove* LMB checks

[Daniel Henrique Barboza <danielhb413@gmail.com>]

On 5/3/21 10:02 PM, David Gibson wrote:
> On Fri, Apr 30, 2021 at 09:09:16AM -0300, Daniel Henrique Barboza wrote:
>> dlpar_memory_remove_by_ic() validates the amount of LMBs to be removed
>> by checking !DRCONF_MEM_RESERVED, and in the following loop before
>> dlpar_remove_lmb() a check for DRCONF_MEM_ASSIGNED is made before
>> removing it. This means that a LMB that is both !DRCONF_MEM_RESERVED and
>> !DRCONF_MEM_ASSIGNED will be counted as valid, but then not being
>> removed.  The function will end up not removing all 'lmbs_to_remove'
>> LMBs while also not reporting any errors.
>>
>> Comparing it to dlpar_memory_remove_by_count(), the validation is done
>> via lmb_is_removable(), which checks for DRCONF_MEM_ASSIGNED and fadump
>> constraints. No additional check is made afterwards, and
>> DRCONF_MEM_RESERVED is never checked before dlpar_remove_lmb(). The
>> function doesn't have the same 'check A for validation, then B for
>> removal' issue as remove_by_ic(), but it's not checking if the LMB is
>> reserved.
>>
>> There is no reason for these functions to validate the same operation in
>> two different manners.
> 
> Actually, I think there is: remove_by_ic() is handling a request to
> remove a specific range of LMBs.  If any are reserved, they can't be
> removed and so this needs to fail.  But if they are !ASSIGNED, that
> essentially means they're *already* removed (or never added), so
> "removing" them is, correctly, a no-op.
> 
> remove_by_count(), in contrast, is being asked to remove a fixed
> number of LMBs from wherever they can be found, and for that it needs
> to find LMBs that haven't already been removed.
> 
> Basically remove_by_ic() is an absolute request: "make this set of
> LMBs be not-plugged", whereas remove_by_count() is a relative request
> "make N less LMBs be plugged".
> 
> 
> So I think remove_by_ic()s existing handling is correct.  I'm less
> sure if remove_by_count() ignoring RESERVED is correct - I couldn't
> quickly find under what circumstances RESERVED gets set.

RESERVED is never set by the kernel. It is written in the DT by the
firmware/hypervisor and the kernel just checks its value. QEMU sets it in
spapr_dt_dynamic_memory() with the following comment:


             /*
              * LMB information for RMA, boot time RAM and gap b/n RAM and
              * device memory region -- all these are marked as reserved
              * and as having no valid DRC.
              */
             dynamic_memory[0] = cpu_to_be32(addr >> 32);
             dynamic_memory[1] = cpu_to_be32(addr & 0xffffffff);
             dynamic_memory[2] = cpu_to_be32(0);
             dynamic_memory[3] = cpu_to_be32(0); /* reserved */
             dynamic_memory[4] = cpu_to_be32(-1);
             dynamic_memory[5] = cpu_to_be32(SPAPR_LMB_FLAGS_RESERVED |
                                             SPAPR_LMB_FLAGS_DRC_INVALID);


The flag is formally described in LOPAR section 4.2.8, "Reserved Memory":

"Memory nodes marked with the special value of the “status” property of
“reserved” is not to be used or altered by the base OS."


This makes me confident that we should check DRCONF_MEM_RESERVED in
remove_by_count() as well, since phyp needs do adhere to these semantics and
shouldn't be able to remove a LMB marked as RESERVED.



Thanks,



Daniel




> 
> 
>> This patch addresses that by changing
>> lmb_is_removable() to also check for DRCONF_MEM_RESERVED to tell if a
>> lmb is removable, making dlpar_memory_remove_by_count() take the
>> reservation state into account when counting the LMBs.
>> lmb_is_removable() is then used in the validation step of
>> dlpar_memory_remove_by_ic(), which is already checking for both states
>> but in different stages, to avoid counting a LMB that is not assigned as
>> eligible for removal. We can then skip the check before
>> dlpar_remove_lmb() since we're validating all LMBs beforehand.
>>
>> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
>> ---
>>   arch/powerpc/platforms/pseries/hotplug-memory.c | 8 +++-----
>>   1 file changed, 3 insertions(+), 5 deletions(-)
>>
>> diff --git a/arch/powerpc/platforms/pseries/hotplug-memory.c b/arch/powerpc/platforms/pseries/hotplug-memory.c
>> index bb98574a84a2..4e6d162c3f1a 100644
>> --- a/arch/powerpc/platforms/pseries/hotplug-memory.c
>> +++ b/arch/powerpc/platforms/pseries/hotplug-memory.c
>> @@ -348,7 +348,8 @@ static int pseries_remove_mem_node(struct device_node *np)
>>   
>>   static bool lmb_is_removable(struct drmem_lmb *lmb)
>>   {
>> -	if (!(lmb->flags & DRCONF_MEM_ASSIGNED))
>> +	if ((lmb->flags & DRCONF_MEM_RESERVED) ||
>> +		!(lmb->flags & DRCONF_MEM_ASSIGNED))
>>   		return false;
>>   
>>   #ifdef CONFIG_FA_DUMP
>> @@ -523,7 +524,7 @@ static int dlpar_memory_remove_by_ic(u32 lmbs_to_remove, u32 drc_index)
>>   
>>   	/* Validate that there are enough LMBs to satisfy the request */
>>   	for_each_drmem_lmb_in_range(lmb, start_lmb, end_lmb) {
>> -		if (lmb->flags & DRCONF_MEM_RESERVED)
>> +		if (!lmb_is_removable(lmb))
>>   			break;
>>   
>>   		lmbs_available++;
>> @@ -533,9 +534,6 @@ static int dlpar_memory_remove_by_ic(u32 lmbs_to_remove, u32 drc_index)
>>   		return -EINVAL;
>>   
>>   	for_each_drmem_lmb_in_range(lmb, start_lmb, end_lmb) {
>> -		if (!(lmb->flags & DRCONF_MEM_ASSIGNED))
>> -			continue;
>> -
>>   		rc = dlpar_remove_lmb(lmb);
>>   		if (rc)
>>   			break;
>