Re: [PATCH 5/9] mm, page_alloc: make per_cpu_pageset accessible only after init

Vlastimil Babka
2020-10-06T22:28:07Z

On 10/5/20 3:24 PM, Michal Hocko wrote:
> On Tue 22-09-20 16:37:08, Vlastimil Babka wrote:
>> setup_zone_pageset() replaces the boot_pageset by allocating and initializing a
>> proper percpu one. Currently it assigns zone->pageset with the newly allocated
>> one before initializing it. That's currently not an issue, because the zone
>> should not be in any zonelist, thus not visible to allocators at this point.
>>
>> Memory ordering between the pcplist contents and its visibility is also not
>> guaranteed here, but that also shouldn't be an issue because online_pages()
>> does a spin_unlock(pgdat->node_size_lock) before building the zonelists.
>>
>> However it's best that we don't silently rely on operations that can be changed
>> in the future. Make sure only properly initialized pcplists are visible, using
>> smp_store_release(). The read side has a data dependency via the zone->pageset
>> pointer instead of an explicit read barrier.
>
> Heh, this looks like inveting a similar trap the previous patch was
> removing. But more seriously considering that we need a locking for the
> whole setup, wouldn't it be better to simply document the locking
> requirements rather than adding scary looking barriers future ourselves
> or somebody else will need to scratch heads about. I am pretty sure we
> don't do anything like that when initializating numa node or other data
> structures that might be allocated during the memory hotadd.

Yeah it will be best to drop this for now. I just looked closed at
build_zonelist() implementation and got scared how it just modifies the list
that others might be reading at the same time. zoneref_set_zone() sets
zoneref->zone and zoneref->zone_idx without any sync, what if somebody, e.g.
for_next_zone_zonelist_nodemask() makes decision based on zone_idx and then
picks wrong zone pointer? etc.

>> Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
>> ---
>> mm/page_alloc.c | 6 ++++--
>> 1 file changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index 99b74c1c2b0a..de3b48bda45c 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -6246,15 +6246,17 @@ static void zone_set_pageset_high_and_batch(struct zone *zone)
>>
>> void __meminit setup_zone_pageset(struct zone *zone)
>> {
>> + struct per_cpu_pageset __percpu * new_pageset;
>> struct per_cpu_pageset *p;
>> int cpu;
>>
>> - zone->pageset = alloc_percpu(struct per_cpu_pageset);
>> + new_pageset = alloc_percpu(struct per_cpu_pageset);
>> for_each_possible_cpu(cpu) {
>> - p = per_cpu_ptr(zone->pageset, cpu);
>> + p = per_cpu_ptr(new_pageset, cpu);
>> pageset_init(p);
>> }
>>
>> + smp_store_release(&zone->pageset, new_pageset);
>> zone_set_pageset_high_and_batch(zone);
>> }
>>
>> --
>> 2.28.0
>