Re: [PATCH 1/3] powerpc: Allocate per-cpu areas for node IDs for SLQB to use as per-node areas

Mel Gorman
2009-09-22T09:32:18Z

On Tue, Sep 22, 2009 at 09:01:55AM +0900, Tejun Heo wrote:
> Mel Gorman wrote:
> > diff --git a/arch/powerpc/kernel/setup_64.c b/arch/powerpc/kernel/setup_64.c
> > index 1f68160..a5f52d4 100644
> > --- a/arch/powerpc/kernel/setup_64.c
> > +++ b/arch/powerpc/kernel/setup_64.c
> > @@ -588,6 +588,26 @@ void __init setup_per_cpu_areas(void)
> > paca[i].data_offset = ptr - __per_cpu_start;
> > memcpy(ptr, __per_cpu_start, __per_cpu_end - __per_cpu_start);
> > }
> > +#ifdef CONFIG_SLQB
> > + /*
> > + * SLQB abuses DEFINE_PER_CPU to setup a per-node area. This trick
> > + * assumes that ever node ID will have a CPU of that ID to match.
> > + * On systems with memoryless nodes, this may not hold true. Hence,
> > + * we take a second pass initialising a "per-cpu" area for node-ids
> > + * that SLQB can use
> > + */
> > + for_each_node_state(i, N_NORMAL_MEMORY) {
> > +
> > + /* Skip node IDs that a valid CPU id exists for */
> > + if (paca[i].data_offset)
> > + continue;
> > +
> > + ptr = alloc_bootmem_pages_node(NODE_DATA(cpu_to_node(i)), size);
> > +
> > + paca[i].data_offset = ptr - __per_cpu_start;
> > + memcpy(ptr, __per_cpu_start, __per_cpu_end - __per_cpu_start);
> > + }
> > +#endif /* CONFIG_SLQB */
> > }
> > #endif
>
> Eh... I don't know. This seems too hacky to me.

I've come around to this opinion as well. There are probably too many
other architectures and corners where this is gotten wrong and a more
fundamental fix is needed for SLQB to be able to use this hack.

> Why not just
> allocate pointer array of MAX_NUMNODES and allocate per-node memory
> there?

That's what patch 1 from V1 did. I'll make it Patch 1 for V3.

> This will be slightly more expensive but I doubt it will be
> noticeable. The only extra overhead is the cachline footprint for the
> extra array.
>

I'll compare the vmlinux's to quantify the exact penalty but basically,
I don't think it can be avoided at this point.

Thanks.

--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab