Re: [PATCH 14/35] autonuma: knuma_migrated per NUMA node queues

Andrea Arcangeli
2012-05-30T00:15:57Z

Hi,

On Tue, May 29, 2012 at 03:51:08PM +0200, Peter Zijlstra wrote:
> On Fri, 2012-05-25 at 19:02 +0200, Andrea Arcangeli wrote:
>
>
> > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> > index 41aa49b..8e578e6 100644
> > --- a/include/linux/mmzone.h
> > +++ b/include/linux/mmzone.h
> > @@ -666,6 +666,12 @@ typedef struct pglist_data {
> > struct task_struct *kswapd;
> > int kswapd_max_order;
> > enum zone_type classzone_idx;
> > +#ifdef CONFIG_AUTONUMA
> > + spinlock_t autonuma_lock;
> > + struct list_head autonuma_migrate_head[MAX_NUMNODES];
> > + unsigned long autonuma_nr_migrate_pages;
> > + wait_queue_head_t autonuma_knuma_migrated_wait;
> > +#endif
> > } pg_data_t;
> >
> > #define node_present_pages(nid) (NODE_DATA(nid)->node_present_pages)
>
> O(nr_nodes^2) data.. ISTR people rewriting a certain slab allocator to
> get rid of that :-)
>
> Also, don't forget that MAX_NUMNODES is an unconditional 512 on distro
> kernels, even when we only have 2.
>
> Now the total wasted space isn't too bad since its only 16 bytes,
> totaling a whole 2M for a 256 node system. But still, something like
> that wants at least a mention somewhere.

I fully agree, I prefer to fix it and I was fully aware about
this. It's not a big deal so it got low priority to be fixed, but I
intended to optimize this.

As long as num_possible_nodes() is initialized before the pgdat is
allocated it shouldn't be difficult to optimize this moving struct
list_head autonuma_migrate_head[0] at the end of the structure.

mm_autonuma and sched_autonuma initially also had MAX_NUMNODES arrays
in them, then I converted to dynamic allocations to be optimal. We
same needs to happen here.

Thanks,
Andrea