Re: [PATCH take2] Memoryless nodes: use "node_memory_map" for cpuset mems_allowed validation

Nishanth Aravamudan
2007-07-25T14:46:33Z

On 24.07.2007 [15:01:33 -0400], Lee Schermerhorn wrote:
> On Tue, 2007-07-24 at 09:19 -0700, Nishanth Aravamudan wrote:
> > On 24.07.2007 [10:15:25 -0400], Lee Schermerhorn wrote:
> > > Memoryless Nodes: use "node_memory_map" for cpusets - take 2
> > >
> > > Against 2.6.22-rc6-mm1 atop Christoph Lameter's memoryless nodes
> > > series
> > >
> > > take 2:
> > > + replaced node_online_map in cpuset_current_mems_allowed()
> > > with node_states[N_MEMORY]
> > > + replaced node_online_map in cpuset_init_smp() with
> > > node_states[N_MEMORY]
> > >
> > > cpusets try to ensure that any node added to a cpuset's
> > > mems_allowed is on-line and contains memory. The assumption
> > > was that online nodes contained memory. Thus, it is possible
> > > to add memoryless nodes to a cpuset and then add tasks to this
> > > cpuset. This results in continuous series of oom-kill and
> > > apparent system hang.
> > >
> > > Change cpusets to use node_states[N_MEMORY] [a.k.a.
> > > node_memory_map] in place of node_online_map when vetting
> > > memories. Return error if admin attempts to write a non-empty
> > > mems_allowed node mask containing only memoryless-nodes.
> >
> > I think you still are missing a few comment changes (anything mentioning
> > 'track'ing node_online_map will need to be changed, I think). Also, I
> > don't see the necessary change in common_cpu_mem_hotplug_unplug()
> > similar to cpuset_init_smp()'s change.
>
> Sorry. Multitasking meltdown... Will fix.
>
> Meanwhile:
>
> I've tested your 3 patches atop Christoph's series [on 22-rc6-mm1],
> with and without my cpuset patch and I can't reproduce the hang I saw
> a couple of days ago :-(. I hate it when that happens! Perhaps some
> system daemon started up during the test that hung.

Hrm, that stinks. I tested on several h/w variations before posting. And
the changes are pretty transparent, so I'm not sure where we'd hang (and
I would think if we were, we'd see it now too). But I'll do another
audit just to be sure.

Thanks,
Nish

--
Nishanth Aravamudan <nacc@us.ibm.com>
IBM Linux Technology Center