Re: How to notify app of changed cpu/mem/io node configuration?

[Paul Jackson <pj@sgi.com>]

Dave wrote:
> (in response to my query of whether you meant "sync" or "async"):
> Some kind of notification that the kernel sends where it can be sure
> that the app received it.

Huh - I'm confused as to what you think is essential here.  I have this
sense that you are saying we need reliably acknowledged notification.
But this seems impossible to me - one cannot have the kernel depending
on some random bit of user code sending a timely acknowledgement, much
less on such code doing something competent with the notice.

Whatever ... in any case ... one element that I consider essential is an
agreed to convention whereby a system service such as a batch manager
can send to the several tasks in a job "fair and adequate" notification
that they (the tasks) had better pack up and move - or be shot if they
don't.  In particular, the mbind() and set_mempolicy() calls can only be
done by a task on itself, not by some third party.   I doubt that the
kernel is party to or even aware of this convention.

> Rudely being migrated to CPU 0 would break the app,
> and could (in theory) cause data corruption.  

Then I am surpised that you accept the following bit of code, now
in Linus' 2.6.7:

  kernel/sched.c:migrate_all_tasks()

                if (dest_cpu = NR_CPUS) {
                        cpus_setall(tsk->cpus_allowed);
                        dest_cpu = any_online_cpu(tsk->cpus_allowed);

The usual affect of this is to force cpus_allowed to all possible CPUs,
and to set "dest_cpu = 0".  It happens in the case of a migration, when
none of the remaining CPUs allowed to a task are online.

Hmmm ... thinking out loud ... why not set cpus_allowed to the single
cpu which is the lowest numbered online cpu, rather than all cpus?
I see no need to give orphans the run of the entire city!

> > ...  before it told the kernel "all clear - remove old Node from service"
> 
> I just worry about the complexity of feeding all of this information
> back to the kernel. 

Huh - what's to tell the kernel at this step?  "Remove CPU or Memory
node X from service, with ruthless abandon to any residual user tasks or
memory left there" -- that's all I see.

> I think we need Rusty to remind us all ...

Good idea !!

-- 
                          I won't rest till it's the best ...
                          Programmer, Linux Scalability
                          Paul Jackson <pj@sgi.com> 1.650.933.1373


-------------------------------------------------------
This SF.Net email sponsored by Black Hat Briefings & Training.
Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
digital self defense, top technical experts, no vendor pitches, 
unmatched networking opportunities. Visit www.blackhat.com
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel