Re: Use of AI for process scheduling

[Timothy Miller <miller@techsource.com>]

Jeff Sipek wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> I think that this makes sense. It would definitely help with designing the 
> perfect scheduler. One thing tho, I wouldn't use kgdb or any other debugger, 
> instead I would say that /proc or /sys interface would make more sense. 
> Simply copy the weights somewhere else, dissect them, and then act 
> accordingly.
> 
> Jeff.


Well, we have this to deal with:  Someone is exercising the scheduler 
and notices some kind of misscheduling which causes the system to crawl.

How are they going to get to the /proc and /sys directories to do much 
of anything?  The system is completely unresponsive.  Furthermore, even 
if the system IS responsive, we need some way to for the user to hit a 
key and freeze the current state for examination.  Some slow-downs last 
only seconds, but we need to be able to catch them.


You talk about weights.  Would the linux community be willing to put a 
neural net into the kernel?  I'm sure we could optimize it to not take a 
lot of processing overhead, but it's an "unknown".  It would be scary to 
some people to be unable to disect the actual workings of it and have no 
way of determining corner-case behavior from examining code.  But if we 
have, say, only a 2-layer neural net, we might still be able to 
reverse-engineer it.


One thing I was thinking of is that if there is some information about a 
process which is inconvenient to get normally but could be accessed by a 
kernel or user-space thread, then this process could feed back info to 
the scheduler AI for it to do automatic tuning.  Some information that 
the scheduler doesn't have direct (or efficient) access to could be 
inferred from information it DOES have access to, and that would be 
learned through adjustment of weights via backpropogation.