RE: RFC: THE OFFLINE SCHEDULER

Chetan.Loke
2009-08-26T15:54:56Z

> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel-
> owner@vger.kernel.org] On Behalf Of raz ben yehuda
> Sent: Wednesday, August 26, 2009 10:54 AM
> To: Maxim Levitsky
> Cc: Christoph Lameter; Peter Zijlstra; Chris Friesen; Mike Galbraith;
> riel@redhat.com; mingo@elte.hu; andrew motron; wiseman@macs.biu.ac.il;
> lkml; linux-rt-users@vger.kernel.org
> Subject: Re: RFC: THE OFFLINE SCHEDULER
>
>
> On Wed, 2009-08-26 at 17:45 +0300, Maxim Levitsky wrote:
> > On Wed, 2009-08-26 at 09:47 -0400, Christoph Lameter wrote:
> > > On Wed, 26 Aug 2009, raz ben yehuda wrote:
> > >
> > > > How will the kernel is going to handle 32 processors machines ?
> These
> > > > numbers are no longer a science-fiction.
> > >
> > > The kernel is already running on 4096 processor machines. Dont worry
> about
> > > that.
> > >
> > > > What i am suggesting is merely a different approach of how to handle
> > > > multiple core systems. instead of thinking in processes, threads and
> so
> > > > on i am thinking in services. Why not take a processor and define
> this
> > > > processor to do just firewalling ? encryption ? routing ?
> transmission ?
> > > > video processing... and so on...
> > >
> > > I think that is a valuable avenue to explore. What we do so far is
> > > treating each processor equally. Dedicating a processor has benefits
> in
> > > terms of cache hotness and limits OS noise.
> > >
> > > Most of the large processor configurations already partition the
> system
> > > using cpusets in order to limit the disturbance by OS processing. A
> set of
> > > cpus is used for OS activities and system daemons are put into that
> set.
> > > But what can be done is limited because the OS threads as well as
> > > interrupt and timer processing etc cannot currently be moved. The
> ideas
> > > that you are proposing are particularly usedful for applications that
> > > require low latencies and cannot tolerate OS noise easily (Infiniband
> MPI
> > > base jobs f.e.)
> >
> > My 0.2 cents:
> >
> > I have always been fascinated by the idea of controlling another cpu
> > from the main CPU.
> >
> > Usually these cpus are custom, run proprietary software, and have no
> > datasheet on their I/O interfaces.
> >
> > But, being able to turn an ordinary CPU into something like that seems
> > to be very nice.
> >
> > For example, It might help with profiling. Think about a program that
> > can run uninterrupted how much it wants.
> >
> > I might even be better, if the dedicated CPU would use a predefined
> > reserved memory range (I wish there was a way to actually lock it to
> > that range)
> >
> > On the other hand, I could see this as a jump platform for more
> > proprietary code, something like that: we use linux in out server
> > platform, but out "insert buzzword here" network stack pro+ can handle
> > 100% more load that linux does, and it runs on a dedicated core....
> >
> > In the other words, we might see 'firmwares' that take an entire cpu for
> > their usage.
> This is exactly what offsched (sos) is. you got it. SOS was partly
> inspired by the notion of a GPU.
> Processors are to become more and more redundant and Linux as an
> evolutionary system must use it. why not offload raid5 write engine ?
> why not encrypt in a different processor ?

RAID/Encryption + GPU. You got it. This is what one of our teams did but by offloading it on a PCIe I/O module and using couple(Protocol+Application core) of cores. One core could run SAS/SATA and other could run your home grown application f/w and/or a linux distro and you could make it do whatever you want. But that was then. Multi-Core systems are now a commodity.

Chetan Loke