Trying out SCHED_BATCH

Trying out SCHED_BATCH

[Rajsekar]

I would like to try out the SCHED_BATCH.  Unfortunately, I am not able to
find a patch for my kernel.  Could someone enlighten me on this?

I am running 2.6.10-rc1-mm2 with staircase scheduler patch.  My `uname -a'
output is:

Linux rajsekar.pc 2.6.10-rc1-mm2staircase #2 Sat Dec 4 10:49:31 IST 2004 i686 AuthenticAMD unknown GNU/Linux

-- 
    Rajsekar

Re: Trying out SCHED_BATCH

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 808 bytes --]

Rajsekar wrote:
> I would like to try out the SCHED_BATCH.  Unfortunately, I am not able to
> find a patch for my kernel.  Could someone enlighten me on this?
> 
> I am running 2.6.10-rc1-mm2 with staircase scheduler patch.  My `uname -a'
> output is:
> 
> Linux rajsekar.pc 2.6.10-rc1-mm2staircase #2 Sat Dec 4 10:49:31 IST 2004 i686 AuthenticAMD unknown GNU/Linux
> 

Only the staircase scheduler currently has an implementation of 
sched_batch and you need 2 more patches on top of the staircase patch 
for it to work. The most current version for 2.6.10 you can get with 
this patch sequence from the latest -ck patchset (or you could just use 
the full -ck patch)

http://ck.kolivas.org/patches/2.6/2.6.10/2.6.10-ck1/patches/

2.6.10_to_staircase9.2.diff
schedrange.diff
schedbatch2.6.diff

Cheers,
Con

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: Trying out SCHED_BATCH

[Matan Peled]

[-- Attachment #1: Type: text/plain, Size: 676 bytes --]

Rajsekar wrote:
> I would like to try out the SCHED_BATCH.  Unfortunately, I am not able to
> find a patch for my kernel.  Could someone enlighten me on this?
> 
> I am running 2.6.10-rc1-mm2 with staircase scheduler patch.  My `uname -a'
> output is:
> 
> Linux rajsekar.pc 2.6.10-rc1-mm2staircase #2 Sat Dec 4 10:49:31 IST 2004 i686 AuthenticAMD unknown GNU/Linux
> 

2.6.10-ck1 can be found here (It has SCHED_BATCH):

http://members.optusnet.com.au/ckolivas/kernel/

-- 
[Name      ]   ::  [Matan I. Peled    ]
[Location  ]   ::  [Israel            ]
[Public Key]   ::  [0xD6F42CA5        ]
[Keyserver ]   ::  [keyserver.kjsl.com]
encrypted/signed  plaintext   preffered


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Trying out SCHED_BATCH

[Bill Davidsen]

Rajsekar wrote:
> I would like to try out the SCHED_BATCH.  Unfortunately, I am not able to
> find a patch for my kernel.  Could someone enlighten me on this?
> 
> I am running 2.6.10-rc1-mm2 with staircase scheduler patch.  My `uname -a'
> output is:
> 
> Linux rajsekar.pc 2.6.10-rc1-mm2staircase #2 Sat Dec 4 10:49:31 IST 2004 i686 AuthenticAMD unknown GNU/Linux
> 
See the announcement of 2.6.10-ck1, that has what you want. However, you 
want to read the whole thread, as there is one patch WRT swap_token 
which you may want to revert.

-- 
bill davidsen <davidsen@tmr.com>
   CTO TMR Associates, Inc
   Doing interesting things with small computers since 1979

Re: Trying out SCHED_BATCH

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 658 bytes --]

Bill Davidsen wrote:
> Rajsekar wrote:
> 
>> I would like to try out the SCHED_BATCH.  Unfortunately, I am not able to
>> find a patch for my kernel.  Could someone enlighten me on this?
>>
>> I am running 2.6.10-rc1-mm2 with staircase scheduler patch.  My `uname 
>> -a'
>> output is:
>>
>> Linux rajsekar.pc 2.6.10-rc1-mm2staircase #2 Sat Dec 4 10:49:31 IST 
>> 2004 i686 AuthenticAMD unknown GNU/Linux
>>
> See the announcement of 2.6.10-ck1, that has what you want. However, you 
> want to read the whole thread, as there is one patch WRT swap_token 
> which you may want to revert.

That patch is harmless so there is no need to revert it.

Cheers,
Con

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: Trying out SCHED_BATCH

[Maciej Soltysiak]

Hi

Con wrote:
> Only the staircase scheduler currently has an implementation of
> sched_batch and you need 2 more patches on top of the staircase patch
> for it to work.
Hmm, Is it feasable to write a sched_batch policy for the current linux 
schedulers?

I mean, if there are people that want it bad, maybe it would be nice to be 
able
to use a version of sched_batch that would work without the staircase 
scheduler.
It is still experimental, right?

Regards,
Maciej

Re: Trying out SCHED_BATCH

[Con Kolivas]

[-- Attachment #1: Type: text/plain, Size: 1322 bytes --]

Maciej Soltysiak wrote:
> Hi
> 
> Con wrote:
> 
>> Only the staircase scheduler currently has an implementation of
>> sched_batch and you need 2 more patches on top of the staircase patch
>> for it to work.
> 
> Hmm, Is it feasable to write a sched_batch policy for the current linux 
> schedulers?

Yes.

The proper way to make a sched_batch implementation is more 
comprehensive than what is made for staircase to prevent a deadlock 
based on a batch task getting an important lock in the kernel and not 
being able to release it due to a sched_normal task being higher 
priority than it that is actually trying to get the lock. There is code 
in the staircase version to prevent this from happening but probably not 
complete enough in design to prevent everything. However it works and I 
haven't had any reports of lockups since I implemented the extra checking.

Would you like me to create a version like that? I don't have the time 
to try and make a more comprehensive solution and follow the debugging 
of such a beast.

 > I mean, if there are people that want it bad, maybe it would be nice to
 > be able
 > to use a version of sched_batch that would work without the staircase
 > scheduler.
 > It is still experimental, right?

No it's not experimental. It is very stable and used in production systems.

Con

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: Trying out SCHED_BATCH

[Paolo Ciarrocchi]

On Thu, 30 Dec 2004 07:28:35 +1100, Con Kolivas <kernel@kolivas.org> wrote:
> Maciej Soltysiak wrote:
> > Hi
> >
> > Con wrote:
> >
> >> Only the staircase scheduler currently has an implementation of
> >> sched_batch and you need 2 more patches on top of the staircase patch
> >> for it to work.
> >
> > Hmm, Is it feasable to write a sched_batch policy for the current linux
> > schedulers?
> 
> Yes.
> 
> The proper way to make a sched_batch implementation is more
> comprehensive than what is made for staircase to prevent a deadlock
> based on a batch task getting an important lock in the kernel and not
> being able to release it due to a sched_normal task being higher
> priority than it that is actually trying to get the lock. There is code
> in the staircase version to prevent this from happening but probably not
> complete enough in design to prevent everything. However it works and I
> haven't had any reports of lockups since I implemented the extra checking.
> 
> Would you like me to create a version like that? I don't have the time
> to try and make a more comprehensive solution and follow the debugging
> of such a beast.
> 
> > I mean, if there are people that want it bad, maybe it would be nice to
> > be able
> > to use a version of sched_batch that would work without the staircase
> > scheduler.
> > It is still experimental, right?
> 
> No it's not experimental. It is very stable and used in production systems.

Are you gointo  to push to Linus/Andrew ?

-- 
Paolo

Re: Trying out SCHED_BATCH

[Con Kolivas]

Paolo Ciarrocchi wrote:
> On Thu, 30 Dec 2004 07:28:35 +1100, Con Kolivas <kernel@kolivas.org> wrote:
> 
>>Maciej Soltysiak wrote:
>>
>>>Hi
>>>
>>>Con wrote:
>>>
>>>
>>>>Only the staircase scheduler currently has an implementation of
>>>>sched_batch and you need 2 more patches on top of the staircase patch
>>>>for it to work.
>>>
>>>Hmm, Is it feasable to write a sched_batch policy for the current linux
>>>schedulers?
>>
>>Yes.
>>
>>The proper way to make a sched_batch implementation is more
>>comprehensive than what is made for staircase to prevent a deadlock
>>based on a batch task getting an important lock in the kernel and not
>>being able to release it due to a sched_normal task being higher
>>priority than it that is actually trying to get the lock. There is code
>>in the staircase version to prevent this from happening but probably not
>>complete enough in design to prevent everything. However it works and I
>>haven't had any reports of lockups since I implemented the extra checking.
>>
>>Would you like me to create a version like that? I don't have the time
>>to try and make a more comprehensive solution and follow the debugging
>>of such a beast.
>>
>>
>>>I mean, if there are people that want it bad, maybe it would be nice to
>>>be able
>>>to use a version of sched_batch that would work without the staircase
>>>scheduler.
>>>It is still experimental, right?
>>
>>No it's not experimental. It is very stable and used in production systems.
> 
> 
> Are you gointo  to push to Linus/Andrew ?

Staircase? I'm still in pain from the last time I tried to push it in a 
more palatable form via the plugsched architecture which took me a long 
time to do. I don't have the fortitude to go through that again in a hurry.

Cheers,
Con

Re: Trying out SCHED_BATCH

[Paolo Ciarrocchi]

On Thu, 30 Dec 2004 09:56:03 +1100, Con Kolivas <kernel@kolivas.org> wrote:
> Paolo Ciarrocchi wrote:
> > Are you gointo  to push to Linus/Andrew ?
> 
> Staircase? I'm still in pain from the last time I tried to push it in a
> more palatable form via the plugsched architecture which took me a long
> time to do. I don't have the fortitude to go through that again in a hurry.

Yup, I remember your tentative.

Andrew, 
what's your plan for the staircase scheduler ?

-- 
Paolo

Re: Trying out SCHED_BATCH

[Andrew Morton]

Paolo Ciarrocchi <paolo.ciarrocchi@gmail.com> wrote:
>
> Andrew, 
>  what's your plan for the staircase scheduler ?

I have none, frankly.  I haven't seen any complaints about the current
scheduler.

If someone can identify bad behaviour in the current scheduler which
staircase improves then please describe a tescase which the scheduler
developers can use to reproduce the situation.

If, after that, we deem that the problem cannot be feasibly fixed within the
context of the current scheduler and that the problem is sufficiently
serious to justify wholesale replacement of the scheduler then sure,
staircase is an option.

Re: Trying out SCHED_BATCH

[Paolo Ciarrocchi]

On Wed, 29 Dec 2004 23:20:28 -0800, Andrew Morton <akpm@osdl.org> wrote:
> Paolo Ciarrocchi <paolo.ciarrocchi@gmail.com> wrote:
> >
> > Andrew,
> >  what's your plan for the staircase scheduler ?
> 
> I have none, frankly.  I haven't seen any complaints about the current
> scheduler.
> 
> If someone can identify bad behaviour in the current scheduler which
> staircase improves then please describe a tescase which the scheduler
> developers can use to reproduce the situation.
> 
> If, after that, we deem that the problem cannot be feasibly fixed within the
> context of the current scheduler and that the problem is sufficiently
> serious to justify wholesale replacement of the scheduler then sure,
> staircase is an option.

Your answer makes lot of sense.
I think Con can explain the pro and cons of the staircase scheduler.

Best,

-- 
Paolo

Re: Trying out SCHED_BATCH

[Con Kolivas]

Paolo Ciarrocchi wrote:
> On Wed, 29 Dec 2004 23:20:28 -0800, Andrew Morton <akpm@osdl.org> wrote:
> 
>>Paolo Ciarrocchi <paolo.ciarrocchi@gmail.com> wrote:
>>
>>>Andrew,
>>> what's your plan for the staircase scheduler ?
>>
>>I have none, frankly.  I haven't seen any complaints about the current
>>scheduler.
>>
>>If someone can identify bad behaviour in the current scheduler which
>>staircase improves then please describe a tescase which the scheduler
>>developers can use to reproduce the situation.
>>
>>If, after that, we deem that the problem cannot be feasibly fixed within the
>>context of the current scheduler and that the problem is sufficiently
>>serious to justify wholesale replacement of the scheduler then sure,
>>staircase is an option.
> 
> 
> Your answer makes lot of sense.
> I think Con can explain the pro and cons of the staircase scheduler.

I agree fully with Andrew. I'm not going there while we have 2.6 forever 
development.

Cheers,
Con

Re: Trying out SCHED_BATCH

[Bill Davidsen]

Andrew Morton wrote:
> Paolo Ciarrocchi <paolo.ciarrocchi@gmail.com> wrote:
> 
>>Andrew, 
>> what's your plan for the staircase scheduler ?
> 
> 
> I have none, frankly.  I haven't seen any complaints about the current
> scheduler.
> 
> If someone can identify bad behaviour in the current scheduler which
> staircase improves then please describe a tescase which the scheduler
> developers can use to reproduce the situation.

Of course that may result in just another band-aid on the current 
scheduler rather than a change.
> 
> If, after that, we deem that the problem cannot be feasibly fixed within the
> context of the current scheduler and that the problem is sufficiently
> serious to justify wholesale replacement of the scheduler then sure,
> staircase is an option.

More to the point, was there a problem with plugable schedulers? It 
would be both technically and politically better to let people try, use, 
and write schedulers for special case loads, just as we have for io 
scheduling.

I didn't find staircase to be the solution to any of my problems, but it 
would be nice to let all the people who are improving schedulers have an 
easy way to try new ideas (easier than building a whole new kernel, that 
is).

-- 
bill davidsen <davidsen@tmr.com>
   CTO TMR Associates, Inc
   Doing interesting things with small computers since 1979

Re: Trying out SCHED_BATCH

[Peter Williams]

Bill Davidsen wrote:
> Andrew Morton wrote:
> 
>> Paolo Ciarrocchi <paolo.ciarrocchi@gmail.com> wrote:
>>
>>> Andrew, what's your plan for the staircase scheduler ?
>>
>>
>>
>> I have none, frankly.  I haven't seen any complaints about the current
>> scheduler.
>>
>> If someone can identify bad behaviour in the current scheduler which
>> staircase improves then please describe a tescase which the scheduler
>> developers can use to reproduce the situation.
> 
> 
> Of course that may result in just another band-aid on the current 
> scheduler rather than a change.
> 
>>
>> If, after that, we deem that the problem cannot be feasibly fixed 
>> within the
>> context of the current scheduler and that the problem is sufficiently
>> serious to justify wholesale replacement of the scheduler then sure,
>> staircase is an option.
> 
> 
> More to the point, was there a problem with plugable schedulers? It 
> would be both technically and politically better to let people try, use, 
> and write schedulers for special case loads, just as we have for io 
> scheduling.
> 
> I didn't find staircase to be the solution to any of my problems, but it 
> would be nice to let all the people who are improving schedulers have an 
> easy way to try new ideas (easier than building a whole new kernel, that 
> is).
> 

At Con's request I've taken over responsibility for plugsched but I've 
been away visiting relatives for the last week or so and, therefore, I'm 
a little behind.  I hope to release a plugsched patch for 2.6.10 in the 
next few days with a (work in progress) modification to share a lot more 
code between schedulers so that the amount of work required to implement 
new schedulers is reduced.

Peter
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce