Doubt about scheduler

Doubt about scheduler

[Raúl Baena]

Hello!!!, I´m a student of computer science and I´m doing my final 
degree job in linux. It is about "linux kernel modules" , I have to know 
some things of the scheduler. The runqueue struct, and so on. The 
problem is that in the last linux kernel version in the "sched.h" isn´t 
defined these structs (prio_array, runqueue...), and I cann´t access to 
runqueue or prio_array fields. I know that in the 2.6.5 kernel version 
these fields were accessible and now don´t, could you tell me what is 
the reason please?
 I think that I´m going to do it (the module) in the 2.6.5 kernel 
version and will try to explain why, and for this I need your help.

Please answer to my email, because I´m not subscribe to mailing list.

Thank you very much for your help, sorry for my poor english, it´s not 
my mother tongue.

PS: If you think that I can do the module in the last kernel version, 
could you tell me how, please?, thank you very much.

Re: Doubt about scheduler

[Jon Masters]

On 3/2/06, Raúl Baena <raul_baena@ya.com> wrote:

> Hello!!!, I´m a student of computer science and I´m doing my final
> degree job in linux. It is about "linux kernel modules" , I have to know
> some things of the scheduler. The runqueue struct, and so on. The
> problem is that in the last linux kernel version in the "sched.h" isn´t
> defined these structs (prio_array, runqueue...), and I cann´t access to
> runqueue or prio_array fields. I know that in the 2.6.5 kernel version
> these fields were accessible and now don´t, could you tell me what is
> the reason please?

Deliberately, these aren't available outside of the scheduler so that
they can't be played with. Much as things like the symbol table aren't
exported to modules, some things in the kernel aren't even available
to other parts of the core kernel :-)

>  I think that I´m going to do it (the module) in the 2.6.5 kernel
> version and will try to explain why, and for this I need your help.

If you really need to play with this stuff, then why not just make
your own kernel tree with this hacked up in the scheduler code itself?
If it's just for you, then that'll work fine. If you would explain
what it is that you need to do, then someone might be able to offer
you advice on the general direction to take - see also
http://www.kernelnewbies.org/

Jon.

Re: Doubt about scheduler

[Raúl Baena]

Thank you very much Jon. But I think I haven´t explained very well.

I know that now the prio_array and runqueues structs aren´t accesible 
for modules, but in the 2.6.5 version they were. I would like to know 
the reason, why before they were accesible and now they don´t? If you 
could answer me, it would be great. I could to write the reason in my 
university job. (In Spain we have to make a final degree job, and mine 
is about modules in linux (I chose this), I would like to show 
information of the new scheduler, a scheduler monitor, and these fields 
are indispensable for me)
I thought in your solution (own kernel tree), but I would like to make a 
module that worked in standard distributions. I prefer to make a module 
that worked in 2.6.5 version instead to make one that worked in 2.6.12 
but with my own kernel tree. So I suposse that I will do that, but 
knowing the reason why could serve me to make a better investigation 
document.

Thank you again for your help, I will mention you in my final degree job 
acknowledge. Please, keep helping me!!! :)



Jon Masters escribió:

>On 3/2/06, Raúl Baena <raul_baena@ya.com> wrote:
>
>  
>
>>Hello!!!, I´m a student of computer science and I´m doing my final
>>degree job in linux. It is about "linux kernel modules" , I have to know
>>some things of the scheduler. The runqueue struct, and so on. The
>>problem is that in the last linux kernel version in the "sched.h" isn´t
>>defined these structs (prio_array, runqueue...), and I cann´t access to
>>runqueue or prio_array fields. I know that in the 2.6.5 kernel version
>>these fields were accessible and now don´t, could you tell me what is
>>the reason please?
>>    
>>
>
>Deliberately, these aren't available outside of the scheduler so that
>they can't be played with. Much as things like the symbol table aren't
>exported to modules, some things in the kernel aren't even available
>to other parts of the core kernel :-)
>
>  
>
>> I think that I´m going to do it (the module) in the 2.6.5 kernel
>>version and will try to explain why, and for this I need your help.
>>    
>>
>
>If you really need to play with this stuff, then why not just make
>your own kernel tree with this hacked up in the scheduler code itself?
>If it's just for you, then that'll work fine. If you would explain
>what it is that you need to do, then someone might be able to offer
>you advice on the general direction to take - see also
>http://www.kernelnewbies.org/
>
>Jon.
>
>
>
>  
>

Re: Doubt about scheduler

[Nick Piggin]

Raúl Baena wrote:
> Thank you very much Jon. But I think I haven´t explained very well.
> 
> I know that now the prio_array and runqueues structs aren´t accesible 
> for modules, but in the 2.6.5 version they were. I would like to know 
> the reason, why before they were accesible and now they don´t? If you 
> could answer me, it would be great.

I don't remember them being available in 2.6.5... but as to why they
aren't available now: it is much cleaner this way. It even benefits
you because now nobody will break your module when they change the
data structure.

> I could to write the reason in my 
> university job. (In Spain we have to make a final degree job, and mine 
> is about modules in linux (I chose this), I would like to show 
> information of the new scheduler, a scheduler monitor, and these fields 
> are indispensable for me)

If your task is about modules in Linux, then I don't see how that
involves the scheduler at all?

On the other hand, if you want a scheduler monitor then I can't see
why it would be appropriate to implement as a module (we have schedstats,
which you can read from a userspace program or daemon).

Nick

-- 
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 

Re: Doubt about scheduler

[Raúl Baena]

>> I could to write the reason in my university job. (In Spain we have 
>> to make a final degree job, and mine is about modules in linux (I 
>> chose this), I would like to show information of the new scheduler, a 
>> scheduler monitor, and these fields are indispensable for me)
>
>
> If your task is about modules in Linux, then I don't see how that
> involves the scheduler at all?
>
> On the other hand, if you want a scheduler monitor then I can't see
> why it would be appropriate to implement as a module (we have schedstats,
> which you can read from a userspace program or daemon).
>
> Nick
>
Thank you very much Nick.

My task is free, I can do everything that I want that involves modules, 
I thought that to make the module about the new O(k) scheduler would be 
a good idea. I think that it´s not enough for me schedstats, because I 
want to make a visual scheduler, I mean, using GTK+ , a module and 
something else to make a visual scheduler monitor, how the tasks move 
between "active" and "expired", where the task are in prio_array with 
the bitmap fields...this module isn´t usefull, only in a didactic way. 
This is the reason because I have to make a module althought ( according 
with your recomendations, recomendations of a person who knows 
infinitely much than me) probably would be better to make another thing.

At any rate, I will search "schedstats" references, in case it was usefull.

Thank all of you very much again , with your help I hope to be able to 
make a good final degree job.

Re: Doubt about scheduler

[Jon Masters]

On 3/5/06, Raúl Baena <raul_baena@ya.com> wrote:

> I thought that to make the module about the new O(k) scheduler would be
> a good idea. I think that it´s not enough for me schedstats, because I
> want to make a visual scheduler, I mean, using GTK+ , a module and
> something else to make a visual scheduler monitor, how the tasks move
> between "active" and "expired", where the task are in prio_array with
> the bitmap fields...this module isn´t usefull, only in a didactic way.

If you're seriously interested in this then cool. Let me know how you get on.

I looked at hacking something into gtop etc. previously to use
/proc/kcore and pull out task information - I'd certainly like to see
a visual process monitor that could pull all of this stuff out and
display it for educational interest (page tables, vmas, other
resources). But then, it's probably been done - I didn't look to see
what else is out there.

Jon.

Re: Doubt about scheduler

[Steven Rostedt]

On Sun, 5 Mar 2006, Jon Masters wrote:

> On 3/5/06, Raúl Baena <raul_baena@ya.com> wrote:
>
> > I thought that to make the module about the new O(k) scheduler would be
> > a good idea. I think that it´s not enough for me schedstats, because I
> > want to make a visual scheduler, I mean, using GTK+ , a module and
> > something else to make a visual scheduler monitor, how the tasks move
> > between "active" and "expired", where the task are in prio_array with
> > the bitmap fields...this module isn´t usefull, only in a didactic way.
>
> If you're seriously interested in this then cool. Let me know how you get on.
>
> I looked at hacking something into gtop etc. previously to use
> /proc/kcore and pull out task information - I'd certainly like to see
> a visual process monitor that could pull all of this stuff out and
> display it for educational interest (page tables, vmas, other
> resources). But then, it's probably been done - I didn't look to see
> what else is out there.
>

Raul, Also take a look at relayfs. It's a fast way to record data in the
kernel and pass it back to a userland process.  You'll have to patch the
kernel as it is said that the data needed is private to sched.c

Look into Documentation/filesystems/relayfs.txt

relayfs entered the kernel in 2.6.14.

-- Steve

Re: Doubt about scheduler

[Raul]

Steven Rostedt wrote:

>On Sun, 5 Mar 2006, Jon Masters wrote:
>
>  
>
>>On 3/5/06, Raúl Baena <raul_baena@ya.com> wrote:
>>
>>    
>>
>>>I thought that to make the module about the new O(k) scheduler would be
>>>a good idea. I think that it´s not enough for me schedstats, because I
>>>want to make a visual scheduler, I mean, using GTK+ , a module and
>>>something else to make a visual scheduler monitor, how the tasks move
>>>between "active" and "expired", where the task are in prio_array with
>>>the bitmap fields...this module isn´t usefull, only in a didactic way.
>>>      
>>>
>>If you're seriously interested in this then cool. Let me know how you get on.
>>
>>I looked at hacking something into gtop etc. previously to use
>>/proc/kcore and pull out task information - I'd certainly like to see
>>a visual process monitor that could pull all of this stuff out and
>>display it for educational interest (page tables, vmas, other
>>resources). But then, it's probably been done - I didn't look to see
>>what else is out there.
>>
>>    
>>
>
>Raul, Also take a look at relayfs. It's a fast way to record data in the
>kernel and pass it back to a userland process.  You'll have to patch the
>kernel as it is said that the data needed is private to sched.c
>
>Look into Documentation/filesystems/relayfs.txt
>
>relayfs entered the kernel in 2.6.14.
>
>-- Steve
>
>  
>
Thank you very much, I'll see it. I'll tell you my progress!!!

Re: Doubt about scheduler

[Randy.Dunlap]

On Mon, 06 Mar 2006 09:52:36 +0100 Raul wrote:

> Steven Rostedt wrote:
> 
> >On Sun, 5 Mar 2006, Jon Masters wrote:
> >
> >  
> >
> >>On 3/5/06, Raúl Baena <raul_baena@ya.com> wrote:
> >>
> >>    
> >>
> >>>I thought that to make the module about the new O(k) scheduler would be
> >>>a good idea. I think that it´s not enough for me schedstats, because I
> >>>want to make a visual scheduler, I mean, using GTK+ , a module and
> >>>something else to make a visual scheduler monitor, how the tasks move
> >>>between "active" and "expired", where the task are in prio_array with
> >>>the bitmap fields...this module isn´t usefull, only in a didactic way.
> >>>      
> >>>
> >>If you're seriously interested in this then cool. Let me know how you get on.
> >>
> >>I looked at hacking something into gtop etc. previously to use
> >>/proc/kcore and pull out task information - I'd certainly like to see
> >>a visual process monitor that could pull all of this stuff out and
> >>display it for educational interest (page tables, vmas, other
> >>resources). But then, it's probably been done - I didn't look to see
> >>what else is out there.
> >>
> >>    
> >>
> >
> >Raul, Also take a look at relayfs. It's a fast way to record data in the
> >kernel and pass it back to a userland process.  You'll have to patch the
> >kernel as it is said that the data needed is private to sched.c
> >
> >Look into Documentation/filesystems/relayfs.txt
> >
> >relayfs entered the kernel in 2.6.14.
> >
> >-- Steve
> >
> >  
> >
> Thank you very much, I'll see it. I'll tell you my progress!!!

note that in recent kernels it has changed to:

config RELAY
	bool "Kernel->user space relay support (formerly relayfs)"
	help
	  This option enables support for relay interface support in
	  certain file systems (such as sysfs, and trivially debugfs).
	  It is designed to provide an efficient mechanism for tools and
	  facilities to relay large amounts of data from kernel space to
	  user space.



---
~Randy

Re: Doubt about scheduler

[Mike Galbraith]

On Mon, 2006-03-06 at 00:30 +1100, Nick Piggin wrote:
> Raúl Baena wrote:
> > Thank you very much Jon. But I think I haven´t explained very well.
> > 
> > I know that now the prio_array and runqueues structs aren´t accesible 
> > for modules, but in the 2.6.5 version they were. I would like to know 
> > the reason, why before they were accesible and now they don´t? If you 
> > could answer me, it would be great.
> 
> I don't remember them being available in 2.6.5... but as to why they
> aren't available now: it is much cleaner this way. It even benefits
> you because now nobody will break your module when they change the
> data structure.

I can't recall having ever seen these exposed.. by Ingo.

	-Mike 

Re: Doubt about scheduler

[Peter Williams]

Nick Piggin wrote:
> Raúl Baena wrote:
> 
>> Thank you very much Jon. But I think I haven´t explained very well.
>>
>> I know that now the prio_array and runqueues structs aren´t accesible 
>> for modules, but in the 2.6.5 version they were. I would like to know 
>> the reason, why before they were accesible and now they don´t? If you 
>> could answer me, it would be great.
> 
> 
> I don't remember them being available in 2.6.5... but as to why they
> aren't available now: it is much cleaner this way. It even benefits
> you because now nobody will break your module when they change the
> data structure.

SuSE patched their 2.6.5 kernel to make the run queues visible in a 
header file.  Perhaps Raul was using a SuSE version of 2.6.5.

> 
>> I could to write the reason in my university job. (In Spain we have to 
>> make a final degree job, and mine is about modules in linux (I chose 
>> this), I would like to show information of the new scheduler, a 
>> scheduler monitor, and these fields are indispensable for me)
> 
> 
> If your task is about modules in Linux, then I don't see how that
> involves the scheduler at all?
> 
> On the other hand, if you want a scheduler monitor then I can't see
> why it would be appropriate to implement as a module (we have schedstats,
> which you can read from a userspace program or daemon).
> 
> Nick
> 

Peter
-- 
Peter Williams                                   pwil3058@bigpond.net.au

"Learning, n. The kind of ignorance distinguishing the studious."
  -- Ambrose Bierce

Re: Doubt about scheduler

[Raúl Baena]

Peter Williams escribió:

> SuSE patched their 2.6.5 kernel to make the run queues visible in a 
> header file.  Perhaps Raul was using a SuSE version of 2.6.5.

Exact!!!! . To be honest, I have the 2.6.12 version to debian, but my 
project tutor has this one. And when I argued about my project and I 
told him that those fields weren´t accesible, he sent me his sched.h, 
and I saw that runqueue and prio_array were accesible in it. I didn´t 
think that it was a SUSE patch . Sorry, it's my fault

I have made a litte cheat to be able to see prio_array fields, I have 
created a new struct and later with the "array" field of "task_struct" I 
have made a cast to my struct. Then I have been able to show these 
values, but I don't like this "solution", because I can´t know anything 
of tasks in another state ("active" or "expired"), neither which is my 
state!! For this reason I would like to know how to get this 
information, my project have to be done in June 30 :(.

Thank you very much again for your help, I´m getting a lot of 
information about the kernel that I'll be able to use in my report, of 
course if you agree.

Re: Doubt about scheduler

[Jon Masters]

On 3/5/06, Raúl Baena <raul_baena@ya.com> wrote:

> Thank you very much Jon. But I think I haven´t explained very well.

No, you did. Perhaps I wasn't terse enough in my reply :-)

> I know that now the prio_array and runqueues structs aren´t accesible
> for modules, but in the 2.6.5 version they were. I would like to know
> the reason, why before they were accesible and now they don´t?

Because it's a /bad idea/ to screw around with these. That's why. I
was trying to say that nicely before - it's cleaner and safer if the
scheduler is the only thing playing there as it's the only thing with
a bona fide need.

I'm not trying to be offensive at all - by all means make your own
kernel, perhaps with an exported version of these structure or
introduce your own API for tinkering with them. But just don't
necessarily expect everyone else to jump at it :-)

Jon.