finding out the value of HZ from userspace

finding out the value of HZ from userspace

[Micha Feigin]

Is it possible to find out what the kernel's notion of HZ is from user
space?
It seem to change from system to system and between 2.4 (100 on i386)
to 2.6 (1000 on i386).

Re: finding out the value of HZ from userspace

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 354 bytes --]

On Thu, 2004-03-11 at 15:17, Micha Feigin wrote:
> Is it possible to find out what the kernel's notion of HZ is from user
> space?
> It seem to change from system to system and between 2.4 (100 on i386)
> to 2.6 (1000 on i386).

if you can see 1000 from userspace that is a bad kernel bug; can you say
where you find something in units of 1000 ?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: finding out the value of HZ from userspace

[John Reiser]

Arjan van de Ven wrote:
> On Thu, 2004-03-11 at 15:17, Micha Feigin wrote:
> 
>>Is it possible to find out what the kernel's notion of HZ is from user
>>space?
>>It seem to change from system to system and between 2.4 (100 on i386)
>>to 2.6 (1000 on i386).
> 
> 
> if you can see 1000 from userspace that is a bad kernel bug; can you say
> where you find something in units of 1000 ?

create_elf_tables() in fs/binfmt_elf.c tells every ELF execve():
         NEW_AUX_ENT(AT_CLKTCK, CLOCKS_PER_SEC);
which can be found by crawling through the stack above the pointer
to the last environment variable.

-- 

Re: finding out the value of HZ from userspace

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 922 bytes --]

On Sat, Mar 13, 2004 at 11:34:37AM -0800, John Reiser wrote:
> Arjan van de Ven wrote:
> >On Thu, 2004-03-11 at 15:17, Micha Feigin wrote:
> >
> >>Is it possible to find out what the kernel's notion of HZ is from user
> >>space?
> >>It seem to change from system to system and between 2.4 (100 on i386)
> >>to 2.6 (1000 on i386).
> >
> >
> >if you can see 1000 from userspace that is a bad kernel bug; can you say
> >where you find something in units of 1000 ?
> 
> create_elf_tables() in fs/binfmt_elf.c tells every ELF execve():
>         NEW_AUX_ENT(AT_CLKTCK, CLOCKS_PER_SEC);
> which can be found by crawling through the stack above the pointer
> to the last environment variable.

Ugh that should say 100 on x86....
but..
param.h:# define USER_HZ        100             /* .. some user interfaces are in "ticks" */
param.h:# define CLOCKS_PER_SEC (USER_HZ)       /* like times() */
.....
that looks like 100 to me.


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: finding out the value of HZ from userspace

[Micha Feigin]

On Sat, Mar 13, 2004 at 08:38:52PM +0100, Arjan van de Ven wrote:
> On Sat, Mar 13, 2004 at 11:34:37AM -0800, John Reiser wrote:
> > Arjan van de Ven wrote:
> > >On Thu, 2004-03-11 at 15:17, Micha Feigin wrote:
> > >
> > >>Is it possible to find out what the kernel's notion of HZ is from user
> > >>space?
> > >>It seem to change from system to system and between 2.4 (100 on i386)
> > >>to 2.6 (1000 on i386).
> > >
> > >
> > >if you can see 1000 from userspace that is a bad kernel bug; can you say
> > >where you find something in units of 1000 ?
> > 
> > create_elf_tables() in fs/binfmt_elf.c tells every ELF execve():
> >         NEW_AUX_ENT(AT_CLKTCK, CLOCKS_PER_SEC);
> > which can be found by crawling through the stack above the pointer
> > to the last environment variable.
> 
> Ugh that should say 100 on x86....
> but..
> param.h:# define USER_HZ        100             /* .. some user interfaces are in "ticks" */
> param.h:# define CLOCKS_PER_SEC (USER_HZ)       /* like times() */
> .....
> that looks like 100 to me.
> 

When dealing with bdflush and a few other interfaces the values need to
be in jiffies which requires knowledge of the kernels notion of HZ not
userspace.

The other option is to try to push a change to make the interface in
centisecs instead of jiffies, question is if it will catch.

Re: finding out the value of HZ from userspace

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 1480 bytes --]

On Sat, 2004-03-13 at 23:14, Micha Feigin wrote:
> On Sat, Mar 13, 2004 at 08:38:52PM +0100, Arjan van de Ven wrote:
> > On Sat, Mar 13, 2004 at 11:34:37AM -0800, John Reiser wrote:
> > > Arjan van de Ven wrote:
> > > >On Thu, 2004-03-11 at 15:17, Micha Feigin wrote:
> > > >
> > > >>Is it possible to find out what the kernel's notion of HZ is from user
> > > >>space?
> > > >>It seem to change from system to system and between 2.4 (100 on i386)
> > > >>to 2.6 (1000 on i386).
> > > >
> > > >
> > > >if you can see 1000 from userspace that is a bad kernel bug; can you say
> > > >where you find something in units of 1000 ?
> > > 
> > > create_elf_tables() in fs/binfmt_elf.c tells every ELF execve():
> > >         NEW_AUX_ENT(AT_CLKTCK, CLOCKS_PER_SEC);
> > > which can be found by crawling through the stack above the pointer
> > > to the last environment variable.
> > 
> > Ugh that should say 100 on x86....
> > but..
> > param.h:# define USER_HZ        100             /* .. some user interfaces are in "ticks" */
> > param.h:# define CLOCKS_PER_SEC (USER_HZ)       /* like times() */
> > .....
> > that looks like 100 to me.
> > 
> 
> When dealing with bdflush and a few other interfaces the values need to
> be in jiffies which requires knowledge of the kernels notion of HZ not
> userspace.

Wrong. Any such interface is supposed to convert automatically. Any
interface you can find that doesn't should be reported as a serious bug!


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: finding out the value of HZ from userspace

[Micha Feigin]

On Sat, Mar 13, 2004 at 11:32:39PM +0100, Arjan van de Ven wrote:
> On Sat, 2004-03-13 at 23:14, Micha Feigin wrote:
> > On Sat, Mar 13, 2004 at 08:38:52PM +0100, Arjan van de Ven wrote:
> > > On Sat, Mar 13, 2004 at 11:34:37AM -0800, John Reiser wrote:
> > > > Arjan van de Ven wrote:
> > > > >On Thu, 2004-03-11 at 15:17, Micha Feigin wrote:
> > > > >
> > > > >>Is it possible to find out what the kernel's notion of HZ is from user
> > > > >>space?
> > > > >>It seem to change from system to system and between 2.4 (100 on i386)
> > > > >>to 2.6 (1000 on i386).
> > > > >
> > > > >
> > > > >if you can see 1000 from userspace that is a bad kernel bug; can you say
> > > > >where you find something in units of 1000 ?
> > > > 
> > > > create_elf_tables() in fs/binfmt_elf.c tells every ELF execve():
> > > >         NEW_AUX_ENT(AT_CLKTCK, CLOCKS_PER_SEC);
> > > > which can be found by crawling through the stack above the pointer
> > > > to the last environment variable.
> > > 
> > > Ugh that should say 100 on x86....
> > > but..
> > > param.h:# define USER_HZ        100             /* .. some user interfaces are in "ticks" */
> > > param.h:# define CLOCKS_PER_SEC (USER_HZ)       /* like times() */
> > > .....
> > > that looks like 100 to me.
> > > 
> > 
> > When dealing with bdflush and a few other interfaces the values need to
> > be in jiffies which requires knowledge of the kernels notion of HZ not
> > userspace.
> 
> Wrong. Any such interface is supposed to convert automatically. Any
> interface you can find that doesn't should be reported as a serious bug!
> 

Like I said, look at bdflush in 2.4 (this was fixed with the changed 2.6
interface) and xfs proc interface in both 2.4 and 2.6.
In light of your post then there is a serious bug.

For example for bdflush age_buffer field (true for the other used fields
also), no conversion:
	bh->b_flushtime = jiffies + bdf_prm.b_un.age_buffer;

For xfs flush interval:
if (pbd_active == 1) {
			mod_timer(&pb_daemon_timer,
				  jiffies + pb_params.flush_interval.val);
			interruptible_sleep_on(&pbd_waitq);
		}

xfs should be converted to centisecs, bdflush should also be converted
to centisecs, or the interface from 2.6 should somehow be ported to
exist in parallel to the 2.4 one.

I don't mind making a patch, which approach should be used?

Re: finding out the value of HZ from userspace

[Andrew Morton]

Micha Feigin <michf@post.tau.ac.il> wrote:
>
>  > Wrong. Any such interface is supposed to convert automatically. Any
>  > interface you can find that doesn't should be reported as a serious bug!
>  > 
> 
>  Like I said, look at bdflush in 2.4 (this was fixed with the changed 2.6
>  interface) and xfs proc interface in both 2.4 and 2.6.
>  In light of your post then there is a serious bug.
> 
>  For example for bdflush age_buffer field (true for the other used fields
>  also), no conversion:
>  	bh->b_flushtime = jiffies + bdf_prm.b_un.age_buffer;

I doubt if there's any motivation to fix these things in 2.4.  If you change
HZ in 2.4 you own both pieces.  (alpha has HZ=1024 in 2.4, so presumably
bdflush tuning doesn't work right).

In 2.6, the bdflush parameters do not exist.  They were replaced by
/proc/sys/vm/*_centisecs, which are HZ-independent.

There are, I think, still some /proc tunables in 2.6 which do depend upon
HZ and they should be found and fixed.  If the same tunables are present in
2.4 kernels then they should be converted to take centiseconds in 2.6, so
2.4-based tools continue to work correctly.

We have similar problems where /proc tunables are expressed in terms of
"number of pages".  As PAGE_SIZE varies from 4096 to 65536 this is
sometimes wrong.  Fixing this is more subtle.

Re: finding out the value of HZ from userspace

[Micha Feigin]

On Sat, Mar 13, 2004 at 05:49:29PM -0800, Andrew Morton wrote:
> Micha Feigin <michf@post.tau.ac.il> wrote:
> >
> >  > Wrong. Any such interface is supposed to convert automatically. Any
> >  > interface you can find that doesn't should be reported as a serious bug!
> >  > 
> > 
> >  Like I said, look at bdflush in 2.4 (this was fixed with the changed 2.6
> >  interface) and xfs proc interface in both 2.4 and 2.6.
> >  In light of your post then there is a serious bug.
> > 
> >  For example for bdflush age_buffer field (true for the other used fields
> >  also), no conversion:
> >  	bh->b_flushtime = jiffies + bdf_prm.b_un.age_buffer;
> 
> I doubt if there's any motivation to fix these things in 2.4.  If you change
> HZ in 2.4 you own both pieces.  (alpha has HZ=1024 in 2.4, so presumably
> bdflush tuning doesn't work right).
> 

There is for laptop mode which is now in the kernel so a generic startup
script can be written.

I will right a patch and post it in a new thread and see how it takes.

> In 2.6, the bdflush parameters do not exist.  They were replaced by
> /proc/sys/vm/*_centisecs, which are HZ-independent.
> 
> There are, I think, still some /proc tunables in 2.6 which do depend upon
> HZ and they should be found and fixed.  If the same tunables are present in
> 2.4 kernels then they should be converted to take centiseconds in 2.6, so
> 2.4-based tools continue to work correctly.
> 
> We have similar problems where /proc tunables are expressed in terms of
> "number of pages".  As PAGE_SIZE varies from 4096 to 65536 this is
> sometimes wrong.  Fixing this is more subtle.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>  
>  +++++++++++++++++++++++++++++++++++++++++++
>  This Mail Was Scanned By Mail-seCure System
>  at the Tel-Aviv University CC.
> 

Re: finding out the value of HZ from userspace

[Peter Williams]

Arjan van de Ven wrote:
> On Sat, Mar 13, 2004 at 11:34:37AM -0800, John Reiser wrote:
> 
>>Arjan van de Ven wrote:
>>
>>>On Thu, 2004-03-11 at 15:17, Micha Feigin wrote:
>>>
>>>
>>>>Is it possible to find out what the kernel's notion of HZ is from user
>>>>space?
>>>>It seem to change from system to system and between 2.4 (100 on i386)
>>>>to 2.6 (1000 on i386).
>>>
>>>
>>>if you can see 1000 from userspace that is a bad kernel bug; can you say
>>>where you find something in units of 1000 ?
>>
>>create_elf_tables() in fs/binfmt_elf.c tells every ELF execve():
>>        NEW_AUX_ENT(AT_CLKTCK, CLOCKS_PER_SEC);
>>which can be found by crawling through the stack above the pointer
>>to the last environment variable.
> 
> 
> Ugh that should say 100 on x86....
> but..
> param.h:# define USER_HZ        100             /* .. some user interfaces are in "ticks" */
> param.h:# define CLOCKS_PER_SEC (USER_HZ)       /* like times() */
> .....
> that looks like 100 to me.
> 

This horrible hack of converting all tick values to 100 (from 1000) for 
export to user space because a large number of user space programs 
assume that HZ is 100 would NOT be necessary if there was a mechanism 
whereby user space programs could find out how many ticks there are in a 
second instead of having to make assumptions.

I think that providing such a mechanism should be a priority and when 
it's been available for a reasonable amount time (so that the user space 
programs can be converted to using it) USER_HZ should become equal to HZ.

Another alternative would be to stop exporting time as ticks and use 
some standard unit for all systems.  The chosen unit should be small 
enough (e.g. microseconds or mybe even nanoseconds) so that no 
information is lost (which it is in the current implementation) on 
conversion from ticks to these units.  Of course 64 bit integers would 
be needed.

Peter
-- 
Dr Peter Williams, Chief Scientist                peterw@aurema.com
Aurema Pty Limited                                Tel:+61 2 9698 2322
PO Box 305, Strawberry Hills NSW 2012, Australia  Fax:+61 2 9699 9174
79 Myrtle Street, Chippendale NSW 2008, Australia http://www.aurema.com

Re: finding out the value of HZ from userspace

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 732 bytes --]

On Tue, Mar 16, 2004 at 11:28:18AM +1100, Peter Williams wrote:
> >Ugh that should say 100 on x86....
> >but..
> >param.h:# define USER_HZ        100             /* .. some user interfaces 
> >are in "ticks" */
> >param.h:# define CLOCKS_PER_SEC (USER_HZ)       /* like times() */
> >.....
> >that looks like 100 to me.
> >
> 
> This horrible hack of converting all tick values to 100 (from 1000) for 
> export to user space because a large number of user space programs 
> assume that HZ is 100 would NOT be necessary if there was a mechanism 
> whereby user space programs could find out how many ticks there are in a 
> second instead of having to make assumptions.

there is one. Nothing uses it
(sysconf() provides this info)


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: finding out the value of HZ from userspace

[Peter Williams]

Arjan van de Ven wrote:
> On Tue, Mar 16, 2004 at 11:28:18AM +1100, Peter Williams wrote:
> 
>>>Ugh that should say 100 on x86....
>>>but..
>>>param.h:# define USER_HZ        100             /* .. some user interfaces 
>>>are in "ticks" */
>>>param.h:# define CLOCKS_PER_SEC (USER_HZ)       /* like times() */
>>>.....
>>>that looks like 100 to me.
>>>
>>
>>This horrible hack of converting all tick values to 100 (from 1000) for 
>>export to user space because a large number of user space programs 
>>assume that HZ is 100 would NOT be necessary if there was a mechanism 
>>whereby user space programs could find out how many ticks there are in a 
>>second instead of having to make assumptions.
> 
> 
> there is one. Nothing uses it
> (sysconf() provides this info)

Seems to me that it would be fairly trivial to modify those programs 
(that should use this mechanism but don't) to use it?  So why should 
they be allowed to dictate kernel behaviour?

Peter
-- 
Dr Peter Williams, Chief Scientist                peterw@aurema.com
Aurema Pty Limited                                Tel:+61 2 9698 2322
PO Box 305, Strawberry Hills NSW 2012, Australia  Fax:+61 2 9699 9174
79 Myrtle Street, Chippendale NSW 2008, Australia http://www.aurema.com

Re: finding out the value of HZ from userspace

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 473 bytes --]


On Wed, Mar 17, 2004 at 10:38:03AM +1100, Peter Williams wrote:
> >there is one. Nothing uses it
> >(sysconf() provides this info)
> 
> Seems to me that it would be fairly trivial to modify those programs 
> (that should use this mechanism but don't) to use it?  So why should 
> they be allowed to dictate kernel behaviour?

quality of implementation; for example shell scripts that want to do
echo 500 > /proc/sys/foo/bar/something_in_HZ
...
or /etc/sysctl.conf or ...


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: finding out the value of HZ from userspace

[Stefan Smietanowski]

>>>there is one. Nothing uses it
>>>(sysconf() provides this info)
>>
>>Seems to me that it would be fairly trivial to modify those programs 
>>(that should use this mechanism but don't) to use it?  So why should 
>>they be allowed to dictate kernel behaviour?
> 
> 
> quality of implementation; for example shell scripts that want to do
> echo 500 > /proc/sys/foo/bar/something_in_HZ
> ...
> or /etc/sysctl.conf or ...
> 

Then write a simple program already. How hard is it to write a program
that does a sysconf() and returns (as ascii of course) just the
value of HZ? Then do some trivial calculation off of that.

HZ=$(gethz)

If your 500 was 5 seconds, do

TIME=$[HZ*5]
echo $TIME > /proc/sys/foo/bar/something_in_HZ

I mean, come on.

Then you include it in the default distro of choice so that
everybody can use it and there you are.

If someone doesn't have "gethz" then they can download it.

// Stefan

Re: finding out the value of HZ from userspace

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 267 bytes --]

On Sat, Mar 20, 2004 at 12:28:00PM +0100, Stefan Smietanowski wrote:
> 
> Then you include it in the default distro of choice so that
> everybody can use it and there you are.

but what is the POINT of all this changing/breaking ?
Can someone at least tell me that ?

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: finding out the value of HZ from userspace

[Peter Williams]

Arjan van de Ven wrote:
> On Sat, Mar 20, 2004 at 12:28:00PM +0100, Stefan Smietanowski wrote:
> 
>>Then you include it in the default distro of choice so that
>>everybody can use it and there you are.
> 
> 
> but what is the POINT of all this changing/breaking ?
> Can someone at least tell me that ?

In the 2.6 kernels internal timing and task statistics (for i386 
systems) are now kept in milliseconds where they were previously in 
1/100ths of a second.  By converting these statistics to 1/100ths of a 
second for export to user space an order of magnitude (i.e. a factor of 
10) loss of precision occurs.

Peter
PS I'd like to point out that there are changes in 2.6 kernels that have 
more serious consequences than this that have to be coped with when 
using 2.6 kernels on distributions such as RedHat 9 that were built 
around older kernels.
-- 
Dr Peter Williams, Chief Scientist                peterw@aurema.com
Aurema Pty Limited                                Tel:+61 2 9698 2322
PO Box 305, Strawberry Hills NSW 2012, Australia  Fax:+61 2 9699 9174
79 Myrtle Street, Chippendale NSW 2008, Australia http://www.aurema.com

Re: finding out the value of HZ from userspace

[Tim Schmielau]

On Sun, 21 Mar 2004, Peter Williams wrote:

> In the 2.6 kernels internal timing and task statistics (for i386 
> systems) are now kept in milliseconds where they were previously in 
> 1/100ths of a second.  By converting these statistics to 1/100ths of a 
> second for export to user space an order of magnitude (i.e. a factor of 
> 10) loss of precision occurs.

No. The statistics are not a result of full bookkeeping, but simply
gained by periodically sampling the processor state. So they don't
have a precision of 1/1000th of a second anyways.


Tim

Re: finding out the value of HZ from userspace

[Peter Williams]

Tim Schmielau wrote:
> On Sun, 21 Mar 2004, Peter Williams wrote:
> 
> 
>>In the 2.6 kernels internal timing and task statistics (for i386 
>>systems) are now kept in milliseconds where they were previously in 
>>1/100ths of a second.  By converting these statistics to 1/100ths of a 
>>second for export to user space an order of magnitude (i.e. a factor of 
>>10) loss of precision occurs.
> 
> 
> No. The statistics are not a result of full bookkeeping, but simply
> gained by periodically sampling the processor state. So they don't
> have a precision of 1/1000th of a second anyways.

1/1000th of a second IS the internal timing precision.  The issue of how 
tasks' CPU usage is allocated for reporting is a different matter but 
from a statistical viewpoint this will just effect the variance (or 
standard deviation) of the estimates and NOT their precision. As the 
number of samples the variance (or standard deviation) decrease rapidly 
so to all intents and purposes the statistics are accurate to the 
nearest 1/1000th of a second.

Peter
-- 
Dr Peter Williams, Chief Scientist                peterw@aurema.com
Aurema Pty Limited                                Tel:+61 2 9698 2322
PO Box 305, Strawberry Hills NSW 2012, Australia  Fax:+61 2 9699 9174
79 Myrtle Street, Chippendale NSW 2008, Australia http://www.aurema.com

Re: finding out the value of HZ from userspace

[Kai Henningsen]

stesmi@stesmi.com (Stefan Smietanowski)  wrote on 20.03.04 in <405C2AC0.70605@stesmi.com>:

> >>>there is one. Nothing uses it
> >>>(sysconf() provides this info)
> >>
> >>Seems to me that it would be fairly trivial to modify those programs
> >>(that should use this mechanism but don't) to use it?  So why should
> >>they be allowed to dictate kernel behaviour?
> >
> >
> > quality of implementation; for example shell scripts that want to do
> > echo 500 > /proc/sys/foo/bar/something_in_HZ
> > ...
> > or /etc/sysctl.conf or ...
> >
>
> Then write a simple program already. How hard is it to write a program
> that does a sysconf() and returns (as ascii of course) just the
> value of HZ? Then do some trivial calculation off of that.

How about a slightly more useful utility, like this:

$ getconf CLK_TCK
100
$ getconf OPEN_MAX
1024
$ getconf PATH_MAX /proc/
4096
$

MfG Kai

Re: finding out the value of HZ from userspace

[Stefan Smietanowski]

Hi.

>>>>>there is one. Nothing uses it
>>>>>(sysconf() provides this info)
>>>>
>>>>Seems to me that it would be fairly trivial to modify those programs
>>>>(that should use this mechanism but don't) to use it?  So why should
>>>>they be allowed to dictate kernel behaviour?
>>>
>>>
>>>quality of implementation; for example shell scripts that want to do
>>>echo 500 > /proc/sys/foo/bar/something_in_HZ
>>>...
>>>or /etc/sysctl.conf or ...
>>>
>>
>>Then write a simple program already. How hard is it to write a program
>>that does a sysconf() and returns (as ascii of course) just the
>>value of HZ? Then do some trivial calculation off of that.
> 
> 
> How about a slightly more useful utility, like this:
> 
> $ getconf CLK_TCK
> 100
> $ getconf OPEN_MAX
> 1024
> $ getconf PATH_MAX /proc/
> 4096
> $

Yes, yes, yes, I like that one actually.

It does solve the shell script issues and we've never said that things
don't need to adapt to changes before so I don't see why not now.

And that one would be good to have regardless of the HZ issue.

// Stefan

Re: finding out the value of HZ from userspace

[Micha Feigin]

On Sat, Mar 20, 2004 at 12:28:00PM +0100, Stefan Smietanowski wrote:
> >>>there is one. Nothing uses it
> >>>(sysconf() provides this info)
> >>
> >>Seems to me that it would be fairly trivial to modify those programs 
> >>(that should use this mechanism but don't) to use it?  So why should 
> >>they be allowed to dictate kernel behaviour?
> >
> >
> >quality of implementation; for example shell scripts that want to do
> >echo 500 > /proc/sys/foo/bar/something_in_HZ
> >...
> >or /etc/sysctl.conf or ...
> >
> 
> Then write a simple program already. How hard is it to write a program
> that does a sysconf() and returns (as ascii of course) just the
> value of HZ? Then do some trivial calculation off of that.
> 
> HZ=$(gethz)
> 
> If your 500 was 5 seconds, do
> 
> TIME=$[HZ*5]
> echo $TIME > /proc/sys/foo/bar/something_in_HZ
> 

Will this be USER_HZ or kernel HZ?
Someone earlier suggested it would be USER_HZ which would make it
pointless.

> I mean, come on.
> 
> Then you include it in the default distro of choice so that
> everybody can use it and there you are.
> 
> If someone doesn't have "gethz" then they can download it.
> 
> // Stefan
> 

Re: finding out the value of HZ from userspace

[Peter Williams]

Micha Feigin wrote:
> On Sat, Mar 20, 2004 at 12:28:00PM +0100, Stefan Smietanowski wrote:
> 
>>>>>there is one. Nothing uses it
>>>>>(sysconf() provides this info)
>>>>
>>>>Seems to me that it would be fairly trivial to modify those programs 
>>>>(that should use this mechanism but don't) to use it?  So why should 
>>>>they be allowed to dictate kernel behaviour?
>>>
>>>
>>>quality of implementation; for example shell scripts that want to do
>>>echo 500 > /proc/sys/foo/bar/something_in_HZ
>>>...
>>>or /etc/sysctl.conf or ...
>>>
>>
>>Then write a simple program already. How hard is it to write a program
>>that does a sysconf() and returns (as ascii of course) just the
>>value of HZ? Then do some trivial calculation off of that.
>>
>>HZ=$(gethz)
>>
>>If your 500 was 5 seconds, do
>>
>>TIME=$[HZ*5]
>>echo $TIME > /proc/sys/foo/bar/something_in_HZ
>>
> 
> 
> Will this be USER_HZ or kernel HZ?
> Someone earlier suggested it would be USER_HZ which would make it
> pointless.

It has to be whatever enables user space to correctly interpret values 
sent to user space as "ticks".  That means USER_HZ and it's not useless 
as it enables USER_HZ to be different and/or change without breaking 
programs that use values expressed in "ticks".

> 
> 
>>I mean, come on.
>>
>>Then you include it in the default distro of choice so that
>>everybody can use it and there you are.
>>
>>If someone doesn't have "gethz" then they can download it.
>>
>>// Stefan
>>
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/


-- 
Dr Peter Williams, Chief Scientist                peterw@aurema.com
Aurema Pty Limited                                Tel:+61 2 9698 2322
PO Box 305, Strawberry Hills NSW 2012, Australia  Fax:+61 2 9699 9174
79 Myrtle Street, Chippendale NSW 2008, Australia http://www.aurema.com

Re: finding out the value of HZ from userspace

[Jamie Lokier]

Peter Williams wrote:
> >Will this be USER_HZ or kernel HZ?
> >Someone earlier suggested it would be USER_HZ which would make it
> >pointless.
> 
> It has to be whatever enables user space to correctly interpret values 
> sent to user space as "ticks".  That means USER_HZ and it's not useless 
> as it enables USER_HZ to be different and/or change without breaking 
> programs that use values expressed in "ticks".

It is, however, useless for the _other_ reasons userspace needs to
know kernel HZ, including as I mentioned userspace timer granularity.

(Btw, that usage would be better as a period rather than a frequency,
so that a "tickless" kernel can report zero).

The fundamental problem is that there are two values, and both values
have programs which can usefully use them.

How hard can it be to export both?

-- Jamie

Re: finding out the value of HZ from userspace

[Peter Williams]

Jamie Lokier wrote:
> Peter Williams wrote:
> 
>>>Will this be USER_HZ or kernel HZ?
>>>Someone earlier suggested it would be USER_HZ which would make it
>>>pointless.
>>
>>It has to be whatever enables user space to correctly interpret values 
>>sent to user space as "ticks".  That means USER_HZ and it's not useless 
>>as it enables USER_HZ to be different and/or change without breaking 
>>programs that use values expressed in "ticks".
> 
> 
> It is, however, useless for the _other_ reasons userspace needs to
> know kernel HZ, including as I mentioned userspace timer granularity.

Theoretically, which I know can be a pain, user space timer granularity 
should be in USER_HZ as, theoretically, this is the only one user space 
is supposed to know about.  Because of this, in my view, HZ and USER_HZ 
should be the same or USER_HZ should be greater than HZ.

> 
> (Btw, that usage would be better as a period rather than a frequency,
> so that a "tickless" kernel can report zero).

_SC_CLK_TCK is a POSIX.1 definition and can't be changed.  But I don't 
think that there's any impediment to adding new parameters that can be 
reported by sysconf().

> 
> The fundamental problem is that there are two values,  and both values
> have programs which can usefully use them.
> 
> How hard can it be to export both?
> 

Making HZ == USER_HZ would also solve the problem.

Peter
-- 
Dr Peter Williams, Chief Scientist                peterw@aurema.com
Aurema Pty Limited                                Tel:+61 2 9698 2322
PO Box 305, Strawberry Hills NSW 2012, Australia  Fax:+61 2 9699 9174
79 Myrtle Street, Chippendale NSW 2008, Australia http://www.aurema.com

Re: finding out the value of HZ from userspace

[Jamie Lokier]

Peter Williams wrote:
> Making HZ == USER_HZ would also solve the problem.

They were equal once.

Making them equal now would reintroduce the problem that USER_HZ was
created to resolve: some userspace programs hard-code the value, so it
cannot be changed in interfaces used by those programs.

-- Jamie

Re: finding out the value of HZ from userspace

[Peter Williams]

Jamie Lokier wrote:
> Peter Williams wrote:
> 
>>Making HZ == USER_HZ would also solve the problem.
> 
> 
> They were equal once.
> 
> Making them equal now would reintroduce the problem that USER_HZ was
> created to resolve: some userspace programs hard-code the value, so it
> cannot be changed in interfaces used by those programs.

That was the wrong solution to that particular problem.  The programs 
should have been fixed rather than the kernel being maimed to 
accommodate their shortcomings.

Peter
-- 
Dr Peter Williams, Chief Scientist                peterw@aurema.com
Aurema Pty Limited                                Tel:+61 2 9698 2322
PO Box 305, Strawberry Hills NSW 2012, Australia  Fax:+61 2 9699 9174
79 Myrtle Street, Chippendale NSW 2008, Australia http://www.aurema.com

Re: finding out the value of HZ from userspace

[Jamie Lokier]

Peter Williams wrote:
> >>Making HZ == USER_HZ would also solve the problem.
> >
> >Making them equal now would reintroduce the problem that USER_HZ was
> >created to resolve: some userspace programs hard-code the value, so it
> >cannot be changed in interfaces used by those programs.
> 
> That was the wrong solution to that particular problem.  The programs 
> should have been fixed rather than the kernel being maimed to 
> accommodate their shortcomings.

I agree, and perhaps that should still be done so we can eliminate USER_HZ.

-- Jamie

Re: finding out the value of HZ from userspace

[Micha Feigin]

On Tue, Mar 23, 2004 at 10:04:22AM +1100, Peter Williams wrote:
> Micha Feigin wrote:
> >On Sat, Mar 20, 2004 at 12:28:00PM +0100, Stefan Smietanowski wrote:
> >
> >>>>>there is one. Nothing uses it
> >>>>>(sysconf() provides this info)
> >>>>
> >>>>Seems to me that it would be fairly trivial to modify those programs 
> >>>>(that should use this mechanism but don't) to use it?  So why should 
> >>>>they be allowed to dictate kernel behaviour?
> >>>
> >>>
> >>>quality of implementation; for example shell scripts that want to do
> >>>echo 500 > /proc/sys/foo/bar/something_in_HZ
> >>>...
> >>>or /etc/sysctl.conf or ...
> >>>
> >>
> >>Then write a simple program already. How hard is it to write a program
> >>that does a sysconf() and returns (as ascii of course) just the
> >>value of HZ? Then do some trivial calculation off of that.
> >>
> >>HZ=$(gethz)
> >>
> >>If your 500 was 5 seconds, do
> >>
> >>TIME=$[HZ*5]
> >>echo $TIME > /proc/sys/foo/bar/something_in_HZ
> >>
> >
> >
> >Will this be USER_HZ or kernel HZ?
> >Someone earlier suggested it would be USER_HZ which would make it
> >pointless.
> 
> It has to be whatever enables user space to correctly interpret values 
> sent to user space as "ticks".  That means USER_HZ and it's not useless 
> as it enables USER_HZ to be different and/or change without breaking 
> programs that use values expressed in "ticks".
> 

Unless the kernel is converted to make that conversion possible then it
is useless at the moment since userspace gets USER_HZ and the kernel
proc interface speaks (KERNEL) HZ so userspace really has no idea how
to speak to kernel space with 2.6.

> >
> >
> >>I mean, come on.
> >>
> >>Then you include it in the default distro of choice so that
> >>everybody can use it and there you are.
> >>
> >>If someone doesn't have "gethz" then they can download it.
> >>
> >>// Stefan
> >>
> >
> >-
> >To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> >the body of a message to majordomo@vger.kernel.org
> >More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >Please read the FAQ at  http://www.tux.org/lkml/
> 
> 
> -- 
> Dr Peter Williams, Chief Scientist                peterw@aurema.com
> Aurema Pty Limited                                Tel:+61 2 9698 2322
> PO Box 305, Strawberry Hills NSW 2012, Australia  Fax:+61 2 9699 9174
> 79 Myrtle Street, Chippendale NSW 2008, Australia http://www.aurema.com
> 
> 
> +++++++++++++++++++++++++++++++++++++++++++
> This Mail Was Scanned By Mail-seCure System
> at the Tel-Aviv University CC.
> 

Re: finding out the value of HZ from userspace

[Peter Williams]

Arjan van de Ven wrote:
> On Wed, Mar 17, 2004 at 10:38:03AM +1100, Peter Williams wrote:
> 
>>>there is one. Nothing uses it
>>>(sysconf() provides this info)
>>
>>Seems to me that it would be fairly trivial to modify those programs 
>>(that should use this mechanism but don't) to use it?  So why should 
>>they be allowed to dictate kernel behaviour?
> 
> 
> quality of implementation; for example shell scripts that want to do
> echo 500 > /proc/sys/foo/bar/something_in_HZ
> ...
> or /etc/sysctl.conf or ...
> 

A small utility program secs_to_ticks would solve this problem e.g.:

secs_to_ticks 0.5 > /proc/sys/foo/bar/something_in_HZ

Peter
-- 
Dr Peter Williams, Chief Scientist                peterw@aurema.com
Aurema Pty Limited                                Tel:+61 2 9698 2322
PO Box 305, Strawberry Hills NSW 2012, Australia  Fax:+61 2 9699 9174
79 Myrtle Street, Chippendale NSW 2008, Australia http://www.aurema.com

Re: finding out the value of HZ from userspace

[tabris]

On Saturday 13 March 2004 12:24 pm, Arjan van de Ven wrote:
> On Thu, 2004-03-11 at 15:17, Micha Feigin wrote:
> > Is it possible to find out what the kernel's notion of HZ is from user
> > space?
> > It seem to change from system to system and between 2.4 (100 on i386)
> > to 2.6 (1000 on i386).
>
> if you can see 1000 from userspace that is a bad kernel bug; can you say
> where you find something in units of 1000 ?
2.6.3-rc1-mm1

procinfo gives the timer interrupt counting 1000 ints/sec
tho procinfo is broken for other stuff like 2.4 showed pages swapped, pages 
read in and out.

--
tabris
-
"We never make assertions, Miss Taggart," said Hugh Akston.  "That is
the moral crime peculiar to our enemies.  We do not tell -- we *show*.
We do not claim -- we *prove*."  
-- Ayn Rand, _Atlas Shrugged_

Re: finding out the value of HZ from userspace

[Micha Feigin]

On Sat, Mar 13, 2004 at 06:24:31PM +0100, Arjan van de Ven wrote:
> On Thu, 2004-03-11 at 15:17, Micha Feigin wrote:
> > Is it possible to find out what the kernel's notion of HZ is from user
> > space?
> > It seem to change from system to system and between 2.4 (100 on i386)
> > to 2.6 (1000 on i386).
> 
> if you can see 1000 from userspace that is a bad kernel bug; can you say
> where you find something in units of 1000 ?

I can't see it from user space. Its in the kernel headers. The thing is
I am working on fixes to laptop mode. The problem is it requires
changing bdflush and journaled file systems journal flush times. The
problem is that some of these (bdflush, xfs) expect the value in jiffies
and not seconds or milliseconds so making the initiation script portable
requires knowing the value of HZ.

Re: finding out the value of HZ from userspace

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 1163 bytes --]

On Sat, 2004-03-13 at 23:10, Micha Feigin wrote:
> On Sat, Mar 13, 2004 at 06:24:31PM +0100, Arjan van de Ven wrote:
> > On Thu, 2004-03-11 at 15:17, Micha Feigin wrote:
> > > Is it possible to find out what the kernel's notion of HZ is from user
> > > space?
> > > It seem to change from system to system and between 2.4 (100 on i386)
> > > to 2.6 (1000 on i386).
> > 
> > if you can see 1000 from userspace that is a bad kernel bug; can you say
> > where you find something in units of 1000 ?
> 
> I can't see it from user space. Its in the kernel headers. The thing is
> I am working on fixes to laptop mode. The problem is it requires
> changing bdflush and journaled file systems journal flush times. The
> problem is that some of these (bdflush, xfs) expect the value in jiffies
> and not seconds or milliseconds so making the initiation script portable
> requires knowing the value of HZ.

the kernel side is supposed to use clock_t_to_jiffies() and co for this
to present a unified HZ to userspace. The internal kernel HZ should
*NOT* leak out to usespace. Heck it's quite thinkable that in the future
there will be no such HZ.
 


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: finding out the value of HZ from userspace

[Micha Feigin]

On Sat, Mar 13, 2004 at 11:41:25PM +0100, Arjan van de Ven wrote:
> On Sat, 2004-03-13 at 23:10, Micha Feigin wrote:
> > On Sat, Mar 13, 2004 at 06:24:31PM +0100, Arjan van de Ven wrote:
> > > On Thu, 2004-03-11 at 15:17, Micha Feigin wrote:
> > > > Is it possible to find out what the kernel's notion of HZ is from user
> > > > space?
> > > > It seem to change from system to system and between 2.4 (100 on i386)
> > > > to 2.6 (1000 on i386).
> > > 
> > > if you can see 1000 from userspace that is a bad kernel bug; can you say
> > > where you find something in units of 1000 ?
> > 
> > I can't see it from user space. Its in the kernel headers. The thing is
> > I am working on fixes to laptop mode. The problem is it requires
> > changing bdflush and journaled file systems journal flush times. The
> > problem is that some of these (bdflush, xfs) expect the value in jiffies
> > and not seconds or milliseconds so making the initiation script portable
> > requires knowing the value of HZ.
> 
> the kernel side is supposed to use clock_t_to_jiffies() and co for this
> to present a unified HZ to userspace. The internal kernel HZ should
> *NOT* leak out to usespace. Heck it's quite thinkable that in the future
> there will be no such HZ.
>  
> 

Kernel side doesn't do that at the moment. Even the fixed bdflush
interface in 2.6 which has dirty_writeback_centisecs as an example
converts it as
(dirty_writeback_centisecs * HZ) / 100

Re: finding out the value of HZ from userspace

[John Reiser]

> The internal kernel HZ should *NOT* leak out to usespace.

/proc/interrupts "leaks" the value of HZ.  On x86, for instance:
    ( cat /proc/interrupts; sleep 5; cat /proc/interrupts )  |  grep timer

-- 

Re: finding out the value of HZ from userspace

[Horst von Brand]

Micha Feigin <michf@post.tau.ac.il> said:
> Is it possible to find out what the kernel's notion of HZ is from user
> space?

What for? It should be invisible to userspace...

> It seem to change from system to system and between 2.4 (100 on i386)
> to 2.6 (1000 on i386).

And can also be tweaked when compiling, and depends on architecture, and...
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: finding out the value of HZ from userspace

[Micha Feigin]

On Sat, Mar 13, 2004 at 11:45:17PM -0300, Horst von Brand wrote:
> Micha Feigin <michf@post.tau.ac.il> said:
> > Is it possible to find out what the kernel's notion of HZ is from user
> > space?
> 
> What for? It should be invisible to userspace...
> 

Its not. Some proc interfaces expect time in jiffies, which means
knowing HZ (bdflush in 2.4 or xfs for example).

> > It seem to change from system to system and between 2.4 (100 on i386)
> > to 2.6 (1000 on i386).
> 
> And can also be tweaked when compiling, and depends on architecture, and...
> -- 
> Dr. Horst H. von Brand                   User #22616 counter.li.org
> Departamento de Informatica                     Fono: +56 32 654431
> Universidad Tecnica Federico Santa Maria              +56 32 654239
> Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>  
>  +++++++++++++++++++++++++++++++++++++++++++
>  This Mail Was Scanned By Mail-seCure System
>  at the Tel-Aviv University CC.
> 

Re: finding out the value of HZ from userspace

[Jamie Lokier]

Horst von Brand wrote:
> What for? It should be invisible to userspace...

It's not invisible.  select/poll/epoll/setitimer round their time
argument according to HZ, and programs which do smooth (i.e. low
_jitter_) animation of the kind where the eye is sensitive to the
jitter need to track it and correct for it.

-- Jamie

Re: finding out the value of HZ from userspace

[Mark Gross]

On Monday 15 March 2004 00:17, Jamie Lokier wrote:
> Horst von Brand wrote:
> > What for? It should be invisible to userspace...
>
> It's not invisible.  select/poll/epoll/setitimer round their time
> argument according to HZ, and programs which do smooth (i.e. low
> _jitter_) animation of the kind where the eye is sensitive to the
> jitter need to track it and correct for it.
>

Wouldn't it be better to just use high res timers and associated posix interfaces or low jitter applications?

--mgross

Re: finding out the value of HZ from userspace

[Richard Curnow]

* Horst von Brand <vonbrand@inf.utfsm.cl> [2004-03-14]:
> Micha Feigin <michf@post.tau.ac.il> said:
> > Is it possible to find out what the kernel's notion of HZ is from user
> > space?
> 
> What for? It should be invisible to userspace...
> 

A related issue that's bugged me for a long time is lack of userspace
access to the quantity that's called 'freq_scale' in 2.4, where it's
(1<<SHIFT_HZ)/HZ for HZ!=100 and 128/128.125 for HZ==100.  (I haven't
started to reverse-engineer the equivalent value in 2.6, I took a quick
look once and concluded things had got a little more hairy.)

My interest is that I maintain (in spare-time) an NTP application called
chrony (http://chrony.sunsite.dk/), originally written to be good for
dial-up, i.e. NTP servers accessible for a short window once or twice a
day.  This app wants to tune the parameters it passes to adjtimex() to
take a best shot at keeping the system clock correct over the
potentially 'long' offline period.  To do this well, it has to
reverse-compensate for the freq_scale multiplier that the kernel will
apply to the frequency value passed to adjtimex().  Getting the right
value for this across different kernels has always been a fragile
exercise.

-- 
Richard \\\ SH-4/SH-5 Core & Debug Architect
Curnow  \\\         SuperH (UK) Ltd, Bristol

Call for HRT in 2.6 kernel was Re: finding out the value of HZ from userspace

[Mark Gross]

[-- Attachment #1: Type: text/plain, Size: 5351 bytes --]

On Tuesday 16 March 2004 18:30, Jamie Lokier wrote:
> Mark Gross wrote:
> > What can I do to help get high res timers available in the base kernel?
>
> Patches were written long ago, by George Anzinger I think.
>
> They were not accepted.  You might want to look into why not.  I think
> the reason was nothing to do with the quality of code, but rather that
> the clock programming overhead and code bloat wasn't desirable, and
> the gains not worth it.
>
> That is less true now that the kernel is pre-emptive.  Before,
> high-res timers couldn't provide any useful scheduling guarantee: you
> might receive a timer at exactly 1.56ms, but your code might still not
> run for another 150ms anyway.

Thats not what we see using the HRT patch on 2.4.40, or with the RHT patch on 2.6.

Yes, preemptive kernels are cool.

>
> Now there is still no guarantee, but the statistical response is much
> better.

Its A LOT better in practice with HRT, and more or less good enough for interactive  and multi-media uses.

>
> > I have some internal folks I've been assigned to help out that need
> > a low jitter time base to do some VoIP and need jitter < 250usec on
> > a ~2ms timer for the DSP computations to be do-able in user space.
> >
> > I would love to help get the existing HRT patches updated so as to
> > be acceptible, or failing that perhaps write a simple low jitter
> > time base driver vareant of the rtc driver.
>
> If it's appropriate for your application, just use one of the existing
> real-time linuxes (RTAI etc.).  They all offer high-res timers and
> will give much better VoIP guarantees than any of the high-res timer
> patches for non-real-time linux.
>

Nope, we need something thats in the future typical distribution or this product is not viable.

This isn't an embedded gadgit these guys are working on, so shipping an HRT patched kernel
is a show stopper for the product.

> If, however, you can't do that, consider: on i386 HZ is currently
> 1000.  So your request for 2ms timer is perfectly satisfiable using
> the standard timers.  Remember to turn on CONFIG_PREEMPT, and use a
> SCHED_FIFO sceduling policy.
>

They tested that a long time ago, an I re-tested it recently.  It doesn't work.
The Jitter is just way too high doing this without HR timers.

Check out the non-HRT timer code it rounds up to the next jiffies, always.  

Running with the HRT patch, we get a lot closer to what is being asked for.  

> If you measure the jitter and find it is unacceptable, be aware that
> none of the high-res timer patches for non-real-time linux will
> improve that.

Not true.  The HRT patch does indeed improve things a lot.  In fact it more or less does the job and it
enables the application.  Its not perfect, but its good.

The high res timer patch re-programs the PIT to produce an interrupt as close to the timeout as it 
can, where just the jiffies clock will wake up on the following jiffies tick.  On average 1 jiffies late!  Thats
a LOT of jitter.  If you look at the code and follow through the logic, if you ask for a 2ms sleep, you are basically 
going to get a 3ms sleep.   If you ask for a 1.1ms sleep you get a 2ms (with random larger jitters) sleep.  
To change this without doing some of the things in the HRT patch opens up the timer code to waking up 
the process too early.  Also a bad thing.  

This just isn't good enough for an entire class of applications that could exist on linux if it weren't for this issue.

>
> > Yup, and the human ear is even more sensitive.
> > Busted lip synch on video playback is embarassing under linux.
>
> Lip sync isn't a problem for buffered audio+video, if the playback
> code is able to adapt to the sound card's slight deviation from the
> nominal sample rate.  The sound card itself provides the regular
> clock.  A small difference between audio and video times is ok, as
> long as it stays consistent.
>
> The difficulty occurs with two-way communication, i.e. VoIP and video,
> when you can't buffer much.
>

Communications is definitely a harder problem calling for good low jitter time base services

> > My faverate flash animation web sites have a hard time with this as well.
>
> Btw, I noticed that flash animations seem visually smoother on
> Netscape 4 than Mozilla 1.2, on the same fast box running 2.6.
> Strange -- do they have different flash implementations?
>

I don't know, but my home box doesn't play back flash too well when booted into Mandrake or Fedora, where
you-know-who's OS works just fine.  Its a BP-6 box, not too speedy.  Regardless, my point is that without
an OS standard low jitter time base in the OS all the ISV's will be cobbling together there own hacks to get
low jitter time bases for their applications.  Some will work well, others will be flaky, more will have coexistence
issues sharing time base sources like the /dev/rtc.  

If the OS provides such support they would all tend to do the same thing and programs 
needing this type of time base features would all just work better.

> > Linux needs a low jitter time base standard for desktop multi-media
> > applications of many types.
>
> That's one of the reasons why HZ was changed to 1000 on x86 for 2.6
> kernels, and the major motivation for adding CONFIG_PREEMPT.
>

I know, but the current solution still isn't good enough, on a number of levels.


--mgross

[-- Attachment #2: jitter_test_noHRT.c --]
[-- Type: text/x-csrc, Size: 3240 bytes --]

#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <time.h>
#include <sys/io.h>
#include <sched.h>
#include <sys/time.h>
#include <sys/mman.h>
#include <errno.h>
#include <signal.h>
#include <assert.h>

#define NOF_JITTER (500 * 60) /* 1 minute test 2ms*500*60 */

int jitter[NOF_JITTER];
int ijit;
struct timeval start;

void alrm_handler(int signo, siginfo_t *info, void *context)
{
  struct timeval end;
  int delta;
  static int bla=0;
  static int first = 1;

  printf(".\n");

  ioperm(0x378,3,1);

  gettimeofday(&end, NULL);

  if(bla) {
     outb(0x1,0x378);
     bla = 0;
   } else { 
     outb(0x0,0x378);
     bla = 1;

  }
	

  delta = (end.tv_sec - start.tv_sec) * 1000000 +
    (end.tv_usec - start.tv_usec);
  delta -= 2000;
  start = end;
  if (first) { first = 0; return; }
  if(ijit >= NOF_JITTER)
	printf("out of number already... \n");
  jitter[ijit++] = delta;
}

void print_hist(void)
{
#define HISTSIZE 1000*5 * 10

  int hist[50000+ 1];
  int i;

  printf("jitter[0] = %d\n", jitter[0]);
  memset(hist, 0, sizeof(hist));

  for (i = 1; i < NOF_JITTER; i++) {
    if (jitter[i] >= HISTSIZE/2) {
      printf("sample %d over max hist: %d\n", i, jitter[i]);
      hist[HISTSIZE]++;
    }
    else if (jitter[i] <= -HISTSIZE/2) {
      printf("sample %d over min hist: %d\n", i, jitter[i]);
      hist[0]++;
    }
    else {
      hist[jitter[i] + HISTSIZE/2]++;
    }
  }
  for (i = 1; i < HISTSIZE; i++) {
    if (hist[i]) {
      printf("%d: %d\n", i-HISTSIZE/2, hist[i]);
    }
  }
  printf("HC-: %d\n", hist[0]);
  printf("HC+: %d\n", hist[HISTSIZE]);
}

void print_avg(void)
{
  double sum;
  int i;

  sum = 0;
  for (i = 1; i < NOF_JITTER; i++) {
    sum += jitter[i];
  }

  printf("avg. jitter: %f\n", sum/(NOF_JITTER-1));
}

int main(void)
{
	int retval;
	timer_t t = 0;
	struct itimerspec ispec;
	struct itimerspec ospec;
	struct sigaction sa;
	struct sched_param sched;
#if 1 
	retval = mlockall(MCL_CURRENT|MCL_FUTURE);
	if (retval) {
	  perror("mlockall(MCL_CURRENT|MCL_FUTURE) failed");
	}
	assert(retval == 0);

	sched.sched_priority = 2;
	retval = sched_setscheduler(0, SCHED_FIFO, &sched); 
	if (retval) {
	  perror("sched_setscheduler(SCHED_FIFO)");
	}
	assert(retval == 0);
#endif

	sa.sa_sigaction = alrm_handler;
	sa.sa_flags = SA_SIGINFO;
	sigemptyset(&sa.sa_mask);

	if (sigaction(SIGALRM, &sa, NULL)) {
		perror("sigaction failed");
		exit(1);
	}

	if (sigaction(SIGRTMIN, &sa, NULL)) {
		perror("sigaction failed");
		exit(1);
	}

	retval = timer_create(CLOCK_REALTIME, NULL, &t);
	if (retval) {
		perror("timer_create(CLOCK_REALTIME) failed");
	}
	assert(retval == 0);

	retval = clock_gettime(CLOCK_REALTIME, &ispec.it_value);
	if (retval) {
		perror("clock_gettime(CLOCK_REALTIME) failed");
	}
	ispec.it_value.tv_sec += 1;
	ispec.it_value.tv_nsec = 0;
	ispec.it_interval.tv_sec = 0;
	ispec.it_interval.tv_nsec = 2*1000*1000; /* 100 Hz */

	retval = timer_settime(t, TIMER_ABSTIME, &ispec, &ospec);
	if (retval) {
		perror("timer_settime(TIMER_ABSTIME) failed");
	}

	do { pause(); } while (ijit < NOF_JITTER);

	retval = timer_delete(t);
	if (retval) {
		perror("timer_delete(existing timer) failed");
	}
	assert(retval == 0);

	print_hist();

	print_avg();

	return 0;
}

Re: Call for HRT in 2.6 kernel was Re: finding out the value of HZ from userspace

[Jamie Lokier]

Mark Gross wrote:
> > If, however, you can't do that, consider: on i386 HZ is currently
> > 1000.  So your request for 2ms timer is perfectly satisfiable using
> > the standard timers.  Remember to turn on CONFIG_PREEMPT, and use a
> > SCHED_FIFO sceduling policy.
> 
> Check out the non-HRT timer code it rounds up to the next jiffies, always.  

You said you wanted a _2ms_ timer.  Rounded up to the next jiffie,
that's... 2ms!

> Running with the HRT patch, we get a lot closer to what is being asked for.  

Ok.

> > If you measure the jitter and find it is unacceptable, be aware that
> > none of the high-res timer patches for non-real-time linux will
> > improve that.
> 
> Not true.  The HRT patch does indeed improve things a lot.  In fact
> it more or less does the job and it enables the application.  Its
> not perfect, but its good.

Ok.  My point was theoretical and I got it wrong, you're right and you
tried it. :)

> The high res timer patch re-programs the PIT to produce an interrupt
> as close to the timeout as it can, where just the jiffies clock will
> wake up on the following jiffies tick.  On average 1 jiffies late!
> Thats a LOT of jitter.  If you look at the code and follow through
> the logic, if you ask for a 2ms sleep, you are basically going to
> get a 3ms sleep.  If you ask for a 1.1ms sleep you get a 2ms (with
> random larger jitters) sleep.

Yes.  The point is that the added delay with the standard timers is
predictable, so it is possible to structure your program around that,
synchronising to the jiffies clock: have your program tick every
1.99ms or whatever that _actual_ rate of HZ/2 is on 2.6 x86 kernels.
(I gather the jiffie is slightly shorter than 1ms due to timer chip
limitation).

I don't see that the unpredictable part of the jitter would be
improved with high res timers: the unpredictable part being due to
disabled preemption, other interrupts etc.

That's what I meant by jitter, sorry for the lack of precision (no pun
intended).

> To change this without doing some of the things in the HRT patch opens up the timer code to waking up 
> the process too early.  Also a bad thing.

That's what I did with my old "Snake" program: determine when select()
will round up, and then wake up early and busy-wait in a loop calling
gettimeofday() until the precise time arrives.

It's not good, although the busy wait is limited to the length of 1
jiffie, or less if you can structure your program to compute
synchronised with the jiffie clock.

> This just isn't good enough for an entire class of applications that
> could exist on linux if it weren't for this issue.

Hmm.

For VoIP, I'm wondering why you need a timebase other than the sound
card.  Won't it provide an interrupt for every new sound fragment?

> > > Linux needs a low jitter time base standard for desktop multi-media
> > > applications of many types.
> >
> > That's one of the reasons why HZ was changed to 1000 on x86 for 2.6
> > kernels, and the major motivation for adding CONFIG_PREEMPT.
> 
> I know, but the current solution still isn't good enough, on a
> number of levels.

To demonstrate that 1000Hz ticks aren't good enough, because you need
much smaller jitter than 1ms on "ordinary machines" i.e. standard
distros, you'll have to demonstrate that you really are seeing much
smaller jitter than 1ms in your HRT-patched kernels and that it makes
a useful difference.

The pre-emptive patches was initially rejected, but Linus changed his
mind after a lot of good experimental data showing significant and
consistent improvements in latency statistics, the fact that the
patches were remarkeably non-invasive (because most of the work had
been done to support fine-grained SMP by then), and perhaps most
importantly, and surprisingly, I/O performance improved.

So there is hope with HRT, but it needs more than an implementation to
get into the standard tree (IMHO): it has to be fairly small,
non-invasive, not harm existing performance, and backed by convincing
experimental data showing worthwhile improvements.

On the bright side, HRT makes it possible to eliminate the jiffie tick
entirely, which is quite likely to be good for performance and power
consumption.  The objection to that has been that changing code which
depends on the timer tick to not use it any more would complicate that
code without much gain, and it's just not worth complicating anything
for it.  But maybe, as for kernel pre-emption, it will turn out
simpler than expected.

-- Jamie

Re: Call for HRT in 2.6 kernel was Re: finding out the value of HZ from userspace

[Mark Gross]

[-- Attachment #1: Type: text/plain, Size: 4213 bytes --]

On Wednesday 17 March 2004 12:07, Jamie Lokier wrote:
> That's what I did with my old "Snake" program: determine when select()
> will round up, and then wake up early and busy-wait in a loop calling
> gettimeofday() until the precise time arrives.
>
> It's not good, although the busy wait is limited to the length of 1
> jiffie, or less if you can structure your program to compute
> synchronised with the jiffie clock.
>

Is this not a very strong argument for some type of HRT support in the kernel?

> > This just isn't good enough for an entire class of applications that
> > could exist on linux if it weren't for this issue.
>
> Hmm.
>
> For VoIP, I'm wondering why you need a timebase other than the sound
> card.  Won't it provide an interrupt for every new sound fragment?
>

I don't know the application the team I'm trying to help out well enough to say if thats workable.

We are exploring different "plan -B"s in case we can't get George's HRT patch updated and
into the base.  I would rather put my effort into helping George and the HRT 
implementation than writing another RTC like driver with crappy non-posix interface.

> > > > Linux needs a low jitter time base standard for desktop multi-media
> > > > applications of many types.
> > >
> > > That's one of the reasons why HZ was changed to 1000 on x86 for 2.6
> > > kernels, and the major motivation for adding CONFIG_PREEMPT.
> >
> > I know, but the current solution still isn't good enough, on a
> > number of levels.
>
> To demonstrate that 1000Hz ticks aren't good enough, because you need
> much smaller jitter than 1ms on "ordinary machines" i.e. standard
> distros, you'll have to demonstrate that you really are seeing much
> smaller jitter than 1ms in your HRT-patched kernels and that it makes
> a useful difference.

I'm trying!  

I think I have demostrated that you cannot get a 1ms timer 
by code inspection.

I've also demonstrated that asking for a 2ms periodic wake up will 
result in a 3ms periodic wake up.

The application I'm trying to enable spec's out a 0.25ms jitter on a 2ms 
periodic event clock to support doing some audio dsp.  However; I 
cannot argue the validity of the 0.25ms requierment.  I think its a valid requierment.

Attached is the jitter test using the HRT.  Running on 2.6.3 + a rebase of the source 
forge patch gets me a 2ms wave form with some jitter < 0.25ms.  On my oscilloscope
it "looks" like about +/- 0.2 ms.  

It makes a useful difference today.

I should state the HRT patch for 2.6 on the source forge site is a bit out 
of date WRT 2.6 and needs some updating.  It works for the test application, but has 
some problems that don't happen with the 2.4  version of the patch.  George 
tells me he's rolling in some fixes he has binned up "soon".  Running the same test
using the 2.4.20 + HRT + preemption patch gives less jitter than the 2.6 version.
I'm here to help get that fixed up for 2.6.

The point here is that the rebased version of the current 2.6 HRT patch works good
enough for the application I'm worried about.


>
> The pre-emptive patches was initially rejected, but Linus changed his
> mind after a lot of good experimental data showing significant and
> consistent improvements in latency statistics, the fact that the
> patches were remarkeably non-invasive (because most of the work had
> been done to support fine-grained SMP by then), and perhaps most
> importantly, and surprisingly, I/O performance improved.
>
> So there is hope with HRT, but it needs more than an implementation to
> get into the standard tree (IMHO): it has to be fairly small,
> non-invasive, not harm existing performance, and backed by convincing
> experimental data showing worthwhile improvements.
>

Hope is good.

> On the bright side, HRT makes it possible to eliminate the jiffie tick
> entirely, which is quite likely to be good for performance and power
> consumption.  The objection to that has been that changing code which
> depends on the timer tick to not use it any more would complicate that
> code without much gain, and it's just not worth complicating anything
> for it.  But maybe, as for kernel pre-emption, it will turn out
> simpler than expected.

Perhaps.

--mgross

[-- Attachment #2: jitter_test.c --]
[-- Type: text/x-csrc, Size: 3766 bytes --]

#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <time.h>
#include <sys/io.h>
#include <sched.h>
#include <sys/time.h>
#include <sys/mman.h>
#include <errno.h>
#include <signal.h>
#include <assert.h>

#define __NR_timer_create	259
#define __NR_timer_settime	(__NR_timer_create+1)
#define __NR_timer_gettime	(__NR_timer_create+2)
#define __NR_timer_getoverrun	(__NR_timer_create+3)
#define __NR_timer_delete	(__NR_timer_create+4)
#define __NR_clock_settime	(__NR_timer_create+5)
#define __NR_clock_gettime	(__NR_timer_create+6)
#define __NR_clock_getres	(__NR_timer_create+7)
#define __NR_clock_nanosleep	(__NR_timer_create+8)
#include "high-res-timers/lib/posix_time.h"
#include "high-res-timers/lib/syscall_timer.c"


#define NOF_JITTER (500*60) /* 1 minute test 2ms * 500 * 60 */

int jitter[NOF_JITTER];
int ijit;
struct timeval start;

void alrm_handler(int signo, siginfo_t *info, void *context)
{
  struct timeval end;
  int delta;
  static int bla=0;
  static int first = 1;

  printf(".\n");

  ioperm(0x378,3,1);

  gettimeofday(&end, NULL);

  if(bla) {
     outb(0x1,0x378);
     bla = 0;
   } else { 
     outb(0x0,0x378);
     bla = 1;

  }
	

  delta = (end.tv_sec - start.tv_sec) * 1000000 +
    (end.tv_usec - start.tv_usec);
  delta -= 2000;
  start = end;
  if (first) { first = 0; return; }
  if(ijit >= NOF_JITTER)
	printf("out of number already... \n");
  jitter[ijit++] = delta;
}

void print_hist(void)
{
#define HISTSIZE 1000*5 * 10

  int hist[50000+ 1];
  int i;

  printf("jitter[0] = %d\n", jitter[0]);
  memset(hist, 0, sizeof(hist));

  for (i = 1; i < NOF_JITTER; i++) {
    if (jitter[i] >= HISTSIZE/2) {
      printf("sample %d over max hist: %d\n", i, jitter[i]);
      hist[HISTSIZE]++;
    }
    else if (jitter[i] <= -HISTSIZE/2) {
      printf("sample %d over min hist: %d\n", i, jitter[i]);
      hist[0]++;
    }
    else {
      hist[jitter[i] + HISTSIZE/2]++;
    }
  }
  for (i = 1; i < HISTSIZE; i++) {
    if (hist[i]) {
      printf("%d: %d\n", i-HISTSIZE/2, hist[i]);
    }
  }
  printf("HC-: %d\n", hist[0]);
  printf("HC+: %d\n", hist[HISTSIZE]);
}

void print_avg(void)
{
  double sum;
  int i;

  sum = 0;
  for (i = 1; i < NOF_JITTER; i++) {
    sum += jitter[i];
  }

  printf("avg. jitter: %f\n", sum/(NOF_JITTER-1));
}

int main(void)
{
	int retval;
	timer_t t = 0;
	struct itimerspec ispec;
	struct itimerspec ospec;
	struct sigaction sa;
	struct sched_param sched;
#if 1 
	retval = mlockall(MCL_CURRENT|MCL_FUTURE);
	if (retval) {
	  perror("mlockall(MCL_CURRENT|MCL_FUTURE) failed");
	}
	assert(retval == 0);

	sched.sched_priority = 2;
	retval = sched_setscheduler(0, SCHED_FIFO, &sched); 
	if (retval) {
	  perror("sched_setscheduler(SCHED_FIFO)");
	}
	assert(retval == 0);
#endif

	sa.sa_sigaction = alrm_handler;
	sa.sa_flags = SA_SIGINFO;
	sigemptyset(&sa.sa_mask);

	if (sigaction(SIGALRM, &sa, NULL)) {
		perror("sigaction failed");
		exit(1);
	}

	if (sigaction(SIGRTMIN, &sa, NULL)) {
		perror("sigaction failed");
		exit(1);
	}

	retval = timer_create(CLOCK_REALTIME_HR, NULL, &t);
	if (retval) {
		perror("timer_create(CLOCK_REALTIME) failed");
	}
	assert(retval == 0);

	retval = clock_gettime(CLOCK_REALTIME_HR, &ispec.it_value);
	if (retval) {
		perror("clock_gettime(CLOCK_REALTIME) failed");
	}
	ispec.it_value.tv_sec += 1;
	ispec.it_value.tv_nsec = 0;
	ispec.it_interval.tv_sec = 0;
	ispec.it_interval.tv_nsec = 2*1000*1000; /* 100 Hz */

	retval = timer_settime(t, TIMER_ABSTIME, &ispec, &ospec);
	if (retval) {
		perror("timer_settime(TIMER_ABSTIME) failed");
	}

	do { pause(); } while (ijit < NOF_JITTER);

	retval = timer_delete(t);
	if (retval) {
		perror("timer_delete(existing timer) failed");
	}
	assert(retval == 0);

	print_hist();

	print_avg();

	return 0;
}

Re: Call for HRT in 2.6 kernel was Re: finding out the value of HZ from userspace

[Karim Yaghmour]

Jamie Lokier wrote:
> So there is hope with HRT, but it needs more than an implementation to
> get into the standard tree (IMHO): it has to be fairly small,
> non-invasive, not harm existing performance, and backed by convincing
> experimental data showing worthwhile improvements.

Looking at the high-res timer stuff, it's somewhat similar to what
RTAI does (reprogram timer chip, use nanoseconds for computations,
provide API for ease of use, etc.) except that it can't guarantee
hard-rt, and it's integrated directly with Linux's scheduling
whereas RTAI has its own scheduler and lives as a module.

It's not my intention to debate which is better, I'll leave that
for another day. What I'm interested in, however, is that there is
a common functionality that all such facilities can use to achieve/
deliver hard-rt.

I'm thinking here of Adeos. It's the smallest subset of services
required for obtaining hard-rt in the kernel, and it's fairly
non-invasive (not to mention that configuring it out results in no
changes to the kernel.) So while Adeos doesn't provide abstract
services such as "tasks" or "timers", it does provide the basic
mechanism for all add-ons that want to provide these to obtain
the hard-rt from Adeos using an architecture-independent API.

Here are a few examples of software that can obtain hard-rt with
Adeos:
- RT Executives: Currently RTAI uses Adeos on x86 and a port of
RTLinux/GPL to Adeos is planned. Adeos, however, isn't limited to
either of these executives, and could easily be used by any other
RT executive to sit side-by-side with Linux and other kernels.
- hard-rt drivers: It's fairly trivial for a driver to hook into the
Adeos pipeline and obtain hard-rt without using either RTAI or
RTLinux. In that case, there's just the standard Linux kernel with
a hard-rt driver side-by-side atop Adeos.
- HRT: Any mechanism that modifies the PIT can then export timer
services to drivers for providing them with deterministic timer
response times. Granted there would be no integration with the
current Linux scheduler, but the added advantage is that HRT can
live as a loadable module. Scheduling/notifying of user-space
processes using such HRT is possible; RTAI's hard-rt scheduling
of normal Linux processes being an example.

Actually, most software that needs hard-rt can live as loadable
modules once Adeos is integrated in the kernel.

The Adeos patches are available here for those who are interested:
http://download.gna.org/adeos/patches/

P.S.: Before anyone comes back shouting after looking at the #ifdefs
used to modify _existing_ kernel files in the current Adeos patches,
please keep in mind that they are mainly there because they make it
fairly trivial to create patches for new kernels. When cleaning up
the patches for inclusion, most of these would disappear.

Karim
-- 
Author, Speaker, Developer, Consultant
Pushing Embedded and Real-Time Linux Systems Beyond the Limits
http://www.opersys.com || karim@opersys.com || 1-866-677-4546

Re: Call for HRT in 2.6 kernel was Re: finding out the value of HZ from userspace

[Jamie Lokier]

I see we have gone from a desire for soft-rt high-res timers to
pushing hard-rt :)

Karim Yaghmour wrote:
> I'm thinking here of Adeos. It's the smallest subset of services
> required for obtaining hard-rt in the kernel,

In this case, it's not clear that hard-rt is desirable.  VoIP doesn't
like occasional glitches, but it can tolerate them and must do so when
a machine is overloaded, e.g. trying to handle too many scheduling
objectives at once.  I don't know much about the original poster's
problem, that's just my take on VoIP.

> and it's fairly non-invasive (not to mention that configuring it out
> results in no changes to the kernel.) So while Adeos doesn't provide
> abstract services such as "tasks" or "timers", it does provide the
> basic mechanism for all add-ons that want to provide these to obtain
> the hard-rt from Adeos using an architecture-independent API.

There is also Bernard Kuhn's recent "real-time interrupts" patch for
2.6 which could be utilised:

http://home.t-online.de/home/Bernhard_Kuhn/rtirq/20040304/rtirq.html
http://home.t-online.de/home/Bernhard_Kuhn/rtirq/20040304/rtirq-2.6.2-20040304.tar.bz2

> Actually, most software that needs hard-rt can live as loadable
> modules once Adeos is integrated in the kernel.

A couple of questions.

Can Adeos-registed timer callbacks call the same functions as normal
timer callbacks, schedule userspace, and kick network I/O with near-RT
guarantees?  Or do they run in a non-kernel context?

(Mark can say whether a normal context, i.e. with system calls, memory
allocation and network I/O, is required for Intel's VoIP application.)

Can Adeos itself be loaded as a module which overrides normal non-RT
kernel interrupt and timer functions?  If it can be kept out of the
standard kernel, but loaded when needed, that would be nice.

One more thing would help, IMHO, in getting any fancy interrupt system
in: if it balanced the different execution contexts, i.e. limit total
CPU taken in high priority, low priority interrupts, task queues
etc. in an efficient yet fair way, such that overall throughput was
improved over standard kernels in cases such as network overload.
NAPI does this at the network card level, but there is no reason why
balancing CPU among contexts cannot be done at the generic interrupt
scheduling level, making it work for all I/O devices without special
driver support.

-- Jamie

Re: Call for HRT in 2.6 kernel was Re: finding out the value of HZ from userspace

[Karim Yaghmour]

Jamie Lokier wrote:
> I see we have gone from a desire for soft-rt high-res timers to
> pushing hard-rt :)

:D

> In this case, it's not clear that hard-rt is desirable.  VoIP doesn't
> like occasional glitches, but it can tolerate them and must do so when
> a machine is overloaded, e.g. trying to handle too many scheduling
> objectives at once.  I don't know much about the original poster's
> problem, that's just my take on VoIP.

I agree that VoIP doesn't "require" hard-rt, I have no issue with that.
I was expanding on the specific discussion of the HRT mechanism, not
the intended usage itself. Let me know if you'd rather see this discussed
separately.

> There is also Bernard Kuhn's recent "real-time interrupts" patch for
> 2.6 which could be utilised:

The first implementation of such a mechanism for Linux was done by David
Schleef a few years back and it was called the "no-warm air" patch (long
story.) Basically, though, I think this is the wrong approach because the
resulting behavior is very much CPU-dependent (interrupt priority
mechanisms being CPU-specific.) With Adeos, you get the same API and
the same behavior regardless of underlying architecture. Adeos' pipeline
actually provides the same net effect as playing with the int controllers,
except that it's totally hardware independent. No to mention that a
nanokernel's behavior can be modified/extended while a CPU's behavior is
pretty much ... hmmm, well, fixed in silicone ...

> A couple of questions.

Sure.

> Can Adeos-registed timer callbacks call the same functions as normal
> timer callbacks, schedule userspace, and kick network I/O with near-RT
> guarantees?  Or do they run in a non-kernel context?

The Adeos-registered callbacks cannot _directly_ call kernel functions.
They can, however, trigger virtual IRQs that, once propagated to Linux,
can then take care of such calls. Here are the relevant Adeos functions
(see http://home.gna.org/adeos/doc/api/interface_8h.html for the full
API and its usage):

unsigned adeos_alloc_irq (void)
> Allocates a system-wide pipelined virtual interrupt. Virtual interrupts
> are pseudo-interrupt channels which are handled in exactly the same way
> than their hardware-generated counterparts. This is a basic, one-way
> only, inter-domain communication system allowing a domain to trigger the
> execution of a handler inside cooperating domains using the interrupt
> semantics (see adeos_trigger_irq()). A domain can trap the virtual
> interrupt using the adeos_virtualize_irq() service. Any domain can use
> adeos_virtualize_irq() on a virtual interrupt even if such interrupt was
> allocated by another domain.

int adeos_trigger_irq (unsigned irq)
> Simulates the occurrence of an interrupt. Adeos acts as if the specified
> interrupt had been received from the underlying hardware, and starts
> propagating it down the pipeline. The calling domain might be immediately
> preempted on behalf of this routine if the interrupt is delivered to a more
> prioritary domain.

Using these basic functions and the relevant callbacks it's fairly simple
to have the hard-rt component do the critical stuff and immediately signal
a Linux-aware handler to "call the same functions as normal timer callbacks,
schedule userspace, and kick network I/O with near-RT guarantees." It
should also be possible to wrap these primitives around higher-level
functions such as HRT, for example.

Side note: virtual IRQs is something CPU int controller silicone is
incapable of.

> Can Adeos itself be loaded as a module which overrides normal non-RT
> kernel interrupt and timer functions?  If it can be kept out of the
> standard kernel, but loaded when needed, that would be nice.

Sure, Adeos can be compiled as a module and loaded at runtime, but it does
require a few basic things to be modified within the kernel in order to
allow runtime loading.

> One more thing would help, IMHO, in getting any fancy interrupt system
> in: if it balanced the different execution contexts, i.e. limit total
> CPU taken in high priority, low priority interrupts, task queues
> etc. in an efficient yet fair way, such that overall throughput was
> improved over standard kernels in cases such as network overload.
> NAPI does this at the network card level, but there is no reason why
> balancing CPU among contexts cannot be done at the generic interrupt
> scheduling level, making it work for all I/O devices without special
> driver support.

Hmm... IOW, Adeos wouldn't implement a pipeline, but an interrupt
scheduler? I guess that's possible, but I can see things getting very
confusing very fast for driver developers because of the hard-rt issues,
not to mention code complexity in the interrupt scheduler for avoiding
starvation, missed deadlines, overruns, etc. Given that Adeos itself is
a nanokernel it really would make little sense for it to act as the
scheduler of any interrupts within any of its clients. It could certainly
have an OS scheduler, but that's different from you're asking for. There
is no reason, however, for not having an additional Linux-specific
mechanism for prioritizing incoming Linux interrupts. For example, if I
understand it correctly TimeSys has implemented some form of interrupt
threading in their kernel. Maybe that can be used at the Linux level in
addition to having a basic bare-bones hard-RT mechanism for delivering
interrupts. While I can't expand on this, since I haven't seen their
code, I think something similar may be better adapted in order to
preserve existing driver behavior.

Karim
-- 
Author, Speaker, Developer, Consultant
Pushing Embedded and Real-Time Linux Systems Beyond the Limits
http://www.opersys.com || karim@opersys.com || 1-866-677-4546

Re: Call for HRT in 2.6 kernel was Re: finding out the value of HZ from userspace

[Erik Andersen]

On Thu Mar 18, 2004 at 10:23:55AM -0500, Karim Yaghmour wrote:
> except that it's totally hardware independent. No to mention that a
> nanokernel's behavior can be modified/extended while a CPU's behavior is
> pretty much ... hmmm, well, fixed in silicone ...

Silicone?  You expect CPU behavior to jiggle around
a lot I suppose.  ;-)

 -Erik

--
Erik B. Andersen             http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

Re: Call for HRT in 2.6 kernel was Re: finding out the value of HZ from userspace

[Karim Yaghmour]

Erik Andersen wrote:
> Silicone?  You expect CPU behavior to jiggle around
> a lot I suppose.  ;-)

... I'm sure that it would sometimes help explain odd system behavior :)

Speaking of sexy, though, you gotta love these virtual interrupts.

Karim
-- 
Author, Speaker, Developer, Consultant
Pushing Embedded and Real-Time Linux Systems Beyond the Limits
http://www.opersys.com || karim@opersys.com || 1-866-677-4546