nanosleep resolution, jiffies vs microseconds

nanosleep resolution, jiffies vs microseconds

[Darren Hart]

I am looking at trying to improve the latency of nanosleep for short
sleep times (~1ms).  After reading Martin Schwidefsky's post for cputime
on s390 (Message-ID:
<20041111171439.GA4900@mschwid3.boeblingen.de.ibm.com>), it seems to me
that we may be able to accomplish this by storing the expire time in
microseconds rather than jiffies.  Here is an example for context:

Say we want to sleep for 1ms on i386, we call nanosleep(1000000).
Unfortunately on i386 a jiffy is slightly less than 1ms (as one might
expect with HZ = 1000).  So when sys_nanosleep calls
timespec_to_jiffies, it returns 2.  Now to allow for the corner case
when my 1ms sleep request gets called at the very tail end of a clock
period (see ascii diagram below), nanosleep adds 1 to that and calls
schedule_timeout with 3.  So a 1 ms sleep correctly turns into 3
jiffies.

If we were to store the expire value in microseconds, this corner case
would still exist and still span two full tick periods.  However, the
large majority of the time, nanosleep(1000000) could pause for only 2
jiffies, instead of 3.  Before I dug to deep into the relevant code I
wanted to hear some opinions on this approach.


Worst case scenario for a 1ms sleep:

TICK @ 1000000000 ns ------------------------   (X jiffies)


    nanosleep(1000000) // this can't correctly wake until 1001999849
TICK @ 1000999849 ns ------------------------   (X jiffies + 1)



TICK @ 1001999698 ns ------------------------   (X jiffies + 2)
    at 1001999849 nanosleep call can wake up
    (but since this is after X jiffies + 2, we can't actually wake
     until X jiffies + 3)

TICK @ 1002999547 ns ------------------------   (X jiffies + 3)
    wake from nanosleep


Thanks,

-- 
Darren Hart <darren@dvhart.com>

Re: nanosleep resolution, jiffies vs microseconds

[Mike Kravetz]

On Wed, Dec 08, 2004 at 08:47:48AM -0800, Darren Hart wrote:
> I am looking at trying to improve the latency of nanosleep for short
> sleep times (~1ms).  After reading Martin Schwidefsky's post for cputime
> on s390 (Message-ID:
> <20041111171439.GA4900@mschwid3.boeblingen.de.ibm.com>), it seems to me
> that we may be able to accomplish this by storing the expire time in
> microseconds rather than jiffies.

My only question would be 'why'?  Is there some environment where this
is an issue? -OR- Is this just 'something to do'?  Seems to me that the
only environment where this could be an issue is for 'realtime' tasks.
For non-realtime, I would guess that the variability of preemption/scheduling 
makes this almost a non-issue.  In environments where I have seen heavy
use of nanosleep, there were other scheduling issues that almost always
cause one to 'sleep' longer than the specified time.  I'm not opposed to
work in this area.  Just curious as to why?

-- 
Mike

Re: nanosleep resolution, jiffies vs microseconds

[Paul E. McKenney]

Hello, Darren,

Thank you very much for getting to the bottom of this!

This is mostly an issue when sleeping for small numbers of ticks,
so if HZ was 10000, a nanosleep(1000000) would get bumped by
a couple hundred microseconds rather than the current milliseconds,
right?

Further, if one were to do nanosleep(900000) given HZ of 1024,
the expected sleep time would be 2 milliseconds, right?

						Thanx, Paul

On Wed, Dec 08, 2004 at 08:47:48AM -0800, Darren Hart wrote:
> I am looking at trying to improve the latency of nanosleep for short
> sleep times (~1ms).  After reading Martin Schwidefsky's post for cputime
> on s390 (Message-ID:
> <20041111171439.GA4900@mschwid3.boeblingen.de.ibm.com>), it seems to me
> that we may be able to accomplish this by storing the expire time in
> microseconds rather than jiffies.  Here is an example for context:
> 
> Say we want to sleep for 1ms on i386, we call nanosleep(1000000).
> Unfortunately on i386 a jiffy is slightly less than 1ms (as one might
> expect with HZ = 1000).  So when sys_nanosleep calls
> timespec_to_jiffies, it returns 2.  Now to allow for the corner case
> when my 1ms sleep request gets called at the very tail end of a clock
> period (see ascii diagram below), nanosleep adds 1 to that and calls
> schedule_timeout with 3.  So a 1 ms sleep correctly turns into 3
> jiffies.
> 
> If we were to store the expire value in microseconds, this corner case
> would still exist and still span two full tick periods.  However, the
> large majority of the time, nanosleep(1000000) could pause for only 2
> jiffies, instead of 3.  Before I dug to deep into the relevant code I
> wanted to hear some opinions on this approach.
> 
> 
> Worst case scenario for a 1ms sleep:
> 
> TICK @ 1000000000 ns ------------------------   (X jiffies)
> 
> 
>     nanosleep(1000000) // this can't correctly wake until 1001999849
> TICK @ 1000999849 ns ------------------------   (X jiffies + 1)
> 
> 
> 
> TICK @ 1001999698 ns ------------------------   (X jiffies + 2)
>     at 1001999849 nanosleep call can wake up
>     (but since this is after X jiffies + 2, we can't actually wake
>      until X jiffies + 3)
> 
> TICK @ 1002999547 ns ------------------------   (X jiffies + 3)
>     wake from nanosleep
> 
> 
> Thanks,
> 
> -- 
> Darren Hart <darren@dvhart.com>
> 
> 
> 

Re: nanosleep resolution, jiffies vs microseconds

[Paul E. McKenney]

On Wed, Dec 08, 2004 at 09:05:04AM -0800, Mike Kravetz wrote:
> On Wed, Dec 08, 2004 at 08:47:48AM -0800, Darren Hart wrote:
> > I am looking at trying to improve the latency of nanosleep for short
> > sleep times (~1ms).  After reading Martin Schwidefsky's post for cputime
> > on s390 (Message-ID:
> > <20041111171439.GA4900@mschwid3.boeblingen.de.ibm.com>), it seems to me
> > that we may be able to accomplish this by storing the expire time in
> > microseconds rather than jiffies.
> 
> My only question would be 'why'?  Is there some environment where this
> is an issue? -OR- Is this just 'something to do'?  Seems to me that the
> only environment where this could be an issue is for 'realtime' tasks.
> For non-realtime, I would guess that the variability of preemption/scheduling 
> makes this almost a non-issue.  In environments where I have seen heavy
> use of nanosleep, there were other scheduling issues that almost always
> cause one to 'sleep' longer than the specified time.  I'm not opposed to
> work in this area.  Just curious as to why?

This is indeed for realtime work.

						Thanx, Paul

Re: nanosleep resolution, jiffies vs microseconds

[Darren Hart]

On Wed, 2004-12-08 at 09:14 -0800, Paul E. McKenney wrote:
> Hello, Darren,
> 
> Thank you very much for getting to the bottom of this!
> 
> This is mostly an issue when sleeping for small numbers of ticks,
> so if HZ was 10000, a nanosleep(1000000) would get bumped by
> a couple hundred microseconds rather than the current milliseconds,
> right?

yes.

> 
> Further, if one were to do nanosleep(900000) given HZ of 1024,
> the expected sleep time would be 2 milliseconds, right?

and yes.

-- 
Darren Hart <dvhltc@us.ibm.com>
IBM Linux Technology Center

Re: nanosleep resolution, jiffies vs microseconds

[Jan Engelhardt]

>I am looking at trying to improve the latency of nanosleep for short
>sleep times (~1ms).  After reading Martin Schwidefsky's post for cputime
>on s390 (Message-ID:
><20041111171439.GA4900@mschwid3.boeblingen.de.ibm.com>), it seems to me
>that we may be able to accomplish this by storing the expire time in
>microseconds rather than jiffies.  Here is an example for context:
>
>Say we want to sleep for 1ms on i386, we call nanosleep(1000000).
[...]

There was once a busy-wait patch to allow nanosleep have resolution up to 2 ns.
Unfortunately, it was reverted by Linus...

---------------------
PatchSet 3949
Date: 2002/09/26 05:04:43
Author: torvalds
Branch: HEAD
Tag: (none)
Log:
Remove busy-wait for short RT nanosleeps. It's a random special case
and does the wrong thing for higher HZ values anyway.

BKrev: 3d92875bgaJQe6_FSRDwHLDYHwPTgw

Members:
        ChangeSet:1.3949->1.3950
        kernel/timer.c:1.22->1.23


>Unfortunately on i386 a jiffy is slightly less than 1ms (as one might
>expect with HZ = 1000).  So when sys_nanosleep calls
>timespec_to_jiffies, it returns 2.  Now to allow for the corner case
>when my 1ms sleep request gets called at the very tail end of a clock
>period (see ascii diagram below), nanosleep adds 1 to that and calls
>schedule_timeout with 3.  So a 1 ms sleep correctly turns into 3
>jiffies.
>
>If we were to store the expire value in microseconds, this corner case
>would still exist and still span two full tick periods.  However, the
>large majority of the time, nanosleep(1000000) could pause for only 2
>jiffies, instead of 3.  Before I dug to deep into the relevant code I
>wanted to hear some opinions on this approach.
>

Jan Engelhardt
-- 
ENOSPC