Hard real time in user space with preempt_rt patch

Hard real time in user space with preempt_rt patch

[Anisha Kaul]

From: https://rt.wiki.kernel.org/articles/f/r/e/Frequently_Asked_Questions_7407.html

> Real-time only has impact on the kernel; Userspace does not notice the difference except for better real time behavior.

Does it mean that if we write the applications in user space, they
won't get the hard real time effect?
The threads running in the userspace won't get the hard real time effect?

I am not able to understand the English behind the quote. Please clarify.

Regards,
Anisha

Re: Hard real time in user space with preempt_rt patch

[raz ben yehuda]

On Tue, 2012-04-24 at 11:16 +0530, Anisha Kaul wrote:
> From: https://rt.wiki.kernel.org/articles/f/r/e/Frequently_Asked_Questions_7407.html
> 
> > Real-time only has impact on the kernel; Userspace does not notice the difference except for better real time behavior.
> 
> Does it mean that if we write the applications in user space, they
> won't get the hard real time effect?
> The threads running in the userspace won't get the hard real time effect?
user DOES get real time. The quote means that you need not to change the
application. in short, you u usleep for 2 ms, you wake after 2ms if you
have the system correctly tuned.
> I am not able to understand the English behind the quote. Please clarify.
> 
> Regards,
> Anisha
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Hard real time in user space with preempt_rt patch

[Mark Hounschell]

On 04/24/2012 01:46 AM, Anisha Kaul wrote:
> From: https://rt.wiki.kernel.org/articles/f/r/e/Frequently_Asked_Questions_7407.html
>
>> Real-time only has impact on the kernel; Userspace does not notice the difference except for better real time behavior.
>
> Does it mean that if we write the applications in user space, they
> won't get the hard real time effect?
> The threads running in the userspace won't get the hard real time effect?
>

You use the term "hard real time". The RT patch set does not even come 
close to providing a "hard real time" environment, and isn't even 
attempting to. It does however provide user land applications a much better 
chance for a "soft real time" environment. The phrase you quot above just 
means the patches are applied to the kernel and there are no patches 
required for user land glibc or your application.

Regards
Mark

Re: Hard real time in user space with preempt_rt patch

[Mark Hounschell]

On 04/24/2012 05:08 AM, Lars Segerlund wrote:
>   This is not based on facts, rt-preempt does provide hard realtime and
> strive to provide hard realtime, where have you come up with the
> notion that it does not ?
>
>   Please, don't spread misinformation, this is pure FUD .......
>
>   Check osadl.org and their test rack, it will perhaps shed some light
> on the quality assurance they try to do.
>   It is hard to argue with numbers, also check a recent kernel and a
> 'good' system.
>   Some system have latency problems, but most are fine, atleast a
> worstcase of 50 usor so under hard load and normal times in the low 10
> to 20 us range.
>
>   / regards, Lars Segerlund.
>
>
> 2012/4/24 Mark Hounschell<dmarkh@cfl.rr.com>:
>> On 04/24/2012 01:46 AM, Anisha Kaul wrote:
>>>
>>> From:
>>> https://rt.wiki.kernel.org/articles/f/r/e/Frequently_Asked_Questions_7407.html
>>>
>>>> Real-time only has impact on the kernel; Userspace does not notice the
>>>> difference except for better real time behavior.
>>>
>>>
>>> Does it mean that if we write the applications in user space, they
>>> won't get the hard real time effect?
>>> The threads running in the userspace won't get the hard real time effect?
>>>
>>
>> You use the term "hard real time". The RT patch set does not even come close
>> to providing a "hard real time" environment, and isn't even attempting to.
>> It does however provide user land applications a much better chance for a
>> "soft real time" environment. The phrase you quot above just means the
>> patches are applied to the kernel and there are no patches required for user
>> land glibc or your application.
>>

Your in lala land. The Linux kernel, even with the RT patch has so much 
"per CPU" crap in it, there is no way to prevent it from steeling usecs 
from your application. The per CPU timer interrupt alone takes a few usecs 
away from your application every HZ. A hard RT env is one in which you can 
always, every time, do a predefined work in the same amount of time. Fast 
or slow isn't the key. It's determinism. The timer interrupt alone prevents 
that. And it's not the only thing. I've got 8 cpus on my machine but the 
kernel has to have a piece of every one of them. Until there is isolation 
from the kernel, there cannot be "Hard RT". This is fact.

Mark

Re: Hard real time in user space with preempt_rt patch

[Sven-Thorsten Dietrich]

> 
> Your in lala land. The Linux kernel, even with the RT patch has so much "per CPU" crap in it, there is no way to prevent it from steeling usecs from your application. The per CPU timer interrupt alone takes a few usecs away from your application every HZ. A hard RT env is one in which you can always, every time, do a predefined work in the same amount of time. Fast or slow isn't the key. It's determinism. The timer interrupt alone prevents that. And it's not the only thing. I've got 8 cpus on my machine but the kernel has to have a piece of every one of them. Until there is isolation from the kernel, there cannot be "Hard RT". This is fact.
> 


Mark,

there is a big difference between real-time processing and bare-metal processing.

In bare metal, we do care about individual cycles, but not all real-time is bare-metal.

Determinism , or "RT requirements" is defined by the application. 

Many applications, with critical functionality, that we an define as hard real-time, 
can exist fine using the Linux kernel as infrastructure and meet deadlines in a time-critical manner.

The OS meets deterministic hard-real-time requirements, if the application is not late to execute.

Generally, hard real-time is defined as the value of the computation being at most 0 when executed late.

Soft real time implies a diminished value.

Bare metal today, since you bring up counting cycles, is a bit hard to reach with all the caching layers, pipelining, memory and bus architecture and so on, all aimed at throughput.

You will be hard pressed to find predictable behavior in anything but the simplest loops, bare-bones CPUs, and even then, pressure to differentiate in the market ends up with odd designs making CPUs do funky things.

Please do refrain from making such broad and sweeping statements, such as calling people in lala land (I have been there and its quite nice btw).

Such statements carry no value, they can always be picked apart, and I do agree, bring a heavy toll of mis-understanding and FUD amongst those of us who are not as deeply involved in the intricacies.

Thanks

Sven


> Mark
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Hard real time in user space with preempt_rt patch

[Mark Hounschell]

On 04/24/2012 12:26 PM, Sven-Thorsten Dietrich wrote:
>>
>> Your in lala land. The Linux kernel, even with the RT patch has so much "per CPU" crap in it, there is no way to prevent it from steeling usecs from your application. The per CPU timer interrupt alone takes a few usecs away from your application every HZ. A hard RT env is one in which you can always, every time, do a predefined work in the same amount of time. Fast or slow isn't the key. It's determinism. The timer interrupt alone prevents that. And it's not the only thing. I've got 8 cpus on my machine but the kernel has to have a piece of every one of them. Until there is isolation from the kernel, there cannot be "Hard RT". This is fact.
>>
>
>
> Mark,
>
> there is a big difference between real-time processing and bare-metal processing.
>
> In bare metal, we do care about individual cycles, but not all real-time is bare-metal.
>
> Determinism , or "RT requirements" is defined by the application.
>
> Many applications, with critical functionality, that we an define as hard real-time,
> can exist fine using the Linux kernel as infrastructure and meet deadlines in a time-critical manner.
>
> The OS meets deterministic hard-real-time requirements, if the application is not late to execute.
>
> Generally, hard real-time is defined as the value of the computation being at most 0 when executed late.
>
> Soft real time implies a diminished value.
>
> Bare metal today, since you bring up counting cycles, is a bit hard to reach with all the caching layers, pipelining, memory and bus architecture and so on, all aimed at throughput.
>
> You will be hard pressed to find predictable behavior in anything but the simplest loops, bare-bones CPUs, and even then, pressure to differentiate in the market ends up with odd designs making CPUs do funky things.
>
> Please do refrain from making such broad and sweeping statements, such as calling people in lala land (I have been there and its quite nice btw).
>

Yea, I've been there too. But go back to the OP. I then was simply 
informing him of what Hard RT (you may call it bare-bones if you like) 
was and still is in my opinion. I was then accused of spreading FUD by 
someone named Lars. The response you quoted above was my response to 
being accused of spreading FUD when I gave my opinion of what Hard-RT 
and the RT patch set is.

I've been in the Hard and Soft RT business since the 70s and understand 
what it is, why it is, how programmers in the past have done things 
considered wrong now, simply because they could then, and that there 
really isn't any hardware designed for it these days. But even if there 
was, what you quoted above would still be true.

I will have to admit though that processing speed has certainly helped 
in recent years. We are able to replace _some_ of these old "designed 
for HRT" machines with modern boxes simply because they are so fast. 
Maybe when they are fast enough to handle timer interrupts in less than 
a usec or so, I'll be able to replace more of them.

But, as my original response to the original OP was, the RT kernel patch 
is NOT making the kernel capable of HARD RT but certainly gives good 
Soft RT results. If ya lower the bar, you'll never get to the top of the 
pole.

Regards
Mark

Re: Hard real time in user space with preempt_rt patch

[Lars Segerlund]

 Hi , I'm the nasty person that pointed out that the rt-preempt
patches DOES provide and strive to provide deterministic HARD realtime
for the linux kernel.

 Hard realtime is as you pointed out bounded by being deterministic,
so all talk about microseconds and relative latencies are not really
relevant per your own definition.

 On a good system worst case latencies of around 10-20 usec is on par
with RTAI or RT-Linux, they have lower averages, but more jitter so
about the same worst cases, and since worst case is what limits the
system they are somewhat comparable, hardware makes a bigger
difference.

 Now if were talking hardware, no regular PC with virtual memory,
superscalar execution and multitasking will ever provide much less
than some usec's ... due to hardware latencies.

 You said that preempt-rt doesn't provide hard realtime and never
will, to and I pointed out in an email that that was your opinion and
that I considered that FUD, and sent a link to the osadl.org site wich
have a test rack to track worst case latencies on systems run under
long time and high load, if that is not deterministic I don't know
what is, as far as discussing a common PC.

 I also pointed out that is was quite rude to make such a statement on
the linux-rt list, since most people here are working on getting
rt-preempt as good as possible with the goal of exactly hard realtime.

 So difference of opinion is totally acceptable, but stating that
opinions are the truth is something else, now statements like
lala-land is just degoratory and are clearly meant to be insulting.

 I do consider you misinformed and wrong in opinion based on your
statement that the definition of hard realtime is if it has a
deterministic worst case behaviour or not, but I do hope that I
haven't been rude.

 / regards, Lars Segerlund.


2012/4/24 Mark Hounschell <markh@compro.net>:
> On 04/24/2012 12:26 PM, Sven-Thorsten Dietrich wrote:
>>>
>>>
>>> Your in lala land. The Linux kernel, even with the RT patch has so much
>>> "per CPU" crap in it, there is no way to prevent it from steeling usecs from
>>> your application. The per CPU timer interrupt alone takes a few usecs away
>>> from your application every HZ. A hard RT env is one in which you can
>>> always, every time, do a predefined work in the same amount of time. Fast or
>>> slow isn't the key. It's determinism. The timer interrupt alone prevents
>>> that. And it's not the only thing. I've got 8 cpus on my machine but the
>>> kernel has to have a piece of every one of them. Until there is isolation
>>> from the kernel, there cannot be "Hard RT". This is fact.
>>>
>>
>>
>> Mark,
>>
>> there is a big difference between real-time processing and bare-metal
>> processing.
>>
>> In bare metal, we do care about individual cycles, but not all real-time
>> is bare-metal.
>>
>> Determinism , or "RT requirements" is defined by the application.
>>
>> Many applications, with critical functionality, that we an define as hard
>> real-time,
>> can exist fine using the Linux kernel as infrastructure and meet deadlines
>> in a time-critical manner.
>>
>> The OS meets deterministic hard-real-time requirements, if the application
>> is not late to execute.
>>
>> Generally, hard real-time is defined as the value of the computation being
>> at most 0 when executed late.
>>
>> Soft real time implies a diminished value.
>>
>> Bare metal today, since you bring up counting cycles, is a bit hard to
>> reach with all the caching layers, pipelining, memory and bus architecture
>> and so on, all aimed at throughput.
>>
>> You will be hard pressed to find predictable behavior in anything but the
>> simplest loops, bare-bones CPUs, and even then, pressure to differentiate in
>> the market ends up with odd designs making CPUs do funky things.
>>
>> Please do refrain from making such broad and sweeping statements, such as
>> calling people in lala land (I have been there and its quite nice btw).
>>
>
> Yea, I've been there too. But go back to the OP. I then was simply informing
> him of what Hard RT (you may call it bare-bones if you like) was and still
> is in my opinion. I was then accused of spreading FUD by someone named Lars.
> The response you quoted above was my response to being accused of spreading
> FUD when I gave my opinion of what Hard-RT and the RT patch set is.
>
> I've been in the Hard and Soft RT business since the 70s and understand what
> it is, why it is, how programmers in the past have done things considered
> wrong now, simply because they could then, and that there really isn't any
> hardware designed for it these days. But even if there was, what you quoted
> above would still be true.
>
> I will have to admit though that processing speed has certainly helped in
> recent years. We are able to replace _some_ of these old "designed for HRT"
> machines with modern boxes simply because they are so fast. Maybe when they
> are fast enough to handle timer interrupts in less than a usec or so, I'll
> be able to replace more of them.
>
> But, as my original response to the original OP was, the RT kernel patch is
> NOT making the kernel capable of HARD RT but certainly gives good Soft RT
> results. If ya lower the bar, you'll never get to the top of the pole.
>
> Regards
>
> Mark
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Hard real time in user space with preempt_rt patch

[Esben Nielsen]

On Tue, Apr 24, 2012 at 9:20 PM, Lars Segerlund
<lars.segerlund@gmail.com> wrote:
>  Hi , I'm the nasty person that pointed out that the rt-preempt
> patches DOES provide and strive to provide deterministic HARD realtime
> for the linux kernel.
>
>  Hard realtime is as you pointed out bounded by being deterministic,
> so all talk about microseconds and relative latencies are not really
> relevant per your own definition.
>
> <a lot of other stuff cut>

I have seen these discussions before:
One camp equal hard real time with highly safety critical,
deterministic systems. The Linux kernel can of course never be that.
But Linux with preempt realtime is "hard realtime" in the sense
meeting deadlines are deterministic and testable. I.e. you can use it
as a platform for a system where not meeting a deadline is "bug" of
equal severity as say a segmentation fault: You can be as sure to meet
your deadlines if you have tested it in the lab as you can be sure not
to have any other kind of bug. And no, it is not mathematically
proven, and the kernel and your applications might have bugs still to
be discovered.

>
>  / regards, Lars Segerlund.
>

Esben
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Hard real time in user space with preempt_rt patch

[Armin Steinhoff]

Mark Hounschell wrote:
> On 04/24/2012 05:08 AM, Lars Segerlund wrote:
>>   This is not based on facts, rt-preempt does provide hard realtime and
>> strive to provide hard realtime, where have you come up with the
>> notion that it does not ?
>>
>>   Please, don't spread misinformation, this is pure FUD .......
>>
>>   Check osadl.org and their test rack, it will perhaps shed some light
>> on the quality assurance they try to do.
>>   It is hard to argue with numbers, also check a recent kernel and a
>> 'good' system.
>>   Some system have latency problems, but most are fine, atleast a
>> worstcase of 50 usor so under hard load and normal times in the low 10
>> to 20 us range.
>>
>>   / regards, Lars Segerlund.
>>
>>
>> 2012/4/24 Mark Hounschell<dmarkh@cfl.rr.com>:
>>> On 04/24/2012 01:46 AM, Anisha Kaul wrote:
>>>>
>>>> From:
>>>> https://rt.wiki.kernel.org/articles/f/r/e/Frequently_Asked_Questions_7407.html 
>>>>
>>>>
>>>>> Real-time only has impact on the kernel; Userspace does not notice 
>>>>> the
>>>>> difference except for better real time behavior.
>>>>
>>>>
>>>> Does it mean that if we write the applications in user space, they
>>>> won't get the hard real time effect?
>>>> The threads running in the userspace won't get the hard real time 
>>>> effect?
>>>>
>>>
>>> You use the term "hard real time". The RT patch set does not even 
>>> come close
>>> to providing a "hard real time" environment, and isn't even 
>>> attempting to.
>>> It does however provide user land applications a much better chance 
>>> for a
>>> "soft real time" environment. The phrase you quot above just means the
>>> patches are applied to the kernel and there are no patches required 
>>> for user
>>> land glibc or your application.
>>>
>
> Your in lala land. The Linux kernel, even with the RT patch has so 
> much "per CPU" crap in it, there is no way to prevent it from steeling 
> usecs from your application.


Not only the kernel steals CPU time :)  Also the SMI interrupt can steal 
a lot of time from any CPU core!  And these effects are much higher then 
the actions at the kernel level.
The OS AND the hardware must provide the environment for hard real-time 
processing. The definition of the a deadline includes also a time 
resolution (or variation) for meeting the deadlines ... and this is 
defined by the real-time application and not by theOS. The OS has just 
to profide the technological base to support this requirement.  In some 
cases is a time resolution of 1ms OK for meeting a deadline ...  and 
this still a hard-realtime application.


> The per CPU timer interrupt alone takes a few usecs away from your 
> application every HZ. A hard RT env is one in which you can always, 
> every time, do a predefined work in the same amount of time. Fast or 
> slow isn't the key. It's determinism. The timer interrupt alone 
> prevents that. And it's not the only thing. I've got 8 cpus on my 
> machine but the kernel has to have a piece of every one of them. Until 
> there is isolation from the kernel, there cannot be "Hard RT". This is 
> fact.

Your understanding of hard real-time is far too narrow ...

--armin

>
> Mark
> -- 
> To unsubscribe from this list: send the line "unsubscribe 
> linux-rt-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Hard real time in user space with preempt_rt patch

[John Kacur]

On Tue, 24 Apr 2012, Mark Hounschell wrote:

> On 04/24/2012 01:46 AM, Anisha Kaul wrote:
> > From:
> > https://rt.wiki.kernel.org/articles/f/r/e/Frequently_Asked_Questions_7407.html
> > 
> > > Real-time only has impact on the kernel; Userspace does not notice the
> > > difference except for better real time behavior.
> > 
> > Does it mean that if we write the applications in user space, they
> > won't get the hard real time effect?
> > The threads running in the userspace won't get the hard real time effect?
> > 
> 
> You use the term "hard real time". The RT patch set does not even come close
> to providing a "hard real time" environment, and isn't even attempting to.

Wrong. It is attempting to come as close to hard real-time responses as 
possible. There is however, no mathematical guarantee that it will do so, 
that some would require of "hard real time". What exactly is meant by hard 
real time is also a bit hard to pin down. For a good easy to 
read discussion of it, 
check out http://www.linuxjournal.com/article/9361

> does however provide user land applications a much better chance for a "soft
> real time" environment. The phrase you quot above just means the patches are
> applied to the kernel and there are no patches required for user land glibc or
> your application.

That sounds right to me.