[Xenomai-core] Realtime gettimeofday()

[Xenomai-core] Realtime gettimeofday()

[Wolfgang Mauerer]

Hi,

we'd like to implement a gettimeofday() mechanism that works
in realtime context, both from user- and kernelland. Most importantly,
the correctins made to the wall time by the NTP protcol on Linux
must be transferred into the Xenomai domain. We have a preliminary
implementation (with several weaknesses) ready, but before we go
into the details of writing a complete solution, it would be
great to have the community's opinion on how to proceed best -- there
are a number of alternatives, each with specific strenghts and
weaknesses. Any comments on the following proposals are very welcome!

Essentially, getting the NTP-corrected time requires three ingredients:

- A clock source.
- A wall time t_{0} as basis, together with the value of the
  clocksource at that moment.
- An NTP-corrected conversion factor between clock ticks and walltime.

Since the NTP correction information holds wrt. a specific clock source,
we have to use the clock source used by Linux.

The Linux kernel provides a vsyscall based solution to the problem:
The base data are periodically copied to a userland-accessible page,
and a vsyscall reads the information using seqlock protection (i.e.,
rereading the information when it was changed during the read
operation). This way, the data can be locklessly relayed into userland,
and perturbation for the kernel is minimal as no lock to write the data
is required.

However, things get a little more involved when realtime comes into
play: Using a simple seqlock that might require rereading the data an
arbitrary number of times contrary to the determinism requirements of
realtime. So far, we can think of three alternatives:

1.) Re-use the vsyscall data from the kernel with a buffered seqlock
    replacement. We cannot directly use the data provided by the Linux
    kernel because Xenomai might have interrupted the update process,
    which leads to an inconsistent state. Instead, we would use a
    "buffered" seqlock that uses two more copies of the data that are
    accompanied by a flag which indicates if some other Xenomai thread
    is currently reading or writing to the data: They are both filled
    with an initial valid copy of the data during Xenomai
    initialisation. When the data are requested in primary mode, Xenomai
    first checks if the Linux seqlock is taken:

    - If the lock is taken, the data provided by Linux may be in an
      inconsistent state, and the backup copies are used.
    - If the lock is not taken, check with the aforementioned
      flag if another Xenomai task is currently updating one of
      the copies.
      - (A) If yes, then read the data from the other copy.
      - (B) If no, perform at most one read try of the Linux data --
        another CPU running the Linux kernel might update them in the
        meantime.
	If they have been read without perturbation,
	mark one copy as being currently updated (naturally,
        we need to flip the flag atomically), update the buffer, and
	use the data. If the data were modified during the read phase,
	discard the result and continue as in (A)

    Note that the flags may also be extended with version
    information to check which buffer has been updated more recently.

    While Xenomai might block the update process for a while,
    we always have at least a consistent copy of the data, and there is
    a fixed upper bound on the time required to compute the current
    NTP-corrected wall time.

2.) Use a transactional mechanism as outlined above to relay the data
    into userland, make sure that the triple (sec,nsec,ticks) is copied
    atomically to avoid an invalid state, and live with the fact that if
    the correction factor is not uptodate, we are only in danger of
    using outdated information, but won't get a completely bogous result
    (a userland application might start to read a buffer that is
    concurrently being updated by another CPU)

3.) Use explicit locking between the Linux kernel and Xenomai to update
    respectively access the data, and partially eliminate the
    performance advantages of the vsyscall mechanism. This would have
    the usual drawbacks associated with locks in primary mode. Besides,
    since a kernel lock is involved, it would not allow for a pure
    userland solution without kernel intervention.

Are there preferences and/or arguments for/against any of these
solutions? Or any better ideas? Thanks in advance for your comments!

Best, Wolfgang

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Wolfgang Mauerer wrote:
> Hi,
> 
> we'd like to implement a gettimeofday() mechanism that works
> in realtime context, both from user- and kernelland. Most importantly,
> the correctins made to the wall time by the NTP protcol on Linux
> must be transferred into the Xenomai domain.

Yes, the real issue is NTP, because other than that, you can simply
implement gettimeofday in terms of clock_gettime(CLOCK_REALTIME).

This issue has been discussed several times, but never a lot. We have a
simple solution: starting with 2.4 (if I remember correctly) xenomai
provides clock_settime, so you can simply rely on clock_gettime, and
call clock_settime from time to time to resync the Xenomai clock with
Linux NTP-enhanced clock. This is not pretty, you get a drift increasing
over time, then suddenly been reset. But I guess this has been enough
for current users, until now. You control the maximum drift by how often
you call clock_settime.

Would not it be enough for your use of xenomai?

-- 
                                          Gilles

Re: [Xenomai-core] Realtime gettimeofday()

[Jan Kiszka]

Gilles Chanteperdrix wrote:
> Wolfgang Mauerer wrote:
>> Hi,
>>
>> we'd like to implement a gettimeofday() mechanism that works
>> in realtime context, both from user- and kernelland. Most importantly,
>> the correctins made to the wall time by the NTP protcol on Linux
>> must be transferred into the Xenomai domain.
> 
> Yes, the real issue is NTP, because other than that, you can simply
> implement gettimeofday in terms of clock_gettime(CLOCK_REALTIME).

Not truly as Linux processes that tune the host clock do not affect the
Xenomai world.

> 
> This issue has been discussed several times, but never a lot. We have a
> simple solution: starting with 2.4 (if I remember correctly) xenomai
> provides clock_settime, so you can simply rely on clock_gettime, and
> call clock_settime from time to time to resync the Xenomai clock with
> Linux NTP-enhanced clock. This is not pretty, you get a drift increasing
> over time, then suddenly been reset. But I guess this has been enough
> for current users, until now. You control the maximum drift by how often
> you call clock_settime.
> 
> Would not it be enough for your use of xenomai?

Nope for the reasons you mentioned.

Moreover, reading the Xenomai real-time clock requires a syscall,
gettimeofday can work without that penalty (given non-broken TSC).

Next issue is that the different clock bases of
clock_gettime(CLOCK_REALTIME) and (real) gettimeofday + the switch-back
or even lock-up side-effects of the latter have been quite a PITA in our
project.

And finally, the vision is to have NTP-accurate (or whatever the sync
source is, eg. IEEE1588) absolute timing also for the nucleus one day.
Having an RT-safe way to obtain the NTP parameters from any context is
the first step.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Gilles Chanteperdrix wrote:
>> Wolfgang Mauerer wrote:
>>> Hi,
>>>
>>> we'd like to implement a gettimeofday() mechanism that works
>>> in realtime context, both from user- and kernelland. Most importantly,
>>> the correctins made to the wall time by the NTP protcol on Linux
>>> must be transferred into the Xenomai domain.
>> Yes, the real issue is NTP, because other than that, you can simply
>> implement gettimeofday in terms of clock_gettime(CLOCK_REALTIME).
> 
> Not truly as Linux processes that tune the host clock do not affect the
> Xenomai world.
> 
>> This issue has been discussed several times, but never a lot. We have a
>> simple solution: starting with 2.4 (if I remember correctly) xenomai
>> provides clock_settime, so you can simply rely on clock_gettime, and
>> call clock_settime from time to time to resync the Xenomai clock with
>> Linux NTP-enhanced clock. This is not pretty, you get a drift increasing
>> over time, then suddenly been reset. But I guess this has been enough
>> for current users, until now. You control the maximum drift by how often
>> you call clock_settime.
>>
>> Would not it be enough for your use of xenomai?
> 
> Nope for the reasons you mentioned.
> 
> Moreover, reading the Xenomai real-time clock requires a syscall,
> gettimeofday can work without that penalty (given non-broken TSC).
> 
> Next issue is that the different clock bases of
> clock_gettime(CLOCK_REALTIME) and (real) gettimeofday + the switch-back
> or even lock-up side-effects of the latter have been quite a PITA in our
> project.
> 
> And finally, the vision is to have NTP-accurate (or whatever the sync
> source is, eg. IEEE1588) absolute timing also for the nucleus one day.
> Having an RT-safe way to obtain the NTP parameters from any context is
> the first step.

clock_gettime(CLOCK_MONOTONIC) works without syscall too. And note that
the real-time clock could work without a syscall for the current scheme,
but you were the one to explicitly refuse this.

My opinion with regard to these issues is that the I-pipe patch should
send us an event when the parameters are adjusted (the wall clock time
changed by Linux, or the slow down/acceleration caused by the NTP
daemon). And that we should duplicate the processing done by the Linux
kernel for the Xenomai clock. Doing this in user-space without a syscall
is another issue that will be possible to handle when we have solved the
kernel-space clock issue. Of course, it supposes that Xenomai should use
the same clocksource as Linux, so, that we should implement the HPET
clock source for x86 (on other architectures, we already use the same
clock source).

Another possibility is to implement syscalls to allow adjusting the
nucleus real-time clock parameters the same way ntp does, and to require
a special ntp daemon that would use these syscalls instead of the linux
kernel syscalls. After all, if we adjust Xenomai clock, Linux clock will
follow without any adjustment. On an embedded system, requiring a
recompiled NTP is not much of an issue.

So, to summarize, here is what I think we should do:
- implement the HPET clock source
- implement the I-pipe events signalling wall clock adjustment and NTP
related adjustments
- or implement syscalls to allow NTP related adjustments (we already
have the syscall allowing to change wall clock)
- rework the nucleus real-time clock to use these adjustements
- implement the user-space syscall-less version of the real-time clock read.

Trying to go directly to the last point looks premature to me.

-- 
                                          Gilles

Re: [Xenomai-core] Realtime gettimeofday()

[Jan Kiszka]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Gilles Chanteperdrix wrote:
>>> Wolfgang Mauerer wrote:
>>>> Hi,
>>>>
>>>> we'd like to implement a gettimeofday() mechanism that works
>>>> in realtime context, both from user- and kernelland. Most importantly,
>>>> the correctins made to the wall time by the NTP protcol on Linux
>>>> must be transferred into the Xenomai domain.
>>> Yes, the real issue is NTP, because other than that, you can simply
>>> implement gettimeofday in terms of clock_gettime(CLOCK_REALTIME).
>> Not truly as Linux processes that tune the host clock do not affect the
>> Xenomai world.
>>
>>> This issue has been discussed several times, but never a lot. We have a
>>> simple solution: starting with 2.4 (if I remember correctly) xenomai
>>> provides clock_settime, so you can simply rely on clock_gettime, and
>>> call clock_settime from time to time to resync the Xenomai clock with
>>> Linux NTP-enhanced clock. This is not pretty, you get a drift increasing
>>> over time, then suddenly been reset. But I guess this has been enough
>>> for current users, until now. You control the maximum drift by how often
>>> you call clock_settime.
>>>
>>> Would not it be enough for your use of xenomai?
>> Nope for the reasons you mentioned.
>>
>> Moreover, reading the Xenomai real-time clock requires a syscall,
>> gettimeofday can work without that penalty (given non-broken TSC).
>>
>> Next issue is that the different clock bases of
>> clock_gettime(CLOCK_REALTIME) and (real) gettimeofday + the switch-back
>> or even lock-up side-effects of the latter have been quite a PITA in our
>> project.
>>
>> And finally, the vision is to have NTP-accurate (or whatever the sync
>> source is, eg. IEEE1588) absolute timing also for the nucleus one day.
>> Having an RT-safe way to obtain the NTP parameters from any context is
>> the first step.
> 
> clock_gettime(CLOCK_MONOTONIC) works without syscall too. And note that
> the real-time clock could work without a syscall for the current scheme,
> but you were the one to explicitly refuse this.

Yes, because we need a scheme like Wolfgang suggests to ensure
consistent states during updates. And we didn't had such thing back then.

> 
> My opinion with regard to these issues is that the I-pipe patch should
> send us an event when the parameters are adjusted (the wall clock time
> changed by Linux, or the slow down/acceleration caused by the NTP
> daemon). And that we should duplicate the processing done by the Linux
> kernel for the Xenomai clock.

That may be a solution for only half of the problem.

> Doing this in user-space without a syscall
> is another issue that will be possible to handle when we have solved the
> kernel-space clock issue. Of course, it supposes that Xenomai should use
> the same clocksource as Linux, so, that we should implement the HPET
> clock source for x86 (on other architectures, we already use the same
> clock source).

Solving the user space transfer automatically includes a solution for
the Linux-to-Xenomai transfer. I really suggest that we discuss a
complete solution to avoid leaving design issues behind that require
another ABI change later on.

> 
> Another possibility is to implement syscalls to allow adjusting the
> nucleus real-time clock parameters the same way ntp does, and to require
> a special ntp daemon that would use these syscalls instead of the linux
> kernel syscalls. After all, if we adjust Xenomai clock, Linux clock will
> follow without any adjustment. On an embedded system, requiring a
> recompiled NTP is not much of an issue.
> 
> So, to summarize, here is what I think we should do:
> - implement the HPET clock source

What for? The days of HPET are counted (both as clock and event source),
modern systems provide proper TSCs.

> - implement the I-pipe events signalling wall clock adjustment and NTP
> related adjustments
> - or implement syscalls to allow NTP related adjustments (we already
> have the syscall allowing to change wall clock)
> - rework the nucleus real-time clock to use these adjustements
> - implement the user-space syscall-less version of the real-time clock read.
> 
> Trying to go directly to the last point looks premature to me.

Why? It delivers us the core mechanism we need for the rest as well -
and it does not require fancy I-pipe hooks.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Why? It delivers us the core mechanism we need for the rest as well -
> and it does not require fancy I-pipe hooks.

Because relying on the vdso/vsyscall only works on x86. Whereas
implementing clock slew down/acceleration at nucleus level and simply
sharing data between kernel and user through the the global sem heap,
works for all architectures.

-- 
                                          Gilles

Re: [Xenomai-core] Realtime gettimeofday()

[Jan Kiszka]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Why? It delivers us the core mechanism we need for the rest as well -
>> and it does not require fancy I-pipe hooks.
> 
> Because relying on the vdso/vsyscall only works on x86. Whereas
> implementing clock slew down/acceleration at nucleus level and simply
> sharing data between kernel and user through the the global sem heap,
> works for all architectures.

There are three kind of archs:
 - those that already support vgettimeofday & friends (x86, powerpc,
   maybe more)
 - those that do not yet though they could (I strongly suspect arm falls
   into this category as well)
 - those that never will (due to lacking user-readable time sources)

We need temporary/permanent i-pipe workarounds for the last two, but I
see no point in complicating the first category. This design aims at a
longer term.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Gilles Chanteperdrix wrote:
>> Jan Kiszka wrote:
>>> Why? It delivers us the core mechanism we need for the rest as well -
>>> and it does not require fancy I-pipe hooks.
>> Because relying on the vdso/vsyscall only works on x86. Whereas
>> implementing clock slew down/acceleration at nucleus level and simply
>> sharing data between kernel and user through the the global sem heap,
>> works for all architectures.
> 
> There are three kind of archs:
>  - those that already support vgettimeofday & friends (x86, powerpc,
>    maybe more)
>  - those that do not yet though they could (I strongly suspect arm falls
>    into this category as well)
>  - those that never will (due to lacking user-readable time sources)
> 
> We need temporary/permanent i-pipe workarounds for the last two, but I
> see no point in complicating the first category. This design aims at a
> longer term.

Well, I may be wrong, but I prefer generic code to arch-specific code.
Nucleus code to handle clock slow down/acceleration would be generic;
I-pipe code to signal NTP syscalls would be generic (and yes, even if
I-pipe patches are generated for all architectures, whether the code is
generic or specific makes a big difference);
User-space code to implement clock_gettime(CLOCK_REALTIME) using data
shared through the global sem heap would be generic.

So, I think this design is future proof and easy to maintain. And I do
not see how it complicates x86 situation, since it is only made of
generic code.

Looks like we found a troll to occupy this afternoon :-).

-- 
                                          Gilles

Re: [Xenomai-core] Realtime gettimeofday()

[Jan Kiszka]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Gilles Chanteperdrix wrote:
>>> Jan Kiszka wrote:
>>>> Why? It delivers us the core mechanism we need for the rest as well -
>>>> and it does not require fancy I-pipe hooks.
>>> Because relying on the vdso/vsyscall only works on x86. Whereas
>>> implementing clock slew down/acceleration at nucleus level and simply
>>> sharing data between kernel and user through the the global sem heap,
>>> works for all architectures.
>> There are three kind of archs:
>>  - those that already support vgettimeofday & friends (x86, powerpc,
>>    maybe more)
>>  - those that do not yet though they could (I strongly suspect arm falls
>>    into this category as well)
>>  - those that never will (due to lacking user-readable time sources)
>>
>> We need temporary/permanent i-pipe workarounds for the last two, but I
>> see no point in complicating the first category. This design aims at a
>> longer term.
> 
> Well, I may be wrong, but I prefer generic code to arch-specific code.
> Nucleus code to handle clock slow down/acceleration would be generic;
> I-pipe code to signal NTP syscalls would be generic (and yes, even if
> I-pipe patches are generated for all architectures, whether the code is
> generic or specific makes a big difference);
> User-space code to implement clock_gettime(CLOCK_REALTIME) using data
> shared through the global sem heap would be generic.
> 
> So, I think this design is future proof and easy to maintain. And I do
> not see how it complicates x86 situation, since it is only made of
> generic code.

Well, OK, then place a small optional I-pipe hook into that part that
normally writes the update into the vdso page (I think that is
arch-specific anyway), replicating it into a specified page the nucleus
may set up on a globally shared heap. That hook also has to maintain a
seqlock like Linux does, ie. generating the same layout and semantics.
It's just the transport mechanism, we can easily select it based on the
arch's level of support.

But I'm against any needless redirection through the nucleus (including
potential nklocks etc.).

> 
> Looks like we found a troll to occupy this afternoon :-).
> 

Yeah, would be a nice one if there weren't that qemu-kvm migration
issues I have to trace down in parallel... :)

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Gilles Chanteperdrix wrote:
>> Jan Kiszka wrote:
>>> Gilles Chanteperdrix wrote:
>>>> Jan Kiszka wrote:
>>>>> Why? It delivers us the core mechanism we need for the rest as well -
>>>>> and it does not require fancy I-pipe hooks.
>>>> Because relying on the vdso/vsyscall only works on x86. Whereas
>>>> implementing clock slew down/acceleration at nucleus level and simply
>>>> sharing data between kernel and user through the the global sem heap,
>>>> works for all architectures.
>>> There are three kind of archs:
>>>  - those that already support vgettimeofday & friends (x86, powerpc,
>>>    maybe more)
>>>  - those that do not yet though they could (I strongly suspect arm falls
>>>    into this category as well)
>>>  - those that never will (due to lacking user-readable time sources)
>>>
>>> We need temporary/permanent i-pipe workarounds for the last two, but I
>>> see no point in complicating the first category. This design aims at a
>>> longer term.
>> Well, I may be wrong, but I prefer generic code to arch-specific code.
>> Nucleus code to handle clock slow down/acceleration would be generic;
>> I-pipe code to signal NTP syscalls would be generic (and yes, even if
>> I-pipe patches are generated for all architectures, whether the code is
>> generic or specific makes a big difference);
>> User-space code to implement clock_gettime(CLOCK_REALTIME) using data
>> shared through the global sem heap would be generic.
>>
>> So, I think this design is future proof and easy to maintain. And I do
>> not see how it complicates x86 situation, since it is only made of
>> generic code.
> 
> Well, OK, then place a small optional I-pipe hook into that part that
> normally writes the update into the vdso page (I think that is
> arch-specific anyway), replicating it into a specified page the nucleus
> may set up on a globally shared heap. That hook also has to maintain a
> seqlock like Linux does, ie. generating the same layout and semantics.
> It's just the transport mechanism, we can easily select it based on the
> arch's level of support.
> 
> But I'm against any needless redirection through the nucleus (including
> potential nklocks etc.).

I do not really understand why you want to use the vdso page, since we
have the global sem heap anyway. clock_gettime already has a mean to
read a clock source and the frequency of this clock source, this is
guaranteed on all platforms, so I think the correction code can be made
generic.

However, duplicating the ntp related kernel code may be the real issue.
I have to look at that code to see how complex it is.

For the locking, well, if we have variables on a shared area to update
every time ntp corrects the clock, we will have problems doing it under
nklock anyway (the irq locking prevents from preemption on the local
cpu, but does not prevent the remote cpus running user-space programs
from accessing the shared area). So, we will have to devise some locking
mechanisms. But I do not see the reason for making this linux
compatible. If we keep the nucleus business separated, the nucleus
shared area will never have to be accessed by plain Linux, which will
access its area in the vdso, the Linux kernel doing its house keeping.

For the HPET clocksource, well, I do not share your enthusiasm, Linux
keeps complaining about the stability of the tsc of my laptop, though it
is a fairly recent core2 with the "constant_tsc" flag.


-- 
                                          Gilles

Re: [Xenomai-core] Realtime gettimeofday()

[Jan Kiszka]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Gilles Chanteperdrix wrote:
>>> Jan Kiszka wrote:
>>>> Gilles Chanteperdrix wrote:
>>>>> Jan Kiszka wrote:
>>>>>> Why? It delivers us the core mechanism we need for the rest as well -
>>>>>> and it does not require fancy I-pipe hooks.
>>>>> Because relying on the vdso/vsyscall only works on x86. Whereas
>>>>> implementing clock slew down/acceleration at nucleus level and simply
>>>>> sharing data between kernel and user through the the global sem heap,
>>>>> works for all architectures.
>>>> There are three kind of archs:
>>>>  - those that already support vgettimeofday & friends (x86, powerpc,
>>>>    maybe more)
>>>>  - those that do not yet though they could (I strongly suspect arm falls
>>>>    into this category as well)
>>>>  - those that never will (due to lacking user-readable time sources)
>>>>
>>>> We need temporary/permanent i-pipe workarounds for the last two, but I
>>>> see no point in complicating the first category. This design aims at a
>>>> longer term.
>>> Well, I may be wrong, but I prefer generic code to arch-specific code.
>>> Nucleus code to handle clock slow down/acceleration would be generic;
>>> I-pipe code to signal NTP syscalls would be generic (and yes, even if
>>> I-pipe patches are generated for all architectures, whether the code is
>>> generic or specific makes a big difference);
>>> User-space code to implement clock_gettime(CLOCK_REALTIME) using data
>>> shared through the global sem heap would be generic.
>>>
>>> So, I think this design is future proof and easy to maintain. And I do
>>> not see how it complicates x86 situation, since it is only made of
>>> generic code.
>> Well, OK, then place a small optional I-pipe hook into that part that
>> normally writes the update into the vdso page (I think that is
>> arch-specific anyway), replicating it into a specified page the nucleus
>> may set up on a globally shared heap. That hook also has to maintain a
>> seqlock like Linux does, ie. generating the same layout and semantics.
>> It's just the transport mechanism, we can easily select it based on the
>> arch's level of support.
>>
>> But I'm against any needless redirection through the nucleus (including
>> potential nklocks etc.).
> 
> I do not really understand why you want to use the vdso page, since we
> have the global sem heap anyway. clock_gettime already has a mean to
> read a clock source and the frequency of this clock source, this is
> guaranteed on all platforms, so I think the correction code can be made
> generic.

Again, the page is not the point, what it contains is important. Our
currently published data is not sufficient to support dynamic updates,
but the vdso _contains_ all the data we could reuse with an enhanced
algorithm to provide a dynamically adjusted time base.

> 
> However, duplicating the ntp related kernel code may be the real issue.
> I have to look at that code to see how complex it is.

I surely don't want to duplicate ntp code, just the results it spits out
(e.g. into the vdso page).

> 
> For the locking, well, if we have variables on a shared area to update
> every time ntp corrects the clock, we will have problems doing it under
> nklock anyway (the irq locking prevents from preemption on the local
> cpu, but does not prevent the remote cpus running user-space programs
> from accessing the shared area). So, we will have to devise some locking
> mechanisms. But I do not see the reason for making this linux
> compatible. If we keep the nucleus business separated, the nucleus
> shared area will never have to be accessed by plain Linux, which will
> access its area in the vdso, the Linux kernel doing its house keeping.

The kernel need to synchronize with user land, so the problem is
(almost) the same as with obtaining the original data from Linux
directly: user land can only use a retry or a smart fall-back mechanism
to obtain consistent data.

> 
> For the HPET clocksource, well, I do not share your enthusiasm, Linux
> keeps complaining about the stability of the tsc of my laptop, though it
> is a fairly recent core2 with the "constant_tsc" flag.

My point is that we offer Xenomai without a real alternative to tsc for
many moons, and no one really stood up so far and complained about
missing hpet support. Just like you cannot use every SMI-infected box
for RT, you can't do so when the tsc is unfixably broken. I'm not sure
how hard hpet addition and maintenance would actually be, if it's
trivial, I'm fine. But I think we have more important issues to solve.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Gilles Chanteperdrix wrote:
>> Jan Kiszka wrote:
>>> Gilles Chanteperdrix wrote:
>>>> Jan Kiszka wrote:
>>>>> Gilles Chanteperdrix wrote:
>>>>>> Jan Kiszka wrote:
>>>>>>> Why? It delivers us the core mechanism we need for the rest as well -
>>>>>>> and it does not require fancy I-pipe hooks.
>>>>>> Because relying on the vdso/vsyscall only works on x86. Whereas
>>>>>> implementing clock slew down/acceleration at nucleus level and simply
>>>>>> sharing data between kernel and user through the the global sem heap,
>>>>>> works for all architectures.
>>>>> There are three kind of archs:
>>>>>  - those that already support vgettimeofday & friends (x86, powerpc,
>>>>>    maybe more)
>>>>>  - those that do not yet though they could (I strongly suspect arm falls
>>>>>    into this category as well)
>>>>>  - those that never will (due to lacking user-readable time sources)
>>>>>
>>>>> We need temporary/permanent i-pipe workarounds for the last two, but I
>>>>> see no point in complicating the first category. This design aims at a
>>>>> longer term.
>>>> Well, I may be wrong, but I prefer generic code to arch-specific code.
>>>> Nucleus code to handle clock slow down/acceleration would be generic;
>>>> I-pipe code to signal NTP syscalls would be generic (and yes, even if
>>>> I-pipe patches are generated for all architectures, whether the code is
>>>> generic or specific makes a big difference);
>>>> User-space code to implement clock_gettime(CLOCK_REALTIME) using data
>>>> shared through the global sem heap would be generic.
>>>>
>>>> So, I think this design is future proof and easy to maintain. And I do
>>>> not see how it complicates x86 situation, since it is only made of
>>>> generic code.
>>> Well, OK, then place a small optional I-pipe hook into that part that
>>> normally writes the update into the vdso page (I think that is
>>> arch-specific anyway), replicating it into a specified page the nucleus
>>> may set up on a globally shared heap. That hook also has to maintain a
>>> seqlock like Linux does, ie. generating the same layout and semantics.
>>> It's just the transport mechanism, we can easily select it based on the
>>> arch's level of support.
>>>
>>> But I'm against any needless redirection through the nucleus (including
>>> potential nklocks etc.).
>> I do not really understand why you want to use the vdso page, since we
>> have the global sem heap anyway. clock_gettime already has a mean to
>> read a clock source and the frequency of this clock source, this is
>> guaranteed on all platforms, so I think the correction code can be made
>> generic.
> 
> Again, the page is not the point, what it contains is important. Our
> currently published data is not sufficient to support dynamic updates,
> but the vdso _contains_ all the data we could reuse with an enhanced
> algorithm to provide a dynamically adjusted time base.
> 
>> However, duplicating the ntp related kernel code may be the real issue.
>> I have to look at that code to see how complex it is.
> 
> I surely don't want to duplicate ntp code, just the results it spits out
> (e.g. into the vdso page).

Ok, good point, we can avoid duplicating ntp code, but the vdso page
trick only works on two architectures. So, IMO, the nucleus should get
the infos, and copies them to the shared area. The question is to know
if the infos are portable/in usable units such as nanoseconds or clock
source ticks.

> 
>> For the locking, well, if we have variables on a shared area to update
>> every time ntp corrects the clock, we will have problems doing it under
>> nklock anyway (the irq locking prevents from preemption on the local
>> cpu, but does not prevent the remote cpus running user-space programs
>> from accessing the shared area). So, we will have to devise some locking
>> mechanisms. But I do not see the reason for making this linux
>> compatible. If we keep the nucleus business separated, the nucleus
>> shared area will never have to be accessed by plain Linux, which will
>> access its area in the vdso, the Linux kernel doing its house keeping.
> 
> The kernel need to synchronize with user land, so the problem is
> (almost) the same as with obtaining the original data from Linux
> directly: user land can only use a retry or a smart fall-back mechanism
> to obtain consistent data.

Well, ok, I imagine something like a revision counter. But I do not see
how it would work on an SMP system (kernel writing on one CPU,
user-space reading on another cpu).

> 
>> For the HPET clocksource, well, I do not share your enthusiasm, Linux
>> keeps complaining about the stability of the tsc of my laptop, though it
>> is a fairly recent core2 with the "constant_tsc" flag.
> 
> My point is that we offer Xenomai without a real alternative to tsc for
> many moons, and no one really stood up so far and complained about
> missing hpet support. Just like you cannot use every SMI-infected box
> for RT, you can't do so when the tsc is unfixably broken. I'm not sure
> how hard hpet addition and maintenance would actually be, if it's
> trivial, I'm fine. But I think we have more important issues to solve.

If we do not support HPET, we should have a way to know that Linux is
currently not using tsc as its clock source, and so that the nucleus
should ignore the corrections, otherwise we may end up with a system
drifting even more than if Linux was not running ntp.

>From the ABI point of view, we can add a member in the sysinfo
structure, giving the address where the kernel updates the clock related
data, if that member is NULL, the kernel does not update the clock
related data (it can be a compile-time decision because using a too old
I-pipe patch, or a run-time decision because Linux does not use the tsc
as its clocksource), and clock_gettime(CLOCK_REALTIME) uses the syscall.
We would then postpone the implementation of the non-NULL case (both in
user-space and kernel-space) to after the initial 2.5 release. Mixing
kernel and user from either version would not cause any issue.

-- 
                                          Gilles

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> I surely don't want to duplicate ntp code, just the results it spits out
>> (e.g. into the vdso page).
> 
> Ok, good point, we can avoid duplicating ntp code, but the vdso page
> trick only works on two architectures. So, IMO, the nucleus should get
> the infos, and copies them to the shared area. The question is to know
> if the infos are portable/in usable units such as nanoseconds or clock
> source ticks.

Ok. Had a look, the info is pretty much generic. We can use it on all
architectures. The point which would generate the I-pipe event is
vsyscall_update.

>> The kernel need to synchronize with user land, so the problem is
>> (almost) the same as with obtaining the original data from Linux
>> directly: user land can only use a retry or a smart fall-back mechanism
>> to obtain consistent data.
> 
> Well, ok, I imagine something like a revision counter. But I do not see
> how it would work on an SMP system (kernel writing on one CPU,
> user-space reading on another cpu).

Ok, I had a look at seqlock.h, we can do it the same way, in an area of
the global heap. We will have to provide our own implementation since we
can not include linux/seqlock.h in user-space. update_vsyscall on x86_64
turns off the irqs, to protect against the same code or some reader
being called in an interrupt handler. We will have to do the same.

> 
>>> For the HPET clocksource, well, I do not share your enthusiasm, Linux
>>> keeps complaining about the stability of the tsc of my laptop, though it
>>> is a fairly recent core2 with the "constant_tsc" flag.
>> My point is that we offer Xenomai without a real alternative to tsc for
>> many moons, and no one really stood up so far and complained about
>> missing hpet support. Just like you cannot use every SMI-infected box
>> for RT, you can't do so when the tsc is unfixably broken. I'm not sure
>> how hard hpet addition and maintenance would actually be, if it's
>> trivial, I'm fine. But I think we have more important issues to solve.
> 
> If we do not support HPET, we should have a way to know that Linux is
> currently not using tsc as its clock source, and so that the nucleus
> should ignore the corrections, otherwise we may end up with a system
> drifting even more than if Linux was not running ntp.
> 
> From the ABI point of view, we can add a member in the sysinfo
> structure, giving the address where the kernel updates the clock related
> data, if that member is NULL, the kernel does not update the clock
> related data (it can be a compile-time decision because using a too old
> I-pipe patch, or a run-time decision because Linux does not use the tsc
> as its clocksource), and clock_gettime(CLOCK_REALTIME) uses the syscall.
> We would then postpone the implementation of the non-NULL case (both in
> user-space and kernel-space) to after the initial 2.5 release. Mixing
> kernel and user from either version would not cause any issue.
> 

This still stands. But we have to handle the disabled tsc differently,
probably with a flag in the shared area. Because this can happen after
the sys info have been retrieved.

-- 
					    Gilles.

Re: [Xenomai-core] Realtime gettimeofday()

[Wolfgang Mauerer]

Gilles Chanteperdrix wrote:
> Gilles Chanteperdrix wrote:
>> Jan Kiszka wrote:
>>> I surely don't want to duplicate ntp code, just the results it spits out
>>> (e.g. into the vdso page).
>> Ok, good point, we can avoid duplicating ntp code, but the vdso page
>> trick only works on two architectures. So, IMO, the nucleus should get
>> the infos, and copies them to the shared area. The question is to know
>> if the infos are portable/in usable units such as nanoseconds or clock
>> source ticks.
> 
> Ok. Had a look, the info is pretty much generic. We can use it on all
> architectures. The point which would generate the I-pipe event is
> vsyscall_update.
completely agreed. But this required arch-specific changes to the
I-pipe patch and cannot be done in a generic fashion.
> 
>>> The kernel need to synchronize with user land, so the problem is
>>> (almost) the same as with obtaining the original data from Linux
>>> directly: user land can only use a retry or a smart fall-back mechanism
>>> to obtain consistent data.
>> Well, ok, I imagine something like a revision counter. But I do not see
>> how it would work on an SMP system (kernel writing on one CPU,
>> user-space reading on another cpu).
> 
> Ok, I had a look at seqlock.h, we can do it the same way, in an area of
> the global heap. We will have to provide our own implementation since we
> can not include linux/seqlock.h in user-space. update_vsyscall on x86_64
> turns off the irqs, to protect against the same code or some reader
> being called in an interrupt handler. We will have to do the same.

But then the code can loop potentially unbounded, can't it? It may be
that I'm missing something, but this was the initial reasaon for the
three-buffer idea I've presented initially.

Best, Wolfgang

> 
>>>> For the HPET clocksource, well, I do not share your enthusiasm, Linux
>>>> keeps complaining about the stability of the tsc of my laptop, though it
>>>> is a fairly recent core2 with the "constant_tsc" flag.
>>> My point is that we offer Xenomai without a real alternative to tsc for
>>> many moons, and no one really stood up so far and complained about
>>> missing hpet support. Just like you cannot use every SMI-infected box
>>> for RT, you can't do so when the tsc is unfixably broken. I'm not sure
>>> how hard hpet addition and maintenance would actually be, if it's
>>> trivial, I'm fine. But I think we have more important issues to solve.
>> If we do not support HPET, we should have a way to know that Linux is
>> currently not using tsc as its clock source, and so that the nucleus
>> should ignore the corrections, otherwise we may end up with a system
>> drifting even more than if Linux was not running ntp.
>>
>> From the ABI point of view, we can add a member in the sysinfo
>> structure, giving the address where the kernel updates the clock related
>> data, if that member is NULL, the kernel does not update the clock
>> related data (it can be a compile-time decision because using a too old
>> I-pipe patch, or a run-time decision because Linux does not use the tsc
>> as its clocksource), and clock_gettime(CLOCK_REALTIME) uses the syscall.
>> We would then postpone the implementation of the non-NULL case (both in
>> user-space and kernel-space) to after the initial 2.5 release. Mixing
>> kernel and user from either version would not cause any issue.
>>
> 
> This still stands. But we have to handle the disabled tsc differently,
> probably with a flag in the shared area. Because this can happen after
> the sys info have been retrieved.
> 

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Wolfgang Mauerer wrote:
> Gilles Chanteperdrix wrote:
>> Gilles Chanteperdrix wrote:
>>> Jan Kiszka wrote:
>>>> I surely don't want to duplicate ntp code, just the results it spits out
>>>> (e.g. into the vdso page).
>>> Ok, good point, we can avoid duplicating ntp code, but the vdso page
>>> trick only works on two architectures. So, IMO, the nucleus should get
>>> the infos, and copies them to the shared area. The question is to know
>>> if the infos are portable/in usable units such as nanoseconds or clock
>>> source ticks.
>> Ok. Had a look, the info is pretty much generic. We can use it on all
>> architectures. The point which would generate the I-pipe event is
>> vsyscall_update.
> completely agreed. But this required arch-specific changes to the
> I-pipe patch and cannot be done in a generic fashion.

Well, it can be done in many ways. One of them is to create a function

static inline ipipe_vsyscall_update(whatever)
{
	vsyscall_update(whatever);
	ipipe_dispatch_event(whatever);
}

in linux/clocksource.h

and replace calls to vsyscall_update in generic code with a call to
ipipe_vsyscall_update.

So, no, this does not necessarily requires arch-specific changes.
However, changing it in the arch-specific way is probably the easier to
maintain on the long run (if we use the method just sketched, we will
have to check with every release if a new call to vsyscall_update
appears). Besides, for the architectures which do not implement
vsyscall_update (the majority, xenomai-wise, only x86 and ppc implement
vsyscall_update), the call to ipipe_dispatch_event may be put in the
empty implementation which may be found in linux/clocksource.h.

>>>> The kernel need to synchronize with user land, so the problem is
>>>> (almost) the same as with obtaining the original data from Linux
>>>> directly: user land can only use a retry or a smart fall-back mechanism
>>>> to obtain consistent data.
>>> Well, ok, I imagine something like a revision counter. But I do not see
>>> how it would work on an SMP system (kernel writing on one CPU,
>>> user-space reading on another cpu).
>> Ok, I had a look at seqlock.h, we can do it the same way, in an area of
>> the global heap. We will have to provide our own implementation since we
>> can not include linux/seqlock.h in user-space. update_vsyscall on x86_64
>> turns off the irqs, to protect against the same code or some reader
>> being called in an interrupt handler. We will have to do the same.
> 
> But then the code can loop potentially unbounded, can't it? It may be
> that I'm missing something, but this was the initial reasaon for the
> three-buffer idea I've presented initially.

Well, on a normally running system, the update takes place for every
Linux timer interrupt (or is it only for the periodic tick ?), anyway,
if real-time code is running, there is no chance that it can be
preempted by a Linux timer interrupt, and if the code is called from non
real-time context, we do not care much.

Note that we already use such code for tsc emulation on some platforms,
and well, it works. And this is normal, the chances to get preempted
several times in a row by a timer interrupt without being able to
complete the loop are pretty low. Otherwise it would mean that there are
plenty of timer interrupts.

-- 
					    Gilles.

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Gilles Chanteperdrix wrote:
> Wolfgang Mauerer wrote:
>> Gilles Chanteperdrix wrote:
>>> Gilles Chanteperdrix wrote:
>>>> Jan Kiszka wrote:
>>>>> I surely don't want to duplicate ntp code, just the results it spits out
>>>>> (e.g. into the vdso page).
>>>> Ok, good point, we can avoid duplicating ntp code, but the vdso page
>>>> trick only works on two architectures. So, IMO, the nucleus should get
>>>> the infos, and copies them to the shared area. The question is to know
>>>> if the infos are portable/in usable units such as nanoseconds or clock
>>>> source ticks.
>>> Ok. Had a look, the info is pretty much generic. We can use it on all
>>> architectures. The point which would generate the I-pipe event is
>>> vsyscall_update.
>> completely agreed. But this required arch-specific changes to the
>> I-pipe patch and cannot be done in a generic fashion.
> 
> Well, it can be done in many ways. One of them is to create a function
> 
> static inline ipipe_vsyscall_update(whatever)
> {
> 	vsyscall_update(whatever);
> 	ipipe_dispatch_event(whatever);
> }
> 
> in linux/clocksource.h
> 
> and replace calls to vsyscall_update in generic code with a call to
> ipipe_vsyscall_update.
> 
> So, no, this does not necessarily requires arch-specific changes.
> However, changing it in the arch-specific way is probably the easier to
> maintain on the long run (if we use the method just sketched, we will
> have to check with every release if a new call to vsyscall_update
> appears). Besides, for the architectures which do not implement
> vsyscall_update (the majority, xenomai-wise, only x86 and ppc implement
> vsyscall_update), the call to ipipe_dispatch_event may be put in the
> empty implementation which may be found in linux/clocksource.h.
> 
>>>>> The kernel need to synchronize with user land, so the problem is
>>>>> (almost) the same as with obtaining the original data from Linux
>>>>> directly: user land can only use a retry or a smart fall-back mechanism
>>>>> to obtain consistent data.
>>>> Well, ok, I imagine something like a revision counter. But I do not see
>>>> how it would work on an SMP system (kernel writing on one CPU,
>>>> user-space reading on another cpu).
>>> Ok, I had a look at seqlock.h, we can do it the same way, in an area of
>>> the global heap. We will have to provide our own implementation since we
>>> can not include linux/seqlock.h in user-space. update_vsyscall on x86_64
>>> turns off the irqs, to protect against the same code or some reader
>>> being called in an interrupt handler. We will have to do the same.
>> But then the code can loop potentially unbounded, can't it? It may be
>> that I'm missing something, but this was the initial reasaon for the
>> three-buffer idea I've presented initially.
> 
> Well, on a normally running system, the update takes place for every
> Linux timer interrupt (or is it only for the periodic tick ?), anyway,
> if real-time code is running, there is no chance that it can be
> preempted by a Linux timer interrupt, and if the code is called from non
> real-time context, we do not care much.
> 
> Note that we already use such code for tsc emulation on some platforms,
> and well, it works. And this is normal, the chances to get preempted
> several times in a row by a timer interrupt without being able to
> complete the loop are pretty low. Otherwise it would mean that there are
> plenty of timer interrupts.
> 

There is another more important issue. Currently, the timer code is
designed around the idea that the parameters used for conversion between
cpu time and wall clock time do not change often, namely only when
xnpod_set_time is called. If we start using dynamic parameters for the
conversion, it will break the assumption, and break the current timer
code (namely, real-time timers expiry date will be approximate). So, we
will have to rework the real-time timers handling code.

So, I think we should focus on the ABI issue and find a way to keep
these changes for later.

-- 
					    Gilles.

Re: [Xenomai-core] Realtime gettimeofday()

[Wolfgang Mauerer]

Hi,

On 02.12.2009, at 23:20, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org 
 > wrote:

> Wolfgang Mauerer wrote:
>> Gilles Chanteperdrix wrote:
>>> Gilles Chanteperdrix wrote:
>>>> Jan Kiszka wrote:
>>>>> I surely don't want to duplicate ntp code, just the results it  
>>>>> spits out
>>>>> (e.g. into the vdso page).
>>>> Ok, good point, we can avoid duplicating ntp code, but the vdso  
>>>> page
>>>> trick only works on two architectures. So, IMO, the nucleus  
>>>> should get
>>>> the infos, and copies them to the shared area. The question is to  
>>>> know
>>>> if the infos are portable/in usable units such as nanoseconds or  
>>>> clock
>>>> source ticks.
>>> Ok. Had a look, the info is pretty much generic. We can use it on  
>>> all
>>> architectures. The point which would generate the I-pipe event is
>>> vsyscall_update.
>> completely agreed. But this required arch-specific changes to the
>> I-pipe patch and cannot be done in a generic fashion.
>
> Well, it can be done in many ways. One of them is to create a function
>
> static inline ipipe_vsyscall_update(whatever)
> {
>    vsyscall_update(whatever);
>    ipipe_dispatch_event(whatever);
> }
>
> in linux/clocksource.h
>
> and replace calls to vsyscall_update in generic code with a call to
> ipipe_vsyscall_update.
>
> So, no, this does not necessarily requires arch-specific changes.
> However, changing it in the arch-specific way is probably the easier  
> to
> maintain on the long run (if we use the method just sketched, we will
> have to check with every release if a new call to vsyscall_update
> appears). Besides, for the architectures which do not implement
> vsyscall_update (the majority, xenomai-wise, only x86 and ppc  
> implement
> vsyscall_update), the call to ipipe_dispatch_event may be put in the
> empty implementation which may be found in linux/clocksource.h.
>
>>>>> The kernel need to synchronize with user land, so the problem is
>>>>> (almost) the same as with obtaining the original data from Linux
>>>>> directly: user land can only use a retry or a smart fall-back  
>>>>> mechanism
>>>>> to obtain consistent data.
>>>> Well, ok, I imagine something like a revision counter. But I do  
>>>> not see
>>>> how it would work on an SMP system (kernel writing on one CPU,
>>>> user-space reading on another cpu).
>>> Ok, I had a look at seqlock.h, we can do it the same way, in an  
>>> area of
>>> the global heap. We will have to provide our own implementation  
>>> since we
>>> can not include linux/seqlock.h in user-space. update_vsyscall on  
>>> x86_64
>>> turns off the irqs, to protect against the same code or some reader
>>> being called in an interrupt handler. We will have to do the same.
>>
>> But then the code can loop potentially unbounded, can't it? It may be
>> that I'm missing something, but this was the initial reasaon for the
>> three-buffer idea I've presented initially.
>
> Well, on a normally running system, the update takes place for every
> Linux timer interrupt (or is it only for the periodic tick ?), anyway,
> if real-time code is running, there is no chance that it can be
> preempted by a Linux timer interrupt, and if the code is called from  
> non
> real-time context, we do not care much.

For a tickless System, it's only the periodic Tick, Not every Timer  
Interrupt.
> Note that we already use such code for tsc emulation on some  
> platforms,
> and well, it works. And this is normal, the chances to get preempted
> several times in a row by a timer interrupt without being able to
> complete the loop are pretty low. Otherwise it would mean that there  
> are
> plenty of timer interrupts.

So that means, in essence, that you would accept probabilistic  
algorithms in realtime context?

Thanks,

Wolfgang

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Wolfgang Mauerer wrote:
>>>>>> The kernel need to synchronize with user land, so the problem is
>>>>>> (almost) the same as with obtaining the original data from Linux
>>>>>> directly: user land can only use a retry or a smart fall-back  
>>>>>> mechanism
>>>>>> to obtain consistent data.
>>>>> Well, ok, I imagine something like a revision counter. But I do  
>>>>> not see
>>>>> how it would work on an SMP system (kernel writing on one CPU,
>>>>> user-space reading on another cpu).
>>>> Ok, I had a look at seqlock.h, we can do it the same way, in an  
>>>> area of
>>>> the global heap. We will have to provide our own implementation  
>>>> since we
>>>> can not include linux/seqlock.h in user-space. update_vsyscall on  
>>>> x86_64
>>>> turns off the irqs, to protect against the same code or some reader
>>>> being called in an interrupt handler. We will have to do the same.
>>> But then the code can loop potentially unbounded, can't it? It may be
>>> that I'm missing something, but this was the initial reasaon for the
>>> three-buffer idea I've presented initially.
>> Well, on a normally running system, the update takes place for every
>> Linux timer interrupt (or is it only for the periodic tick ?), anyway,
>> if real-time code is running, there is no chance that it can be
>> preempted by a Linux timer interrupt, and if the code is called from  
>> non
>> real-time context, we do not care much.
> 
> For a tickless System, it's only the periodic Tick, Not every Timer  
> Interrupt.
>> Note that we already use such code for tsc emulation on some  
>> platforms,
>> and well, it works. And this is normal, the chances to get preempted
>> several times in a row by a timer interrupt without being able to
>> complete the loop are pretty low. Otherwise it would mean that there  
>> are
>> plenty of timer interrupts.
> 
> So that means, in essence, that you would accept probabilistic  
> algorithms in realtime context?

Ah, today's troll!

As I think I explained, the use of a seqlock in real-time context when
the seqlock writer only happens in linux context is not probabilistic.
It will work every time the first pass.

But other than that, yes, we do:
- RAM ECC algorithms only work for a given maximum number of bit changes
per cycle, and the source of bit changes is essentially random;
- cache behaviour, even though deterministic system-wise is
probabilistic from a real-time thread point of view, you never now how
many of your cache lines the non real-time threads will evict;
- fast user-space mutexes use compare-and-swap and retry if it failed
- user-space access to tsc on arm, powerpc, blackfin, nios, that is on
anything but x86 use a retry strategy.

We live with it.

The latency test allow you to measure the probabilistic effect of cache.

For the mutexes and tsc, if you happen to retry too often, it means that
the system starves the task which has the problem. It may be because you
designed your system that way, but in that case, you will expect it to
run slowly anyway, and it would be strange to specifically mandates the
mutex lock and tsc calls not to be slow. Or it may be because of a
design error, and in that case the design should be fixed, not mutex
lock and tsc.

-- 
                                          Gilles

Re: [Xenomai-core] Realtime gettimeofday()

[Wolfgang Mauerer]

Hi,

Gilles Chanteperdrix wrote:
> Wolfgang Mauerer wrote:

>> So that means, in essence, that you would accept probabilistic  
>> algorithms in realtime context?
> 
> Ah, today's troll!

though it seems that I have to replace Jan this time ;-)
> 
> As I think I explained, the use of a seqlock in real-time context when
> the seqlock writer only happens in linux context is not probabilistic.
> It will work every time the first pass.

I still don't see why it should succeed every time: What about
the case that the Linux kernel on CPU0 updates the data, while
Xenomai accesses them on another CPU? This can lead to
inconsistent data, and they must be reread on the Xenomai side.

I'm asking because if this case can not happen, then there's
nothing left to to as I have the code already at hand.

Thanks, Wolfgang

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Wolfgang Mauerer wrote:
> Hi,
> 
> Gilles Chanteperdrix wrote:
>> Wolfgang Mauerer wrote:
> 
>>> So that means, in essence, that you would accept probabilistic  
>>> algorithms in realtime context?
>> Ah, today's troll!
> 
> though it seems that I have to replace Jan this time ;-)
>> As I think I explained, the use of a seqlock in real-time context when
>> the seqlock writer only happens in linux context is not probabilistic.
>> It will work every time the first pass.
> 
> I still don't see why it should succeed every time: What about
> the case that the Linux kernel on CPU0 updates the data, while
> Xenomai accesses them on another CPU? This can lead to
> inconsistent data, and they must be reread on the Xenomai side.

Yeah, right. I was not thinking about SMP. But admit that in this case,
there will be only one retry, there is nothing pathological.

> 
> I'm asking because if this case can not happen, then there's
> nothing left to to as I have the code already at hand.

You have reworked the nucleus timers handling to adapt to this new
real-time clock ?

-- 
                                          Gilles

Re: [Xenomai-core] Realtime gettimeofday()

[Wolfgang Mauerer]

Hi,

On 03.12.2009, at 14:14, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org 
 > wrote:

> Wolfgang Mauerer wrote:
>> Hi,
>>
>> Gilles Chanteperdrix wrote:
>>> Wolfgang Mauerer wrote:
>>
>>>> So that means, in essence, that you would accept probabilistic
>>>> algorithms in realtime context?
>>> Ah, today's troll!
>>
>> though it seems that I have to replace Jan this time ;-)
>>> As I think I explained, the use of a seqlock in real-time context  
>>> when
>>> the seqlock writer only happens in linux context is not  
>>> probabilistic.
>>> It will work every time the first pass.
>>
>> I still don't see why it should succeed every time: What about
>> the case that the Linux kernel on CPU0 updates the data, while
>> Xenomai accesses them on another CPU? This can lead to
>> inconsistent data, and they must be reread on the Xenomai side.
>
> Yeah, right. I was not thinking about SMP. But admit that in this  
> case,
> there will be only one retry, there is nothing pathological.
>
>>
>> I'm asking because if this case can not happen, then there's
>> nothing left to to as I have the code already at hand.
>
> You have reworked the nucleus timers handling to adapt to this new
> real-time clock ?

Nope. Sorry, I was a bit unclear: I'm just referring to the gtod  
syscall that does the timer handling, Not any other adaptions.

Thanks, Wolfgang

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Wolfgang Mauerer wrote:
> Hi,
> 
> On 03.12.2009, at 14:14, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org 
>  > wrote:
> 
>> Wolfgang Mauerer wrote:
>>> Hi,
>>>
>>> Gilles Chanteperdrix wrote:
>>>> Wolfgang Mauerer wrote:
>>>>> So that means, in essence, that you would accept probabilistic
>>>>> algorithms in realtime context?
>>>> Ah, today's troll!
>>> though it seems that I have to replace Jan this time ;-)
>>>> As I think I explained, the use of a seqlock in real-time context  
>>>> when
>>>> the seqlock writer only happens in linux context is not  
>>>> probabilistic.
>>>> It will work every time the first pass.
>>> I still don't see why it should succeed every time: What about
>>> the case that the Linux kernel on CPU0 updates the data, while
>>> Xenomai accesses them on another CPU? This can lead to
>>> inconsistent data, and they must be reread on the Xenomai side.
>> Yeah, right. I was not thinking about SMP. But admit that in this  
>> case,
>> there will be only one retry, there is nothing pathological.
>>
>>> I'm asking because if this case can not happen, then there's
>>> nothing left to to as I have the code already at hand.
>> You have reworked the nucleus timers handling to adapt to this new
>> real-time clock ?
> 
> Nope. Sorry, I was a bit unclear: I'm just referring to the gtod  
> syscall that does the timer handling, Not any other adaptions.

Ok, but what good is the gtod syscall if you can not use it as a time
reference for other timing related services?

-- 
                                          Gilles

Re: [Xenomai-core] Realtime gettimeofday()

[Wolfgang Mauerer]

Gilles Chanteperdrix wrote:
> Wolfgang Mauerer wrote:
>> Hi,
>>
>> On 03.12.2009, at 14:14, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org 
>>  > wrote:
>>
>>> Wolfgang Mauerer wrote:
>>>> Hi,
>>>>
>>>> Gilles Chanteperdrix wrote:
>>>>> Wolfgang Mauerer wrote:
>>>>>> So that means, in essence, that you would accept probabilistic
>>>>>> algorithms in realtime context?
>>>>> Ah, today's troll!
>>>> though it seems that I have to replace Jan this time ;-)
>>>>> As I think I explained, the use of a seqlock in real-time context  
>>>>> when
>>>>> the seqlock writer only happens in linux context is not  
>>>>> probabilistic.
>>>>> It will work every time the first pass.
>>>> I still don't see why it should succeed every time: What about
>>>> the case that the Linux kernel on CPU0 updates the data, while
>>>> Xenomai accesses them on another CPU? This can lead to
>>>> inconsistent data, and they must be reread on the Xenomai side.
>>> Yeah, right. I was not thinking about SMP. But admit that in this  
>>> case,
>>> there will be only one retry, there is nothing pathological.

that's right. Which makes it bounded again, so it's maybe
the best way to go.
>>>
>>>> I'm asking because if this case can not happen, then there's
>>>> nothing left to to as I have the code already at hand.
>>> You have reworked the nucleus timers handling to adapt to this new
>>> real-time clock ?
>> Nope. Sorry, I was a bit unclear: I'm just referring to the gtod  
>> syscall that does the timer handling, Not any other adaptions.
> 
> Ok, but what good is the gtod syscall if you can not use it as a time
> reference for other timing related services?

it suffices for our project's current purposes ;-)

But it's certainly not the full solution. Before, we
should have a decision wrt. the design issues, but I
won't be able to continue working on this before
mid of next week to look at the changed required for timer
handling and come up with code.

Cheers, Wolfgang

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Wolfgang Mauerer wrote:
> Gilles Chanteperdrix wrote:
>> Wolfgang Mauerer wrote:
>>> Hi,
>>>
>>> On 03.12.2009, at 14:14, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org 
>>>  > wrote:
>>>
>>>> Wolfgang Mauerer wrote:
>>>>> Hi,
>>>>>
>>>>> Gilles Chanteperdrix wrote:
>>>>>> Wolfgang Mauerer wrote:
>>>>>>> So that means, in essence, that you would accept probabilistic
>>>>>>> algorithms in realtime context?
>>>>>> Ah, today's troll!
>>>>> though it seems that I have to replace Jan this time ;-)
>>>>>> As I think I explained, the use of a seqlock in real-time context  
>>>>>> when
>>>>>> the seqlock writer only happens in linux context is not  
>>>>>> probabilistic.
>>>>>> It will work every time the first pass.
>>>>> I still don't see why it should succeed every time: What about
>>>>> the case that the Linux kernel on CPU0 updates the data, while
>>>>> Xenomai accesses them on another CPU? This can lead to
>>>>> inconsistent data, and they must be reread on the Xenomai side.
>>>> Yeah, right. I was not thinking about SMP. But admit that in this  
>>>> case,
>>>> there will be only one retry, there is nothing pathological.
> 
> that's right. Which makes it bounded again, so it's maybe
> the best way to go.
>>>>> I'm asking because if this case can not happen, then there's
>>>>> nothing left to to as I have the code already at hand.
>>>> You have reworked the nucleus timers handling to adapt to this new
>>>> real-time clock ?
>>> Nope. Sorry, I was a bit unclear: I'm just referring to the gtod  
>>> syscall that does the timer handling, Not any other adaptions.
>> Ok, but what good is the gtod syscall if you can not use it as a time
>> reference for other timing related services?
> 
> it suffices for our project's current purposes ;-)
> 
> But it's certainly not the full solution. Before, we
> should have a decision wrt. the design issues, but I
> won't be able to continue working on this before
> mid of next week to look at the changed required for timer
> handling and come up with code.

Ok. To summarize what we have said, here is how I see we could implement
the NTP synchronized clock fully, and portably:
1- allocate at nucleus init time, an area in the global sem heap for
this clock house-keeping
2- add an event to the I-pipe patch when vsyscall_update is called
3- implement the nucleus callback for the I-pipe event which copies
relevant data with our own version of seqlock called with hardware irqs
off, to the area allocated in 1 if the current clock source is the tsc
4- rework the nucleus clocks and timers handling to use these data
5- pass the offset of the data allocated in 1 to user-space through the
xnsysinfo, or xnfeatinfo structures
6- rework clock_gettime to use these data, using the user-space
counterpart of the seqlock used in 3

The real hard work is 4, and note that something which I did not mention
yesterday, is that we not only have to change the real-time clock
implementation, we also have to change the monotonic clock
implementation, otherwise the two clocks will drift apart.

I think making such a change now is unreasonable.

So, solution 1, we can implement 5, passing a null offset to mean that
the support is unimplemented by the kernel and not even use it in
user-space. Keeping the work for later in the 2.5 life cycle.

Solution 2, we keep this change for 3.0.

Solution 3, we implement a way to read that clock without synchronizing
the nucleus with it (that is, everything but 4). One way to do this,
which I do not like, is to add a dummy clock id to the posix skin, for
instance CLOCK_LINUX_NTP_REALTIME, and implement the clock reading for
that clock id in clock_gettime. This clock id, when passed to any other
service, causes EINVAL to be returned, making it clear that this clock
can not be used for anything else. Note that if we do that, even if we
implement the full support later, we will have to keep that dummy clock
id forever.

My preference goes to solution 1. Philippe, what do you think?

-- 
					    Gilles.

Re: [Xenomai-core] Realtime gettimeofday()

[Philippe Gerum]

On Thu, 2009-12-03 at 22:42 +0100, Gilles Chanteperdrix wrote:
> Wolfgang Mauerer wrote:
> > Gilles Chanteperdrix wrote:
> >> Wolfgang Mauerer wrote:
> >>> Hi,
> >>>
> >>> On 03.12.2009, at 14:14, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org 
> >>>  > wrote:
> >>>
> >>>> Wolfgang Mauerer wrote:
> >>>>> Hi,
> >>>>>
> >>>>> Gilles Chanteperdrix wrote:
> >>>>>> Wolfgang Mauerer wrote:
> >>>>>>> So that means, in essence, that you would accept probabilistic
> >>>>>>> algorithms in realtime context?
> >>>>>> Ah, today's troll!
> >>>>> though it seems that I have to replace Jan this time ;-)
> >>>>>> As I think I explained, the use of a seqlock in real-time context  
> >>>>>> when
> >>>>>> the seqlock writer only happens in linux context is not  
> >>>>>> probabilistic.
> >>>>>> It will work every time the first pass.
> >>>>> I still don't see why it should succeed every time: What about
> >>>>> the case that the Linux kernel on CPU0 updates the data, while
> >>>>> Xenomai accesses them on another CPU? This can lead to
> >>>>> inconsistent data, and they must be reread on the Xenomai side.
> >>>> Yeah, right. I was not thinking about SMP. But admit that in this  
> >>>> case,
> >>>> there will be only one retry, there is nothing pathological.
> > 
> > that's right. Which makes it bounded again, so it's maybe
> > the best way to go.
> >>>>> I'm asking because if this case can not happen, then there's
> >>>>> nothing left to to as I have the code already at hand.
> >>>> You have reworked the nucleus timers handling to adapt to this new
> >>>> real-time clock ?
> >>> Nope. Sorry, I was a bit unclear: I'm just referring to the gtod  
> >>> syscall that does the timer handling, Not any other adaptions.
> >> Ok, but what good is the gtod syscall if you can not use it as a time
> >> reference for other timing related services?
> > 
> > it suffices for our project's current purposes ;-)
> > 
> > But it's certainly not the full solution. Before, we
> > should have a decision wrt. the design issues, but I
> > won't be able to continue working on this before
> > mid of next week to look at the changed required for timer
> > handling and come up with code.
> 
> Ok. To summarize what we have said, here is how I see we could implement
> the NTP synchronized clock fully, and portably:
> 1- allocate at nucleus init time, an area in the global sem heap for
> this clock house-keeping
> 2- add an event to the I-pipe patch when vsyscall_update is called
> 3- implement the nucleus callback for the I-pipe event which copies
> relevant data with our own version of seqlock called with hardware irqs
> off, to the area allocated in 1 if the current clock source is the tsc
> 4- rework the nucleus clocks and timers handling to use these data
> 5- pass the offset of the data allocated in 1 to user-space through the
> xnsysinfo, or xnfeatinfo structures
> 6- rework clock_gettime to use these data, using the user-space
> counterpart of the seqlock used in 3
> 
> The real hard work is 4, and note that something which I did not mention
> yesterday, is that we not only have to change the real-time clock
> implementation, we also have to change the monotonic clock
> implementation, otherwise the two clocks will drift apart.
> 
> I think making such a change now is unreasonable.
> 
> So, solution 1, we can implement 5, passing a null offset to mean that
> the support is unimplemented by the kernel and not even use it in
> user-space. Keeping the work for later in the 2.5 life cycle.
> 
> Solution 2, we keep this change for 3.0.
> 
> Solution 3, we implement a way to read that clock without synchronizing
> the nucleus with it (that is, everything but 4). One way to do this,
> which I do not like, is to add a dummy clock id to the posix skin, for
> instance CLOCK_LINUX_NTP_REALTIME, and implement the clock reading for
> that clock id in clock_gettime. This clock id, when passed to any other
> service, causes EINVAL to be returned, making it clear that this clock
> can not be used for anything else. Note that if we do that, even if we
> implement the full support later, we will have to keep that dummy clock
> id forever.
> 
> My preference goes to solution 1. Philippe, what do you think?

Way too late for this kind of change; 2.5.0 will be out this month.
Timer related issues are prone to introduce nasty subtle regressions.
Let's plan this for 3.0. The earlier 2.5.0 will be out, the earlier 3.0
will start.

> 
> -- 
> 					    Gilles.
> 
> _______________________________________________
> Xenomai-core mailing list
> Xenomai-core@domain.hid
> https://mail.gna.org/listinfo/xenomai-core


-- 
Philippe.

Re: [Xenomai-core] Realtime gettimeofday()

[Jan Kiszka]

Philippe Gerum wrote:
> On Thu, 2009-12-03 at 22:42 +0100, Gilles Chanteperdrix wrote:
>> Wolfgang Mauerer wrote:
>>> Gilles Chanteperdrix wrote:
>>>> Wolfgang Mauerer wrote:
>>>>> Hi,
>>>>>
>>>>> On 03.12.2009, at 14:14, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org 
>>>>>  > wrote:
>>>>>
>>>>>> Wolfgang Mauerer wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> Gilles Chanteperdrix wrote:
>>>>>>>> Wolfgang Mauerer wrote:
>>>>>>>>> So that means, in essence, that you would accept probabilistic
>>>>>>>>> algorithms in realtime context?
>>>>>>>> Ah, today's troll!
>>>>>>> though it seems that I have to replace Jan this time ;-)
>>>>>>>> As I think I explained, the use of a seqlock in real-time context  
>>>>>>>> when
>>>>>>>> the seqlock writer only happens in linux context is not  
>>>>>>>> probabilistic.
>>>>>>>> It will work every time the first pass.
>>>>>>> I still don't see why it should succeed every time: What about
>>>>>>> the case that the Linux kernel on CPU0 updates the data, while
>>>>>>> Xenomai accesses them on another CPU? This can lead to
>>>>>>> inconsistent data, and they must be reread on the Xenomai side.
>>>>>> Yeah, right. I was not thinking about SMP. But admit that in this  
>>>>>> case,
>>>>>> there will be only one retry, there is nothing pathological.
>>> that's right. Which makes it bounded again, so it's maybe
>>> the best way to go.
>>>>>>> I'm asking because if this case can not happen, then there's
>>>>>>> nothing left to to as I have the code already at hand.
>>>>>> You have reworked the nucleus timers handling to adapt to this new
>>>>>> real-time clock ?
>>>>> Nope. Sorry, I was a bit unclear: I'm just referring to the gtod  
>>>>> syscall that does the timer handling, Not any other adaptions.
>>>> Ok, but what good is the gtod syscall if you can not use it as a time
>>>> reference for other timing related services?
>>> it suffices for our project's current purposes ;-)
>>>
>>> But it's certainly not the full solution. Before, we
>>> should have a decision wrt. the design issues, but I
>>> won't be able to continue working on this before
>>> mid of next week to look at the changed required for timer
>>> handling and come up with code.
>> Ok. To summarize what we have said, here is how I see we could implement
>> the NTP synchronized clock fully, and portably:
>> 1- allocate at nucleus init time, an area in the global sem heap for
>> this clock house-keeping
>> 2- add an event to the I-pipe patch when vsyscall_update is called
>> 3- implement the nucleus callback for the I-pipe event which copies
>> relevant data with our own version of seqlock called with hardware irqs
>> off, to the area allocated in 1 if the current clock source is the tsc
>> 4- rework the nucleus clocks and timers handling to use these data
>> 5- pass the offset of the data allocated in 1 to user-space through the
>> xnsysinfo, or xnfeatinfo structures
>> 6- rework clock_gettime to use these data, using the user-space
>> counterpart of the seqlock used in 3
>>
>> The real hard work is 4, and note that something which I did not mention
>> yesterday, is that we not only have to change the real-time clock
>> implementation, we also have to change the monotonic clock
>> implementation, otherwise the two clocks will drift apart.
>>
>> I think making such a change now is unreasonable.
>>
>> So, solution 1, we can implement 5, passing a null offset to mean that
>> the support is unimplemented by the kernel and not even use it in
>> user-space. Keeping the work for later in the 2.5 life cycle.
>>
>> Solution 2, we keep this change for 3.0.
>>
>> Solution 3, we implement a way to read that clock without synchronizing
>> the nucleus with it (that is, everything but 4). One way to do this,
>> which I do not like, is to add a dummy clock id to the posix skin, for
>> instance CLOCK_LINUX_NTP_REALTIME, and implement the clock reading for
>> that clock id in clock_gettime. This clock id, when passed to any other
>> service, causes EINVAL to be returned, making it clear that this clock
>> can not be used for anything else. Note that if we do that, even if we
>> implement the full support later, we will have to keep that dummy clock
>> id forever.
>>
>> My preference goes to solution 1. Philippe, what do you think?
> 
> Way too late for this kind of change; 2.5.0 will be out this month.
> Timer related issues are prone to introduce nasty subtle regressions.
> Let's plan this for 3.0. The earlier 2.5.0 will be out, the earlier 3.0
> will start.

No, this was not about turning the nucleus timer handling upside down
for 2.5.0, this was first of all about establishing the required kernel
to user space ABI for 2.5. Can we agree on the latter?

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Jan Kiszka wrote:
> Philippe Gerum wrote:
>> On Thu, 2009-12-03 at 22:42 +0100, Gilles Chanteperdrix wrote:
>>> Wolfgang Mauerer wrote:
>>>> Gilles Chanteperdrix wrote:
>>>>> Wolfgang Mauerer wrote:
>>>>>> Hi,
>>>>>>
>>>>>> On 03.12.2009, at 14:14, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org 
>>>>>>  > wrote:
>>>>>>
>>>>>>> Wolfgang Mauerer wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Gilles Chanteperdrix wrote:
>>>>>>>>> Wolfgang Mauerer wrote:
>>>>>>>>>> So that means, in essence, that you would accept probabilistic
>>>>>>>>>> algorithms in realtime context?
>>>>>>>>> Ah, today's troll!
>>>>>>>> though it seems that I have to replace Jan this time ;-)
>>>>>>>>> As I think I explained, the use of a seqlock in real-time context  
>>>>>>>>> when
>>>>>>>>> the seqlock writer only happens in linux context is not  
>>>>>>>>> probabilistic.
>>>>>>>>> It will work every time the first pass.
>>>>>>>> I still don't see why it should succeed every time: What about
>>>>>>>> the case that the Linux kernel on CPU0 updates the data, while
>>>>>>>> Xenomai accesses them on another CPU? This can lead to
>>>>>>>> inconsistent data, and they must be reread on the Xenomai side.
>>>>>>> Yeah, right. I was not thinking about SMP. But admit that in this  
>>>>>>> case,
>>>>>>> there will be only one retry, there is nothing pathological.
>>>> that's right. Which makes it bounded again, so it's maybe
>>>> the best way to go.
>>>>>>>> I'm asking because if this case can not happen, then there's
>>>>>>>> nothing left to to as I have the code already at hand.
>>>>>>> You have reworked the nucleus timers handling to adapt to this new
>>>>>>> real-time clock ?
>>>>>> Nope. Sorry, I was a bit unclear: I'm just referring to the gtod  
>>>>>> syscall that does the timer handling, Not any other adaptions.
>>>>> Ok, but what good is the gtod syscall if you can not use it as a time
>>>>> reference for other timing related services?
>>>> it suffices for our project's current purposes ;-)
>>>>
>>>> But it's certainly not the full solution. Before, we
>>>> should have a decision wrt. the design issues, but I
>>>> won't be able to continue working on this before
>>>> mid of next week to look at the changed required for timer
>>>> handling and come up with code.
>>> Ok. To summarize what we have said, here is how I see we could implement
>>> the NTP synchronized clock fully, and portably:
>>> 1- allocate at nucleus init time, an area in the global sem heap for
>>> this clock house-keeping
>>> 2- add an event to the I-pipe patch when vsyscall_update is called
>>> 3- implement the nucleus callback for the I-pipe event which copies
>>> relevant data with our own version of seqlock called with hardware irqs
>>> off, to the area allocated in 1 if the current clock source is the tsc
>>> 4- rework the nucleus clocks and timers handling to use these data
>>> 5- pass the offset of the data allocated in 1 to user-space through the
>>> xnsysinfo, or xnfeatinfo structures
>>> 6- rework clock_gettime to use these data, using the user-space
>>> counterpart of the seqlock used in 3
>>>
>>> The real hard work is 4, and note that something which I did not mention
>>> yesterday, is that we not only have to change the real-time clock
>>> implementation, we also have to change the monotonic clock
>>> implementation, otherwise the two clocks will drift apart.
>>>
>>> I think making such a change now is unreasonable.
>>>
>>> So, solution 1, we can implement 5, passing a null offset to mean that
>>> the support is unimplemented by the kernel and not even use it in
>>> user-space. Keeping the work for later in the 2.5 life cycle.
>>>
>>> Solution 2, we keep this change for 3.0.
>>>
>>> Solution 3, we implement a way to read that clock without synchronizing
>>> the nucleus with it (that is, everything but 4). One way to do this,
>>> which I do not like, is to add a dummy clock id to the posix skin, for
>>> instance CLOCK_LINUX_NTP_REALTIME, and implement the clock reading for
>>> that clock id in clock_gettime. This clock id, when passed to any other
>>> service, causes EINVAL to be returned, making it clear that this clock
>>> can not be used for anything else. Note that if we do that, even if we
>>> implement the full support later, we will have to keep that dummy clock
>>> id forever.
>>>
>>> My preference goes to solution 1. Philippe, what do you think?
>> Way too late for this kind of change; 2.5.0 will be out this month.
>> Timer related issues are prone to introduce nasty subtle regressions.
>> Let's plan this for 3.0. The earlier 2.5.0 will be out, the earlier 3.0
>> will start.
> 
> No, this was not about turning the nucleus timer handling upside down
> for 2.5.0, this was first of all about establishing the required kernel
> to user space ABI for 2.5. Can we agree on the latter?

Ok. It is solution 1 then. In order to get things evolving more easily
than my first solution, here is what is proposed.

We define a structure which will be shared between kernel and user,
which contains all data exported by kernel to user, as well as flags
indicating which member of the structure is available. The definition of
the struct for 2.5.0 will be:

struct xnshared  {
	unsigned long long features;
};

This struct will be allocated in the global sem heap, and features will
be null for the time being.

Every time we need to share some data between kernel and user (including
for the ntp support), we will add data to the structure, and use a
"features" bit to mean that the data are available from kernel. It will
work as long as we add data from release to release and never remove it.

So, for 2.5.0, the xnshared structure will be allocated, but the
features member will be null. We will pass the offset of this structure
in the global sem heap in the sysinfo structure.

-- 
                                          Gilles

Re: [Xenomai-core] Realtime gettimeofday()

[Jan Kiszka]

Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
>> Philippe Gerum wrote:
>>> On Thu, 2009-12-03 at 22:42 +0100, Gilles Chanteperdrix wrote:
>>>> Wolfgang Mauerer wrote:
>>>>> Gilles Chanteperdrix wrote:
>>>>>> Wolfgang Mauerer wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> On 03.12.2009, at 14:14, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org 
>>>>>>>  > wrote:
>>>>>>>
>>>>>>>> Wolfgang Mauerer wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Gilles Chanteperdrix wrote:
>>>>>>>>>> Wolfgang Mauerer wrote:
>>>>>>>>>>> So that means, in essence, that you would accept probabilistic
>>>>>>>>>>> algorithms in realtime context?
>>>>>>>>>> Ah, today's troll!
>>>>>>>>> though it seems that I have to replace Jan this time ;-)
>>>>>>>>>> As I think I explained, the use of a seqlock in real-time context  
>>>>>>>>>> when
>>>>>>>>>> the seqlock writer only happens in linux context is not  
>>>>>>>>>> probabilistic.
>>>>>>>>>> It will work every time the first pass.
>>>>>>>>> I still don't see why it should succeed every time: What about
>>>>>>>>> the case that the Linux kernel on CPU0 updates the data, while
>>>>>>>>> Xenomai accesses them on another CPU? This can lead to
>>>>>>>>> inconsistent data, and they must be reread on the Xenomai side.
>>>>>>>> Yeah, right. I was not thinking about SMP. But admit that in this  
>>>>>>>> case,
>>>>>>>> there will be only one retry, there is nothing pathological.
>>>>> that's right. Which makes it bounded again, so it's maybe
>>>>> the best way to go.
>>>>>>>>> I'm asking because if this case can not happen, then there's
>>>>>>>>> nothing left to to as I have the code already at hand.
>>>>>>>> You have reworked the nucleus timers handling to adapt to this new
>>>>>>>> real-time clock ?
>>>>>>> Nope. Sorry, I was a bit unclear: I'm just referring to the gtod  
>>>>>>> syscall that does the timer handling, Not any other adaptions.
>>>>>> Ok, but what good is the gtod syscall if you can not use it as a time
>>>>>> reference for other timing related services?
>>>>> it suffices for our project's current purposes ;-)
>>>>>
>>>>> But it's certainly not the full solution. Before, we
>>>>> should have a decision wrt. the design issues, but I
>>>>> won't be able to continue working on this before
>>>>> mid of next week to look at the changed required for timer
>>>>> handling and come up with code.
>>>> Ok. To summarize what we have said, here is how I see we could implement
>>>> the NTP synchronized clock fully, and portably:
>>>> 1- allocate at nucleus init time, an area in the global sem heap for
>>>> this clock house-keeping
>>>> 2- add an event to the I-pipe patch when vsyscall_update is called
>>>> 3- implement the nucleus callback for the I-pipe event which copies
>>>> relevant data with our own version of seqlock called with hardware irqs
>>>> off, to the area allocated in 1 if the current clock source is the tsc
>>>> 4- rework the nucleus clocks and timers handling to use these data
>>>> 5- pass the offset of the data allocated in 1 to user-space through the
>>>> xnsysinfo, or xnfeatinfo structures
>>>> 6- rework clock_gettime to use these data, using the user-space
>>>> counterpart of the seqlock used in 3
>>>>
>>>> The real hard work is 4, and note that something which I did not mention
>>>> yesterday, is that we not only have to change the real-time clock
>>>> implementation, we also have to change the monotonic clock
>>>> implementation, otherwise the two clocks will drift apart.
>>>>
>>>> I think making such a change now is unreasonable.
>>>>
>>>> So, solution 1, we can implement 5, passing a null offset to mean that
>>>> the support is unimplemented by the kernel and not even use it in
>>>> user-space. Keeping the work for later in the 2.5 life cycle.
>>>>
>>>> Solution 2, we keep this change for 3.0.
>>>>
>>>> Solution 3, we implement a way to read that clock without synchronizing
>>>> the nucleus with it (that is, everything but 4). One way to do this,
>>>> which I do not like, is to add a dummy clock id to the posix skin, for
>>>> instance CLOCK_LINUX_NTP_REALTIME, and implement the clock reading for
>>>> that clock id in clock_gettime. This clock id, when passed to any other
>>>> service, causes EINVAL to be returned, making it clear that this clock
>>>> can not be used for anything else. Note that if we do that, even if we
>>>> implement the full support later, we will have to keep that dummy clock
>>>> id forever.
>>>>
>>>> My preference goes to solution 1. Philippe, what do you think?
>>> Way too late for this kind of change; 2.5.0 will be out this month.
>>> Timer related issues are prone to introduce nasty subtle regressions.
>>> Let's plan this for 3.0. The earlier 2.5.0 will be out, the earlier 3.0
>>> will start.
>> No, this was not about turning the nucleus timer handling upside down
>> for 2.5.0, this was first of all about establishing the required kernel
>> to user space ABI for 2.5. Can we agree on the latter?
> 
> Ok. It is solution 1 then. In order to get things evolving more easily
> than my first solution, here is what is proposed.
> 
> We define a structure which will be shared between kernel and user,
> which contains all data exported by kernel to user, as well as flags
> indicating which member of the structure is available. The definition of
> the struct for 2.5.0 will be:
> 
> struct xnshared  {
> 	unsigned long long features;
> };
> 
> This struct will be allocated in the global sem heap, and features will
> be null for the time being.
> 
> Every time we need to share some data between kernel and user (including
> for the ntp support), we will add data to the structure, and use a
> "features" bit to mean that the data are available from kernel. It will
> work as long as we add data from release to release and never remove it.

...while features can still be removed by clearing their bits again.
Sounds good to me.

> 
> So, for 2.5.0, the xnshared structure will be allocated, but the
> features member will be null. We will pass the offset of this structure
> in the global sem heap in the sysinfo structure.

Ack.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] Realtime gettimeofday()

[Philippe Gerum]

On Mon, 2009-12-07 at 14:43 +0100, Gilles Chanteperdrix wrote:
> Jan Kiszka wrote:
> > Philippe Gerum wrote:
> >> On Thu, 2009-12-03 at 22:42 +0100, Gilles Chanteperdrix wrote:
> >>> Wolfgang Mauerer wrote:
> >>>> Gilles Chanteperdrix wrote:
> >>>>> Wolfgang Mauerer wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> On 03.12.2009, at 14:14, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org 
> >>>>>>  > wrote:
> >>>>>>
> >>>>>>> Wolfgang Mauerer wrote:
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> Gilles Chanteperdrix wrote:
> >>>>>>>>> Wolfgang Mauerer wrote:
> >>>>>>>>>> So that means, in essence, that you would accept probabilistic
> >>>>>>>>>> algorithms in realtime context?
> >>>>>>>>> Ah, today's troll!
> >>>>>>>> though it seems that I have to replace Jan this time ;-)
> >>>>>>>>> As I think I explained, the use of a seqlock in real-time context  
> >>>>>>>>> when
> >>>>>>>>> the seqlock writer only happens in linux context is not  
> >>>>>>>>> probabilistic.
> >>>>>>>>> It will work every time the first pass.
> >>>>>>>> I still don't see why it should succeed every time: What about
> >>>>>>>> the case that the Linux kernel on CPU0 updates the data, while
> >>>>>>>> Xenomai accesses them on another CPU? This can lead to
> >>>>>>>> inconsistent data, and they must be reread on the Xenomai side.
> >>>>>>> Yeah, right. I was not thinking about SMP. But admit that in this  
> >>>>>>> case,
> >>>>>>> there will be only one retry, there is nothing pathological.
> >>>> that's right. Which makes it bounded again, so it's maybe
> >>>> the best way to go.
> >>>>>>>> I'm asking because if this case can not happen, then there's
> >>>>>>>> nothing left to to as I have the code already at hand.
> >>>>>>> You have reworked the nucleus timers handling to adapt to this new
> >>>>>>> real-time clock ?
> >>>>>> Nope. Sorry, I was a bit unclear: I'm just referring to the gtod  
> >>>>>> syscall that does the timer handling, Not any other adaptions.
> >>>>> Ok, but what good is the gtod syscall if you can not use it as a time
> >>>>> reference for other timing related services?
> >>>> it suffices for our project's current purposes ;-)
> >>>>
> >>>> But it's certainly not the full solution. Before, we
> >>>> should have a decision wrt. the design issues, but I
> >>>> won't be able to continue working on this before
> >>>> mid of next week to look at the changed required for timer
> >>>> handling and come up with code.
> >>> Ok. To summarize what we have said, here is how I see we could implement
> >>> the NTP synchronized clock fully, and portably:
> >>> 1- allocate at nucleus init time, an area in the global sem heap for
> >>> this clock house-keeping
> >>> 2- add an event to the I-pipe patch when vsyscall_update is called
> >>> 3- implement the nucleus callback for the I-pipe event which copies
> >>> relevant data with our own version of seqlock called with hardware irqs
> >>> off, to the area allocated in 1 if the current clock source is the tsc
> >>> 4- rework the nucleus clocks and timers handling to use these data
> >>> 5- pass the offset of the data allocated in 1 to user-space through the
> >>> xnsysinfo, or xnfeatinfo structures
> >>> 6- rework clock_gettime to use these data, using the user-space
> >>> counterpart of the seqlock used in 3
> >>>
> >>> The real hard work is 4, and note that something which I did not mention
> >>> yesterday, is that we not only have to change the real-time clock
> >>> implementation, we also have to change the monotonic clock
> >>> implementation, otherwise the two clocks will drift apart.
> >>>
> >>> I think making such a change now is unreasonable.
> >>>
> >>> So, solution 1, we can implement 5, passing a null offset to mean that
> >>> the support is unimplemented by the kernel and not even use it in
> >>> user-space. Keeping the work for later in the 2.5 life cycle.
> >>>
> >>> Solution 2, we keep this change for 3.0.
> >>>
> >>> Solution 3, we implement a way to read that clock without synchronizing
> >>> the nucleus with it (that is, everything but 4). One way to do this,
> >>> which I do not like, is to add a dummy clock id to the posix skin, for
> >>> instance CLOCK_LINUX_NTP_REALTIME, and implement the clock reading for
> >>> that clock id in clock_gettime. This clock id, when passed to any other
> >>> service, causes EINVAL to be returned, making it clear that this clock
> >>> can not be used for anything else. Note that if we do that, even if we
> >>> implement the full support later, we will have to keep that dummy clock
> >>> id forever.
> >>>
> >>> My preference goes to solution 1. Philippe, what do you think?
> >> Way too late for this kind of change; 2.5.0 will be out this month.
> >> Timer related issues are prone to introduce nasty subtle regressions.
> >> Let's plan this for 3.0. The earlier 2.5.0 will be out, the earlier 3.0
> >> will start.
> > 
> > No, this was not about turning the nucleus timer handling upside down
> > for 2.5.0, this was first of all about establishing the required kernel
> > to user space ABI for 2.5. Can we agree on the latter?
> 
> Ok. It is solution 1 then. In order to get things evolving more easily
> than my first solution, here is what is proposed.
> 
> We define a structure which will be shared between kernel and user,
> which contains all data exported by kernel to user, as well as flags
> indicating which member of the structure is available. The definition of
> the struct for 2.5.0 will be:
> 
> struct xnshared  {
> 	unsigned long long features;
> };
> 
> This struct will be allocated in the global sem heap, and features will
> be null for the time being.
> 
> Every time we need to share some data between kernel and user (including
> for the ntp support), we will add data to the structure, and use a
> "features" bit to mean that the data are available from kernel. It will
> work as long as we add data from release to release and never remove it.
> 
> So, for 2.5.0, the xnshared structure will be allocated, but the
> features member will be null. We will pass the offset of this structure
> in the global sem heap in the sysinfo structure.
> 

Go for this. Ack.

-- 
Philippe.

Re: [Xenomai-core] Realtime gettimeofday()

[Wolfgang Mauerer]

Quoting Philippe Gerum <rpm@xenomai.org>:

> On Mon, 2009-12-07 at 14:43 +0100, Gilles Chanteperdrix wrote:
>> Jan Kiszka wrote:
>> > Philippe Gerum wrote:
>> >> On Thu, 2009-12-03 at 22:42 +0100, Gilles Chanteperdrix wrote:
>> >>> Wolfgang Mauerer wrote:
>> >>>> Gilles Chanteperdrix wrote:
>> >>>>> Wolfgang Mauerer wrote:

>> We define a structure which will be shared between kernel and user,
>> which contains all data exported by kernel to user, as well as flags
>> indicating which member of the structure is available. The definition of
>> the struct for 2.5.0 will be:
>>
>> struct xnshared  {
>> 	unsigned long long features;
>> };
>>
>> This struct will be allocated in the global sem heap, and features will
>> be null for the time being.
>>
>> Every time we need to share some data between kernel and user (including
>> for the ntp support), we will add data to the structure, and use a
>> "features" bit to mean that the data are available from kernel. It will
>> work as long as we add data from release to release and never remove it.
>>
>> So, for 2.5.0, the xnshared structure will be allocated, but the
>> features member will be null. We will pass the offset of this structure
>> in the global sem heap in the sysinfo structure.
>>
>
> Go for this. Ack.

That's also fine for me.

Best, Wolfgang

Re: [Xenomai-core] Realtime gettimeofday()

[Richard Cochran]

On Wed, Dec 02, 2009 at 02:23:39PM +0100, Gilles Chanteperdrix wrote:
> This issue has been discussed several times, but never a lot. We have a
> simple solution: starting with 2.4 (if I remember correctly) xenomai
> provides clock_settime, so you can simply rely on clock_gettime, and
> call clock_settime from time to time to resync the Xenomai clock with
> Linux NTP-enhanced clock. This is not pretty, you get a drift increasing
> over time, then suddenly been reset. But I guess this has been enough
> for current users, until now. You control the maximum drift by how often
> you call clock_settime.

Well, it looks like I will need to use this method, too, until the
"real" solution comes along. But that leaves me with more questions:

1. My program is written using the native skin, and I need to time
stamp some data while in primary mode. Can I safely call the posix
skin's clock_gettime() function from my native thread?

2. I will need a non-critical thread to go the gettimeofday() -
clock_settime() adjustment hack once in a while. Can I safely start a
both a posix skin thread and a native thread from the same main
program?

Thanks,
Richard

Re: [Xenomai-core] Realtime gettimeofday()

[Gilles Chanteperdrix]

Richard Cochran wrote:
> On Wed, Dec 02, 2009 at 02:23:39PM +0100, Gilles Chanteperdrix wrote:
>> This issue has been discussed several times, but never a lot. We have a
>> simple solution: starting with 2.4 (if I remember correctly) xenomai
>> provides clock_settime, so you can simply rely on clock_gettime, and
>> call clock_settime from time to time to resync the Xenomai clock with
>> Linux NTP-enhanced clock. This is not pretty, you get a drift increasing
>> over time, then suddenly been reset. But I guess this has been enough
>> for current users, until now. You control the maximum drift by how often
>> you call clock_settime.
> 
> Well, it looks like I will need to use this method, too, until the
> "real" solution comes along. But that leaves me with more questions:
> 
> 1. My program is written using the native skin, and I need to time
> stamp some data while in primary mode. Can I safely call the posix
> skin's clock_gettime() function from my native thread?

Yes.

> 
> 2. I will need a non-critical thread to go the gettimeofday() -
> clock_settime() adjustment hack once in a while. Can I safely start a
> both a posix skin thread and a native thread from the same main
> program?

Yes. But you should better use __real_clock_gettime than gettimeofday,
since it returns a struct timespec which you may feed into clock_settime
without any further adjustments.

-- 
                                          Gilles