[Xenomai-help] What has highest priority RTcap or TDMA-synchronization reception?

[Xenomai-help] What has highest priority RTcap or TDMA-synchronization reception?

[Benjamin Biegel]

What has highest priority RTcap or TDMA-synchronization reception?

We have a PC with two NICs, so one has TDMA, and one other captures with 
RTcap. If two frames arrive at the same time on both NICS, which is 
handled? Is it the first one, or does one have higher priority than the 
other?

Any help directly or where to look is appreciated.

Thanks :)

Benjamin

Re: [Xenomai-help] What has highest priority RTcap or TDMA-synchronization reception?

[Jan Kiszka]

Benjamin Biegel wrote:
> What has highest priority RTcap or TDMA-synchronization reception?
> 
> We have a PC with two NICs, so one has TDMA, and one other captures with 
> RTcap. If two frames arrive at the same time on both NICS, which is 
> handled? Is it the first one, or does one have higher priority than the 
> other?
> 
> Any help directly or where to look is appreciated.

Timestamping and packet queuing of both sources work at exactly the same
priority and even go through a common path, the Stack Manager task. So
the ordering is more or less random. But the final delivery happens at
different level: TDMA is handled at high RT prio, RTcap captured frames
are forwarded to Linux, thus are then saved at the according low prio.

But why are you running two NICs instead of attaching RTcap to the
TDMA-managed interface? Or are they connected to different Ethernets?

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux

Re: [Xenomai-help] What has highest priority RTcap or TDMA-synchronization reception?

[Benjamin Biegel]

[-- Attachment #1: Type: text/html, Size: 2236 bytes --]

Re: [Xenomai-help] What has highest priority RTcap or TDMA-synchronization reception?

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1972 bytes --]

Benjamin Biegel wrote:
> Thanks Jan
> 
> We use this setup: Two computers are RTnet-slaves synchronized to provide them common time. These two computers also have a second NIC which tap a target Ethernet network using RTcap. In this way we can measure Ethernet travel times on the target network.
> 
> This works well. The only problem is that we don't know exactly how to handle if a TDMA sync arrives at the same time as RTcap captures a frame on the target network.
> 
> We ran a test where we only read the system time from kernel space using rt_timer_read(). This reading and saving to a file takes approximately 1 us. However this time increases to approx 16 us when the reading is made when a synchronization TDMA packet is being handled.
> 
> We would like to detect when this happens so we can discard the frame travel times that collide with the TDMA sync.
> 
> I hope this makes sense

Note that the synchronization accuracy via TDMA is not in the order of a
few microseconds, rather a few 10 us. That's because no clever approach
to compensate jitters of the reception interrupt is applied. So your
worrying about the jitter caused by the second NIC actually has to be
extended by worries about jitter of other host interrupts. They might be
smaller as I-pipe will suppress non-RT IRQs far more quickly, but they
are more and can line up and cause larger jitters as well.

If you want to get rid of most of them: use a multicore box and isolate
the RTnet NIC IRQ on a dedicated core. All Linux IRQs except for the
timer IRQ can then be avoided.

And I still see no point in using a second NIC if the first one is only
_receiving_ TDMA sync frames. It's true that timestamps taken by RTcap
on outgoing frames will naturally slightly differ from those taken by a
listen-only observer - but to my understanding this scenarios does not
apply to you case. So RTcap will work just fine when attached already to
the first NIC.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] What has highest priority RTcap or TDMA-synchronization reception?

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1899 bytes --]

Jan Kiszka wrote:
> Benjamin Biegel wrote:
>> Thanks Jan
>>
>> We use this setup: Two computers are RTnet-slaves synchronized to provide them common time. These two computers also have a second NIC which tap a target Ethernet network using RTcap. In this way we can measure Ethernet travel times on the target network.
>>
>> This works well. The only problem is that we don't know exactly how to handle if a TDMA sync arrives at the same time as RTcap captures a frame on the target network.
>>
>> We ran a test where we only read the system time from kernel space using rt_timer_read(). This reading and saving to a file takes approximately 1 us. However this time increases to approx 16 us when the reading is made when a synchronization TDMA packet is being handled.
>>
>> We would like to detect when this happens so we can discard the frame travel times that collide with the TDMA sync.
>>
>> I hope this makes sense
> 
> Note that the synchronization accuracy via TDMA is not in the order of a
> few microseconds, rather a few 10 us. That's because no clever approach
> to compensate jitters of the reception interrupt is applied. So your
> worrying about the jitter caused by the second NIC actually has to be
> extended by worries about jitter of other host interrupts. They might be
> smaller as I-pipe will suppress non-RT IRQs far more quickly, but they
> are more and can line up and cause larger jitters as well.
> 
> If you want to get rid of most of them: use a multicore box and isolate
> the RTnet NIC IRQ on a dedicated core. All Linux IRQs except for the
> timer IRQ can then be avoided.

Well, there are actually more unavoidable Linux IRQs: You may still see
occasional rescheduling IRQs, and some other IPIs may be unavoidable
with current Linux as well. So no perfect world here, but at least some
sources of IRQ jitters can be avoided.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

[Xenomai-help] How to capture frames with RTcap without TDMA?

[Benjamin Biegel]

Hello all

We want to send data from a NIC and capture what we send using RTcap, 
but we do not want any TDMA discipline running.

So we want the VNIC but not TDMA running. The solution seems to be using 
nomac.

My attempt was to exchange 'insmod ../tdma.ko' with 'insmod ../nomac.ko' 
in the rtnet bash script, but it gives an '-1 Unknown symbol in module' 
error. How then is the nomac-discipline loaded?

I assume this module somehow must be loaded, where after i run 'nomaccfg 
rteth1attach'?

Thanks for any help

Benjamin

[Xenomai-help] Solved - Re: How to capture frames with RTcap without TDMA?

[Benjamin Biegel]

Solution was to delete all old modules before re-installing RTnet (with 
the parameter --enable-nomac).

Benjamin

Benjamin Biegel wrote:
> Hello all
>
> We want to send data from a NIC and capture what we send using RTcap, 
> but we do not want any TDMA discipline running.
>
> So we want the VNIC but not TDMA running. The solution seems to be using 
> nomac.
>
> My attempt was to exchange 'insmod ../tdma.ko' with 'insmod ../nomac.ko' 
> in the rtnet bash script, but it gives an '-1 Unknown symbol in module' 
> error. How then is the nomac-discipline loaded?
>
> I assume this module somehow must be loaded, where after i run 'nomaccfg 
> rteth1attach'?
>
> Thanks for any help
>
> Benjamin
>
> _______________________________________________
> Xenomai-help mailing list
> Xenomai-help@domain.hid
> https://mail.gna.org/listinfo/xenomai-help
>   

Re: [Xenomai-help] What has highest priority RTcap or TDMA-synchronization reception?

[Benjamin Biegel]

[-- Attachment #1: Type: text/html, Size: 3301 bytes --]

Re: [Xenomai-help] What has highest priority RTcap or TDMA-synchronization reception?

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 1520 bytes --]

Benjamin Biegel wrote:
> Reading 'Life With Adeos' casts some light over the whole interrupt question. 
> However I have some further questions.
> 
> Why is it that the 'host interrupts' that you (Jan) mention below, can stall and 
> cause jitter in the xenomai threads, when the host interrupts have not been 
> given xenomai domain priorities? If such a host interrupt is played, Life With 
> Adeos makes it sound that this interrupt is preemptable, given an interrupt with 
> higher priority occurs.

Not sure what you mean with "stall", but even low-prio interrupts still
involve basic I-pipe activity to acknowledge and record them when they
first occur during high-prio Xenomai activities. And this can cause
(though small) disturbances to the real-time jobs.

> 
> Is it true that when RTcap is enabled, then every incoming frame will play an 
> interrupt of highest possible priority (real time nucleus), as to timestamp the 
> incoming frame?

This has nothing to do with RTcap, it is a property of RTnet NIC
drivers. Their RX interrupt handlers will run at high prio and perform
the timestamping. RTcap is just pushing all frames into Linux space for
collecting them via libpcap etc.

> 
> Is it true, that when a TDMA synchronization frame arrives, this will also play 
> an interrupt of highest possible priority (real time nucleus), as to calculate 
> the offset between local and global clock?

Same IRQ (or if there are >1 NIC: IRQ of identical priority) as above.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]

Re: [Xenomai-help] What has highest priority RTcap or TDMA-synchronization reception?

[Benjamin Biegel]

[-- Attachment #1: Type: text/html, Size: 3021 bytes --]

Re: [Xenomai-help] What has highest priority RTcap or TDMA-synchronization reception?

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 963 bytes --]

Benjamin Biegel wrote:
> Again thanks!
> 
> I will look into the RTnet isolation. How would you suggest I obtain knowledge 
> about multicore box etc?

linux/Documentation/cgroups/cpusets.txt and
linux/Documentation/IRQ-affinity.txt for the beginning. Xenomai manages
affinity via explicit API calls (not done by RTnet, though) or
/proc/xenomai/affinity.

> 
> Just to explain the use of two NICs, if I can. The reason is that our goal is to 
> measure travel times on any Ethernet, without altering this Ethernet. To do this 
> we plug in two transparent y-splitters the two places we want to measure at 
> ravel time between (e.g. on either side of a switch). These two y-splitter 
> outputs are fed to two NICs on each PC. They cannot be plugged into the same NIC 
> as the TDMA is on, then it would be through a switch, which would increase 
> further delay as I see it...

I see, they are attached to quasi different Ethernets.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 257 bytes --]