* Networking latency issue on 2.6.33.7-rt29 kernel
@ 2010-11-23 17:51 Jean-Michel Hautbois
[not found] ` <0016e65a0784be92790495bd73d5@google.com>
0 siblings, 1 reply; 4+ messages in thread
From: Jean-Michel Hautbois @ 2010-11-23 17:51 UTC (permalink / raw)
To: linux-rt-users; +Cc: rostedt
Hi,
I have a latency issue using the linux-2.6.33.7-rt29 kernel.
Here is my setup :
| Board 1| | Board 2|
PC --> --> eth->wifi->wifi->eth --> --> PC
|_ tap ----------DAG------- tap_|
The measure is done thanks to external TAP devices.
The DAG card is on another PC and does the measure.
But it is an "out of the box" measure and this is done only after I
stop the capture by merging pcap files and using timestamps.
The issue, now...
As you can see on the following two pictures, at least two events
occur during my test.
http://rostedt.homelinux.com/rt-jitter/latencies_1500us_rt_cable_noip7.png
http://rostedt.homelinux.com/rt-jitter/latencies_1500us_rt_cable_noip8.png
Each graph is representing the latency in µs vs. the packet number.
Each packet is separated from the previous one by 1500µs.
I don't have any IP address configured (I removed the loopback and all
other interfaces are configured using ifconfig 0.0.0.0).
The problem occurs in PREEMPT_RT and PREEMPT_DESKTOP.
The period is bigger when I change HZ (I am currently using HZ=1000).
Hrtimers are configured, but even when not using it, the problem is the same.
I have done many many tests, removing timers, changing frequencies, etc.
I also tried to change priority of my processes associated with
sirq-rx, sirq-tx, and the interrupts.
I can't find the cause.
If you have *any* idea, thanks in advance !
Here are the timers after the test associated to the picture named
"latencies_1500us_rt_cable_noip8.png" :
Board 1 (eth -> wifi) :
-----------------------
Sample period: 1654.752 s
1654753, 4 sirq-timer/0 hrtimer_start_range_ns (tick_sched_timer)
1655, 4 sirq-timer/0 __enqueue_rt_entity (sched_rt_period_timer)
1649, 15 events/0 phy_state_machine (delayed_work_timer_fn)
1650, 15 events/0 msleep (process_timeout)
824, 15 events/0 msleep (process_timeout)
275, 4 sirq-timer/0 arm_supers_timer (sync_supers_timer_fn)
330, 80 bdi-default bdi_forker_task (process_timeout)
109D, 15 events/0 neigh_periodic_work (delayed_work_timer_fn)
27D, 15 events/0 rt_worker_func (delayed_work_timer_fn)
26, 4 sirq-timer/0 dev_watchdog (dev_watchdog)
26, 4 sirq-timer/0 dev_watchdog (dev_watchdog)
13, 4 sirq-timer/0 peer_check_expire (peer_check_expire)
5, 15 events/0 rekey_seq_generator (delayed_work_timer_fn)
2, 4 sirq-timer/0 inet_frag_secret_rebuild
(inet_frag_secret_rebuild)
1D, 4 sirq-timer/0 rt_secret_rebuild (rt_secret_rebuild)
1661345 total events, 1003.107 events/sec
Board 2 (wifi -> eth) :
-----------------------
Sample period: 1649.636 s
1649636, 4 sirq-timer/0 hrtimer_start_range_ns (tick_sched_timer)
1650, 4 sirq-timer/0 __enqueue_rt_entity (sched_rt_period_timer)
1644, 15 events/0 phy_state_machine (delayed_work_timer_fn)
1644, 15 events/0 msleep (process_timeout)
822, 15 events/0 msleep (process_timeout)
329, 80 bdi-default bdi_forker_task (process_timeout)
274, 4 sirq-timer/0 arm_supers_timer (sync_supers_timer_fn)
108D, 15 events/0 neigh_periodic_work (delayed_work_timer_fn)
27, 4 sirq-timer/0 dev_watchdog (dev_watchdog)
27, 4 sirq-timer/0 dev_watchdog (dev_watchdog)
26D, 15 events/0 rt_worker_func (delayed_work_timer_fn)
13, 4 sirq-timer/0 peer_check_expire (peer_check_expire)
4, 15 events/0 rekey_seq_generator (delayed_work_timer_fn)
1, 4 sirq-timer/0 inet_frag_secret_rebuild
(inet_frag_secret_rebuild)
1D, 4 sirq-timer/0 rt_secret_rebuild (rt_secret_rebuild)
1656207 total events, 1003.603 events/sec
And here is my ifconfig on each board :
Board 1 (eth -> wifi) :
-----------------------
ath0 Link encap:Ethernet HWaddr 00:0B:6B:2F:9F:8A
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:3866892 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:777245292 (741.2 MiB)
br0 Link encap:Ethernet HWaddr 00:0B:6B:2F:9F:8A
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth0 Link encap:Ethernet HWaddr 00:10:02:0F:B8:06
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3866892 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:792712860 (755.9 MiB) TX bytes:0 (0.0 B)
Interrupt:133 Base address:0x3000
wifi0 Link encap:UNSPEC HWaddr
00-0B-6B-2F-9F-8A-00-00-00-00-00-00-00-00-00-00
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:3866892 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:199
RX bytes:0 (0.0 B) TX bytes:912586512 (870.3 MiB)
Interrupt:66
Board 2 (wifi -> eth) :
-----------------------
ath0 Link encap:Ethernet HWaddr 00:0B:6B:2F:A0:14
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3818800 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:767578800 (732.0 MiB) TX bytes:0 (0.0 B)
br0 Link encap:Ethernet HWaddr 00:0B:6B:2F:A0:14
UP BROADCAST RUNNING NOARP MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth0 Link encap:Ethernet HWaddr 00:10:02:0F:B8:05
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:3818800 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:782854000 (746.5 MiB)
Interrupt:133 Base address:0x3000
wifi0 Link encap:UNSPEC HWaddr
00-0B-6B-2F-A0-14-00-00-00-00-00-00-00-00-00-00
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3818800 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:199
RX bytes:901236800 (859.4 MiB) TX bytes:0 (0.0 B)
Interrupt:66
JM
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 4+ messages in thread[parent not found: <0016e65a0784be92790495bd73d5@google.com>]
* Re: Networking latency issue on 2.6.33.7-rt29 kernel [not found] ` <0016e65a0784be92790495bd73d5@google.com> @ 2010-11-24 8:54 ` Jean-Michel Hautbois 2010-11-24 13:35 ` Jean-Michel Hautbois 0 siblings, 1 reply; 4+ messages in thread From: Jean-Michel Hautbois @ 2010-11-24 8:54 UTC (permalink / raw) To: linux-rt-users; +Cc: rostedt 2010/11/23 <jhautbois@gmail.com>: > Some precisions : > > I am working on a MPC5200 board. > Packets are transfered thanks to a transparent bridge. > This is an L2 proprietary protocol. > > Le , Jean-Michel Hautbois <jhautbois@gmail.com> a écrit : >> Hi, >> >> >> >> I have a latency issue using the linux-2.6.33.7-rt29 kernel. >> >> Here is my setup : >> >> >> >> | Board 1| | Board 2| >> >> PC --> --> eth->wifi->wifi->eth --> --> PC >> >> |_ tap ----------DAG------- tap_| >> >> >> >> The measure is done thanks to external TAP devices. >> >> >> >> The DAG card is on another PC and does the measure. >> >> But it is an "out of the box" measure and this is done only after I >> >> stop the capture by merging pcap files and using timestamps. >> >> >> >> The issue, now... >> >> >> >> As you can see on the following two pictures, at least two events >> >> occur during my test. >> >> http://rostedt.homelinux.com/rt-jitter/latencies_1500us_rt_cable_noip7.png >> >> http://rostedt.homelinux.com/rt-jitter/latencies_1500us_rt_cable_noip8.png >> >> >> >> Each graph is representing the latency in µs vs. the packet number. >> >> Each packet is separated from the previous one by 1500µs. >> >> I don't have any IP address configured (I removed the loopback and all >> >> other interfaces are configured using ifconfig 0.0.0.0). >> >> >> >> The problem occurs in PREEMPT_RT and PREEMPT_DESKTOP. >> >> The period is bigger when I change HZ (I am currently using HZ=1000). >> >> Hrtimers are configured, but even when not using it, the problem is the >> same. >> >> >> >> I have done many many tests, removing timers, changing frequencies, etc. >> >> I also tried to change priority of my processes associated with >> >> sirq-rx, sirq-tx, and the interrupts. >> >> >> >> I can't find the cause. >> >> If you have *any* idea, thanks in advance ! >> >> >> >> Here are the timers after the test associated to the picture named >> >> "latencies_1500us_rt_cable_noip8.png" : >> >> >> >> Board 1 (eth -> wifi) : >> >> ----------------------- >> >> Sample period: 1654.752 s >> >> 1654753, 4 sirq-timer/0 hrtimer_start_range_ns (tick_sched_timer) >> >> 1655, 4 sirq-timer/0 __enqueue_rt_entity (sched_rt_period_timer) >> >> 1649, 15 events/0 phy_state_machine (delayed_work_timer_fn) >> >> 1650, 15 events/0 msleep (process_timeout) >> >> 824, 15 events/0 msleep (process_timeout) >> >> 275, 4 sirq-timer/0 arm_supers_timer (sync_supers_timer_fn) >> >> 330, 80 bdi-default bdi_forker_task (process_timeout) >> >> 109D, 15 events/0 neigh_periodic_work (delayed_work_timer_fn) >> >> 27D, 15 events/0 rt_worker_func (delayed_work_timer_fn) >> >> 26, 4 sirq-timer/0 dev_watchdog (dev_watchdog) >> >> 26, 4 sirq-timer/0 dev_watchdog (dev_watchdog) >> >> 13, 4 sirq-timer/0 peer_check_expire (peer_check_expire) >> >> 5, 15 events/0 rekey_seq_generator (delayed_work_timer_fn) >> >> 2, 4 sirq-timer/0 inet_frag_secret_rebuild >> >> (inet_frag_secret_rebuild) >> >> 1D, 4 sirq-timer/0 rt_secret_rebuild (rt_secret_rebuild) >> >> 1661345 total events, 1003.107 events/sec >> >> >> >> Board 2 (wifi -> eth) : >> >> ----------------------- >> >> Sample period: 1649.636 s >> >> 1649636, 4 sirq-timer/0 hrtimer_start_range_ns (tick_sched_timer) >> >> 1650, 4 sirq-timer/0 __enqueue_rt_entity (sched_rt_period_timer) >> >> 1644, 15 events/0 phy_state_machine (delayed_work_timer_fn) >> >> 1644, 15 events/0 msleep (process_timeout) >> >> 822, 15 events/0 msleep (process_timeout) >> >> 329, 80 bdi-default bdi_forker_task (process_timeout) >> >> 274, 4 sirq-timer/0 arm_supers_timer (sync_supers_timer_fn) >> >> 108D, 15 events/0 neigh_periodic_work (delayed_work_timer_fn) >> >> 27, 4 sirq-timer/0 dev_watchdog (dev_watchdog) >> >> 27, 4 sirq-timer/0 dev_watchdog (dev_watchdog) >> >> 26D, 15 events/0 rt_worker_func (delayed_work_timer_fn) >> >> 13, 4 sirq-timer/0 peer_check_expire (peer_check_expire) >> >> 4, 15 events/0 rekey_seq_generator (delayed_work_timer_fn) >> >> 1, 4 sirq-timer/0 inet_frag_secret_rebuild >> >> (inet_frag_secret_rebuild) >> >> 1D, 4 sirq-timer/0 rt_secret_rebuild (rt_secret_rebuild) >> >> 1656207 total events, 1003.603 events/sec >> >> >> >> And here is my ifconfig on each board : >> >> >> >> Board 1 (eth -> wifi) : >> >> ----------------------- >> >> ath0 Link encap:Ethernet HWaddr 00:0B:6B:2F:9F:8A >> >> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >> >> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >> >> TX packets:3866892 errors:0 dropped:0 overruns:0 carrier:0 >> >> collisions:0 txqueuelen:0 >> >> RX bytes:0 (0.0 B) TX bytes:777245292 (741.2 MiB) >> >> >> >> br0 Link encap:Ethernet HWaddr 00:0B:6B:2F:9F:8A >> >> UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 >> >> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >> >> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >> >> collisions:0 txqueuelen:0 >> >> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) >> >> >> >> eth0 Link encap:Ethernet HWaddr 00:10:02:0F:B8:06 >> >> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >> >> RX packets:3866892 errors:0 dropped:0 overruns:0 frame:0 >> >> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >> >> collisions:0 txqueuelen:1000 >> >> RX bytes:792712860 (755.9 MiB) TX bytes:0 (0.0 B) >> >> Interrupt:133 Base address:0x3000 >> >> >> >> wifi0 Link encap:UNSPEC HWaddr >> >> 00-0B-6B-2F-9F-8A-00-00-00-00-00-00-00-00-00-00 >> >> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >> >> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >> >> TX packets:3866892 errors:0 dropped:0 overruns:0 carrier:0 >> >> collisions:0 txqueuelen:199 >> >> RX bytes:0 (0.0 B) TX bytes:912586512 (870.3 MiB) >> >> Interrupt:66 >> >> >> >> >> >> Board 2 (wifi -> eth) : >> >> ----------------------- >> >> ath0 Link encap:Ethernet HWaddr 00:0B:6B:2F:A0:14 >> >> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >> >> RX packets:3818800 errors:0 dropped:0 overruns:0 frame:0 >> >> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >> >> collisions:0 txqueuelen:0 >> >> RX bytes:767578800 (732.0 MiB) TX bytes:0 (0.0 B) >> >> >> >> br0 Link encap:Ethernet HWaddr 00:0B:6B:2F:A0:14 >> >> UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 >> >> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >> >> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >> >> collisions:0 txqueuelen:0 >> >> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) >> >> >> >> eth0 Link encap:Ethernet HWaddr 00:10:02:0F:B8:05 >> >> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >> >> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >> >> TX packets:3818800 errors:0 dropped:0 overruns:0 carrier:0 >> >> collisions:0 txqueuelen:1000 >> >> RX bytes:0 (0.0 B) TX bytes:782854000 (746.5 MiB) >> >> Interrupt:133 Base address:0x3000 >> >> >> >> wifi0 Link encap:UNSPEC HWaddr >> >> 00-0B-6B-2F-A0-14-00-00-00-00-00-00-00-00-00-00 >> >> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >> >> RX packets:3818800 errors:0 dropped:0 overruns:0 frame:0 >> >> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >> >> collisions:0 txqueuelen:199 >> >> RX bytes:901236800 (859.4 MiB) TX bytes:0 (0.0 B) >> >> Interrupt:66 >> >> >> >> JM >> I have done some tests using two different HZ values with CONFIG_PREEMPT_DESKTOP. Here are the results : With HZ=1000 http://picasaweb.google.com/lh/photo/ltb6hZvfssCFN0uF1iscP5MU3g7ObZTPqzWaaCIpUqI?feat=directlink With HZ=250 http://picasaweb.google.com/lh/photo/H9mKcM-Tarfumezbu8IJUZMU3g7ObZTPqzWaaCIpUqI?feat=directlink It seems to be related to a timer, maybe the scheduler itself... Here is my ps result (on busybox) after the HZ=250 test : PID USER TIME COMMAND 1 root 0:02 init 2 root 0:00 [kthreadd] 3 root 0:00 [sirq-high/0] 4 root 0:21 [sirq-timer/0] 5 root 0:00 [sirq-net-tx/0] 6 root 3:08 [sirq-net-rx/0] 7 root 0:00 [sirq-block/0] 8 root 0:00 [sirq-block-iopo] 9 root 1:56 [sirq-tasklet/0] 10 root 0:00 [sirq-sched/0] 11 root 0:00 [sirq-hrtimer/0] 12 root 0:22 [sirq-rcu/0] 13 root 0:00 [posixcputmr/0] 14 root 0:00 [desched/0] 15 root 0:00 [events/0] 16 root 0:00 [khelper] 19 root 0:00 [async/mgr] 78 root 0:00 [sync_supers] 80 root 0:00 [bdi-default] 81 root 0:00 [kblockd/0] 105 root 0:00 [kswapd0] 106 root 0:00 [aio/0] 107 root 0:00 [crypto/0] 128 root 0:00 [mtdblockd] 188 root 0:00 [irq/144-i2c-mpc] 192 root 0:00 [kstriped] 194 root 0:00 [ksnapd] 198 root 0:00 [irq/139-mpc52xx] 216 root 0:00 [jffs2_gcd_mtd14] 221 root 0:00 /usr/bin/update 1 240 root 1:00 [irq/66-wifi%d] 257 root 0:00 [irq/133-mpc52xx] 258 root 1:10 [irq/192-mpc52xx] 259 root 0:00 [irq/193-mpc52xx] 280 root 0:00 dropbear 281 root 0:00 -/bin/sh 308 root 0:00 ps -- To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Networking latency issue on 2.6.33.7-rt29 kernel 2010-11-24 8:54 ` Jean-Michel Hautbois @ 2010-11-24 13:35 ` Jean-Michel Hautbois 2010-12-01 8:27 ` Andre Puschmann 0 siblings, 1 reply; 4+ messages in thread From: Jean-Michel Hautbois @ 2010-11-24 13:35 UTC (permalink / raw) To: linux-rt-users; +Cc: rostedt 2010/11/24 Jean-Michel Hautbois <jhautbois@gmail.com>: > 2010/11/23 <jhautbois@gmail.com>: >> Some precisions : >> >> I am working on a MPC5200 board. >> Packets are transfered thanks to a transparent bridge. >> This is an L2 proprietary protocol. >> >> Le , Jean-Michel Hautbois <jhautbois@gmail.com> a écrit : >>> Hi, >>> >>> >>> >>> I have a latency issue using the linux-2.6.33.7-rt29 kernel. >>> >>> Here is my setup : >>> >>> >>> >>> | Board 1| | Board 2| >>> >>> PC --> --> eth->wifi->wifi->eth --> --> PC >>> >>> |_ tap ----------DAG------- tap_| >>> >>> >>> >>> The measure is done thanks to external TAP devices. >>> >>> >>> >>> The DAG card is on another PC and does the measure. >>> >>> But it is an "out of the box" measure and this is done only after I >>> >>> stop the capture by merging pcap files and using timestamps. >>> >>> >>> >>> The issue, now... >>> >>> >>> >>> As you can see on the following two pictures, at least two events >>> >>> occur during my test. >>> >>> http://rostedt.homelinux.com/rt-jitter/latencies_1500us_rt_cable_noip7.png >>> >>> http://rostedt.homelinux.com/rt-jitter/latencies_1500us_rt_cable_noip8.png >>> >>> >>> >>> Each graph is representing the latency in µs vs. the packet number. >>> >>> Each packet is separated from the previous one by 1500µs. >>> >>> I don't have any IP address configured (I removed the loopback and all >>> >>> other interfaces are configured using ifconfig 0.0.0.0). >>> >>> >>> >>> The problem occurs in PREEMPT_RT and PREEMPT_DESKTOP. >>> >>> The period is bigger when I change HZ (I am currently using HZ=1000). >>> >>> Hrtimers are configured, but even when not using it, the problem is the >>> same. >>> >>> >>> >>> I have done many many tests, removing timers, changing frequencies, etc. >>> >>> I also tried to change priority of my processes associated with >>> >>> sirq-rx, sirq-tx, and the interrupts. >>> >>> >>> >>> I can't find the cause. >>> >>> If you have *any* idea, thanks in advance ! >>> >>> >>> >>> Here are the timers after the test associated to the picture named >>> >>> "latencies_1500us_rt_cable_noip8.png" : >>> >>> >>> >>> Board 1 (eth -> wifi) : >>> >>> ----------------------- >>> >>> Sample period: 1654.752 s >>> >>> 1654753, 4 sirq-timer/0 hrtimer_start_range_ns (tick_sched_timer) >>> >>> 1655, 4 sirq-timer/0 __enqueue_rt_entity (sched_rt_period_timer) >>> >>> 1649, 15 events/0 phy_state_machine (delayed_work_timer_fn) >>> >>> 1650, 15 events/0 msleep (process_timeout) >>> >>> 824, 15 events/0 msleep (process_timeout) >>> >>> 275, 4 sirq-timer/0 arm_supers_timer (sync_supers_timer_fn) >>> >>> 330, 80 bdi-default bdi_forker_task (process_timeout) >>> >>> 109D, 15 events/0 neigh_periodic_work (delayed_work_timer_fn) >>> >>> 27D, 15 events/0 rt_worker_func (delayed_work_timer_fn) >>> >>> 26, 4 sirq-timer/0 dev_watchdog (dev_watchdog) >>> >>> 26, 4 sirq-timer/0 dev_watchdog (dev_watchdog) >>> >>> 13, 4 sirq-timer/0 peer_check_expire (peer_check_expire) >>> >>> 5, 15 events/0 rekey_seq_generator (delayed_work_timer_fn) >>> >>> 2, 4 sirq-timer/0 inet_frag_secret_rebuild >>> >>> (inet_frag_secret_rebuild) >>> >>> 1D, 4 sirq-timer/0 rt_secret_rebuild (rt_secret_rebuild) >>> >>> 1661345 total events, 1003.107 events/sec >>> >>> >>> >>> Board 2 (wifi -> eth) : >>> >>> ----------------------- >>> >>> Sample period: 1649.636 s >>> >>> 1649636, 4 sirq-timer/0 hrtimer_start_range_ns (tick_sched_timer) >>> >>> 1650, 4 sirq-timer/0 __enqueue_rt_entity (sched_rt_period_timer) >>> >>> 1644, 15 events/0 phy_state_machine (delayed_work_timer_fn) >>> >>> 1644, 15 events/0 msleep (process_timeout) >>> >>> 822, 15 events/0 msleep (process_timeout) >>> >>> 329, 80 bdi-default bdi_forker_task (process_timeout) >>> >>> 274, 4 sirq-timer/0 arm_supers_timer (sync_supers_timer_fn) >>> >>> 108D, 15 events/0 neigh_periodic_work (delayed_work_timer_fn) >>> >>> 27, 4 sirq-timer/0 dev_watchdog (dev_watchdog) >>> >>> 27, 4 sirq-timer/0 dev_watchdog (dev_watchdog) >>> >>> 26D, 15 events/0 rt_worker_func (delayed_work_timer_fn) >>> >>> 13, 4 sirq-timer/0 peer_check_expire (peer_check_expire) >>> >>> 4, 15 events/0 rekey_seq_generator (delayed_work_timer_fn) >>> >>> 1, 4 sirq-timer/0 inet_frag_secret_rebuild >>> >>> (inet_frag_secret_rebuild) >>> >>> 1D, 4 sirq-timer/0 rt_secret_rebuild (rt_secret_rebuild) >>> >>> 1656207 total events, 1003.603 events/sec >>> >>> >>> >>> And here is my ifconfig on each board : >>> >>> >>> >>> Board 1 (eth -> wifi) : >>> >>> ----------------------- >>> >>> ath0 Link encap:Ethernet HWaddr 00:0B:6B:2F:9F:8A >>> >>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>> >>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >>> >>> TX packets:3866892 errors:0 dropped:0 overruns:0 carrier:0 >>> >>> collisions:0 txqueuelen:0 >>> >>> RX bytes:0 (0.0 B) TX bytes:777245292 (741.2 MiB) >>> >>> >>> >>> br0 Link encap:Ethernet HWaddr 00:0B:6B:2F:9F:8A >>> >>> UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 >>> >>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >>> >>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >>> >>> collisions:0 txqueuelen:0 >>> >>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) >>> >>> >>> >>> eth0 Link encap:Ethernet HWaddr 00:10:02:0F:B8:06 >>> >>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>> >>> RX packets:3866892 errors:0 dropped:0 overruns:0 frame:0 >>> >>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >>> >>> collisions:0 txqueuelen:1000 >>> >>> RX bytes:792712860 (755.9 MiB) TX bytes:0 (0.0 B) >>> >>> Interrupt:133 Base address:0x3000 >>> >>> >>> >>> wifi0 Link encap:UNSPEC HWaddr >>> >>> 00-0B-6B-2F-9F-8A-00-00-00-00-00-00-00-00-00-00 >>> >>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>> >>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >>> >>> TX packets:3866892 errors:0 dropped:0 overruns:0 carrier:0 >>> >>> collisions:0 txqueuelen:199 >>> >>> RX bytes:0 (0.0 B) TX bytes:912586512 (870.3 MiB) >>> >>> Interrupt:66 >>> >>> >>> >>> >>> >>> Board 2 (wifi -> eth) : >>> >>> ----------------------- >>> >>> ath0 Link encap:Ethernet HWaddr 00:0B:6B:2F:A0:14 >>> >>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>> >>> RX packets:3818800 errors:0 dropped:0 overruns:0 frame:0 >>> >>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >>> >>> collisions:0 txqueuelen:0 >>> >>> RX bytes:767578800 (732.0 MiB) TX bytes:0 (0.0 B) >>> >>> >>> >>> br0 Link encap:Ethernet HWaddr 00:0B:6B:2F:A0:14 >>> >>> UP BROADCAST RUNNING NOARP MTU:1500 Metric:1 >>> >>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >>> >>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >>> >>> collisions:0 txqueuelen:0 >>> >>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) >>> >>> >>> >>> eth0 Link encap:Ethernet HWaddr 00:10:02:0F:B8:05 >>> >>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>> >>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0 >>> >>> TX packets:3818800 errors:0 dropped:0 overruns:0 carrier:0 >>> >>> collisions:0 txqueuelen:1000 >>> >>> RX bytes:0 (0.0 B) TX bytes:782854000 (746.5 MiB) >>> >>> Interrupt:133 Base address:0x3000 >>> >>> >>> >>> wifi0 Link encap:UNSPEC HWaddr >>> >>> 00-0B-6B-2F-A0-14-00-00-00-00-00-00-00-00-00-00 >>> >>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>> >>> RX packets:3818800 errors:0 dropped:0 overruns:0 frame:0 >>> >>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 >>> >>> collisions:0 txqueuelen:199 >>> >>> RX bytes:901236800 (859.4 MiB) TX bytes:0 (0.0 B) >>> >>> Interrupt:66 >>> >>> >>> >>> JM >>> > > I have done some tests using two different HZ values with > CONFIG_PREEMPT_DESKTOP. > Here are the results : > With HZ=1000 > http://picasaweb.google.com/lh/photo/ltb6hZvfssCFN0uF1iscP5MU3g7ObZTPqzWaaCIpUqI?feat=directlink > With HZ=250 > http://picasaweb.google.com/lh/photo/H9mKcM-Tarfumezbu8IJUZMU3g7ObZTPqzWaaCIpUqI?feat=directlink > > It seems to be related to a timer, maybe the scheduler itself... > Here is my ps result (on busybox) after the HZ=250 test : > PID USER TIME COMMAND > 1 root 0:02 init > 2 root 0:00 [kthreadd] > 3 root 0:00 [sirq-high/0] > 4 root 0:21 [sirq-timer/0] > 5 root 0:00 [sirq-net-tx/0] > 6 root 3:08 [sirq-net-rx/0] > 7 root 0:00 [sirq-block/0] > 8 root 0:00 [sirq-block-iopo] > 9 root 1:56 [sirq-tasklet/0] > 10 root 0:00 [sirq-sched/0] > 11 root 0:00 [sirq-hrtimer/0] > 12 root 0:22 [sirq-rcu/0] > 13 root 0:00 [posixcputmr/0] > 14 root 0:00 [desched/0] > 15 root 0:00 [events/0] > 16 root 0:00 [khelper] > 19 root 0:00 [async/mgr] > 78 root 0:00 [sync_supers] > 80 root 0:00 [bdi-default] > 81 root 0:00 [kblockd/0] > 105 root 0:00 [kswapd0] > 106 root 0:00 [aio/0] > 107 root 0:00 [crypto/0] > 128 root 0:00 [mtdblockd] > 188 root 0:00 [irq/144-i2c-mpc] > 192 root 0:00 [kstriped] > 194 root 0:00 [ksnapd] > 198 root 0:00 [irq/139-mpc52xx] > 216 root 0:00 [jffs2_gcd_mtd14] > 221 root 0:00 /usr/bin/update 1 > 240 root 1:00 [irq/66-wifi%d] > 257 root 0:00 [irq/133-mpc52xx] > 258 root 1:10 [irq/192-mpc52xx] > 259 root 0:00 [irq/193-mpc52xx] > 280 root 0:00 dropbear > 281 root 0:00 -/bin/sh > 308 root 0:00 ps > For those who may be interested, CONFIG_TINY_RCU changes the behaviour (it was CONFIG_TREE_PREEMPT_RCU) : http://picasaweb.google.com/lh/photo/DmKJyIv04fvnN_y0wR0_9ZMU3g7ObZTPqzWaaCIpUqI?feat=directlink -- To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Networking latency issue on 2.6.33.7-rt29 kernel 2010-11-24 13:35 ` Jean-Michel Hautbois @ 2010-12-01 8:27 ` Andre Puschmann 0 siblings, 0 replies; 4+ messages in thread From: Andre Puschmann @ 2010-12-01 8:27 UTC (permalink / raw) To: linux-rt-users Hi, On 11/24/2010 02:35 PM, Jean-Michel Hautbois wrote: > For those who may be interested, CONFIG_TINY_RCU changes the behaviour > (it was CONFIG_TREE_PREEMPT_RCU) : > http://picasaweb.google.com/lh/photo/DmKJyIv04fvnN_y0wR0_9ZMU3g7ObZTPqzWaaCIpUqI?feat=directlink In init/Kconfig, CONFIG_TINY_RCU is configured to depend on "!SMP && !PREEMPT_RT". Did you just remove the !PREEMPT_RT dependency then? If so, I am afraid this may cause side effects. Regards, Andre ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2010-12-01 8:27 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-11-23 17:51 Networking latency issue on 2.6.33.7-rt29 kernel Jean-Michel Hautbois
[not found] ` <0016e65a0784be92790495bd73d5@google.com>
2010-11-24 8:54 ` Jean-Michel Hautbois
2010-11-24 13:35 ` Jean-Michel Hautbois
2010-12-01 8:27 ` Andre Puschmann
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).