* Conntrack checksumming problem with pv_ops dom0
@ 2010-04-28 13:08 Dmitriy Novikov
2010-05-11 7:42 ` Dmitry Novikov
0 siblings, 1 reply; 3+ messages in thread
From: Dmitriy Novikov @ 2010-04-28 13:08 UTC (permalink / raw)
To: xen-devel; +Cc: xen-devel
Hello.
I have 2.6.32.11 pv_ops dom0 kernel, xen 3.4.3.
dom0 has 2 x Broadcom NICs:
02:05.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704
Gigabit Ethernet (rev 10)
02:05.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704
Gigabit Ethernet (rev 10)
On second interface 4,6 tagged vlan configured and connected to
corresponding bridges:
br4 8000.0030485a308f no eth1.4
vif1.1
br6 8000.0030485a308f no eth1.6
vif1.0
domU configured with 2 interfeces (WAN,LAN) which connected to
corresponding bridges:
vif = ['mac=00:16:3e:00:06:11,bridge=br6','mac=00:16:3e:00:04:11,bridge=br4']
domU used for OpenVPN. OpenVPN runs over tun1 interface. OpenVPN
traffic comes from eth0
and forwarded to local network via eth1 interface. All checksumming
disabled on xen-netfront interfaces:
# ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
# ethtool -k eth1
Offload parameters for eth1:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
All checksumming disabled on real and vif/dom0 interfaces:
# ethtool -k eth1
Offload parameters for eth1:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
# ethtool -k vif1.1
Offload parameters for vif1.1:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
# ethtool -k vif1.0
Offload parameters for vif1.0:
rx-checksumming: off
tx-checksumming: off
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: off
large-receive-offload: off
Broadcom offloading firmware removed from /lib/firmware:
mv /lib/firmware/2.6.32.11-xen-dom0/tigon ~/fw_backup
Apr 27 18:26:09 xenon kernel: [ 0.562660] tg3.c:v3.102 (September 1, 2009)
Apr 27 18:26:09 xenon kernel: [ 0.562800] xen: registering gsi 26
triggering 0 polarity 1
Apr 27 18:26:09 xenon kernel: [ 0.562817] alloc irq_desc for 26 on node 0
Apr 27 18:26:09 xenon kernel: [ 0.562821] alloc kstat_irqs on node 0
Apr 27 18:26:09 xenon kernel: [ 0.562831] xen: --> irq=26
Apr 27 18:26:09 xenon kernel: [ 0.562841] tg3 0000:02:05.0: PCI INT
A -> GSI 26 (level, low) -> IRQ 26
...
Apr 27 18:26:09 xenon kernel: [ 0.607550] eth0: Tigon3
[partno(BCM95704A6) rev 2100] (PCIX:100MHz:64-bit) MAC address
00:30:48:5a:30:8e
Apr 27 18:26:09 xenon kernel: [ 0.607708] eth0: attached PHY is
5704 (10/100/1000Base-T Ethernet) (WireSpeed[1])
Apr 27 18:26:09 xenon kernel: [ 0.607856] eth0: RXcsums[1]
LinkChgREG[0] MIirq[0] ASF[1] TSOcap[0] <---------------RXcsums[1]
???
Apr 27 18:26:09 xenon kernel: [ 0.607965] eth0:
dma_rwctrl[769f4000] dma_mask[64-bit]
Apr 27 18:26:09 xenon kernel: [ 0.608111] xen: registering gsi 19
triggering 0 polarity 1
Apr 27 18:26:09 xenon kernel: [ 0.608128] alloc irq_desc for 19 on node 0
Apr 27 18:26:09 xenon kernel: [ 0.608132] alloc kstat_irqs on node 0
Apr 27 18:26:09 xenon kernel: [ 0.608142] xen: --> irq=19
...
Apr 27 18:26:09 xenon kernel: [ 0.615909] tg3 0000:02:05.1: PCI INT
B -> GSI 27 (level, low) -> IRQ 27
Apr 27 18:26:09 xenon kernel: [ 0.636546] eth1: Tigon3
[partno(BCM95704A6) rev 2100] (PCIX:100MHz:64-bit) MAC address
00:30:48:5a:30:8f
Apr 27 18:26:09 xenon kernel: [ 0.636704] eth1: attached PHY is
5704 (10/100/1000Base-T Ethernet) (WireSpeed[1])
Apr 27 18:26:09 xenon kernel: [ 0.636851] eth1: RXcsums[1]
LinkChgREG[0] MIirq[0] ASF[0] TSOcap[1]
Apr 27 18:26:09 xenon kernel: [ 0.636961] eth1:
dma_rwctrl[769f4000] dma_mask[64-bit]
...
Apr 27 18:26:09 xenon kernel: [ 7.340398] tg3 0000:02:05.1:
firmware: requesting tigon/tg3_tso.bin
Apr 27 18:26:09 xenon kernel: [ 7.369999] eth1: Failed to load
firmware "tigon/tg3_tso.bin"
Apr 27 18:26:09 xenon kernel: [ 7.370123] eth1: TSO capability disabled.
Before modprobing nf_conntrack_ipv4 forwarding works fine:
# tcpdump -ni tun1 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun1, link-type RAW (Raw IP), capture size 65535 bytes
15:13:19.525889 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 17, length 64
15:13:19.540332 IP 10.10.16.211 > 192.168.96.4: ICMP echo reply, id
2401, seq 17, length 64
15:13:20.526379 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 18, length 64
15:13:20.540641 IP 10.10.16.211 > 192.168.96.4: ICMP echo reply, id
2401, seq 18, length 64
# tcpdump -ni eth1 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
15:13:41.550283 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 39, length 64
15:13:41.563788 IP 10.10.16.211 > 192.168.96.4: ICMP echo reply, id
2401, seq 39, length 64
15:13:42.551962 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 40, length 64
15:13:42.565110 IP 10.10.16.211 > 192.168.96.4: ICMP echo reply, id
2401, seq 40, length 64
# tcpdump -ni vif1.1 icmp
tcpdump: WARNING: vif1.1: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vif1.1, link-type EN10MB (Ethernet), capture size 96 bytes
15:14:23.823167 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 81, length 64
15:14:23.836664 IP 10.10.16.211 > 192.168.96.4: ICMP echo reply, id
2401, seq 81, length 64
15:14:24.823795 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 82, length 64
15:14:24.836965 IP 10.10.16.211 > 192.168.96.4: ICMP echo reply, id
2401, seq 82, length 64
But after modprobing nf_conntrack_ipv4 traffic destinated to local
network disappears on vif1.1 while traffic is visible on domU's lan
interface.
A lot of "Attempting to checksum a non-TCP/UDP packet, dropping a
protocol 1 packet" messages seen on dom0.
# tcpdump -ni eth1 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
15:15:25.603809 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 143, length 64
15:15:25.618697 IP 10.10.16.211 > 192.168.96.4: ICMP echo reply, id
2401, seq 143, length 64
15:15:26.610258 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 144, length 64
15:15:26.626101 IP 10.10.16.211 > 192.168.96.4: ICMP echo reply, id
2401, seq 144, length 64
# tcpdump -ni tun1 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun1, link-type RAW (Raw IP), capture size 65535 bytes
15:15:41.680256 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 159, length 64
15:15:41.694291 IP 10.10.16.211 > 192.168.96.4: ICMP echo reply, id
2401, seq 159, length 64
15:15:42.681677 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 160, length 64
15:15:42.697395 IP 10.10.16.211 > 192.168.96.4: ICMP echo reply, id
2401, seq 160, length 64
# tcpdump -ni vif1.1 icmp
tcpdump: WARNING: vif1.1: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vif1.1, link-type EN10MB (Ethernet), capture size 96 bytes
15:15:58.036979 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 175, length 64
15:15:59.043942 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 176, length 64
15:16:00.052912 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 177, length 64
15:16:01.053850 IP 192.168.96.4 > 10.10.16.211: ICMP echo request, id
2401, seq 178, length 64
BUT when I set 'sysctl -w net.netfilter.nf_conntrack_checksum=0'
traffic starts to run again.
Any ideas?
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Conntrack checksumming problem with pv_ops dom0
2010-04-28 13:08 Conntrack checksumming problem with pv_ops dom0 Dmitriy Novikov
@ 2010-05-11 7:42 ` Dmitry Novikov
2010-05-22 11:35 ` Pasi Kärkkäinen
0 siblings, 1 reply; 3+ messages in thread
From: Dmitry Novikov @ 2010-05-11 7:42 UTC (permalink / raw)
To: xen-devel
2010/4/28 Dmitriy Novikov <dimetrios@gmail.com>:
> Hello.
> I have 2.6.32.11 pv_ops dom0 kernel, xen 3.4.3.
>
> dom0 has 2 x Broadcom NICs:
> 02:05.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 10)
> 02:05.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 10)
I had tested on other server with two Intel 82574L nics and have same problem.
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Conntrack checksumming problem with pv_ops dom0
2010-05-11 7:42 ` Dmitry Novikov
@ 2010-05-22 11:35 ` Pasi Kärkkäinen
0 siblings, 0 replies; 3+ messages in thread
From: Pasi Kärkkäinen @ 2010-05-22 11:35 UTC (permalink / raw)
To: Dmitry Novikov; +Cc: xen-devel
On Tue, May 11, 2010 at 10:42:52AM +0300, Dmitry Novikov wrote:
> 2010/4/28 Dmitriy Novikov <dimetrios@gmail.com>:
> > Hello.
> > I have 2.6.32.11 pv_ops dom0 kernel, xen 3.4.3.
> >
> > dom0 has 2 x Broadcom NICs:
> > 02:05.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 10)
> > 02:05.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 10)
>
> I had tested on other server with two Intel 82574L nics and have same problem.
>
The checksumming problems should be fixed now in the latest xen/stable-2.6.32.x tree.
It was a bug in netback driver.
-- Pasi
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2010-05-22 11:35 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-04-28 13:08 Conntrack checksumming problem with pv_ops dom0 Dmitriy Novikov
2010-05-11 7:42 ` Dmitry Novikov
2010-05-22 11:35 ` Pasi Kärkkäinen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).