xen-netfront does not properly transmit forwarded packets

[Rafal Wojtczuk <rafal@invisiblethingslab.com>]

Hello,

There is a very weird issue with xen-netfront (I think it is the frontend
problem, not backend). The problem manifests itself with drivers from the SUSE 
kernel-xen-2.6.34.1; I don't know whether it affects vanilla code as well.
For completeness, xen is 3.4.3, all 64bit.

The problem seems to be - xen-netfront does not properly transmit forwarded
packets (locally generated packets are txed fine).

The network looks like this (of course eth0s are xen-netfront) :

testVM             FirewallVM                NetVM
|  eth0  | <---> | vifF.0 eth0 | <---> | vifN.0 wlan0 | <---> Internet

If I do "ping someInternetIP" in FirewallVM, "tcpdump -n -i eth0" 
running in FirewallVM shows outgoing icmp packets, and "tcpdump -n -i vifN.0"
running in NetVM shows incoming packets - all fine.

If I do "ping someInternetIP" in testVM, packets arrive fine on vifF.0 and
are SNATed. Then "tcpdump -n -i eth0" running in FirewallVM shows outgoing icmp 
packets, BUT "tcpdump -n -i vifN.0" running in NetVM shows NOTHING.

The important thing is that during the latter experiment, the /proc/interrupts
line for vifN.0 shows one new interrupt per second - so vifN.0 is notified by
FirewallVM's eth0 about packet transmission, yet packets are not seen by
vifN.0. The TX bytes counter for FirewallVM's eth0 increases normally; no errors
reported by any interface; nothing in the logs.

In case it matters: there is no bridging used at all, just "bare" vifX.Y. Proxy 
arp is activated for both vifs. No IP is assigned to vifs. Turning SNAT off
in FirewallVM does not change anything. The issue has been reproduced by two
different persons on two different machines.

Does anyone have an idea why this is happening ? What is the difference in
frontend's handling of forwarded packets in comparison to locally generated
ones ? Maybe some function does not work properly in interrupt context ?
I guess not many people use netfront in a router machine, so this issue may
have survived unnoticed for a long time.

Regards,
RW