Guest 8021q VLAN tags and macvtap

[Matthew Rosato <mjrosato@linux.vnet.ibm.com>]

Prior to commit 6acf54f1cf (macvtap: Add support of packet capture on
macvtap device), I was able to setup an environment where qemu guests
connected via macvtaps (eg, guest eth0 --> host macvtap0 --> host eth0)
could configure guest 8021q tagging and communicate with each other over
the vlan, with the guests being responsible for tagging/untagging. I
accomplished this by configuring the desired vlan id on both the guest
interface (guest eth0.123) as well as by adding the vlan to the
underlying host macvtap (macvtap0.123).  Configuring the id on the
macvtap was done to allow the tagged packets to get past the lowerdev.

Now, post 6acf54f1cf, guest-tagged traffic never arrives on the target
guest (untagged traffic is fine).  I did some tracing as a tagged packet
comes in on the target host, summarized:

1) __netif_receive_skb calls untag_vlan, vlan_tci is stashed,
vlan_do_receive returns false
2) rx_handler (macvlan_handle_frame) is called
3) netif_rx_ni is called, skb queued/dequeued (pass to macvtap)
4) __netif_receive_skb called again - vlan_do_receive is called
successfully, causing us to lose our stashed vlan_tci
5) rx_handler (macvtap_handle_frame) is called

My understanding is that macvtap expects packets to get untagged, and
intends to re-tag them later at macvtap_put_user.  But, because the
macvtap is vlan-enabled, the stashed vlan_tci is always gone because of
vlan_do_receive.

I think it boils down to the old macvtap_receive/forward logic not
calling vlan_do_receive, but now we do since we're using netif.  Was my
configuration wrong in the first place, or is this a bug?

FWIW, I hacked __netif_receive_skb to skip the vlan_do_receive call for
IFF_MACVLAN, and sure enough that restores my expected behavior, but
I'm not sure if that's the right solution.

Thanks,
Matt