[Qemu-devel] Redundant VDE network

[Qemu-devel] Redundant VDE network

[Dmitry Antipov]

(This is a partial repost from qemu-discuss@ list since I'm suspecting a bug)

I'm using QEMU 2.1.1 to emulate SPARC system and have vde network between two
VMs and host system, organized as shown:

               host
               tap0
         + 192.168.100.254 +
         |                 |
         |                 |
         vm0               vm1
         eth0              eth0
     192.168.100.1 --- 192.168.100.2

On host, I'm running vde_switch and extra stuff as:

vde_switch -tap tap0 -daemon -mod 660 -group [group]
ip addr add 192.168.100.254/24 dev tap0
ip link set tap0 up
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -s 192.168.100.0/24 -o [external network iface] -j MASQUERADE

VMs are running with:

qemu-system-sparc -m 256 -net nic -net vde -hda vm0.img
qemu-system-sparc -m 256 -net nic -net vde -hda vm1.img

Everything looks good, but...

...the problem: I need to build redundant network by using the same method, i.e.
improve the network shown above with:

               host
               tap1
         + 192.168.101.254 +
         |                 |
         |                 |
         vm0               vm1
         eth1              eth1
     192.168.101.1 --- 192.168.101.2

I'm trying to run two vde switches:

vde_switch -sock /tmp/vde0 -tap tap0 -daemon -mod 660 -group [group]
vde_switch -sock /tmp/vde1 -tap tap1 -daemon -mod 660 -group [group]

and run VMs with:

qemu-system-sparc -m 256 -net nic,vlan=0 -net vde,sock=/tmp/vde0,vlan=0 -net nic,vlan=1 -net vde,sock=/tmp/vde1,vlan=1 -hda vm0.img

but the result is:

Warning: hub port hub1port0 has no peer
Warning: vlan 1 with no nics
Warning: netdev hub1port0 has no peer
Warning: requested NIC (anonymous, model unspecified) was not created (not supported by this machine?)

Is this a bug? In general, what's the best method to build redundant network?

Thanks in advance,
Dmitry

Re: [Qemu-devel] Redundant VDE network

[Stefan Hajnoczi]

On Mon, Dec 8, 2014 at 3:00 PM, Dmitry Antipov <dmantipov@yandex.ru> wrote:
> (This is a partial repost from qemu-discuss@ list since I'm suspecting a
> bug)
>
> I'm using QEMU 2.1.1 to emulate SPARC system and have vde network between
> two
> VMs and host system, organized as shown:
>
>               host
>               tap0
>         + 192.168.100.254 +
>         |                 |
>         |                 |
>         vm0               vm1
>         eth0              eth0
>     192.168.100.1 --- 192.168.100.2
>
> On host, I'm running vde_switch and extra stuff as:
>
> vde_switch -tap tap0 -daemon -mod 660 -group [group]
> ip addr add 192.168.100.254/24 dev tap0
> ip link set tap0 up
> echo 1 > /proc/sys/net/ipv4/ip_forward
> iptables -t nat -A POSTROUTING -s 192.168.100.0/24 -o [external network
> iface] -j MASQUERADE
>
> VMs are running with:
>
> qemu-system-sparc -m 256 -net nic -net vde -hda vm0.img
> qemu-system-sparc -m 256 -net nic -net vde -hda vm1.img
>
> Everything looks good, but...
>
> ...the problem: I need to build redundant network by using the same method,
> i.e.
> improve the network shown above with:

If you care about reliability, be aware that VDE is a seldomly used
feature.  There are probably more bugs lurking there than the popular
tap networking code path.

>
>               host
>               tap1
>         + 192.168.101.254 +
>         |                 |
>         |                 |
>         vm0               vm1
>         eth1              eth1
>     192.168.101.1 --- 192.168.101.2
>
> I'm trying to run two vde switches:
>
> vde_switch -sock /tmp/vde0 -tap tap0 -daemon -mod 660 -group [group]
> vde_switch -sock /tmp/vde1 -tap tap1 -daemon -mod 660 -group [group]
>
> and run VMs with:
>
> qemu-system-sparc -m 256 -net nic,vlan=0 -net vde,sock=/tmp/vde0,vlan=0 -net
> nic,vlan=1 -net vde,sock=/tmp/vde1,vlan=1 -hda vm0.img

Try:

qemu-system-sparc -m 256 \
    -netdev vde,sock=/tmp/vde0,id=vde0 \
    -device ne2k_pci,netdev=vde0 \
    -netdev vde,sock=/tmp/vde1,id=vde1 \
    -device ne2k_pci,netdev=vde1 \
    -hda vm0.img

The is the newer syntax and it should work better.

> but the result is:
>
> Warning: hub port hub1port0 has no peer
> Warning: vlan 1 with no nics
> Warning: netdev hub1port0 has no peer
> Warning: requested NIC (anonymous, model unspecified) was not created (not
> supported by this machine?)
>
> Is this a bug?

Yes, this looks like a bug in the legacy -net option code.

Stefan

Re: [Qemu-devel] Redundant VDE network

[Dmitry Antipov]

On 12/08/2014 08:53 PM, Stefan Hajnoczi wrote:

> Try:
>
> qemu-system-sparc -m 256 \
>      -netdev vde,sock=/tmp/vde0,id=vde0 \
>      -device ne2k_pci,netdev=vde0 \
>      -netdev vde,sock=/tmp/vde1,id=vde1 \
>      -device ne2k_pci,netdev=vde1 \
>      -hda vm0.img
>
> The is the newer syntax and it should work better.

qemu-system-sparc -m 256 \
        -netdev vde,sock=/tmp/vde0,id=vde0 -device ne2k_pci,netdev=vde0 \
        -netdev vde,sock=/tmp/vde1,id=vde1 -device ne2k_pci,netdev=vde1 \
        -hda vm0.img

produces:

qemu-system-sparc: -device ne2k_pci,netdev=vde0: 'ne2k_pci' is not a valid device model name

IIUC hw/net/lance.c is the only emulated NIC on SPARC systems; but

qemu-system-sparc -m 256 \
         -netdev vde,sock=/tmp/vde0,id=vde0 -device lance,netdev=vde0 \
         -netdev vde,sock=/tmp/vde1,id=vde1 -device lance,netdev=vde1 \
         -hda vm0.img

produces totally cryptic:

qemu-system-sparc: -device lance,netdev=vde0: Parameter 'driver' expects pluggable device type

Dmitry

Re: [Qemu-devel] Redundant VDE network

[Dmitry Antipov]

On 12/08/2014 08:53 PM, Stefan Hajnoczi wrote:

> Try:
>
> qemu-system-sparc -m 256 \
>      -netdev vde,sock=/tmp/vde0,id=vde0 \
>      -device ne2k_pci,netdev=vde0 \
>      -netdev vde,sock=/tmp/vde1,id=vde1 \
>      -device ne2k_pci,netdev=vde1 \
>      -hda vm0.img

This works just fine for qemu-system-x86, but not for SPARC with lance NIC:

qemu-system-sparc -m 256 \
       -netdev vde,sock=/tmp/vde0,id=vde0 -device lance,netdev=vde0 \
       -netdev vde,sock=/tmp/vde1,id=vde1 -device lance,netdev=vde1 \
       -hda vm0.img

qemu-system-sparc: -device lance,netdev=vde0: Parameter 'driver' expects pluggable device type

It looks like PCI/sysbus emulation is incomplete/broken on SPARC.
Assuming that x86 is OK, what SPARC-specific stuff should I check?

Thanks,
Dmitry

Re: [Qemu-devel] Redundant VDE network

[Mark Cave-Ayland]

On 09/12/14 06:29, Dmitry Antipov wrote:

> On 12/08/2014 08:53 PM, Stefan Hajnoczi wrote:
> 
>> Try:
>>
>> qemu-system-sparc -m 256 \
>>      -netdev vde,sock=/tmp/vde0,id=vde0 \
>>      -device ne2k_pci,netdev=vde0 \
>>      -netdev vde,sock=/tmp/vde1,id=vde1 \
>>      -device ne2k_pci,netdev=vde1 \
>>      -hda vm0.img
> 
> This works just fine for qemu-system-x86, but not for SPARC with lance NIC:
> 
> qemu-system-sparc -m 256 \
>       -netdev vde,sock=/tmp/vde0,id=vde0 -device lance,netdev=vde0 \
>       -netdev vde,sock=/tmp/vde1,id=vde1 -device lance,netdev=vde1 \
>       -hda vm0.img
> 
> qemu-system-sparc: -device lance,netdev=vde0: Parameter 'driver' expects
> pluggable device type
> 
> It looks like PCI/sysbus emulation is incomplete/broken on SPARC.
> Assuming that x86 is OK, what SPARC-specific stuff should I check?

It's not that the emulation is broken (it's actually working as
expected), it's due to the way in which the BIOS probes sbus devices.

On real hardware, the device tree is generated by running bytecode
within the ROM contained upon each card by probing a known offset within
each slot, which is exactly how the cg3 and tcx cards are detected upon
boot.

However the since the lance device is on-board, it is hard-coded at a
fixed address in both Sun's OBP and OpenBIOS rather than being probed
via execution of an external ROM.

It should be a fairly trivial exercise to tweak QEMU and OpenBIOS to add
a second lance instance at a spare sbus slot address, but as you can see
it would require some extra smarts to dynamically patch a compiled ROM
within QEMU to insert the relevant properties from the device tree when
trying to add another network card via -device.


ATB,

Mark.

Re: [Qemu-devel] Redundant VDE network

[Stefan Hajnoczi]

On Tue, Dec 9, 2014 at 6:29 AM, Dmitry Antipov <dmantipov@yandex.ru> wrote:
> On 12/08/2014 08:53 PM, Stefan Hajnoczi wrote:
>
>> Try:
>>
>> qemu-system-sparc -m 256 \
>>      -netdev vde,sock=/tmp/vde0,id=vde0 \
>>      -device ne2k_pci,netdev=vde0 \
>>      -netdev vde,sock=/tmp/vde1,id=vde1 \
>>      -device ne2k_pci,netdev=vde1 \
>>      -hda vm0.img
>
>
> This works just fine for qemu-system-x86, but not for SPARC with lance NIC:
>
> qemu-system-sparc -m 256 \
>       -netdev vde,sock=/tmp/vde0,id=vde0 -device lance,netdev=vde0 \
>       -netdev vde,sock=/tmp/vde1,id=vde1 -device lance,netdev=vde1 \
>       -hda vm0.img
>
> qemu-system-sparc: -device lance,netdev=vde0: Parameter 'driver' expects
> pluggable device type
>
> It looks like PCI/sysbus emulation is incomplete/broken on SPARC.
> Assuming that x86 is OK, what SPARC-specific stuff should I check?

Is there a reason why you are using the old 32-bit SPARC machine types
instead of the more modern 64-bit SPARC machine types?

$ sparc-softmmu/qemu-system-sparc -M help
Supported machines are:
LX                   Sun4m platform, SPARCstation LX
SPARCClassic         Sun4m platform, SPARCClassic
SPARCbook            Sun4m platform, SPARCbook
SS-10                Sun4m platform, SPARCstation 10
SS-20                Sun4m platform, SPARCstation 20
SS-4                 Sun4m platform, SPARCstation 4
SS-5                 Sun4m platform, SPARCstation 5 (default)
SS-600MP             Sun4m platform, SPARCserver 600MP
Voyager              Sun4m platform, SPARCstation Voyager
leon3_generic        Leon-3 generic
none                 empty machine

$ sparc64-softmmu/qemu-system-sparc64 -M help
Supported machines are:
Niagara              Sun4v platform, Niagara
none                 empty machine
sun4u                Sun4u platform (default)
sun4v                Sun4v platform

I guess the SS-5 default machine type you are running doesn't have a PCI bus?

By the way, I have not tested the command-line because Fedora does not
have a vde package.  You are in niche territory with both SPARC and
vde, expect to either debug things yourself or file bugs (they may not
be addressed in a timely fashion).  Just want to set expectations for
what you will find.

Stefan

Re: [Qemu-devel] Redundant VDE network

[Dmitry Antipov]

On 12/09/2014 12:37 PM, Stefan Hajnoczi wrote:

> Is there a reason why you are using the old 32-bit SPARC machine types
> instead of the more modern 64-bit SPARC machine types?

Unfortunately yes, this is the closest match to the real (legacy)
hardware I need to use :-(.

> I guess the SS-5 default machine type you are running doesn't have a PCI bus?

Yes.

> By the way, I have not tested the command-line because Fedora does not
> have a vde package.

FYI, there is one I build myself: http://37.139.80.10/tmp/vde-2.3.2-0.587.fc20.src.rpm

Dmitry