[Qemu-devel] Two taps, same IP?

[Qemu-devel] Two taps, same IP?

[David Barrett]

I'm considering a tap-based alternative to the -redir patch I proposed 
earlier, but I'm just not quite getting how it works.  In particular, 
I'm able to access the webserver on one image just fine, but not the 
other: wget fails with "Connecting to 172.20.0.3:80... failed: No route 
to host."

Can you explain why and set me straight?

Specifically, I have two Debian qemu images (0 and 1), identical in all 
respects except that image0 and image1 are configured to use static IPs 
172.20.0.2 and 172.20.0.3, respectively.  I've launched both 
simultaneously with the following commands:

sudo qemu -kernel-kqemu -net nic,vlan=0 -net tap,vlan=0 image0.raw
sudo qemu -kernel-kqemu -net nic,vlan=0 -net tap,vlan=0 image1.raw

Each image is configured with the following /etc/network/interfaces:

auto lo
iface lo inet loopback
allow-hotplug eth0
iface eth0 inet static
address 172.20.0.2  <--- image1 has: address 172.20.0.3
netmask 255.255.0.0
gateway 172.20.0.1

This creates two tap interfaces (0 and 1) on the Ubuntu host, curiously 
with the same IP:

tap0      Link encap:Ethernet  HWaddr 00:ff:84:12:9d:72
           inet addr:172.20.0.1  Bcast:172.20.255.255  Mask:255.255.0.0
           inet6 addr: fe80::2ff:84ff:fe12:9d72/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:18 errors:0 dropped:0 overruns:0 frame:0
           TX packets:36 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:500
           RX bytes:1336 (1.3 KB)  TX bytes:4704 (4.5 KB)

tap1      Link encap:Ethernet  HWaddr 00:ff:af:9a:48:29
           inet addr:172.20.0.1  Bcast:172.20.255.255  Mask:255.255.0.0
           inet6 addr: fe80::2ff:afff:fe9a:4829/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:24 errors:0 dropped:0 overruns:0 frame:0
           TX packets:34 errors:0 dropped:0 overruns:0 carrier:0
           collisions:0 txqueuelen:500
           RX bytes:1656 (1.6 KB)  TX bytes:4664 (4.5 KB)

"wget http://172.20.0.2" and "wget http://172.20.0.3" each work fine 
inside their respective VMs.  But each is unable to wget the other's 
webserver.

Furthermore, and most unusual, the host is able to wget image0's 
webserver fine, but not image1.  Specifically, the second wget fails as 
follows:

david@SonOfLappy:/svn/staging$ wget http://172.20.0.3
--18:17:12--  http://172.20.0.3/
            => `index.html.1'
Connecting to 172.20.0.3:80... failed: No route to host.
david@SonOfLappy:/svn/staging$

The error message suggests some sort of routing problem, and the routing 
table is:

david@SonOfLappy:/svn/staging$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use 
Iface
68.28.57.85     *               255.255.255.255 UH    0      0        0 ppp0
172.20.0.0      *               255.255.0.0     U     0      0        0 tap0
172.20.0.0      *               255.255.0.0     U     0      0        0 tap1
default         *               0.0.0.0         U     0      0        0 ppp0
david@SonOfLappy:/svn/staging$

However, I'll admit I don't know much about the routing layer and thus 
I'm not sure how to diagnose beyond that.  But it seems very strange to 
me to have two network interfaces with the same IP.

With this in mind, if I shut down image0, the tap0 interface goes away, 
and now the wget to image1 works fine.  Again, this is suggesting 
there's some kind of conflict where the second tap interface is somehow 
"blocked" by the first.

Anyway, that's as far as I can get.  Is this supposed to work and am I 
doing something wrong?  Or am I supposed to do launch the second image 
with some other kind of command line?  Should I manually create my own 
tap devices before launching either image (and if so, any pointers on 
how I go about doing that)?

(Incidentally, I've tried putting the second image onto a different vlan 
by replacing both "vlan=0" with "vlan=1" in image1's launch command, but 
that had no effect -- the results were identical.)

Thanks for any tips you can provide!

-david

Re: [Qemu-devel] Two taps, same IP?

[Anthony Liguori]

David Barrett wrote:
> I'm considering a tap-based alternative to the -redir patch I proposed 
> earlier, but I'm just not quite getting how it works.  In particular, 
> I'm able to access the webserver on one image just fine, but not the 
> other: wget fails with "Connecting to 172.20.0.3:80... failed: No 
> route to host."
>
> Can you explain why and set me straight?
>
> Specifically, I have two Debian qemu images (0 and 1), identical in 
> all respects except that image0 and image1 are configured to use 
> static IPs 172.20.0.2 and 172.20.0.3, respectively.  I've launched 
> both simultaneously with the following commands:
>
> sudo qemu -kernel-kqemu -net nic,vlan=0 -net tap,vlan=0 image0.raw
> sudo qemu -kernel-kqemu -net nic,vlan=0 -net tap,vlan=0 image1.raw

You need to pass a unique mac address for each guest.  You're probably 
getting mac address collisions.

Regards,

Anthony Liguori

> Each image is configured with the following /etc/network/interfaces:
>
> auto lo
> iface lo inet loopback
> allow-hotplug eth0
> iface eth0 inet static
> address 172.20.0.2  <--- image1 has: address 172.20.0.3
> netmask 255.255.0.0
> gateway 172.20.0.1
>
> This creates two tap interfaces (0 and 1) on the Ubuntu host, 
> curiously with the same IP:
>
> tap0      Link encap:Ethernet  HWaddr 00:ff:84:12:9d:72
>           inet addr:172.20.0.1  Bcast:172.20.255.255  Mask:255.255.0.0
>           inet6 addr: fe80::2ff:84ff:fe12:9d72/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:18 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:36 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:500
>           RX bytes:1336 (1.3 KB)  TX bytes:4704 (4.5 KB)
>
> tap1      Link encap:Ethernet  HWaddr 00:ff:af:9a:48:29
>           inet addr:172.20.0.1  Bcast:172.20.255.255  Mask:255.255.0.0
>           inet6 addr: fe80::2ff:afff:fe9a:4829/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:24 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:34 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:500
>           RX bytes:1656 (1.6 KB)  TX bytes:4664 (4.5 KB)
>
> "wget http://172.20.0.2" and "wget http://172.20.0.3" each work fine 
> inside their respective VMs.  But each is unable to wget the other's 
> webserver.
>
> Furthermore, and most unusual, the host is able to wget image0's 
> webserver fine, but not image1.  Specifically, the second wget fails 
> as follows:
>
> david@SonOfLappy:/svn/staging$ wget http://172.20.0.3
> --18:17:12--  http://172.20.0.3/
>            => `index.html.1'
> Connecting to 172.20.0.3:80... failed: No route to host.
> david@SonOfLappy:/svn/staging$
>
> The error message suggests some sort of routing problem, and the 
> routing table is:
>
> david@SonOfLappy:/svn/staging$ route
> Kernel IP routing table
> Destination     Gateway         Genmask         Flags Metric Ref    
> Use Iface
> 68.28.57.85     *               255.255.255.255 UH    0      0        
> 0 ppp0
> 172.20.0.0      *               255.255.0.0     U     0      0        
> 0 tap0
> 172.20.0.0      *               255.255.0.0     U     0      0        
> 0 tap1
> default         *               0.0.0.0         U     0      0        
> 0 ppp0
> david@SonOfLappy:/svn/staging$
>
> However, I'll admit I don't know much about the routing layer and thus 
> I'm not sure how to diagnose beyond that.  But it seems very strange 
> to me to have two network interfaces with the same IP.
>
> With this in mind, if I shut down image0, the tap0 interface goes 
> away, and now the wget to image1 works fine.  Again, this is 
> suggesting there's some kind of conflict where the second tap 
> interface is somehow "blocked" by the first.
>
> Anyway, that's as far as I can get.  Is this supposed to work and am I 
> doing something wrong?  Or am I supposed to do launch the second image 
> with some other kind of command line?  Should I manually create my own 
> tap devices before launching either image (and if so, any pointers on 
> how I go about doing that)?
>
> (Incidentally, I've tried putting the second image onto a different 
> vlan by replacing both "vlan=0" with "vlan=1" in image1's launch 
> command, but that had no effect -- the results were identical.)
>
> Thanks for any tips you can provide!
>
> -david
>
>
>

Re: [Qemu-devel] Two taps, same IP?

[David Barrett]

Anthony Liguori wrote:
> David Barrett wrote:
>> I'm considering a tap-based alternative to the -redir patch I proposed 
>> earlier, but I'm just not quite getting how it works.  In particular, 
>> I'm able to access the webserver on one image just fine, but not the 
>> other: wget fails with "Connecting to 172.20.0.3:80... failed: No 
>> route to host."
>>
>> Can you explain why and set me straight?
>>
>> sudo qemu -kernel-kqemu -net nic,vlan=0 -net tap,vlan=0 image0.raw
>> sudo qemu -kernel-kqemu -net nic,vlan=0 -net tap,vlan=0 image1.raw
> 
> You need to pass a unique mac address for each guest.  You're probably 
> getting mac address collisions.

Hm, you're right, they had the same MAC address.  Unfortunately fixing 
that had no effect.  Now I'm launching with:

sudo qemu -kernel-kqemu -net nic,vlan=0,macaddr=00:00:00:00:00:00 -net 
tap,vlan=0 image0.raw

sudo qemu -kernel-kqemu -net nic,vlan=0,macaddr=00:00:00:00:00:11 -net 
tap,vlan=0 image1.raw

I'm still able to wget image0, but so long as it's operational I can't 
wget image1.  Like before, shutting down image0 enables wget access to 
image1.

Looking at ipconfig in the both guests show that the MAC address is 
being picked up correctly, so it must be something else.

Any other ideas?  Thanks!

-david

Re: [Qemu-devel] Two taps, same IP?

[Erik de Castro Lopo]

David Barrett wrote:

> Looking at ipconfig in the both guests show that the MAC address is 
> being picked up correctly, so it must be something else.
> 
> Any other ideas?  Thanks!
> 

Just a thought, but do you need to set up iptables on the host to
bridge between these two networks?

Erik
-- 
-----------------------------------------------------------------
Erik de Castro Lopo
-----------------------------------------------------------------
"Within C++, there is a much smaller and cleaner language struggling
to get out." -- Bjarne Stroustrup

Re: [Qemu-devel] Two taps, same IP?

[Erik de Castro Lopo]

David Barrett wrote:

> I'm considering a tap-based alternative to the -redir patch I proposed 
> earlier, but I'm just not quite getting how it works.  In particular, 
> I'm able to access the webserver on one image just fine, but not the 
> other: wget fails with "Connecting to 172.20.0.3:80... failed: No route 
> to host."
> 
> Can you explain why and set me straight?
> 
> Specifically, I have two Debian qemu images (0 and 1), identical in all 
> respects except that image0 and image1 are configured to use static IPs 
> 172.20.0.2 and 172.20.0.3, respectively.  I've launched both 
> simultaneously with the following commands:
> 
> sudo qemu -kernel-kqemu -net nic,vlan=0 -net tap,vlan=0 image0.raw
> sudo qemu -kernel-kqemu -net nic,vlan=0 -net tap,vlan=0 image1.raw
> 
> Each image is configured with the following /etc/network/interfaces:
> 
> auto lo
> iface lo inet loopback
> allow-hotplug eth0
> iface eth0 inet static
> address 172.20.0.2  <--- image1 has: address 172.20.0.3
> netmask 255.255.0.0
> gateway 172.20.0.1
> 
> This creates two tap interfaces (0 and 1) on the Ubuntu host, curiously 
> with the same IP:
> 
> tap0      Link encap:Ethernet  HWaddr 00:ff:84:12:9d:72
>            inet addr:172.20.0.1  Bcast:172.20.255.255  Mask:255.255.0.0
>            inet6 addr: fe80::2ff:84ff:fe12:9d72/64 Scope:Link
>            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>            RX packets:18 errors:0 dropped:0 overruns:0 frame:0
>            TX packets:36 errors:0 dropped:0 overruns:0 carrier:0
>            collisions:0 txqueuelen:500
>            RX bytes:1336 (1.3 KB)  TX bytes:4704 (4.5 KB)
> 
> tap1      Link encap:Ethernet  HWaddr 00:ff:af:9a:48:29
>            inet addr:172.20.0.1  Bcast:172.20.255.255  Mask:255.255.0.0
>            inet6 addr: fe80::2ff:afff:fe9a:4829/64 Scope:Link
>            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>            RX packets:24 errors:0 dropped:0 overruns:0 frame:0
>            TX packets:34 errors:0 dropped:0 overruns:0 carrier:0
>            collisions:0 txqueuelen:500
>            RX bytes:1656 (1.6 KB)  TX bytes:4664 (4.5 KB)
> 
> "wget http://172.20.0.2" and "wget http://172.20.0.3" each work fine 
> inside their respective VMs.  But each is unable to wget the other's 
> webserver.

I have multiple machines set up to run through one tap device (named
qtap0). Originally I tried a number of different qemu networking options
but in the end, I settled on VDE as that was the only one which worked
reliably.

When I first set it up, I think I went from this document:

    http://lists.gnu.org/archive/html/qemu-devel/2005-06/txtDAJWgugtC1.txt

The clients have unique MAC addresses and request DHCP addresses from
the host. The DHCP client on each guest is set up to send its host name
back to the DHCP/DNS server (I'm using dnsmasq) and all the running
guests can see all the other running guests as well as the host and
other machines in the network.

I have a quite a few guests (32 bit Debian, 64 bit Debian, 64 bit
Fedora, 64 bit Gentoo, 64 bit FreeBSD etc), but don't have enough RAM
to run more than 2 or 3 at a time.

> Thanks for any tips you can provide!

Maybe try the VDE networking option.

Cheers,
Erik
-- 
-----------------------------------------------------------------
Erik de Castro Lopo
-----------------------------------------------------------------
"TLC declared bankruptcy after they received less than 2 percent of the $175
million earned by their CD sales. That was about 40 times less than the
profit that was divided among their management, production and record
companies." -- Courtney Love on the REAL piracy

Re: [Qemu-devel] Two taps, same IP?

[David Barrett]

Erik de Castro Lopo wrote:
> David Barrett wrote:
> 
>> I'm considering a tap-based alternative to the -redir patch I proposed 
>> earlier, but I'm just not quite getting how it works.  In particular, 
>> I'm able to access the webserver on one image just fine, but not the 
>> other: wget fails with "Connecting to 172.20.0.3:80... failed: No route 
>> to host."
>>
>> Can you explain why and set me straight?
>>
>> Specifically, I have two Debian qemu images (0 and 1), identical in all 
>> respects except that image0 and image1 are configured to use static IPs 
>> 172.20.0.2 and 172.20.0.3, respectively.  I've launched both 
>> simultaneously with the following commands:
>>
>> sudo qemu -kernel-kqemu -net nic,vlan=0 -net tap,vlan=0 image0.raw
>> sudo qemu -kernel-kqemu -net nic,vlan=0 -net tap,vlan=0 image1.raw
>>
>> Each image is configured with the following /etc/network/interfaces:
>>
>> auto lo
>> iface lo inet loopback
>> allow-hotplug eth0
>> iface eth0 inet static
>> address 172.20.0.2  <--- image1 has: address 172.20.0.3
>> netmask 255.255.0.0
>> gateway 172.20.0.1
>>
>> This creates two tap interfaces (0 and 1) on the Ubuntu host, curiously 
>> with the same IP:
>>
>> tap0      Link encap:Ethernet  HWaddr 00:ff:84:12:9d:72
>>            inet addr:172.20.0.1  Bcast:172.20.255.255  Mask:255.255.0.0
>>            inet6 addr: fe80::2ff:84ff:fe12:9d72/64 Scope:Link
>>            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>            RX packets:18 errors:0 dropped:0 overruns:0 frame:0
>>            TX packets:36 errors:0 dropped:0 overruns:0 carrier:0
>>            collisions:0 txqueuelen:500
>>            RX bytes:1336 (1.3 KB)  TX bytes:4704 (4.5 KB)
>>
>> tap1      Link encap:Ethernet  HWaddr 00:ff:af:9a:48:29
>>            inet addr:172.20.0.1  Bcast:172.20.255.255  Mask:255.255.0.0
>>            inet6 addr: fe80::2ff:afff:fe9a:4829/64 Scope:Link
>>            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>            RX packets:24 errors:0 dropped:0 overruns:0 frame:0
>>            TX packets:34 errors:0 dropped:0 overruns:0 carrier:0
>>            collisions:0 txqueuelen:500
>>            RX bytes:1656 (1.6 KB)  TX bytes:4664 (4.5 KB)
>>
>> "wget http://172.20.0.2" and "wget http://172.20.0.3" each work fine 
>> inside their respective VMs.  But each is unable to wget the other's 
>> webserver.
> 
> I have multiple machines set up to run through one tap device (named
> qtap0). Originally I tried a number of different qemu networking options
> but in the end, I settled on VDE as that was the only one which worked
> reliably.
> 
> When I first set it up, I think I went from this document:
> 
>     http://lists.gnu.org/archive/html/qemu-devel/2005-06/txtDAJWgugtC1.txt
> 
> The clients have unique MAC addresses and request DHCP addresses from
> the host. The DHCP client on each guest is set up to send its host name
> back to the DHCP/DNS server (I'm using dnsmasq) and all the running
> guests can see all the other running guests as well as the host and
> other machines in the network.
> 
> I have a quite a few guests (32 bit Debian, 64 bit Debian, 64 bit
> Fedora, 64 bit Gentoo, 64 bit FreeBSD etc), but don't have enough RAM
> to run more than 2 or 3 at a time.
> 
>> Thanks for any tips you can provide!
> 
> Maybe try the VDE networking option.
> 
> Cheers,
> Erik

Success!  I didn't go the VDE route, I just fixed it so tap0 and tap1 
have different IPs.  It's so obvious I don't know why it didn't occur to 
me before.

I did this by simply skipping the default "qemu-ifup" script, which is:

david@SonOfLappy:/svn/staging$ cat /etc/qemu-ifup
#!/bin/sh
sudo -p "Password for $0:" /sbin/ifconfig $1 172.20.0.1
david@SonOfLappy:/svn/staging$

And instead creating separate image0-ifup and image1-ifup scripts as:

david@SonOfLappy:/svn/staging$ cat image0-ifup
#!/bin/sh
sudo -p "Password for $0:" /sbin/ifconfig $1 172.20.0.1
david@SonOfLappy:/svn/staging$ cat image1-ifup
#!/bin/sh
sudo -p "Password for $0:" /sbin/ifconfig $1 172.21.0.1
david@SonOfLappy:/svn/staging$

Then I start it with:

sudo qemu -kernel-kqemu -net nic,macad0:00:00:00 -net 
tap,script=image0-ifup image0.raw

sudo qemu -kernel-kqemu -net nic,macad0:00:00:11 -net 
tap,script=image1-ifup image1.raw

Now it works great, I can wget from the host to both guests.  Granted, 
the guests can't talk to each other, but that's actually a good thing as 
I'm explicitly trying to segment them apart.  Hooray for qemu, and 
thanks for all your help!

-david

Re: [Qemu-devel] Two taps, same IP?

[Anthony Liguori]

David Barrett wrote:
> Anthony Liguori wrote:
>> David Barrett wrote:
>>> I'm considering a tap-based alternative to the -redir patch I 
>>> proposed earlier, but I'm just not quite getting how it works.  In 
>>> particular, I'm able to access the webserver on one image just fine, 
>>> but not the other: wget fails with "Connecting to 172.20.0.3:80... 
>>> failed: No route to host."
>>>
>>> Can you explain why and set me straight?
>>>
>>> sudo qemu -kernel-kqemu -net nic,vlan=0 -net tap,vlan=0 image0.raw
>>> sudo qemu -kernel-kqemu -net nic,vlan=0 -net tap,vlan=0 image1.raw
>>
>> You need to pass a unique mac address for each guest.  You're 
>> probably getting mac address collisions.
>
> Hm, you're right, they had the same MAC address.  Unfortunately fixing 
> that had no effect.  Now I'm launching with:
>
> sudo qemu -kernel-kqemu -net nic,vlan=0,macaddr=00:00:00:00:00:00 -net 
> tap,vlan=0 image0.raw
>
> sudo qemu -kernel-kqemu -net nic,vlan=0,macaddr=00:00:00:00:00:11 -net 
> tap,vlan=0 image1.raw
>
> I'm still able to wget image0, but so long as it's operational I can't 
> wget image1.  Like before, shutting down image0 enables wget access to 
> image1.
>
> Looking at ipconfig in the both guests show that the MAC address is 
> being picked up correctly, so it must be something else.
>
> Any other ideas?  Thanks!

What's in your qemu-ifup script?

Regards,

Anthony Liguori

> -david
>
>
>