public inbox for kvm@vger.kernel.org
 help / color / mirror / Atom feed
* VM networking stops working
@ 2007-11-21 20:52 Mike
       [not found] ` <47449A93.5090007-kcIQ9yavhvsV/sYkb9DZNQ@public.gmane.org>
  0 siblings, 1 reply; 12+ messages in thread
From: Mike @ 2007-11-21 20:52 UTC (permalink / raw)
  To: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f


[-- Attachment #1.1: Type: text/plain, Size: 2815 bytes --]

Hello,
I have a problem with my virtual machines. I am using kvm53
on an AMD Athlon(tm) 64 X2 Dual Core Processor 5000+.
As distribution I use Ubuntu server 7.10.
As kernel I use Linux 2.6.22-14-server #1 SMP Sun Oct 14 22:09:15 GMT 
2007 x86_64 GNU/Linux.

I have 3 different vms running. At the beginning they all work without a 
problem.
Then, after one moment to another one or more stop working.
I did a ping test to all the machines, the vm with the failure stops 
pinging.
The result from external ping: Destination host unreachable.
The same result from a ping onto the main server.

Here are my configuration files:
cat /etc/qemu-ifup
#!/bin/sh
/sbin/ifconfig $1 0.0.0.0 promisc up
/usr/sbin/brctl addif br0 $1
sleep 2
------------------------------------------------------------------------
cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto br0
iface br0 inet static
address 195.24.77.169
netmask 255.255.255.0
gateway 195.24.77.1
bridge_ports eth0
bridge_stp off
bridge_maxwait 5
------------------------------------------------------------------------
I use scripts for starting the vms:
kvm -hda apache.img -hdb apache_storage.img -m 512 -boot c -net 
nic,vlan=0,macaddr=00:16:3e:00:00:01 -net tap -nographic -daemonize
kvm -hda postfix.img -m 512 -boot c -net 
nic,vlan=0,macaddr=00:16:3e:00:00:12 -net tap -nographic -daemonize
kvm -hda openvpn.img -m 256 -boot c -net 
nic,vlan=0,macaddr=00:16:3e:00:00:03 -net tap -nographic -daemonize
------------------------------------------------------------------------
The tapN devices are being created at the startup of the vms.

When I run dmesg I have this:
[  112.390943] br0: port 2(tap1) entering learning state
[  112.391919] br0: port 3(tap0) entering learning state
[  112.397605] br0: port 4(tap2) entering learning state
[  122.674313] tap0: no IPv6 routers present
[  122.729185] tap1: no IPv6 routers present
[  122.843901] tap2: no IPv6 routers present
[  123.336081] kvm: emulating exchange as write
[  127.367940] br0: topology change detected, propagating
[  127.367949] br0: port 4(tap2) entering forwarding state
[  127.362971] br0: topology change detected, propagating
[  127.362977] br0: port 2(tap1) entering forwarding state
[  127.362978] br0: topology change detected, propagating
[  127.362980] br0: port 3(tap0) entering forwarding state

No indication of a problem, nowhere in the log files.

I don't know if the vms are freezing, or the network forwarding stops 
working.
I also activated "net.ipv4.conf.default.forwarding=1"

I googled days for that problem, but without a result.
I hope that maybe you have a solution for this problem. That would be 
great, because I prefer using kvm instead of other virtualization systems.

Thank you very much in advance,
Greetings from Luxembourg.

Have a nice day,
Mike


[-- Attachment #1.2: Type: text/html, Size: 3151 bytes --]

[-- Attachment #2: Type: text/plain, Size: 228 bytes --]

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

[-- Attachment #3: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: VM networking stops working
       [not found] ` <47449A93.5090007-kcIQ9yavhvsV/sYkb9DZNQ@public.gmane.org>
@ 2007-11-21 21:22   ` Izik Eidus
       [not found]     ` <4744A18D.2050705-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
  2007-11-21 22:09   ` Lynn Kerby
  1 sibling, 1 reply; 12+ messages in thread
From: Izik Eidus @ 2007-11-21 21:22 UTC (permalink / raw)
  To: Mike; +Cc: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f

Mike wrote:
> Hello,
> I have a problem with my virtual machines. I am using kvm53
> on an AMD Athlon(tm) 64 X2 Dual Core Processor 5000+.
> As distribution I use Ubuntu server 7.10.
> As kernel I use Linux 2.6.22-14-server #1 SMP Sun Oct 14 22:09:15 GMT 
> 2007 x86_64 GNU/Linux.
>
> I have 3 different vms running. At the beginning they all work without 
> a problem.
> Then, after one moment to another one or more stop working.
> I did a ping test to all the machines, the vm with the failure stops 
> pinging.
> The result from external ping: Destination host unreachable.
> The same result from a ping onto the main server.
>
> Here are my configuration files:
> cat /etc/qemu-ifup
> #!/bin/sh
> /sbin/ifconfig $1 0.0.0.0 promisc up
> /usr/sbin/brctl addif br0 $1
> sleep 2
> ------------------------------------------------------------------------
> cat /etc/network/interfaces
> auto lo
> iface lo inet loopback
>
> auto br0
> iface br0 inet static
> address 195.24.77.169
> netmask 255.255.255.0
> gateway 195.24.77.1
> bridge_ports eth0
> bridge_stp off
> bridge_maxwait 5
> ------------------------------------------------------------------------
> I use scripts for starting the vms:
> kvm -hda apache.img -hdb apache_storage.img -m 512 -boot c -net 
> nic,vlan=0,macaddr=00:16:3e:00:00:01 -net tap -nographic -daemonize
> kvm -hda postfix.img -m 512 -boot c -net 
> nic,vlan=0,macaddr=00:16:3e:00:00:12 -net tap -nographic -daemonize
> kvm -hda openvpn.img -m 256 -boot c -net 
> nic,vlan=0,macaddr=00:16:3e:00:00:03 -net tap -nographic -daemonize
> ------------------------------------------------------------------------
> The tapN devices are being created at the startup of the vms.
>
> When I run dmesg I have this:
> [  112.390943] br0: port 2(tap1) entering learning state
> [  112.391919] br0: port 3(tap0) entering learning state
> [  112.397605] br0: port 4(tap2) entering learning state
> [  122.674313] tap0: no IPv6 routers present
> [  122.729185] tap1: no IPv6 routers present
> [  122.843901] tap2: no IPv6 routers present
> [  123.336081] kvm: emulating exchange as write
> [  127.367940] br0: topology change detected, propagating
> [  127.367949] br0: port 4(tap2) entering forwarding state
> [  127.362971] br0: topology change detected, propagating
> [  127.362977] br0: port 2(tap1) entering forwarding state
> [  127.362978] br0: topology change detected, propagating
> [  127.362980] br0: port 3(tap0) entering forwarding state
>
> No indication of a problem, nowhere in the log files.
>
> I don't know if the vms are freezing, or the network forwarding stops 
> working.
> I also activated "net.ipv4.conf.default.forwarding=1"
>
> I googled days for that problem, but without a result.
> I hope that maybe you have a solution for this problem. That would be 
> great, because I prefer using kvm instead of other virtualization systems.
>
> Thank you very much in advance,
> Greetings from Luxembourg.
>
> Have a nice day,
> Mike
it might be not the network that die, but the whole vm
can you run the vms with -vnc ? (first vm -vnc :1 secoend vm -vnc :2 
third vm -vnc :3)
and try to connect to them with vncviewer and see if they are alive?

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: VM networking stops working
       [not found]         ` <4744A5E5.9060902-kcIQ9yavhvsV/sYkb9DZNQ@public.gmane.org>
@ 2007-11-21 21:40           ` Izik Eidus
       [not found]             ` <4744A5CA.8050604-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
  0 siblings, 1 reply; 12+ messages in thread
From: Izik Eidus @ 2007-11-21 21:40 UTC (permalink / raw)
  To: Mike; +Cc: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f

Mike wrote:
> Izik Eidus schrieb:
>   
>> Mike wrote:
>>     
>>> Hello,
>>> I have a problem with my virtual machines. I am using kvm53
>>> on an AMD Athlon(tm) 64 X2 Dual Core Processor 5000+.
>>> As distribution I use Ubuntu server 7.10.
>>> As kernel I use Linux 2.6.22-14-server #1 SMP Sun Oct 14 22:09:15 GMT 
>>> 2007 x86_64 GNU/Linux.
>>>
>>> I have 3 different vms running. At the beginning they all work 
>>> without a problem.
>>> Then, after one moment to another one or more stop working.
>>> I did a ping test to all the machines, the vm with the failure stops 
>>> pinging.
>>> The result from external ping: Destination host unreachable.
>>> The same result from a ping onto the main server.
>>>
>>> Here are my configuration files:
>>> cat /etc/qemu-ifup
>>> #!/bin/sh
>>> /sbin/ifconfig $1 0.0.0.0 promisc up
>>> /usr/sbin/brctl addif br0 $1
>>> sleep 2
>>> ------------------------------------------------------------------------
>>> cat /etc/network/interfaces
>>> auto lo
>>> iface lo inet loopback
>>>
>>> auto br0
>>> iface br0 inet static
>>> address 195.24.77.169
>>> netmask 255.255.255.0
>>> gateway 195.24.77.1
>>> bridge_ports eth0
>>> bridge_stp off
>>> bridge_maxwait 5
>>> ------------------------------------------------------------------------
>>> I use scripts for starting the vms:
>>> kvm -hda apache.img -hdb apache_storage.img -m 512 -boot c -net 
>>> nic,vlan=0,macaddr=00:16:3e:00:00:01 -net tap -nographic -daemonize
>>> kvm -hda postfix.img -m 512 -boot c -net 
>>> nic,vlan=0,macaddr=00:16:3e:00:00:12 -net tap -nographic -daemonize
>>> kvm -hda openvpn.img -m 256 -boot c -net 
>>> nic,vlan=0,macaddr=00:16:3e:00:00:03 -net tap -nographic -daemonize
>>> ------------------------------------------------------------------------
>>> The tapN devices are being created at the startup of the vms.
>>>
>>> When I run dmesg I have this:
>>> [  112.390943] br0: port 2(tap1) entering learning state
>>> [  112.391919] br0: port 3(tap0) entering learning state
>>> [  112.397605] br0: port 4(tap2) entering learning state
>>> [  122.674313] tap0: no IPv6 routers present
>>> [  122.729185] tap1: no IPv6 routers present
>>> [  122.843901] tap2: no IPv6 routers present
>>> [  123.336081] kvm: emulating exchange as write
>>> [  127.367940] br0: topology change detected, propagating
>>> [  127.367949] br0: port 4(tap2) entering forwarding state
>>> [  127.362971] br0: topology change detected, propagating
>>> [  127.362977] br0: port 2(tap1) entering forwarding state
>>> [  127.362978] br0: topology change detected, propagating
>>> [  127.362980] br0: port 3(tap0) entering forwarding state
>>>
>>> No indication of a problem, nowhere in the log files.
>>>
>>> I don't know if the vms are freezing, or the network forwarding stops 
>>> working.
>>> I also activated "net.ipv4.conf.default.forwarding=1"
>>>
>>> I googled days for that problem, but without a result.
>>> I hope that maybe you have a solution for this problem. That would be 
>>> great, because I prefer using kvm instead of other virtualization 
>>> systems.
>>>
>>> Thank you very much in advance,
>>> Greetings from Luxembourg.
>>>
>>> Have a nice day,
>>> Mike
>>>       
>> it might be not the network that die, but the whole vm
>> can you run the vms with -vnc ? (first vm -vnc :1 secoend vm -vnc :2 
>> third vm -vnc :3)
>> and try to connect to them with vncviewer and see if they are alive?
>>
>>     
> Thank you for your reply.
> I can't connect to the vm if it has died, even with vnc. I tried this 
> already, no reply from the host.
> I don't have physical access to the server because its located in a dc. 
> Only ssh to the host.
>   
ok this mean there is bug in kvm for amd and it not realted to the network
what is the guest that you use? is it 64/32?

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: VM networking stops working
       [not found]     ` <4744A18D.2050705-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
@ 2007-11-21 21:40       ` Mike
       [not found]         ` <4744A5E5.9060902-kcIQ9yavhvsV/sYkb9DZNQ@public.gmane.org>
  0 siblings, 1 reply; 12+ messages in thread
From: Mike @ 2007-11-21 21:40 UTC (permalink / raw)
  To: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f

Izik Eidus schrieb:
> Mike wrote:
>> Hello,
>> I have a problem with my virtual machines. I am using kvm53
>> on an AMD Athlon(tm) 64 X2 Dual Core Processor 5000+.
>> As distribution I use Ubuntu server 7.10.
>> As kernel I use Linux 2.6.22-14-server #1 SMP Sun Oct 14 22:09:15 GMT 
>> 2007 x86_64 GNU/Linux.
>>
>> I have 3 different vms running. At the beginning they all work 
>> without a problem.
>> Then, after one moment to another one or more stop working.
>> I did a ping test to all the machines, the vm with the failure stops 
>> pinging.
>> The result from external ping: Destination host unreachable.
>> The same result from a ping onto the main server.
>>
>> Here are my configuration files:
>> cat /etc/qemu-ifup
>> #!/bin/sh
>> /sbin/ifconfig $1 0.0.0.0 promisc up
>> /usr/sbin/brctl addif br0 $1
>> sleep 2
>> ------------------------------------------------------------------------
>> cat /etc/network/interfaces
>> auto lo
>> iface lo inet loopback
>>
>> auto br0
>> iface br0 inet static
>> address 195.24.77.169
>> netmask 255.255.255.0
>> gateway 195.24.77.1
>> bridge_ports eth0
>> bridge_stp off
>> bridge_maxwait 5
>> ------------------------------------------------------------------------
>> I use scripts for starting the vms:
>> kvm -hda apache.img -hdb apache_storage.img -m 512 -boot c -net 
>> nic,vlan=0,macaddr=00:16:3e:00:00:01 -net tap -nographic -daemonize
>> kvm -hda postfix.img -m 512 -boot c -net 
>> nic,vlan=0,macaddr=00:16:3e:00:00:12 -net tap -nographic -daemonize
>> kvm -hda openvpn.img -m 256 -boot c -net 
>> nic,vlan=0,macaddr=00:16:3e:00:00:03 -net tap -nographic -daemonize
>> ------------------------------------------------------------------------
>> The tapN devices are being created at the startup of the vms.
>>
>> When I run dmesg I have this:
>> [  112.390943] br0: port 2(tap1) entering learning state
>> [  112.391919] br0: port 3(tap0) entering learning state
>> [  112.397605] br0: port 4(tap2) entering learning state
>> [  122.674313] tap0: no IPv6 routers present
>> [  122.729185] tap1: no IPv6 routers present
>> [  122.843901] tap2: no IPv6 routers present
>> [  123.336081] kvm: emulating exchange as write
>> [  127.367940] br0: topology change detected, propagating
>> [  127.367949] br0: port 4(tap2) entering forwarding state
>> [  127.362971] br0: topology change detected, propagating
>> [  127.362977] br0: port 2(tap1) entering forwarding state
>> [  127.362978] br0: topology change detected, propagating
>> [  127.362980] br0: port 3(tap0) entering forwarding state
>>
>> No indication of a problem, nowhere in the log files.
>>
>> I don't know if the vms are freezing, or the network forwarding stops 
>> working.
>> I also activated "net.ipv4.conf.default.forwarding=1"
>>
>> I googled days for that problem, but without a result.
>> I hope that maybe you have a solution for this problem. That would be 
>> great, because I prefer using kvm instead of other virtualization 
>> systems.
>>
>> Thank you very much in advance,
>> Greetings from Luxembourg.
>>
>> Have a nice day,
>> Mike
> it might be not the network that die, but the whole vm
> can you run the vms with -vnc ? (first vm -vnc :1 secoend vm -vnc :2 
> third vm -vnc :3)
> and try to connect to them with vncviewer and see if they are alive?
>
Thank you for your reply.
I can't connect to the vm if it has died, even with vnc. I tried this 
already, no reply from the host.
I don't have physical access to the server because its located in a dc. 
Only ssh to the host.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: VM networking stops working
       [not found]                 ` <4744A8CA.5070702-kcIQ9yavhvsV/sYkb9DZNQ@public.gmane.org>
@ 2007-11-21 21:52                   ` Izik Eidus
       [not found]                     ` <4744A881.1060201-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
  0 siblings, 1 reply; 12+ messages in thread
From: Izik Eidus @ 2007-11-21 21:52 UTC (permalink / raw)
  To: Mike; +Cc: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f

Mike wrote:
> Izik Eidus schrieb:
>   
>> Mike wrote:
>>     
>>> Izik Eidus schrieb:
>>>  
>>>       
>>>> Mike wrote:
>>>>    
>>>>         
>>>>> Hello,
>>>>> I have a problem with my virtual machines. I am using kvm53
>>>>> on an AMD Athlon(tm) 64 X2 Dual Core Processor 5000+.
>>>>> As distribution I use Ubuntu server 7.10.
>>>>> As kernel I use Linux 2.6.22-14-server #1 SMP Sun Oct 14 22:09:15 
>>>>> GMT 2007 x86_64 GNU/Linux.
>>>>>
>>>>> I have 3 different vms running. At the beginning they all work 
>>>>> without a problem.
>>>>> Then, after one moment to another one or more stop working.
>>>>> I did a ping test to all the machines, the vm with the failure 
>>>>> stops pinging.
>>>>> The result from external ping: Destination host unreachable.
>>>>> The same result from a ping onto the main server.
>>>>>
>>>>> Here are my configuration files:
>>>>> cat /etc/qemu-ifup
>>>>> #!/bin/sh
>>>>> /sbin/ifconfig $1 0.0.0.0 promisc up
>>>>> /usr/sbin/brctl addif br0 $1
>>>>> sleep 2
>>>>> ------------------------------------------------------------------------ 
>>>>>
>>>>> cat /etc/network/interfaces
>>>>> auto lo
>>>>> iface lo inet loopback
>>>>>
>>>>> auto br0
>>>>> iface br0 inet static
>>>>> address 195.24.77.169
>>>>> netmask 255.255.255.0
>>>>> gateway 195.24.77.1
>>>>> bridge_ports eth0
>>>>> bridge_stp off
>>>>> bridge_maxwait 5
>>>>> ------------------------------------------------------------------------ 
>>>>>
>>>>> I use scripts for starting the vms:
>>>>> kvm -hda apache.img -hdb apache_storage.img -m 512 -boot c -net 
>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:01 -net tap -nographic -daemonize
>>>>> kvm -hda postfix.img -m 512 -boot c -net 
>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:12 -net tap -nographic -daemonize
>>>>> kvm -hda openvpn.img -m 256 -boot c -net 
>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:03 -net tap -nographic -daemonize
>>>>> ------------------------------------------------------------------------ 
>>>>>
>>>>> The tapN devices are being created at the startup of the vms.
>>>>>
>>>>> When I run dmesg I have this:
>>>>> [  112.390943] br0: port 2(tap1) entering learning state
>>>>> [  112.391919] br0: port 3(tap0) entering learning state
>>>>> [  112.397605] br0: port 4(tap2) entering learning state
>>>>> [  122.674313] tap0: no IPv6 routers present
>>>>> [  122.729185] tap1: no IPv6 routers present
>>>>> [  122.843901] tap2: no IPv6 routers present
>>>>> [  123.336081] kvm: emulating exchange as write
>>>>> [  127.367940] br0: topology change detected, propagating
>>>>> [  127.367949] br0: port 4(tap2) entering forwarding state
>>>>> [  127.362971] br0: topology change detected, propagating
>>>>> [  127.362977] br0: port 2(tap1) entering forwarding state
>>>>> [  127.362978] br0: topology change detected, propagating
>>>>> [  127.362980] br0: port 3(tap0) entering forwarding state
>>>>>
>>>>> No indication of a problem, nowhere in the log files.
>>>>>
>>>>> I don't know if the vms are freezing, or the network forwarding 
>>>>> stops working.
>>>>> I also activated "net.ipv4.conf.default.forwarding=1"
>>>>>
>>>>> I googled days for that problem, but without a result.
>>>>> I hope that maybe you have a solution for this problem. That would 
>>>>> be great, because I prefer using kvm instead of other 
>>>>> virtualization systems.
>>>>>
>>>>> Thank you very much in advance,
>>>>> Greetings from Luxembourg.
>>>>>
>>>>> Have a nice day,
>>>>> Mike
>>>>>       
>>>>>           
>>>> it might be not the network that die, but the whole vm
>>>> can you run the vms with -vnc ? (first vm -vnc :1 secoend vm -vnc :2 
>>>> third vm -vnc :3)
>>>> and try to connect to them with vncviewer and see if they are alive?
>>>>
>>>>     
>>>>         
>>> Thank you for your reply.
>>> I can't connect to the vm if it has died, even with vnc. I tried this 
>>> already, no reply from the host.
>>> I don't have physical access to the server because its located in a 
>>> dc. Only ssh to the host.
>>>   
>>>       
>> ok this mean there is bug in kvm for amd and it not realted to the 
>> network
>> what is the guest that you use? is it 64/32?
>>
>>     
> I'm using Debian Etch 32 Bit for all guests
> with Kernel Linux apache 2.6.18-5-486 #1 Tue Oct 2 23:38:54 UTC 2007 
> i686 GNU/Linux
>
> -------------------------------------------------------------------------
>   
ok but what more interesting me is:
if you run just one vm
is it work?


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: VM networking stops working
       [not found]             ` <4744A5CA.8050604-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
@ 2007-11-21 21:53               ` Mike
       [not found]                 ` <4744A8CA.5070702-kcIQ9yavhvsV/sYkb9DZNQ@public.gmane.org>
  0 siblings, 1 reply; 12+ messages in thread
From: Mike @ 2007-11-21 21:53 UTC (permalink / raw)
  To: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f

Izik Eidus schrieb:
> Mike wrote:
>> Izik Eidus schrieb:
>>  
>>> Mike wrote:
>>>    
>>>> Hello,
>>>> I have a problem with my virtual machines. I am using kvm53
>>>> on an AMD Athlon(tm) 64 X2 Dual Core Processor 5000+.
>>>> As distribution I use Ubuntu server 7.10.
>>>> As kernel I use Linux 2.6.22-14-server #1 SMP Sun Oct 14 22:09:15 
>>>> GMT 2007 x86_64 GNU/Linux.
>>>>
>>>> I have 3 different vms running. At the beginning they all work 
>>>> without a problem.
>>>> Then, after one moment to another one or more stop working.
>>>> I did a ping test to all the machines, the vm with the failure 
>>>> stops pinging.
>>>> The result from external ping: Destination host unreachable.
>>>> The same result from a ping onto the main server.
>>>>
>>>> Here are my configuration files:
>>>> cat /etc/qemu-ifup
>>>> #!/bin/sh
>>>> /sbin/ifconfig $1 0.0.0.0 promisc up
>>>> /usr/sbin/brctl addif br0 $1
>>>> sleep 2
>>>> ------------------------------------------------------------------------ 
>>>>
>>>> cat /etc/network/interfaces
>>>> auto lo
>>>> iface lo inet loopback
>>>>
>>>> auto br0
>>>> iface br0 inet static
>>>> address 195.24.77.169
>>>> netmask 255.255.255.0
>>>> gateway 195.24.77.1
>>>> bridge_ports eth0
>>>> bridge_stp off
>>>> bridge_maxwait 5
>>>> ------------------------------------------------------------------------ 
>>>>
>>>> I use scripts for starting the vms:
>>>> kvm -hda apache.img -hdb apache_storage.img -m 512 -boot c -net 
>>>> nic,vlan=0,macaddr=00:16:3e:00:00:01 -net tap -nographic -daemonize
>>>> kvm -hda postfix.img -m 512 -boot c -net 
>>>> nic,vlan=0,macaddr=00:16:3e:00:00:12 -net tap -nographic -daemonize
>>>> kvm -hda openvpn.img -m 256 -boot c -net 
>>>> nic,vlan=0,macaddr=00:16:3e:00:00:03 -net tap -nographic -daemonize
>>>> ------------------------------------------------------------------------ 
>>>>
>>>> The tapN devices are being created at the startup of the vms.
>>>>
>>>> When I run dmesg I have this:
>>>> [  112.390943] br0: port 2(tap1) entering learning state
>>>> [  112.391919] br0: port 3(tap0) entering learning state
>>>> [  112.397605] br0: port 4(tap2) entering learning state
>>>> [  122.674313] tap0: no IPv6 routers present
>>>> [  122.729185] tap1: no IPv6 routers present
>>>> [  122.843901] tap2: no IPv6 routers present
>>>> [  123.336081] kvm: emulating exchange as write
>>>> [  127.367940] br0: topology change detected, propagating
>>>> [  127.367949] br0: port 4(tap2) entering forwarding state
>>>> [  127.362971] br0: topology change detected, propagating
>>>> [  127.362977] br0: port 2(tap1) entering forwarding state
>>>> [  127.362978] br0: topology change detected, propagating
>>>> [  127.362980] br0: port 3(tap0) entering forwarding state
>>>>
>>>> No indication of a problem, nowhere in the log files.
>>>>
>>>> I don't know if the vms are freezing, or the network forwarding 
>>>> stops working.
>>>> I also activated "net.ipv4.conf.default.forwarding=1"
>>>>
>>>> I googled days for that problem, but without a result.
>>>> I hope that maybe you have a solution for this problem. That would 
>>>> be great, because I prefer using kvm instead of other 
>>>> virtualization systems.
>>>>
>>>> Thank you very much in advance,
>>>> Greetings from Luxembourg.
>>>>
>>>> Have a nice day,
>>>> Mike
>>>>       
>>> it might be not the network that die, but the whole vm
>>> can you run the vms with -vnc ? (first vm -vnc :1 secoend vm -vnc :2 
>>> third vm -vnc :3)
>>> and try to connect to them with vncviewer and see if they are alive?
>>>
>>>     
>> Thank you for your reply.
>> I can't connect to the vm if it has died, even with vnc. I tried this 
>> already, no reply from the host.
>> I don't have physical access to the server because its located in a 
>> dc. Only ssh to the host.
>>   
> ok this mean there is bug in kvm for amd and it not realted to the 
> network
> what is the guest that you use? is it 64/32?
>
I'm using Debian Etch 32 Bit for all guests
with Kernel Linux apache 2.6.18-5-486 #1 Tue Oct 2 23:38:54 UTC 2007 
i686 GNU/Linux

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: VM networking stops working
       [not found]                         ` <4744ABD8.8020409-kcIQ9yavhvsV/sYkb9DZNQ@public.gmane.org>
@ 2007-11-21 22:05                           ` Izik Eidus
       [not found]                             ` <4744ABBC.30708-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
  0 siblings, 1 reply; 12+ messages in thread
From: Izik Eidus @ 2007-11-21 22:05 UTC (permalink / raw)
  To: Mike; +Cc: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f

Mike wrote:
> Izik Eidus schrieb:
>   
>> Mike wrote:
>>     
>>> Izik Eidus schrieb:
>>>  
>>>       
>>>> Mike wrote:
>>>>    
>>>>         
>>>>> Izik Eidus schrieb:
>>>>>  
>>>>>      
>>>>>           
>>>>>> Mike wrote:
>>>>>>           
>>>>>>             
>>>>>>> Hello,
>>>>>>> I have a problem with my virtual machines. I am using kvm53
>>>>>>> on an AMD Athlon(tm) 64 X2 Dual Core Processor 5000+.
>>>>>>> As distribution I use Ubuntu server 7.10.
>>>>>>> As kernel I use Linux 2.6.22-14-server #1 SMP Sun Oct 14 22:09:15 
>>>>>>> GMT 2007 x86_64 GNU/Linux.
>>>>>>>
>>>>>>> I have 3 different vms running. At the beginning they all work 
>>>>>>> without a problem.
>>>>>>> Then, after one moment to another one or more stop working.
>>>>>>> I did a ping test to all the machines, the vm with the failure 
>>>>>>> stops pinging.
>>>>>>> The result from external ping: Destination host unreachable.
>>>>>>> The same result from a ping onto the main server.
>>>>>>>
>>>>>>> Here are my configuration files:
>>>>>>> cat /etc/qemu-ifup
>>>>>>> #!/bin/sh
>>>>>>> /sbin/ifconfig $1 0.0.0.0 promisc up
>>>>>>> /usr/sbin/brctl addif br0 $1
>>>>>>> sleep 2
>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>
>>>>>>> cat /etc/network/interfaces
>>>>>>> auto lo
>>>>>>> iface lo inet loopback
>>>>>>>
>>>>>>> auto br0
>>>>>>> iface br0 inet static
>>>>>>> address 195.24.77.169
>>>>>>> netmask 255.255.255.0
>>>>>>> gateway 195.24.77.1
>>>>>>> bridge_ports eth0
>>>>>>> bridge_stp off
>>>>>>> bridge_maxwait 5
>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>
>>>>>>> I use scripts for starting the vms:
>>>>>>> kvm -hda apache.img -hdb apache_storage.img -m 512 -boot c -net 
>>>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:01 -net tap -nographic -daemonize
>>>>>>> kvm -hda postfix.img -m 512 -boot c -net 
>>>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:12 -net tap -nographic -daemonize
>>>>>>> kvm -hda openvpn.img -m 256 -boot c -net 
>>>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:03 -net tap -nographic -daemonize
>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>
>>>>>>> The tapN devices are being created at the startup of the vms.
>>>>>>>
>>>>>>> When I run dmesg I have this:
>>>>>>> [  112.390943] br0: port 2(tap1) entering learning state
>>>>>>> [  112.391919] br0: port 3(tap0) entering learning state
>>>>>>> [  112.397605] br0: port 4(tap2) entering learning state
>>>>>>> [  122.674313] tap0: no IPv6 routers present
>>>>>>> [  122.729185] tap1: no IPv6 routers present
>>>>>>> [  122.843901] tap2: no IPv6 routers present
>>>>>>> [  123.336081] kvm: emulating exchange as write
>>>>>>> [  127.367940] br0: topology change detected, propagating
>>>>>>> [  127.367949] br0: port 4(tap2) entering forwarding state
>>>>>>> [  127.362971] br0: topology change detected, propagating
>>>>>>> [  127.362977] br0: port 2(tap1) entering forwarding state
>>>>>>> [  127.362978] br0: topology change detected, propagating
>>>>>>> [  127.362980] br0: port 3(tap0) entering forwarding state
>>>>>>>
>>>>>>> No indication of a problem, nowhere in the log files.
>>>>>>>
>>>>>>> I don't know if the vms are freezing, or the network forwarding 
>>>>>>> stops working.
>>>>>>> I also activated "net.ipv4.conf.default.forwarding=1"
>>>>>>>
>>>>>>> I googled days for that problem, but without a result.
>>>>>>> I hope that maybe you have a solution for this problem. That 
>>>>>>> would be great, because I prefer using kvm instead of other 
>>>>>>> virtualization systems.
>>>>>>>
>>>>>>> Thank you very much in advance,
>>>>>>> Greetings from Luxembourg.
>>>>>>>
>>>>>>> Have a nice day,
>>>>>>> Mike
>>>>>>>                 
>>>>>>>               
>>>>>> it might be not the network that die, but the whole vm
>>>>>> can you run the vms with -vnc ? (first vm -vnc :1 secoend vm -vnc 
>>>>>> :2 third vm -vnc :3)
>>>>>> and try to connect to them with vncviewer and see if they are alive?
>>>>>>
>>>>>>             
>>>>>>             
>>>>> Thank you for your reply.
>>>>> I can't connect to the vm if it has died, even with vnc. I tried 
>>>>> this already, no reply from the host.
>>>>> I don't have physical access to the server because its located in a 
>>>>> dc. Only ssh to the host.
>>>>>         
>>>>>           
>>>> ok this mean there is bug in kvm for amd and it not realted to the 
>>>> network
>>>> what is the guest that you use? is it 64/32?
>>>>
>>>>     
>>>>         
>>> I'm using Debian Etch 32 Bit for all guests
>>> with Kernel Linux apache 2.6.18-5-486 #1 Tue Oct 2 23:38:54 UTC 2007 
>>> i686 GNU/Linux
>>>
>>> ------------------------------------------------------------------------- 
>>>
>>>   
>>>       
>> ok but what more interesting me is:
>> if you run just one vm
>> is it work?
>>     
> Yes. If I run only 1 vm, it runs days without problems.
> I have sometimes have peaks in reply time from the guest.
> e.g.
> bytes=32 time=34ms TTL=57
> bytes=32 time=34ms TTL=57
> bytes=32 time=119ms TTL=57
> bytes=32 time=400ms TTL=57
> bytes=32 time=540ms TTL=57
> bytes=32 time=947ms TTL=57
> bytes=32 time=754ms TTL=57
> bytes=32 time=850ms TTL=57
> bytes=32 time=925ms TTL=57
> bytes=32 time=2032ms TTL=57
> bytes=32 time=897ms TTL=57
> bytes=32 time=197ms TTL=57
> bytes=32 time=34ms TTL=57
> bytes=32 time=34ms TTL=57
>
> but if I ping the host server the same time, the reply time is normal, 
> around 34ms
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> kvm-devel mailing list
> kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/kvm-devel
>   
ok sound interesting,
you are sure you are using kvm53 modules?
(one issue like that solved at kvm53, but maybe we have another one...)

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: VM networking stops working
       [not found]                     ` <4744A881.1060201-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
@ 2007-11-21 22:06                       ` Mike
       [not found]                         ` <4744ABD8.8020409-kcIQ9yavhvsV/sYkb9DZNQ@public.gmane.org>
  0 siblings, 1 reply; 12+ messages in thread
From: Mike @ 2007-11-21 22:06 UTC (permalink / raw)
  To: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f

Izik Eidus schrieb:
> Mike wrote:
>> Izik Eidus schrieb:
>>  
>>> Mike wrote:
>>>    
>>>> Izik Eidus schrieb:
>>>>  
>>>>      
>>>>> Mike wrote:
>>>>>           
>>>>>> Hello,
>>>>>> I have a problem with my virtual machines. I am using kvm53
>>>>>> on an AMD Athlon(tm) 64 X2 Dual Core Processor 5000+.
>>>>>> As distribution I use Ubuntu server 7.10.
>>>>>> As kernel I use Linux 2.6.22-14-server #1 SMP Sun Oct 14 22:09:15 
>>>>>> GMT 2007 x86_64 GNU/Linux.
>>>>>>
>>>>>> I have 3 different vms running. At the beginning they all work 
>>>>>> without a problem.
>>>>>> Then, after one moment to another one or more stop working.
>>>>>> I did a ping test to all the machines, the vm with the failure 
>>>>>> stops pinging.
>>>>>> The result from external ping: Destination host unreachable.
>>>>>> The same result from a ping onto the main server.
>>>>>>
>>>>>> Here are my configuration files:
>>>>>> cat /etc/qemu-ifup
>>>>>> #!/bin/sh
>>>>>> /sbin/ifconfig $1 0.0.0.0 promisc up
>>>>>> /usr/sbin/brctl addif br0 $1
>>>>>> sleep 2
>>>>>> ------------------------------------------------------------------------ 
>>>>>>
>>>>>> cat /etc/network/interfaces
>>>>>> auto lo
>>>>>> iface lo inet loopback
>>>>>>
>>>>>> auto br0
>>>>>> iface br0 inet static
>>>>>> address 195.24.77.169
>>>>>> netmask 255.255.255.0
>>>>>> gateway 195.24.77.1
>>>>>> bridge_ports eth0
>>>>>> bridge_stp off
>>>>>> bridge_maxwait 5
>>>>>> ------------------------------------------------------------------------ 
>>>>>>
>>>>>> I use scripts for starting the vms:
>>>>>> kvm -hda apache.img -hdb apache_storage.img -m 512 -boot c -net 
>>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:01 -net tap -nographic -daemonize
>>>>>> kvm -hda postfix.img -m 512 -boot c -net 
>>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:12 -net tap -nographic -daemonize
>>>>>> kvm -hda openvpn.img -m 256 -boot c -net 
>>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:03 -net tap -nographic -daemonize
>>>>>> ------------------------------------------------------------------------ 
>>>>>>
>>>>>> The tapN devices are being created at the startup of the vms.
>>>>>>
>>>>>> When I run dmesg I have this:
>>>>>> [  112.390943] br0: port 2(tap1) entering learning state
>>>>>> [  112.391919] br0: port 3(tap0) entering learning state
>>>>>> [  112.397605] br0: port 4(tap2) entering learning state
>>>>>> [  122.674313] tap0: no IPv6 routers present
>>>>>> [  122.729185] tap1: no IPv6 routers present
>>>>>> [  122.843901] tap2: no IPv6 routers present
>>>>>> [  123.336081] kvm: emulating exchange as write
>>>>>> [  127.367940] br0: topology change detected, propagating
>>>>>> [  127.367949] br0: port 4(tap2) entering forwarding state
>>>>>> [  127.362971] br0: topology change detected, propagating
>>>>>> [  127.362977] br0: port 2(tap1) entering forwarding state
>>>>>> [  127.362978] br0: topology change detected, propagating
>>>>>> [  127.362980] br0: port 3(tap0) entering forwarding state
>>>>>>
>>>>>> No indication of a problem, nowhere in the log files.
>>>>>>
>>>>>> I don't know if the vms are freezing, or the network forwarding 
>>>>>> stops working.
>>>>>> I also activated "net.ipv4.conf.default.forwarding=1"
>>>>>>
>>>>>> I googled days for that problem, but without a result.
>>>>>> I hope that maybe you have a solution for this problem. That 
>>>>>> would be great, because I prefer using kvm instead of other 
>>>>>> virtualization systems.
>>>>>>
>>>>>> Thank you very much in advance,
>>>>>> Greetings from Luxembourg.
>>>>>>
>>>>>> Have a nice day,
>>>>>> Mike
>>>>>>                 
>>>>> it might be not the network that die, but the whole vm
>>>>> can you run the vms with -vnc ? (first vm -vnc :1 secoend vm -vnc 
>>>>> :2 third vm -vnc :3)
>>>>> and try to connect to them with vncviewer and see if they are alive?
>>>>>
>>>>>             
>>>> Thank you for your reply.
>>>> I can't connect to the vm if it has died, even with vnc. I tried 
>>>> this already, no reply from the host.
>>>> I don't have physical access to the server because its located in a 
>>>> dc. Only ssh to the host.
>>>>         
>>> ok this mean there is bug in kvm for amd and it not realted to the 
>>> network
>>> what is the guest that you use? is it 64/32?
>>>
>>>     
>> I'm using Debian Etch 32 Bit for all guests
>> with Kernel Linux apache 2.6.18-5-486 #1 Tue Oct 2 23:38:54 UTC 2007 
>> i686 GNU/Linux
>>
>> ------------------------------------------------------------------------- 
>>
>>   
> ok but what more interesting me is:
> if you run just one vm
> is it work?
Yes. If I run only 1 vm, it runs days without problems.
I have sometimes have peaks in reply time from the guest.
e.g.
bytes=32 time=34ms TTL=57
bytes=32 time=34ms TTL=57
bytes=32 time=119ms TTL=57
bytes=32 time=400ms TTL=57
bytes=32 time=540ms TTL=57
bytes=32 time=947ms TTL=57
bytes=32 time=754ms TTL=57
bytes=32 time=850ms TTL=57
bytes=32 time=925ms TTL=57
bytes=32 time=2032ms TTL=57
bytes=32 time=897ms TTL=57
bytes=32 time=197ms TTL=57
bytes=32 time=34ms TTL=57
bytes=32 time=34ms TTL=57

but if I ping the host server the same time, the reply time is normal, 
around 34ms

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: VM networking stops working
       [not found] ` <47449A93.5090007-kcIQ9yavhvsV/sYkb9DZNQ@public.gmane.org>
  2007-11-21 21:22   ` Izik Eidus
@ 2007-11-21 22:09   ` Lynn Kerby
       [not found]     ` <E729CB4A-EA27-46BD-A54D-47D87117DE90-br2HoPxSX4msTnJN9+BGXg@public.gmane.org>
  1 sibling, 1 reply; 12+ messages in thread
From: Lynn Kerby @ 2007-11-21 22:09 UTC (permalink / raw)
  To: Mike; +Cc: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f


On Nov 21, 2007, at 12:52 PM, Mike wrote:

> Hello,
> I have a problem with my virtual machines. I am using kvm53
> on an AMD Athlon(tm) 64 X2 Dual Core Processor 5000+.
> As distribution I use Ubuntu server 7.10.
> As kernel I use Linux 2.6.22-14-server #1 SMP Sun Oct 14 22:09:15  
> GMT 2007 x86_64 GNU/Linux.
>
> I have 3 different vms running. At the beginning they all work  
> without a problem.
> Then, after one moment to another one or more stop working.
> I did a ping test to all the machines, the vm with the failure  
> stops pinging.
> The result from external ping: Destination host unreachable.
> The same result from a ping onto the main server.
>
> Here are my configuration files:
> cat /etc/qemu-ifup
> #!/bin/sh
> /sbin/ifconfig $1 0.0.0.0 promisc up
> /usr/sbin/brctl addif br0 $1
> sleep 2
> cat /etc/network/interfaces
> auto lo
> iface lo inet loopback
>
> auto br0
> iface br0 inet static
> address 195.24.77.169
> netmask 255.255.255.0
> gateway 195.24.77.1
> bridge_ports eth0
> bridge_stp off
> bridge_maxwait 5
> I use scripts for starting the vms:
> kvm -hda apache.img -hdb apache_storage.img -m 512 -boot c -net  
> nic,vlan=0,macaddr=00:16:3e:00:00:01 -net tap -nographic -daemonize
> kvm -hda postfix.img -m 512 -boot c -net  
> nic,vlan=0,macaddr=00:16:3e:00:00:12 -net tap -nographic -daemonize
> kvm -hda openvpn.img -m 256 -boot c -net  
> nic,vlan=0,macaddr=00:16:3e:00:00:03 -net tap -nographic -daemonize
> The tapN devices are being created at the startup of the vms.
>
> When I run dmesg I have this:
> [  112.390943] br0: port 2(tap1) entering learning state
> [  112.391919] br0: port 3(tap0) entering learning state
> [  112.397605] br0: port 4(tap2) entering learning state
> [  122.674313] tap0: no IPv6 routers present
> [  122.729185] tap1: no IPv6 routers present
> [  122.843901] tap2: no IPv6 routers present
> [  123.336081] kvm: emulating exchange as write
> [  127.367940] br0: topology change detected, propagating
> [  127.367949] br0: port 4(tap2) entering forwarding state
> [  127.362971] br0: topology change detected, propagating
> [  127.362977] br0: port 2(tap1) entering forwarding state
> [  127.362978] br0: topology change detected, propagating
> [  127.362980] br0: port 3(tap0) entering forwarding state
>
> No indication of a problem, nowhere in the log files.
>
> I don't know if the vms are freezing, or the network forwarding  
> stops working.
> I also activated "net.ipv4.conf.default.forwarding=1"
>
> I googled days for that problem, but without a result.
> I hope that maybe you have a solution for this problem. That would  
> be great, because I prefer using kvm instead of other  
> virtualization systems.
>
> Thank you very much in advance,
> Greetings from Luxembourg.
>
> Have a nice day,
> Mike

I had similar problems for my first week or so of trying to use KVM  
for my virtualization needs.  My problem was solved by adding a  
"model=rtl8139" to the first -net option on the kvm/qemu command.   
The default ne2k emulation seems to be broken.  Another possibility  
is problems with macaddr.  In my testing I found that several of my  
machines were getting the same macaddr by default so now I specify  
them explicitly for each VM (or generate a dummy one based on the  
time for testing).

An example from one of my KVMMs (hopefully line wrapping won't  
interfere):

/usr/local/bin/qemu-system-x86_64 -name DOM9-BSD62 -hda /vm_img/ 
bsd_in1/dom9-disk1.qcow \
	-hdb /vm_img/bsd_in1/dom9-disk2.raw -m 512 -boot c -smp 2 -usb - 
usbdevice tablet -vnc :9 \
	-serial telnet::4109,server,nowait -monitor telnet:: 
4209,server,nowait \
	-net nic,vlan=0,macaddr=52:54:09:12:34:56,model=rtl8139 \
	-net tap,vlan=0,ifname=tap1,script=/etc/kvm/kvm-ifup


Lynn Kerby

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: VM networking stops working
       [not found]                             ` <4744ABBC.30708-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
@ 2007-11-21 22:15                               ` Mike
  2007-11-21 22:32                               ` Mike
  1 sibling, 0 replies; 12+ messages in thread
From: Mike @ 2007-11-21 22:15 UTC (permalink / raw)
  To: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f

Izik Eidus schrieb:
> Mike wrote:
>> Izik Eidus schrieb:
>>  
>>> Mike wrote:
>>>    
>>>> Izik Eidus schrieb:
>>>>  
>>>>      
>>>>> Mike wrote:
>>>>>           
>>>>>> Izik Eidus schrieb:
>>>>>>  
>>>>>>               
>>>>>>> Mike wrote:
>>>>>>>                      
>>>>>>>> Hello,
>>>>>>>> I have a problem with my virtual machines. I am using kvm53
>>>>>>>> on an AMD Athlon(tm) 64 X2 Dual Core Processor 5000+.
>>>>>>>> As distribution I use Ubuntu server 7.10.
>>>>>>>> As kernel I use Linux 2.6.22-14-server #1 SMP Sun Oct 14 
>>>>>>>> 22:09:15 GMT 2007 x86_64 GNU/Linux.
>>>>>>>>
>>>>>>>> I have 3 different vms running. At the beginning they all work 
>>>>>>>> without a problem.
>>>>>>>> Then, after one moment to another one or more stop working.
>>>>>>>> I did a ping test to all the machines, the vm with the failure 
>>>>>>>> stops pinging.
>>>>>>>> The result from external ping: Destination host unreachable.
>>>>>>>> The same result from a ping onto the main server.
>>>>>>>>
>>>>>>>> Here are my configuration files:
>>>>>>>> cat /etc/qemu-ifup
>>>>>>>> #!/bin/sh
>>>>>>>> /sbin/ifconfig $1 0.0.0.0 promisc up
>>>>>>>> /usr/sbin/brctl addif br0 $1
>>>>>>>> sleep 2
>>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>>
>>>>>>>> cat /etc/network/interfaces
>>>>>>>> auto lo
>>>>>>>> iface lo inet loopback
>>>>>>>>
>>>>>>>> auto br0
>>>>>>>> iface br0 inet static
>>>>>>>> address 195.24.77.169
>>>>>>>> netmask 255.255.255.0
>>>>>>>> gateway 195.24.77.1
>>>>>>>> bridge_ports eth0
>>>>>>>> bridge_stp off
>>>>>>>> bridge_maxwait 5
>>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>>
>>>>>>>> I use scripts for starting the vms:
>>>>>>>> kvm -hda apache.img -hdb apache_storage.img -m 512 -boot c -net 
>>>>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:01 -net tap -nographic 
>>>>>>>> -daemonize
>>>>>>>> kvm -hda postfix.img -m 512 -boot c -net 
>>>>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:12 -net tap -nographic 
>>>>>>>> -daemonize
>>>>>>>> kvm -hda openvpn.img -m 256 -boot c -net 
>>>>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:03 -net tap -nographic 
>>>>>>>> -daemonize
>>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>>
>>>>>>>> The tapN devices are being created at the startup of the vms.
>>>>>>>>
>>>>>>>> When I run dmesg I have this:
>>>>>>>> [  112.390943] br0: port 2(tap1) entering learning state
>>>>>>>> [  112.391919] br0: port 3(tap0) entering learning state
>>>>>>>> [  112.397605] br0: port 4(tap2) entering learning state
>>>>>>>> [  122.674313] tap0: no IPv6 routers present
>>>>>>>> [  122.729185] tap1: no IPv6 routers present
>>>>>>>> [  122.843901] tap2: no IPv6 routers present
>>>>>>>> [  123.336081] kvm: emulating exchange as write
>>>>>>>> [  127.367940] br0: topology change detected, propagating
>>>>>>>> [  127.367949] br0: port 4(tap2) entering forwarding state
>>>>>>>> [  127.362971] br0: topology change detected, propagating
>>>>>>>> [  127.362977] br0: port 2(tap1) entering forwarding state
>>>>>>>> [  127.362978] br0: topology change detected, propagating
>>>>>>>> [  127.362980] br0: port 3(tap0) entering forwarding state
>>>>>>>>
>>>>>>>> No indication of a problem, nowhere in the log files.
>>>>>>>>
>>>>>>>> I don't know if the vms are freezing, or the network forwarding 
>>>>>>>> stops working.
>>>>>>>> I also activated "net.ipv4.conf.default.forwarding=1"
>>>>>>>>
>>>>>>>> I googled days for that problem, but without a result.
>>>>>>>> I hope that maybe you have a solution for this problem. That 
>>>>>>>> would be great, because I prefer using kvm instead of other 
>>>>>>>> virtualization systems.
>>>>>>>>
>>>>>>>> Thank you very much in advance,
>>>>>>>> Greetings from Luxembourg.
>>>>>>>>
>>>>>>>> Have a nice day,
>>>>>>>> Mike
>>>>>>>>                               
>>>>>>> it might be not the network that die, but the whole vm
>>>>>>> can you run the vms with -vnc ? (first vm -vnc :1 secoend vm 
>>>>>>> -vnc :2 third vm -vnc :3)
>>>>>>> and try to connect to them with vncviewer and see if they are 
>>>>>>> alive?
>>>>>>>
>>>>>>>                         
>>>>>> Thank you for your reply.
>>>>>> I can't connect to the vm if it has died, even with vnc. I tried 
>>>>>> this already, no reply from the host.
>>>>>> I don't have physical access to the server because its located in 
>>>>>> a dc. Only ssh to the host.
>>>>>>                   
>>>>> ok this mean there is bug in kvm for amd and it not realted to the 
>>>>> network
>>>>> what is the guest that you use? is it 64/32?
>>>>>
>>>>>             
>>>> I'm using Debian Etch 32 Bit for all guests
>>>> with Kernel Linux apache 2.6.18-5-486 #1 Tue Oct 2 23:38:54 UTC 
>>>> 2007 i686 GNU/Linux
>>>>
>>>> ------------------------------------------------------------------------- 
>>>>
>>>>         
>>> ok but what more interesting me is:
>>> if you run just one vm
>>> is it work?
>>>     
>> Yes. If I run only 1 vm, it runs days without problems.
>> I have sometimes have peaks in reply time from the guest.
>> e.g.
>> bytes=32 time=34ms TTL=57
>> bytes=32 time=34ms TTL=57
>> bytes=32 time=119ms TTL=57
>> bytes=32 time=400ms TTL=57
>> bytes=32 time=540ms TTL=57
>> bytes=32 time=947ms TTL=57
>> bytes=32 time=754ms TTL=57
>> bytes=32 time=850ms TTL=57
>> bytes=32 time=925ms TTL=57
>> bytes=32 time=2032ms TTL=57
>> bytes=32 time=897ms TTL=57
>> bytes=32 time=197ms TTL=57
>> bytes=32 time=34ms TTL=57
>> bytes=32 time=34ms TTL=57
>>
>> but if I ping the host server the same time, the reply time is 
>> normal, around 34ms
>>
>> ------------------------------------------------------------------------- 
>>
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2005.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>> _______________________________________________
>> kvm-devel mailing list
>> kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
>> https://lists.sourceforge.net/lists/listinfo/kvm-devel
>>   
> ok sound interesting,
> you are sure you are using kvm53 modules?
> (one issue like that solved at kvm53, but maybe we have another one...)
>
I checked the version of the modules, here are the results:
/sbin/modinfo kvm
filename:       /lib/modules/2.6.22-14-server/extra/kvm.ko
license:        GPL
author:         Qumranet
version:        kvm-53
srcversion:     185E42DE9D9A49E63243A57
depends:
vermagic:       2.6.22-14-server SMP mod_unload

/sbin/modinfo kvm_amd
filename:       /lib/modules/2.6.22-14-server/extra/kvm-amd.ko
license:        GPL
author:         Qumranet
version:        kvm-53
srcversion:     F215F1AB760E357FF762B4D
depends:        kvm,kvm
vermagic:       2.6.22-14-server SMP mod_unload


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: VM networking stops working
       [not found]                             ` <4744ABBC.30708-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
  2007-11-21 22:15                               ` Mike
@ 2007-11-21 22:32                               ` Mike
  1 sibling, 0 replies; 12+ messages in thread
From: Mike @ 2007-11-21 22:32 UTC (permalink / raw)
  To: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f

Izik Eidus schrieb:
> Mike wrote:
>> Izik Eidus schrieb:
>>  
>>> Mike wrote:
>>>    
>>>> Izik Eidus schrieb:
>>>>  
>>>>      
>>>>> Mike wrote:
>>>>>           
>>>>>> Izik Eidus schrieb:
>>>>>>  
>>>>>>               
>>>>>>> Mike wrote:
>>>>>>>                      
>>>>>>>> Hello,
>>>>>>>> I have a problem with my virtual machines. I am using kvm53
>>>>>>>> on an AMD Athlon(tm) 64 X2 Dual Core Processor 5000+.
>>>>>>>> As distribution I use Ubuntu server 7.10.
>>>>>>>> As kernel I use Linux 2.6.22-14-server #1 SMP Sun Oct 14 
>>>>>>>> 22:09:15 GMT 2007 x86_64 GNU/Linux.
>>>>>>>>
>>>>>>>> I have 3 different vms running. At the beginning they all work 
>>>>>>>> without a problem.
>>>>>>>> Then, after one moment to another one or more stop working.
>>>>>>>> I did a ping test to all the machines, the vm with the failure 
>>>>>>>> stops pinging.
>>>>>>>> The result from external ping: Destination host unreachable.
>>>>>>>> The same result from a ping onto the main server.
>>>>>>>>
>>>>>>>> Here are my configuration files:
>>>>>>>> cat /etc/qemu-ifup
>>>>>>>> #!/bin/sh
>>>>>>>> /sbin/ifconfig $1 0.0.0.0 promisc up
>>>>>>>> /usr/sbin/brctl addif br0 $1
>>>>>>>> sleep 2
>>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>>
>>>>>>>> cat /etc/network/interfaces
>>>>>>>> auto lo
>>>>>>>> iface lo inet loopback
>>>>>>>>
>>>>>>>> auto br0
>>>>>>>> iface br0 inet static
>>>>>>>> address 195.24.77.169
>>>>>>>> netmask 255.255.255.0
>>>>>>>> gateway 195.24.77.1
>>>>>>>> bridge_ports eth0
>>>>>>>> bridge_stp off
>>>>>>>> bridge_maxwait 5
>>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>>
>>>>>>>> I use scripts for starting the vms:
>>>>>>>> kvm -hda apache.img -hdb apache_storage.img -m 512 -boot c -net 
>>>>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:01 -net tap -nographic 
>>>>>>>> -daemonize
>>>>>>>> kvm -hda postfix.img -m 512 -boot c -net 
>>>>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:12 -net tap -nographic 
>>>>>>>> -daemonize
>>>>>>>> kvm -hda openvpn.img -m 256 -boot c -net 
>>>>>>>> nic,vlan=0,macaddr=00:16:3e:00:00:03 -net tap -nographic 
>>>>>>>> -daemonize
>>>>>>>> ------------------------------------------------------------------------ 
>>>>>>>>
>>>>>>>> The tapN devices are being created at the startup of the vms.
>>>>>>>>
>>>>>>>> When I run dmesg I have this:
>>>>>>>> [  112.390943] br0: port 2(tap1) entering learning state
>>>>>>>> [  112.391919] br0: port 3(tap0) entering learning state
>>>>>>>> [  112.397605] br0: port 4(tap2) entering learning state
>>>>>>>> [  122.674313] tap0: no IPv6 routers present
>>>>>>>> [  122.729185] tap1: no IPv6 routers present
>>>>>>>> [  122.843901] tap2: no IPv6 routers present
>>>>>>>> [  123.336081] kvm: emulating exchange as write
>>>>>>>> [  127.367940] br0: topology change detected, propagating
>>>>>>>> [  127.367949] br0: port 4(tap2) entering forwarding state
>>>>>>>> [  127.362971] br0: topology change detected, propagating
>>>>>>>> [  127.362977] br0: port 2(tap1) entering forwarding state
>>>>>>>> [  127.362978] br0: topology change detected, propagating
>>>>>>>> [  127.362980] br0: port 3(tap0) entering forwarding state
>>>>>>>>
>>>>>>>> No indication of a problem, nowhere in the log files.
>>>>>>>>
>>>>>>>> I don't know if the vms are freezing, or the network forwarding 
>>>>>>>> stops working.
>>>>>>>> I also activated "net.ipv4.conf.default.forwarding=1"
>>>>>>>>
>>>>>>>> I googled days for that problem, but without a result.
>>>>>>>> I hope that maybe you have a solution for this problem. That 
>>>>>>>> would be great, because I prefer using kvm instead of other 
>>>>>>>> virtualization systems.
>>>>>>>>
>>>>>>>> Thank you very much in advance,
>>>>>>>> Greetings from Luxembourg.
>>>>>>>>
>>>>>>>> Have a nice day,
>>>>>>>> Mike
>>>>>>>>                               
>>>>>>> it might be not the network that die, but the whole vm
>>>>>>> can you run the vms with -vnc ? (first vm -vnc :1 secoend vm 
>>>>>>> -vnc :2 third vm -vnc :3)
>>>>>>> and try to connect to them with vncviewer and see if they are 
>>>>>>> alive?
>>>>>>>
>>>>>>>                         
>>>>>> Thank you for your reply.
>>>>>> I can't connect to the vm if it has died, even with vnc. I tried 
>>>>>> this already, no reply from the host.
>>>>>> I don't have physical access to the server because its located in 
>>>>>> a dc. Only ssh to the host.
>>>>>>                   
>>>>> ok this mean there is bug in kvm for amd and it not realted to the 
>>>>> network
>>>>> what is the guest that you use? is it 64/32?
>>>>>
>>>>>             
>>>> I'm using Debian Etch 32 Bit for all guests
>>>> with Kernel Linux apache 2.6.18-5-486 #1 Tue Oct 2 23:38:54 UTC 
>>>> 2007 i686 GNU/Linux
>>>>
>>>> ------------------------------------------------------------------------- 
>>>>
>>>>         
>>> ok but what more interesting me is:
>>> if you run just one vm
>>> is it work?
>>>     
>> Yes. If I run only 1 vm, it runs days without problems.
>> I have sometimes have peaks in reply time from the guest.
>> e.g.
>> bytes=32 time=34ms TTL=57
>> bytes=32 time=34ms TTL=57
>> bytes=32 time=119ms TTL=57
>> bytes=32 time=400ms TTL=57
>> bytes=32 time=540ms TTL=57
>> bytes=32 time=947ms TTL=57
>> bytes=32 time=754ms TTL=57
>> bytes=32 time=850ms TTL=57
>> bytes=32 time=925ms TTL=57
>> bytes=32 time=2032ms TTL=57
>> bytes=32 time=897ms TTL=57
>> bytes=32 time=197ms TTL=57
>> bytes=32 time=34ms TTL=57
>> bytes=32 time=34ms TTL=57
>>
>> but if I ping the host server the same time, the reply time is 
>> normal, around 34ms
>>
>> ------------------------------------------------------------------------- 
>>
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2005.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>> _______________________________________________
>> kvm-devel mailing list
>> kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
>> https://lists.sourceforge.net/lists/listinfo/kvm-devel
>>   
> ok sound interesting,
> you are sure you are using kvm53 modules?
> (one issue like that solved at kvm53, but maybe we have another one...)
>

> I had similar problems for my first week or so of trying to use KVM
> for my virtualization needs. My problem was solved by adding a
> "model=rtl8139" to the first -net option on the kvm/qemu command.
> The default ne2k emulation seems to be broken. Another possibility
> is problems with macaddr. In my testing I found that several of my
> machines were getting the same macaddr by default so now I specify
> them explicitly for each VM (or generate a dummy one based on the
> time for testing).
>
> An example from one of my KVMMs (hopefully line wrapping won't
> interfere):
>
> /usr/local/bin/qemu-system-x86_64 -name DOM9-BSD62 -hda /vm_img/
> bsd_in1/dom9-disk1.qcow \
> -hdb /vm_img/bsd_in1/dom9-disk2.raw -m 512 -boot c -smp 2 -usb -
> usbdevice tablet -vnc :9 \
> -serial telnet::4109,server,nowait -monitor telnet::
> 4209,server,nowait \
> -net nic,vlan=0,macaddr=52:54:09:12:34:56,model=rtl8139 \
> -net tap,vlan=0,ifname=tap1,script=/etc/kvm/kvm-ifup
>
>
> Lynn Kerby
Thank you for this hint,
I'll try this immediately.



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: VM networking stops working
       [not found]     ` <E729CB4A-EA27-46BD-A54D-47D87117DE90-br2HoPxSX4msTnJN9+BGXg@public.gmane.org>
@ 2007-11-22 10:51       ` Mike
  0 siblings, 0 replies; 12+ messages in thread
From: Mike @ 2007-11-22 10:51 UTC (permalink / raw)
  To: kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f

Lynn Kerby schrieb:
>
> On Nov 21, 2007, at 12:52 PM, Mike wrote:
>
>> Hello,
>> I have a problem with my virtual machines. I am using kvm53
>> on an AMD Athlon(tm) 64 X2 Dual Core Processor 5000+.
>> As distribution I use Ubuntu server 7.10.
>> As kernel I use Linux 2.6.22-14-server #1 SMP Sun Oct 14 22:09:15 GMT 
>> 2007 x86_64 GNU/Linux.
>>
>> I have 3 different vms running. At the beginning they all work 
>> without a problem.
>> Then, after one moment to another one or more stop working.
>> I did a ping test to all the machines, the vm with the failure stops 
>> pinging.
>> The result from external ping: Destination host unreachable.
>> The same result from a ping onto the main server.
>>
>> Here are my configuration files:
>> cat /etc/qemu-ifup
>> #!/bin/sh
>> /sbin/ifconfig $1 0.0.0.0 promisc up
>> /usr/sbin/brctl addif br0 $1
>> sleep 2
>> cat /etc/network/interfaces
>> auto lo
>> iface lo inet loopback
>>
>> auto br0
>> iface br0 inet static
>> address 195.24.77.169
>> netmask 255.255.255.0
>> gateway 195.24.77.1
>> bridge_ports eth0
>> bridge_stp off
>> bridge_maxwait 5
>> I use scripts for starting the vms:
>> kvm -hda apache.img -hdb apache_storage.img -m 512 -boot c -net 
>> nic,vlan=0,macaddr=00:16:3e:00:00:01 -net tap -nographic -daemonize
>> kvm -hda postfix.img -m 512 -boot c -net 
>> nic,vlan=0,macaddr=00:16:3e:00:00:12 -net tap -nographic -daemonize
>> kvm -hda openvpn.img -m 256 -boot c -net 
>> nic,vlan=0,macaddr=00:16:3e:00:00:03 -net tap -nographic -daemonize
>> The tapN devices are being created at the startup of the vms.
>>
>> When I run dmesg I have this:
>> [  112.390943] br0: port 2(tap1) entering learning state
>> [  112.391919] br0: port 3(tap0) entering learning state
>> [  112.397605] br0: port 4(tap2) entering learning state
>> [  122.674313] tap0: no IPv6 routers present
>> [  122.729185] tap1: no IPv6 routers present
>> [  122.843901] tap2: no IPv6 routers present
>> [  123.336081] kvm: emulating exchange as write
>> [  127.367940] br0: topology change detected, propagating
>> [  127.367949] br0: port 4(tap2) entering forwarding state
>> [  127.362971] br0: topology change detected, propagating
>> [  127.362977] br0: port 2(tap1) entering forwarding state
>> [  127.362978] br0: topology change detected, propagating
>> [  127.362980] br0: port 3(tap0) entering forwarding state
>>
>> No indication of a problem, nowhere in the log files.
>>
>> I don't know if the vms are freezing, or the network forwarding stops 
>> working.
>> I also activated "net.ipv4.conf.default.forwarding=1"
>>
>> I googled days for that problem, but without a result.
>> I hope that maybe you have a solution for this problem. That would be 
>> great, because I prefer using kvm instead of other virtualization 
>> systems.
>>
>> Thank you very much in advance,
>> Greetings from Luxembourg.
>>
>> Have a nice day,
>> Mike
>
> I had similar problems for my first week or so of trying to use KVM 
> for my virtualization needs.  My problem was solved by adding a 
> "model=rtl8139" to the first -net option on the kvm/qemu command.  The 
> default ne2k emulation seems to be broken.  Another possibility is 
> problems with macaddr.  In my testing I found that several of my 
> machines were getting the same macaddr by default so now I specify 
> them explicitly for each VM (or generate a dummy one based on the time 
> for testing).
>
> An example from one of my KVMMs (hopefully line wrapping won't 
> interfere):
>
> /usr/local/bin/qemu-system-x86_64 -name DOM9-BSD62 -hda 
> /vm_img/bsd_in1/dom9-disk1.qcow \
>     -hdb /vm_img/bsd_in1/dom9-disk2.raw -m 512 -boot c -smp 2 -usb 
> -usbdevice tablet -vnc :9 \
>     -serial telnet::4109,server,nowait -monitor 
> telnet::4209,server,nowait \
>     -net nic,vlan=0,macaddr=52:54:09:12:34:56,model=rtl8139 \
>     -net tap,vlan=0,ifname=tap1,script=/etc/kvm/kvm-ifup
>
>
> Lynn Kerby
>
It's working now.
I added model=rtl8139 to the start command of all vms.
Now, the network connection works without problems.
I did a test the whole night long.
Thank you for your help.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2007-11-22 10:51 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-11-21 20:52 VM networking stops working Mike
     [not found] ` <47449A93.5090007-kcIQ9yavhvsV/sYkb9DZNQ@public.gmane.org>
2007-11-21 21:22   ` Izik Eidus
     [not found]     ` <4744A18D.2050705-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
2007-11-21 21:40       ` Mike
     [not found]         ` <4744A5E5.9060902-kcIQ9yavhvsV/sYkb9DZNQ@public.gmane.org>
2007-11-21 21:40           ` Izik Eidus
     [not found]             ` <4744A5CA.8050604-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
2007-11-21 21:53               ` Mike
     [not found]                 ` <4744A8CA.5070702-kcIQ9yavhvsV/sYkb9DZNQ@public.gmane.org>
2007-11-21 21:52                   ` Izik Eidus
     [not found]                     ` <4744A881.1060201-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
2007-11-21 22:06                       ` Mike
     [not found]                         ` <4744ABD8.8020409-kcIQ9yavhvsV/sYkb9DZNQ@public.gmane.org>
2007-11-21 22:05                           ` Izik Eidus
     [not found]                             ` <4744ABBC.30708-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
2007-11-21 22:15                               ` Mike
2007-11-21 22:32                               ` Mike
2007-11-21 22:09   ` Lynn Kerby
     [not found]     ` <E729CB4A-EA27-46BD-A54D-47D87117DE90-br2HoPxSX4msTnJN9+BGXg@public.gmane.org>
2007-11-22 10:51       ` Mike

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox