KVM doesn't send an arp announce after live migrating a domain

KVM doesn't send an arp announce after live migrating a domain

[Nils Cant]

Hey guys,

not sure if this is a bug or a feature request. It's just something 
we've noticed and are having problems with.

We're using the qemu-kvm lenny-backports package on Debian 5.0.5.

When doing a live migration from the virsh shell, the server in question 
becomes unreachable because the ARP cache on our switches still thinks 
the server is on another port.

As soon as the server sends out some traffic, such as a ping, the ARP 
cache get's updated as expected. If it does nothing, the server remains 
unreachable until the ARP cache expires on the switches. (in our case 4 
hours)

We would like to be able to do live migration for customer machines on 
which we have no access, so we really need KVM to send out an ARP 
announcement/gratuitous ARP when doing a live migration.

Could anyone tell me if this is a bug in KVM, libvirt, or the debian 
qemu-kvm package? (or if I'm doing something wrong? :-) )

I've been tcpdumping the bridged network interfaces on the hosts while 
doing the migrate, and couldn't see any ARP broadcasts.

Debian 5.0.5
Kernel: 2.6.32-bpo.3-amd64
qemu-kvm                      0.12.4+dfsg-1~bpo50+2
libvirt0                      0.7.6-1~bpo50+1

migrate --live testserver qemu+ssh://192.168.1.3/system

Thanks in advance,

Nils

Re: KVM doesn't send an arp announce after live migrating a domain

[Michael Tokarev]

25.08.2010 11:52, Nils Cant wrote:
> Hey guys,
> 
> not sure if this is a bug or a feature request. It's just something
> we've noticed and are having problems with.
> 
> We're using the qemu-kvm lenny-backports package on Debian 5.0.5.
> 
> When doing a live migration from the virsh shell, the server in question
> becomes unreachable because the ARP cache on our switches still thinks
> the server is on another port.
> 
> As soon as the server sends out some traffic, such as a ping, the ARP
> cache get's updated as expected. If it does nothing, the server remains
> unreachable until the ARP cache expires on the switches. (in our case 4
> hours)
> 
> We would like to be able to do live migration for customer machines on
> which we have no access, so we really need KVM to send out an ARP
> announcement/gratuitous ARP when doing a live migration.
> 
> Could anyone tell me if this is a bug in KVM, libvirt, or the debian
> qemu-kvm package? (or if I'm doing something wrong? :-) )

It's probably a bug in your understanding ;)

Jokes aside, the thing is that kvm does not know what is
an ARP and what is an IP address.  It emulates a hardware
network card, which never sends any ARP out by its own,
it is the operating system IP stack who's doing that.
That network card as emulated by kvm does not know what
IP addresses are assigned to it inside the guest (there
may be many, or may be none at all), so it just can not
send the ARPs.

These ARPs should be sent by guest.  Another question is
how to force/tell it to do so, and this is, again, depends
on the guest operating system, number of addresses assigned
to the interface and so on.

The mechanism to trigger it may be based on link status
of the card for example - kvm may lower it for a few ms
right after migration, to indicate that the "cord" were
un-plugged and plugged back, to force the guest to do
whatever it needs to do...  But that's just a possibility
for future development.

/mjt

Re: KVM doesn't send an arp announce after live migrating a domain

[Gleb Natapov]

On Wed, Aug 25, 2010 at 09:52:16AM +0200, Nils Cant wrote:
> Hey guys,
> 
> not sure if this is a bug or a feature request. It's just something
> we've noticed and are having problems with.
> 
> We're using the qemu-kvm lenny-backports package on Debian 5.0.5.
> 
> When doing a live migration from the virsh shell, the server in
> question becomes unreachable because the ARP cache on our switches
> still thinks the server is on another port.
> 
> As soon as the server sends out some traffic, such as a ping, the
> ARP cache get's updated as expected. If it does nothing, the server
> remains unreachable until the ARP cache expires on the switches. (in
> our case 4 hours)
> 
> We would like to be able to do live migration for customer machines
> on which we have no access, so we really need KVM to send out an ARP
> announcement/gratuitous ARP when doing a live migration.
> 
> Could anyone tell me if this is a bug in KVM, libvirt, or the debian
> qemu-kvm package? (or if I'm doing something wrong? :-) )
> 
qemu sends gratuitous ARP after migration. Check forward delay setting on your
bridge interface. It should be set to zero.
  
> I've been tcpdumping the bridged network interfaces on the hosts
> while doing the migrate, and couldn't see any ARP broadcasts.
> 
> Debian 5.0.5
> Kernel: 2.6.32-bpo.3-amd64
> qemu-kvm                      0.12.4+dfsg-1~bpo50+2
> libvirt0                      0.7.6-1~bpo50+1
> 
> migrate --live testserver qemu+ssh://192.168.1.3/system
> 
> Thanks in advance,
> 
> Nils
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
			Gleb.

Re: KVM doesn't send an arp announce after live migrating a domain

[Gleb Natapov]

On Wed, Aug 25, 2010 at 12:37:21PM +0400, Michael Tokarev wrote:
> 25.08.2010 11:52, Nils Cant wrote:
> > Hey guys,
> > 
> > not sure if this is a bug or a feature request. It's just something
> > we've noticed and are having problems with.
> > 
> > We're using the qemu-kvm lenny-backports package on Debian 5.0.5.
> > 
> > When doing a live migration from the virsh shell, the server in question
> > becomes unreachable because the ARP cache on our switches still thinks
> > the server is on another port.
> > 
> > As soon as the server sends out some traffic, such as a ping, the ARP
> > cache get's updated as expected. If it does nothing, the server remains
> > unreachable until the ARP cache expires on the switches. (in our case 4
> > hours)
> > 
> > We would like to be able to do live migration for customer machines on
> > which we have no access, so we really need KVM to send out an ARP
> > announcement/gratuitous ARP when doing a live migration.
> > 
> > Could anyone tell me if this is a bug in KVM, libvirt, or the debian
> > qemu-kvm package? (or if I'm doing something wrong? :-) )
> 
> It's probably a bug in your understanding ;)
> 
> Jokes aside, the thing is that kvm does not know what is
> an ARP and what is an IP address.  It emulates a hardware
> network card, which never sends any ARP out by its own,
> it is the operating system IP stack who's doing that.
> That network card as emulated by kvm does not know what
> IP addresses are assigned to it inside the guest (there
> may be many, or may be none at all), so it just can not
> send the ARPs.
> 
True. Although qemu sends gratuitous ARP the IP field there is
incorrect. It is done to update layer 2 topology, not layer 3.

> These ARPs should be sent by guest.  Another question is
> how to force/tell it to do so, and this is, again, depends
> on the guest operating system, number of addresses assigned
> to the interface and so on.
> 
> The mechanism to trigger it may be based on link status
> of the card for example - kvm may lower it for a few ms
> right after migration, to indicate that the "cord" were
> un-plugged and plugged back, to force the guest to do
> whatever it needs to do...  But that's just a possibility
> for future development.
> 
> /mjt
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
			Gleb.

Re: KVM doesn't send an arp announce after live migrating a domain

[Nils Cant]

On 08/25/2010 10:38 AM, Gleb Natapov wrote:
> qemu sends gratuitous ARP after migration. Check forward delay setting on your
> bridge interface. It should be set to zero.
>

Aha! That fixed it. Turns out that debian bridge-utils sets the default 
to 15 for bridges.
Manually setting it to 0 with 'brctl setfd br0 0' or setting the 
'bridge_fd' parameter to 0 in /etc/network/interfaces solves the issue.

Thanks for the help!

Nils

Re: KVM doesn't send an arp announce after live migrating a domain

[Avi Kivity]

  On 08/25/2010 12:21 PM, Nils Cant wrote:
> On 08/25/2010 10:38 AM, Gleb Natapov wrote:
>> qemu sends gratuitous ARP after migration. Check forward delay 
>> setting on your
>> bridge interface. It should be set to zero.
>>
>
> Aha! That fixed it. Turns out that debian bridge-utils sets the 
> default to 15 for bridges.
> Manually setting it to 0 with 'brctl setfd br0 0' or setting the 
> 'bridge_fd' parameter to 0 in /etc/network/interfaces solves the issue.
>

I think libvirt is doing something about this, copying list for further 
info.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: KVM doesn't send an arp announce after live migrating a domain

[Michael Tokarev]

Gleb Natapov wrote:
> On Wed, Aug 25, 2010 at 12:37:21PM +0400, Michael Tokarev wrote:
[]
>> Jokes aside, the thing is that kvm does not know what is
>> an ARP and what is an IP address.  It emulates a hardware
>> network card, which never sends any ARP out by its own,
>> it is the operating system IP stack who's doing that.
>> That network card as emulated by kvm does not know what
>> IP addresses are assigned to it inside the guest (there
>> may be many, or may be none at all), so it just can not
>> send the ARPs.
>>
> True. Although qemu sends gratuitous ARP the IP field there is
> incorrect. It is done to update layer 2 topology, not layer 3.

Actually, the more I think about that, the more it looks
like a job for "external" (for the guest) piece.

For example, we may teach libvirt or kvm about IP addresses
the guest is using, so that kvm will send these ARPs automatically
after migration has completed.

It shouldn't be difficult to implement.  Something like:

 -net nic,model=virtio,arp=1.2.3.4:5.6.7.8,mac=foo:bar

or, even,

 -net tap,arp=...,...

for the command-line interface, and/or a 'sendarp' monitor
command that expects a network device and a list of ip
addresses.

Kvm is the most natural place to do that, I think, and it's
easy to implement there too (it has the tun device which can
inject packets on behalf of the guest)   Yes, the configuration
will be duplicated somehow, but that's not a big problem, and
it will make things much more reliable.

/mjt

Re: KVM doesn't send an arp announce after live migrating a domain

[Daniel P. Berrange]

On Wed, Aug 25, 2010 at 01:40:19PM +0300, Avi Kivity wrote:
>  On 08/25/2010 12:21 PM, Nils Cant wrote:
> >On 08/25/2010 10:38 AM, Gleb Natapov wrote:
> >>qemu sends gratuitous ARP after migration. Check forward delay 
> >>setting on your
> >>bridge interface. It should be set to zero.
> >>
> >
> >Aha! That fixed it. Turns out that debian bridge-utils sets the 
> >default to 15 for bridges.
> >Manually setting it to 0 with 'brctl setfd br0 0' or setting the 
> >'bridge_fd' parameter to 0 in /etc/network/interfaces solves the issue.
> >
> 
> I think libvirt is doing something about this, copying list for further 
> info.

libvirt doesn't set a policy for this. It provides an API for 
configuring host networking, but we don't override the kernel's
forward delay policy, since we don't presume that all bridges 
are going to have VMs attached. In any case the API isn't available
for Debian yet, since no one has ported netcf to Debian, so I 
assume the OP set bridging up manually. The '15' second default is
actually a kernel level default IIRC. 

The two main host network configs recommended for use with libvirt+KVM
(either NAT or bridging) are documented here:

  http://wiki.libvirt.org/page/Networking

Regards,
Daniel
-- 
|: Red Hat, Engineering, London    -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org        -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: KVM doesn't send an arp announce after live migrating a domain

[Gleb Natapov]

On Wed, Aug 25, 2010 at 02:51:31PM +0400, Michael Tokarev wrote:
> Gleb Natapov wrote:
> > On Wed, Aug 25, 2010 at 12:37:21PM +0400, Michael Tokarev wrote:
> []
> >> Jokes aside, the thing is that kvm does not know what is
> >> an ARP and what is an IP address.  It emulates a hardware
> >> network card, which never sends any ARP out by its own,
> >> it is the operating system IP stack who's doing that.
> >> That network card as emulated by kvm does not know what
> >> IP addresses are assigned to it inside the guest (there
> >> may be many, or may be none at all), so it just can not
> >> send the ARPs.
> >>
> > True. Although qemu sends gratuitous ARP the IP field there is
> > incorrect. It is done to update layer 2 topology, not layer 3.
> 
> Actually, the more I think about that, the more it looks
> like a job for "external" (for the guest) piece.
> 
> For example, we may teach libvirt or kvm about IP addresses
> the guest is using, so that kvm will send these ARPs automatically
> after migration has completed.
> 
> It shouldn't be difficult to implement.  Something like:
> 
>  -net nic,model=virtio,arp=1.2.3.4:5.6.7.8,mac=foo:bar
> 
Back to static IP age?

> or, even,
> 
>  -net tap,arp=...,...
> 
> for the command-line interface, and/or a 'sendarp' monitor
> command that expects a network device and a list of ip
> addresses.
> 
> Kvm is the most natural place to do that, I think, and it's
> easy to implement there too (it has the tun device which can
> inject packets on behalf of the guest)   Yes, the configuration
> will be duplicated somehow, but that's not a big problem, and
> it will make things much more reliable.
> 
KVM is certainly not the most natural place to do that. Even gratuitous
ARP we have today will not work if guest changes mac address. KVM
couldn't care less about host network protocols. Management may implement
guest daemon that will take appropriate action to restore networking
after migration on demand (send gratuitous pigeon if IP over pigeons are
used by guest).

--
			Gleb.

Re: KVM doesn't send an arp announce after live migrating a domain

[Avi Kivity]

  On 08/25/2010 01:52 PM, Daniel P. Berrange wrote:
>
>> I think libvirt is doing something about this, copying list for further
>> info.
> libvirt doesn't set a policy for this. It provides an API for
> configuring host networking, but we don't override the kernel's
> forward delay policy, since we don't presume that all bridges
> are going to have VMs attached. In any case the API isn't available
> for Debian yet, since no one has ported netcf to Debian, so I
> assume the OP set bridging up manually. The '15' second default is
> actually a kernel level default IIRC.
>
> The two main host network configs recommended for use with libvirt+KVM
> (either NAT or bridging) are documented here:
>
>    http://wiki.libvirt.org/page/Networking

 From that page:

# virsh net-define /usr/share/libvirt/networks/default.xml

 From my copy of that file:

<network>
   <name>default</name>
   <bridge name="virbr0" />
   <forward/>
   <ip address="192.168.122.1" netmask="255.255.255.0">
     <dhcp>
       <range start="192.168.122.2" end="192.168.122.254" />
     </dhcp>
   </ip>
</network>

So it looks like the default config uses the kernel default?  If libvirt 
uses an existing bridge I agree it shouldn't hack it, but if it creates 
its own can't it use a sensible default?


-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: KVM doesn't send an arp announce after live migrating a domain

[Gleb Natapov]

On Wed, Aug 25, 2010 at 01:59:02PM +0300, Gleb Natapov wrote:
> > Kvm is the most natural place to do that, I think, and it's
> > easy to implement there too (it has the tun device which can
> > inject packets on behalf of the guest)   Yes, the configuration
> > will be duplicated somehow, but that's not a big problem, and
> > it will make things much more reliable.
> > 
> KVM is certainly not the most natural place to do that. Even gratuitous
> ARP we have today will not work if guest changes mac address. KVM
> couldn't care less about host network protocols. Management may implement
Correction: "couldn't care less about _guest_ network protocols"

> guest daemon that will take appropriate action to restore networking
> after migration on demand (send gratuitous pigeon if IP over pigeons are
> used by guest).
> 
> --
> 			Gleb.
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
			Gleb.

Re: KVM doesn't send an arp announce after live migrating a domain

[Daniel P. Berrange]

On Wed, Aug 25, 2010 at 02:05:45PM +0300, Avi Kivity wrote:
>  On 08/25/2010 01:52 PM, Daniel P. Berrange wrote:
> >
> >>I think libvirt is doing something about this, copying list for further
> >>info.
> >libvirt doesn't set a policy for this. It provides an API for
> >configuring host networking, but we don't override the kernel's
> >forward delay policy, since we don't presume that all bridges
> >are going to have VMs attached. In any case the API isn't available
> >for Debian yet, since no one has ported netcf to Debian, so I
> >assume the OP set bridging up manually. The '15' second default is
> >actually a kernel level default IIRC.
> >
> >The two main host network configs recommended for use with libvirt+KVM
> >(either NAT or bridging) are documented here:
> >
> >   http://wiki.libvirt.org/page/Networking
> 
> From that page:
> 
> # virsh net-define /usr/share/libvirt/networks/default.xml
> 
> From my copy of that file:
> 
> <network>
>   <name>default</name>
>   <bridge name="virbr0" />
>   <forward/>
>   <ip address="192.168.122.1" netmask="255.255.255.0">
>     <dhcp>
>       <range start="192.168.122.2" end="192.168.122.254" />
>     </dhcp>
>   </ip>
> </network>
> 
> So it looks like the default config uses the kernel default?  If libvirt 
> uses an existing bridge I agree it shouldn't hack it, but if it creates 
> its own can't it use a sensible default?

That is the NAT virtual network. That one *does* default to a forward
delay of 0, but since it is NAT, it is fairly useless for migration
in anycase. If you do 'virsh net-dumpxml default' you should see that
delay='0' was added

The OP was using bridging rather than NAT though, so this XML example
doesn't apply. My comments about libvirt not overriding kenrel policy
for forward delay were WRT full bridging mode, not the NAT mode[1]

Regards,
Daniel

[1] Yes, the NAT mode uses a bridge as an implementation detail, but
    there's no physical NIC in that bridge - it is merely to connect
    the TAP devices together. Connection to the LAN is forwarded + NAT.
-- 
|: Red Hat, Engineering, London    -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org        -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: KVM doesn't send an arp announce after live migrating a domain

[Avi Kivity]

  On 08/25/2010 02:15 PM, Daniel P. Berrange wrote:
>
>> So it looks like the default config uses the kernel default?  If libvirt
>> uses an existing bridge I agree it shouldn't hack it, but if it creates
>> its own can't it use a sensible default?
> That is the NAT virtual network. That one *does* default to a forward
> delay of 0, but since it is NAT, it is fairly useless for migration
> in anycase. If you do 'virsh net-dumpxml default' you should see that
> delay='0' was added
>
> The OP was using bridging rather than NAT though, so this XML example
> doesn't apply. My comments about libvirt not overriding kenrel policy
> for forward delay were WRT full bridging mode, not the NAT mode[1]

Yes, of course.

Can't libvirt also create a non-NAT bridge?  Looks like it would prevent 
a lot of manual work and opportunity for misconfiguration.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: KVM doesn't send an arp announce after live migrating a domain

[Daniel P. Berrange]

On Wed, Aug 25, 2010 at 02:30:01PM +0300, Avi Kivity wrote:
>  On 08/25/2010 02:15 PM, Daniel P. Berrange wrote:
> >
> >>So it looks like the default config uses the kernel default?  If libvirt
> >>uses an existing bridge I agree it shouldn't hack it, but if it creates
> >>its own can't it use a sensible default?
> >That is the NAT virtual network. That one *does* default to a forward
> >delay of 0, but since it is NAT, it is fairly useless for migration
> >in anycase. If you do 'virsh net-dumpxml default' you should see that
> >delay='0' was added
> >
> >The OP was using bridging rather than NAT though, so this XML example
> >doesn't apply. My comments about libvirt not overriding kenrel policy
> >for forward delay were WRT full bridging mode, not the NAT mode[1]
> 
> Yes, of course.
> 
> Can't libvirt also create a non-NAT bridge?  Looks like it would prevent 
> a lot of manual work and opportunity for misconfiguration.

Yes, it can on latest Fedora/RHEL6, using the netcf library. This is the
new 'virsh iface-XXX' command set (and equivalent APIs). I've not updated
the docs to cover this functionality yet though. It also does bonding,
and vlans, etc

Daniel
-- 
|: Red Hat, Engineering, London    -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org        -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: KVM doesn't send an arp announce after live migrating a domain

[Michael Tokarev]

Gleb Natapov wrote:
> On Wed, Aug 25, 2010 at 02:51:31PM +0400, Michael Tokarev wrote:
[]
>> For example, we may teach libvirt or kvm about IP addresses
>> the guest is using, so that kvm will send these ARPs automatically
>> after migration has completed.
[]
>> Kvm is the most natural place to do that, I think, and it's
>> easy to implement there too (it has the tun device which can
>> inject packets on behalf of the guest)   Yes, the configuration
>> will be duplicated somehow, but that's not a big problem, and
>> it will make things much more reliable.
>>
> KVM is certainly not the most natural place to do that. Even gratuitous
> ARP we have today will not work if guest changes mac address. KVM
> couldn't care less about host network protocols. Management may implement
> guest daemon that will take appropriate action to restore networking
> after migration on demand (send gratuitous pigeon if IP over pigeons are
> used by guest).

I mean something else.  When using standard, the most common configuration,
without fancy settings or technologies like IP over pigeons, the most easy
way to do that is in kvm, it should be just about 20 lines of code or so.
Yes that will not work in some complex setups, where in-guest solution will
be needed, but in that case a guest daemon alone wont help, it will need
to run some script to do custom actions.

For the MAC address changes for example -- the solution is simple: don't
change MAC address in guest.  Or if you do, either teach kvm about that
(so it'll send proper ARP), or implement custom solution in guest, or
don't migrate, or live with delays after migration.  There are multiple
choices.

Yet doing it the simplest way (in kvm) will cover some 99% cases, and doing
it as some daemon in the guest will cover that same 99% cases anyway (for
the rest some custom script will be needed).

So I think it's the best to implement it in kvm in the most stright-forward
and easy way.

Yes, some guest notification is probably needed anyway - not only for this
case with networks but also in order to notify guest about, say, resume
from freeze (after loadvm or migrate from file), afrer migration and so
on, so guest can react to such events in a meaningful way.  But this is
in parallel with the ability to send an ARP after migration.

Just IMHO ofcourse.

/mjt

Re: KVM doesn't send an arp announce after live migrating a domain

[Avi Kivity]

  On 08/25/2010 02:36 PM, Daniel P. Berrange wrote:
>
>> Can't libvirt also create a non-NAT bridge?  Looks like it would prevent
>> a lot of manual work and opportunity for misconfiguration.
> Yes, it can on latest Fedora/RHEL6, using the netcf library. This is the
> new 'virsh iface-XXX' command set (and equivalent APIs). I've not updated
> the docs to cover this functionality yet though. It also does bonding,
> and vlans, etc

Great.

Is virt-manager able to drive this?  it would be great if you could 
drive everything from there.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: KVM doesn't send an arp announce after live migrating a domain

[Daniel P. Berrange]

On Wed, Aug 25, 2010 at 02:38:25PM +0300, Avi Kivity wrote:
>  On 08/25/2010 02:36 PM, Daniel P. Berrange wrote:
> >
> >>Can't libvirt also create a non-NAT bridge?  Looks like it would prevent
> >>a lot of manual work and opportunity for misconfiguration.
> >Yes, it can on latest Fedora/RHEL6, using the netcf library. This is the
> >new 'virsh iface-XXX' command set (and equivalent APIs). I've not updated
> >the docs to cover this functionality yet though. It also does bonding,
> >and vlans, etc
> 
> Great.
> 
> Is virt-manager able to drive this?  it would be great if you could 
> drive everything from there.

Yes, it does now, under the menu Edit -> Host Details -> Network Interfaces
NetworkManager has also finally learnt to ignore ifcfg-XXX files which
have a BRIDGE= setting in them, so it shouldn't totally trash your guest
bridge networking if you leave NM running.

Regards,
Daniel
-- 
|: Red Hat, Engineering, London    -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org        -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|

Re: KVM doesn't send an arp announce after live migrating a domain

[Avi Kivity]

  On 08/25/2010 02:42 PM, Daniel P. Berrange wrote:
>
>> Is virt-manager able to drive this?  it would be great if you could
>> drive everything from there.
> Yes, it does now, under the menu Edit ->  Host Details ->  Network Interfaces
> NetworkManager has also finally learnt to ignore ifcfg-XXX files which
> have a BRIDGE= setting in them, so it shouldn't totally trash your guest
> bridge networking if you leave NM running.

Cool.  I guess what remains is to get people to unlearn all the previous 
hacks.

(also would be nice to have libvirt talk to NetworkManager instead of 
/etc/sysconfig)

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: KVM doesn't send an arp announce after live migrating a domain

[Gleb Natapov]

On Wed, Aug 25, 2010 at 03:36:29PM +0400, Michael Tokarev wrote:
> Gleb Natapov wrote:
> > On Wed, Aug 25, 2010 at 02:51:31PM +0400, Michael Tokarev wrote:
> []
> >> For example, we may teach libvirt or kvm about IP addresses
> >> the guest is using, so that kvm will send these ARPs automatically
> >> after migration has completed.
> []
> >> Kvm is the most natural place to do that, I think, and it's
> >> easy to implement there too (it has the tun device which can
> >> inject packets on behalf of the guest)   Yes, the configuration
> >> will be duplicated somehow, but that's not a big problem, and
> >> it will make things much more reliable.
> >>
> > KVM is certainly not the most natural place to do that. Even gratuitous
> > ARP we have today will not work if guest changes mac address. KVM
> > couldn't care less about host network protocols. Management may implement
> > guest daemon that will take appropriate action to restore networking
> > after migration on demand (send gratuitous pigeon if IP over pigeons are
> > used by guest).
> 
> I mean something else.  When using standard, the most common configuration,
> without fancy settings or technologies like IP over pigeons, the most easy
> way to do that is in kvm, it should be just about 20 lines of code or so.
> Yes that will not work in some complex setups, where in-guest solution will
> be needed, but in that case a guest daemon alone wont help, it will need
> to run some script to do custom actions.
> 
> For the MAC address changes for example -- the solution is simple: don't
> change MAC address in guest.  Or if you do, either teach kvm about that
> (so it'll send proper ARP), or implement custom solution in guest, or
> don't migrate, or live with delays after migration.  There are multiple
> choices.
> 
> Yet doing it the simplest way (in kvm) will cover some 99% cases, and doing
> it as some daemon in the guest will cover that same 99% cases anyway (for
> the rest some custom script will be needed).
> 
> So I think it's the best to implement it in kvm in the most stright-forward
> and easy way.
> 
> Yes, some guest notification is probably needed anyway - not only for this
> case with networks but also in order to notify guest about, say, resume
> from freeze (after loadvm or migrate from file), afrer migration and so
> on, so guest can react to such events in a meaningful way.  But this is
> in parallel with the ability to send an ARP after migration.
> 
> Just IMHO ofcourse.
> 
The most common case is for guest to use DHCP to obtain IP dynamically, so
KVM cannot know what IP to use without sniffing network. And no, we do
not want to pass mac/ip from guest to host. For that guest agent is
needed anyway and it can send gratuitous ARP by itself.

--
			Gleb.