incompatibility of netfront driver with bonding module

incompatibility of netfront driver with bonding module

[Masroor Vettuparambil]

[-- Attachment #1.1: Type: text/plain, Size: 1126 bytes --]

Hi all,
 
I am trying a live migration of pass-through enabled domU. The idea is
configuring a bonding interface and enslave the pass-through NIC to it.
During migration, enslave a vif and detach the NIC. But this is not
working because of the incompatibility of netfront driver with bonding
module.
 
1. Normally, bonding will inherit the mac from the first slave and
assign it to all the other slaves added later. So the mac of vif will be
updated. But during migration, the mac of vif is getting updated from
xenstore(/vm/). So how about having a set_mac_address entry in netfront
that update the xenstore?
2. arp link monitoring method relies on the slave driver's rx/tx
timestamp. But in netfront, dev->last_rx and dev->trans_start are not
being used, and fails to detect the link status.
3. In mii link monitoring, it relies on the netif_carrier_ok status. But
as netfront is using itsown netfront_carrier_* functions, the status is
not getting updated to bonding module.
 
So I need help, especially on #1. Is it ok to update the xenstore /vm/
keys from domU? how to do this?
 
thanks,
Masroor

[-- Attachment #1.2: Type: text/html, Size: 2408 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

incompatibility of netfront driver with bonding module

[Yosuke Iwamatsu]

[-- Attachment #1: Type: text/plain, Size: 1032 bytes --]

Hi,

Masroor Vettuparambil wrote:
> 1. Normally, bonding will inherit the mac from the first slave and 
> assign it to all the other slaves added later. So the mac of vif will be 
> updated. But during migration, the mac of vif is getting updated from 
> xenstore(/vm/). So how about having a set_mac_address entry in netfront 
> that update the xenstore?

> So I need help, especially on #1. Is it ok to update the xenstore 
> /vm/ keys from domU? how to do this?

I tried a bit to find out the way to update /vm/ keys from frontend,
but didn't succeed. Either way, I don't think it is a good idea to
update the xenstore key of the vif mac address, because the original
mac will be lost and we won't be able to reset it e.g. after the guest
reboot.

So the attached patch adds set mac_address() support to netfront.
It doesn't touch xenstore at all, but just preserves the modified mac
address in netfront_info structure and keeps using it after migration.

Thanks,
-----------------------
Yosuke Iwamatsu
         NEC Corporation

[-- Attachment #2: netfront_set_mac.patch --]
[-- Type: all/allfiles, Size: 1946 bytes --]

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: incompatibility of netfront driver with bonding module

[Masroor Vettuparambil]

Thanks for your work.
I could manage the live migration using a simple check in netfront while
resuming on destination.

if (!(netdev->flags & IFF_SLAVE))
	memcpy(netdev->dev_addr, info->mac, ETH_ALEN);
But this will work only if the interface is enslaved.

In your patch, should we let to change the MAC while interface is up? 
I need to include the support for arp link monitoring to netfront.

Regards
Masroor,
 

-----Original Message-----
From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com] 
Sent: Thursday, April 03, 2008 12:00 PM
To: Masroor Vettuparambil
Cc: xen-devel@lists.xensource.com
Subject: incompatibility of netfront driver with bonding module

Hi,

Masroor Vettuparambil wrote:
> 1. Normally, bonding will inherit the mac from the first slave and 
> assign it to all the other slaves added later. So the mac of vif will 
> be updated. But during migration, the mac of vif is getting updated 
> from xenstore(/vm/). So how about having a set_mac_address entry in 
> netfront that update the xenstore?

> So I need help, especially on #1. Is it ok to update the xenstore /vm/

> keys from domU? how to do this?

I tried a bit to find out the way to update /vm/ keys from frontend, but
didn't succeed. Either way, I don't think it is a good idea to update
the xenstore key of the vif mac address, because the original mac will
be lost and we won't be able to reset it e.g. after the guest reboot.

So the attached patch adds set mac_address() support to netfront.
It doesn't touch xenstore at all, but just preserves the modified mac
address in netfront_info structure and keeps using it after migration.

Thanks,
-----------------------
Yosuke Iwamatsu
         NEC Corporation

incompatibility of netfront driver with bonding module

[Yosuke Iwamatsu]

Masroor Vettuparambil wrote:
> Thanks for your work.
> I could manage the live migration using a simple check in netfront while
> resuming on destination.
> 
> if (!(netdev->flags & IFF_SLAVE))
> 	memcpy(netdev->dev_addr, info->mac, ETH_ALEN);
> But this will work only if the interface is enslaved.
> 
> In your patch, should we let to change the MAC while interface is up?

It might be desirable, but I don't know for now if we can achieve it
easily.

> I need to include the support for arp link monitoring to netfront.

Is link monitoring really necessary?
If you mean you use link monitoring to change the active slave of the 
bond0, I think we can do that using ifenslave command inside domU.
  #ifenslave bond0 eth1     -> enslave new netif(eth1) to bonding device
  #ifenslave -d bond0 eth0  -> release old netif(eth0) from bonding

Regards,
   Yosuke

> 
> Regards
> Masroor,
>  
> 
> -----Original Message-----
> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com] 
> Sent: Thursday, April 03, 2008 12:00 PM
> To: Masroor Vettuparambil
> Cc: xen-devel@lists.xensource.com
> Subject: incompatibility of netfront driver with bonding module
> 
> Hi,
> 
> Masroor Vettuparambil wrote:
>> 1. Normally, bonding will inherit the mac from the first slave and 
>> assign it to all the other slaves added later. So the mac of vif will 
>> be updated. But during migration, the mac of vif is getting updated 
>> from xenstore(/vm/). So how about having a set_mac_address entry in 
>> netfront that update the xenstore?
> 
>> So I need help, especially on #1. Is it ok to update the xenstore /vm/
> 
>> keys from domU? how to do this?
> 
> I tried a bit to find out the way to update /vm/ keys from frontend, but
> didn't succeed. Either way, I don't think it is a good idea to update
> the xenstore key of the vif mac address, because the original mac will
> be lost and we won't be able to reset it e.g. after the guest reboot.
> 
> So the attached patch adds set mac_address() support to netfront.
> It doesn't touch xenstore at all, but just preserves the modified mac
> address in netfront_info structure and keeps using it after migration.
> 
> Thanks,
> -----------------------
> Yosuke Iwamatsu
>          NEC Corporation

RE:[PATCH] incompatibility of netfront driver with bonding module

[Masroor Vettuparambil]

[-- Attachment #1: Type: text/plain, Size: 2723 bytes --]

The attached patch provides the arp link monitoring support for
netfront.
Also it prevents setting the mac while the interface is up.

Regards
Masroor 

-----Original Message-----
From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com] 
Sent: Monday, April 07, 2008 12:36 PM
To: Masroor Vettuparambil
Cc: xen-devel@lists.xensource.com
Subject: incompatibility of netfront driver with bonding module

Masroor Vettuparambil wrote:
> Thanks for your work.
> I could manage the live migration using a simple check in netfront 
> while resuming on destination.
> 
> if (!(netdev->flags & IFF_SLAVE))
> 	memcpy(netdev->dev_addr, info->mac, ETH_ALEN); But this will
work 
> only if the interface is enslaved.
> 
> In your patch, should we let to change the MAC while interface is up?

It might be desirable, but I don't know for now if we can achieve it
easily.

> I need to include the support for arp link monitoring to netfront.

Is link monitoring really necessary?
If you mean you use link monitoring to change the active slave of the
bond0, I think we can do that using ifenslave command inside domU.
  #ifenslave bond0 eth1     -> enslave new netif(eth1) to bonding device
  #ifenslave -d bond0 eth0  -> release old netif(eth0) from bonding

Regards,
   Yosuke

> 
> Regards
> Masroor,
>  
> 
> -----Original Message-----
> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com]
> Sent: Thursday, April 03, 2008 12:00 PM
> To: Masroor Vettuparambil
> Cc: xen-devel@lists.xensource.com
> Subject: incompatibility of netfront driver with bonding module
> 
> Hi,
> 
> Masroor Vettuparambil wrote:
>> 1. Normally, bonding will inherit the mac from the first slave and 
>> assign it to all the other slaves added later. So the mac of vif will

>> be updated. But during migration, the mac of vif is getting updated 
>> from xenstore(/vm/). So how about having a set_mac_address entry in 
>> netfront that update the xenstore?
> 
>> So I need help, especially on #1. Is it ok to update the xenstore 
>> /vm/
> 
>> keys from domU? how to do this?
> 
> I tried a bit to find out the way to update /vm/ keys from frontend, 
> but didn't succeed. Either way, I don't think it is a good idea to 
> update the xenstore key of the vif mac address, because the original 
> mac will be lost and we won't be able to reset it e.g. after the guest
reboot.
> 
> So the attached patch adds set mac_address() support to netfront.
> It doesn't touch xenstore at all, but just preserves the modified mac 
> address in netfront_info structure and keeps using it after migration.
> 
> Thanks,
> -----------------------
> Yosuke Iwamatsu
>          NEC Corporation


[-- Attachment #2: netfront_arpmon_fix --]
[-- Type: application/octet-stream, Size: 1225 bytes --]

# HG changeset patch
# User root@xen-152
# Date 1207815591 -19800
# Node ID 56b23a7b67722db681f31b5faf08f1f50a7335a3
# Parent  cf150ce22d4312278b237c2705114a8311bda01f
Included the arp link monitoring support in netfront for bonding module

Signed-off-by: Masroor Vettuparambil <masroor.vettuparambil@neterion.com>

diff -r cf150ce22d43 -r 56b23a7b6772 drivers/xen/netfront/netfront.c
--- a/drivers/xen/netfront/netfront.c	Tue Apr 08 11:30:38 2008 +0100
+++ b/drivers/xen/netfront/netfront.c	Thu Apr 10 13:49:51 2008 +0530
@@ -1031,6 +1031,7 @@ static int network_start_xmit(struct sk_
 
 	np->stats.tx_bytes += skb->len;
 	np->stats.tx_packets++;
+	dev->trans_start = jiffies;
 
 	/* Note: It is not safe to access skb after network_tx_buf_gc()! */
 	network_tx_buf_gc(dev);
@@ -1063,6 +1064,7 @@ static irqreturn_t netif_int(int irq, vo
 			netfront_accelerator_call_stop_napi_irq(np, dev);
 
 			netif_rx_schedule(dev);
+			dev->last_rx = jiffies;
 		}
 	}
 
@@ -1696,7 +1698,9 @@ static int xennet_set_mac_address(struct
 {
 	struct netfront_info *np = netdev_priv(dev);
 	struct sockaddr *addr = p;
-
+	
+	if (netif_running(dev))
+		return -EBUSY;
 	if (!is_valid_ether_addr(addr->sa_data))
 		return -EADDRNOTAVAIL;
 	

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] incompatibility of netfront driver with bonding module

[Yosuke Iwamatsu]

Thank you.

Now I'm curious if you really succeeded the live migration of
passthrough domains, using pci-hotplug and bonding.
(Currently I have one testing machine and can only try
local migration.)

-- Yosuke

Masroor Vettuparambil wrote:
> The attached patch provides the arp link monitoring support for
> netfront.
> Also it prevents setting the mac while the interface is up.
> 
> Regards
> Masroor 
> 
> -----Original Message-----
> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com] 
> Sent: Monday, April 07, 2008 12:36 PM
> To: Masroor Vettuparambil
> Cc: xen-devel@lists.xensource.com
> Subject: incompatibility of netfront driver with bonding module
> 
> Masroor Vettuparambil wrote:
>> Thanks for your work.
>> I could manage the live migration using a simple check in netfront 
>> while resuming on destination.
>>
>> if (!(netdev->flags & IFF_SLAVE))
>> 	memcpy(netdev->dev_addr, info->mac, ETH_ALEN); But this will
> work 
>> only if the interface is enslaved.
>>
>> In your patch, should we let to change the MAC while interface is up?
> 
> It might be desirable, but I don't know for now if we can achieve it
> easily.
> 
>> I need to include the support for arp link monitoring to netfront.
> 
> Is link monitoring really necessary?
> If you mean you use link monitoring to change the active slave of the
> bond0, I think we can do that using ifenslave command inside domU.
>   #ifenslave bond0 eth1     -> enslave new netif(eth1) to bonding device
>   #ifenslave -d bond0 eth0  -> release old netif(eth0) from bonding
> 
> Regards,
>    Yosuke
> 
>> Regards
>> Masroor,
>>  
>>
>> -----Original Message-----
>> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com]
>> Sent: Thursday, April 03, 2008 12:00 PM
>> To: Masroor Vettuparambil
>> Cc: xen-devel@lists.xensource.com
>> Subject: incompatibility of netfront driver with bonding module
>>
>> Hi,
>>
>> Masroor Vettuparambil wrote:
>>> 1. Normally, bonding will inherit the mac from the first slave and 
>>> assign it to all the other slaves added later. So the mac of vif will
> 
>>> be updated. But during migration, the mac of vif is getting updated 
>>> from xenstore(/vm/). So how about having a set_mac_address entry in 
>>> netfront that update the xenstore?
>>> So I need help, especially on #1. Is it ok to update the xenstore 
>>> /vm/
>>> keys from domU? how to do this?
>> I tried a bit to find out the way to update /vm/ keys from frontend, 
>> but didn't succeed. Either way, I don't think it is a good idea to 
>> update the xenstore key of the vif mac address, because the original 
>> mac will be lost and we won't be able to reset it e.g. after the guest
> reboot.
>> So the attached patch adds set mac_address() support to netfront.
>> It doesn't touch xenstore at all, but just preserves the modified mac 
>> address in netfront_info structure and keeps using it after migration.
>>
>> Thanks,
>> -----------------------
>> Yosuke Iwamatsu
>>          NEC Corporation
> 

RE: [PATCH] incompatibility of netfront driver with bonding module

[Masroor Vettuparambil]

Yes, I could do a live migration of pass-through domain using bonding.  

Regards
Masroor 

-----Original Message-----
From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com] 
Sent: Thursday, April 10, 2008 5:35 PM
To: Masroor Vettuparambil
Cc: xen-devel@lists.xensource.com
Subject: Re: [PATCH] incompatibility of netfront driver with bonding
module

Thank you.

Now I'm curious if you really succeeded the live migration of
passthrough domains, using pci-hotplug and bonding.
(Currently I have one testing machine and can only try local migration.)

-- Yosuke

Masroor Vettuparambil wrote:
> The attached patch provides the arp link monitoring support for 
> netfront.
> Also it prevents setting the mac while the interface is up.
> 
> Regards
> Masroor
> 
> -----Original Message-----
> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com]
> Sent: Monday, April 07, 2008 12:36 PM
> To: Masroor Vettuparambil
> Cc: xen-devel@lists.xensource.com
> Subject: incompatibility of netfront driver with bonding module
> 
> Masroor Vettuparambil wrote:
>> Thanks for your work.
>> I could manage the live migration using a simple check in netfront 
>> while resuming on destination.
>>
>> if (!(netdev->flags & IFF_SLAVE))
>> 	memcpy(netdev->dev_addr, info->mac, ETH_ALEN); But this will
> work
>> only if the interface is enslaved.
>>
>> In your patch, should we let to change the MAC while interface is up?
> 
> It might be desirable, but I don't know for now if we can achieve it 
> easily.
> 
>> I need to include the support for arp link monitoring to netfront.
> 
> Is link monitoring really necessary?
> If you mean you use link monitoring to change the active slave of the 
> bond0, I think we can do that using ifenslave command inside domU.
>   #ifenslave bond0 eth1     -> enslave new netif(eth1) to bonding
device
>   #ifenslave -d bond0 eth0  -> release old netif(eth0) from bonding
> 
> Regards,
>    Yosuke
> 
>> Regards
>> Masroor,
>>  
>>
>> -----Original Message-----
>> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com]
>> Sent: Thursday, April 03, 2008 12:00 PM
>> To: Masroor Vettuparambil
>> Cc: xen-devel@lists.xensource.com
>> Subject: incompatibility of netfront driver with bonding module
>>
>> Hi,
>>
>> Masroor Vettuparambil wrote:
>>> 1. Normally, bonding will inherit the mac from the first slave and 
>>> assign it to all the other slaves added later. So the mac of vif 
>>> will
> 
>>> be updated. But during migration, the mac of vif is getting updated 
>>> from xenstore(/vm/). So how about having a set_mac_address entry in 
>>> netfront that update the xenstore?
>>> So I need help, especially on #1. Is it ok to update the xenstore 
>>> /vm/ keys from domU? how to do this?
>> I tried a bit to find out the way to update /vm/ keys from frontend, 
>> but didn't succeed. Either way, I don't think it is a good idea to 
>> update the xenstore key of the vif mac address, because the original 
>> mac will be lost and we won't be able to reset it e.g. after the 
>> guest
> reboot.
>> So the attached patch adds set mac_address() support to netfront.
>> It doesn't touch xenstore at all, but just preserves the modified mac

>> address in netfront_info structure and keeps using it after
migration.
>>
>> Thanks,
>> -----------------------
>> Yosuke Iwamatsu
>>          NEC Corporation
> 

[PATCH] incompatibility of netfront driver with bonding module

[Yosuke Iwamatsu]

Masroor Vettuparambil wrote:
> Yes, I could do a live migration of pass-through domain using bonding.  

That's great!

Thanks,
   Yosuke

> -----Original Message-----
> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com] 
> Sent: Thursday, April 10, 2008 5:35 PM
> To: Masroor Vettuparambil
> Cc: xen-devel@lists.xensource.com
> Subject: Re: [PATCH] incompatibility of netfront driver with bonding
> module
> 
> Thank you.
> 
> Now I'm curious if you really succeeded the live migration of
> passthrough domains, using pci-hotplug and bonding.
> (Currently I have one testing machine and can only try local migration.)
> 
> -- Yosuke
> 
> Masroor Vettuparambil wrote:
>> The attached patch provides the arp link monitoring support for 
>> netfront.
>> Also it prevents setting the mac while the interface is up.
>>
>> Regards
>> Masroor
>>
>> -----Original Message-----
>> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com]
>> Sent: Monday, April 07, 2008 12:36 PM
>> To: Masroor Vettuparambil
>> Cc: xen-devel@lists.xensource.com
>> Subject: incompatibility of netfront driver with bonding module
>>
>> Masroor Vettuparambil wrote:
>>> Thanks for your work.
>>> I could manage the live migration using a simple check in netfront 
>>> while resuming on destination.
>>>
>>> if (!(netdev->flags & IFF_SLAVE))
>>> 	memcpy(netdev->dev_addr, info->mac, ETH_ALEN); But this will
>> work
>>> only if the interface is enslaved.
>>>
>>> In your patch, should we let to change the MAC while interface is up?
>> It might be desirable, but I don't know for now if we can achieve it 
>> easily.
>>
>>> I need to include the support for arp link monitoring to netfront.
>> Is link monitoring really necessary?
>> If you mean you use link monitoring to change the active slave of the 
>> bond0, I think we can do that using ifenslave command inside domU.
>>   #ifenslave bond0 eth1     -> enslave new netif(eth1) to bonding
> device
>>   #ifenslave -d bond0 eth0  -> release old netif(eth0) from bonding
>>
>> Regards,
>>    Yosuke
>>
>>> Regards
>>> Masroor,
>>>  
>>>
>>> -----Original Message-----
>>> From: Yosuke Iwamatsu [mailto:y-iwamatsu@ab.jp.nec.com]
>>> Sent: Thursday, April 03, 2008 12:00 PM
>>> To: Masroor Vettuparambil
>>> Cc: xen-devel@lists.xensource.com
>>> Subject: incompatibility of netfront driver with bonding module
>>>
>>> Hi,
>>>
>>> Masroor Vettuparambil wrote:
>>>> 1. Normally, bonding will inherit the mac from the first slave and 
>>>> assign it to all the other slaves added later. So the mac of vif 
>>>> will
>>>> be updated. But during migration, the mac of vif is getting updated 
>>>> from xenstore(/vm/). So how about having a set_mac_address entry in 
>>>> netfront that update the xenstore?
>>>> So I need help, especially on #1. Is it ok to update the xenstore 
>>>> /vm/ keys from domU? how to do this?
>>> I tried a bit to find out the way to update /vm/ keys from frontend, 
>>> but didn't succeed. Either way, I don't think it is a good idea to 
>>> update the xenstore key of the vif mac address, because the original 
>>> mac will be lost and we won't be able to reset it e.g. after the 
>>> guest
>> reboot.
>>> So the attached patch adds set mac_address() support to netfront.
>>> It doesn't touch xenstore at all, but just preserves the modified mac
> 
>>> address in netfront_info structure and keeps using it after
> migration.
>>> Thanks,
>>> -----------------------
>>> Yosuke Iwamatsu
>>>          NEC Corporation