Experience with netchannel2

Experience with netchannel2

[Sergey T]

[-- Attachment #1.1: Type: text/plain, Size: 841 bytes --]

Hello, Steven!

I'd like to ask you some questions about netchannel2.
All I've done were with
http://xenbits.xensource.com/ext/netchannel2/xen-unstable.hg
http://xenbits.xensource.com/ext/netchannel2/linux-2.6.18.hg

1. During xen-netchannel2 building, vif2 script
tools/hotplug/Linux/vif2 isn't installed to /etc/xen/scripts.
Is this script needed at all?
What variable should be placed in domU config : 'vif' or 'vif2'?

2. I have managed to start domU with netchannel2 succesfully with single
virtual ethernet card.
 But when I added second vif device to the domU configuration, i got black
screen in dom0, during
domU starting. It happens on the on network initialisation stage, so i could
consider netchannel2
affects this somehow. I didn't dig it yet, but it's critical to me to have
more then one NIC.

Thank you in advance.

Sergey.

[-- Attachment #1.2: Type: text/html, Size: 1100 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Experience with netchannel2

[Steven Smith]

[-- Attachment #1.1: Type: text/plain, Size: 1303 bytes --]

> Hello, Steven!
> 
> I'd like to ask you some questions about netchannel2.
> All I've done were with
> http://xenbits.xensource.com/ext/netchannel2/xen-unstable.hg
> http://xenbits.xensource.com/ext/netchannel2/linux-2.6.18.hg
> 
> 1. During xen-netchannel2 building, vif2 script
> tools/hotplug/Linux/vif2 isn't installed to /etc/xen/scripts.
> Is this script needed at all?
It is needed, and it should be installed in /etc/xen/scripts.  The
normal tools makefiles are supposed to do this automatically when you
run ``make install'' (and they do on my test box).  Could you post
the output of ``cd tools/hotplug; make install'', please?

> What variable should be placed in domU config : 'vif' or 'vif2'?
vif2.  e.g.:

vif2 = [ 'bridge=eth1' ]

> 2. I have managed to start domU with netchannel2 succesfully with single virtual ethernet card.
>  But when I added second vif device to the domU configuration, i got black screen in dom0, during
> domU starting. It happens on the on network initialisation stage, so i could consider netchannel2
> affects this somehow. I didn't dig it yet, but it's critical to me to have more then one NIC.
What do you mean by ``black screen''?  That dom0 crashes?  Is there
any chance you could get a serial console on it, please?

Steven.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Experience with netchannel2

[Sergey Tovpeko]

[-- Attachment #1: Type: text/plain, Size: 2774 bytes --]

Hello, Steven.
>> 1. During xen-netchannel2 building, vif2 script
>> tools/hotplug/Linux/vif2 isn't installed to /etc/xen/scripts.
>> Is this script needed at all?
>>     
> It is needed, and it should be installed in /etc/xen/scripts.  The
> normal tools makefiles are supposed to do this automatically when you
> run ``make install'' (and they do on my test box).  Could you post
> the output of ``cd tools/hotplug; make install'', please?
>   
I've attached the log in 'hotplug_makefile' file.
It seems that tools/hotplug/Makefile doesn't know that it should place
'vif2' to /etc/xen/scripts.


>> What variable should be placed in domU config : 'vif' or 'vif2'?
>>     
> vif2.  e.g.:
>
> vif2 = [ 'bridge=eth1' ]
>   
Hmm, there is no key 'bridge' for vif2 in tools/python/xen/xm/create.py
And domU doesn't start with
Error: Invalid vif option: bridge

     for c in vifs2:
         d = comma_sep_kv_to_dict(c)
         config_vif = ['vif2']

         for k in d.keys():
             if k not in ['front_mac', 'back_mac', 'backend', 'pdev',
                          'max_bypasses']:
                 err('Invalid vif option: ' + k)
             config_vif.append([k, d[k]])
         config_devs.append(['device', config_vif])


Moreover, the domU is starting without 'bridge' key with
vif2 =[ '' ]
but what i have, are
1. Virtual network interface is created in dom0 with name 'eth1', while
in old netchannel1,  virtual network intarface
has the name 'vifN.M'. It's very strange that i have 'eth1' instead of
'vifN.M'.
2. Despite the fact that DomU have started, there no connection in domU.
I suppose it's due to the virtual interface have not been attached to
the bridge in Dom0.
I noticed that there are differencies between /etc/xen/scripts/vif and
/etc/xen/scritps/vif2. The old vif uses 'brctl show' command to find the
bridge, but vif2 script doesn't.


>> 2. I have managed to start domU with netchannel2 succesfully with single virtual ethernet card.
>>  But when I added second vif device to the domU configuration, i got black screen in dom0, during
>> domU starting. It happens on the on network initialisation stage, so i could consider netchannel2
>> affects this somehow. I didn't dig it yet, but it's critical to me to have more then one NIC.
>>     
> What do you mean by ``black screen''?  That dom0 crashes?  Is there
> any chance you could get a serial console on it, please?
>   
I mean my dom0 is rebooting, tripple fault have been occured.
I'm going to research this problem. Here my test case
1. Start domU with two netchannel1 NIC
vif = [ '', '' ]
2. In DomU obtain IP-address manually for  first NIC, then for second one.
dhclient eth0
dhclient eth1

Before second dhclient finished, dom0 goes to reboot.


Best regards,
    Sergey Tovpeko.


[-- Attachment #2: hotplug_makefile --]
[-- Type: text/plain, Size: 2411 bytes --]

make[1]: Entering directory `/home/niagara/hghome/netchannel2-xen-unstable.hg/tools/hotplug'
make -C common install
make[2]: Entering directory `/home/niagara/hghome/netchannel2-xen-unstable.hg/tools/hotplug/common'
[ -d //etc/xen/scripts ] || \
		../../..//tools/cross-install -d -m0755 -p //etc/xen/scripts
set -e; for i in ; \
	   do \
	   ../../..//tools/cross-install -m0755 -p $i //etc/xen/scripts; \
	done
set -e; for i in ; \
	   do \
	   ../../..//tools/cross-install -m0644 -p $i //etc/xen/scripts; \
	done
make[2]: Leaving directory `/home/niagara/hghome/netchannel2-xen-unstable.hg/tools/hotplug/common'
make[1]: Leaving directory `/home/niagara/hghome/netchannel2-xen-unstable.hg/tools/hotplug'
make[1]: Entering directory `/home/niagara/hghome/netchannel2-xen-unstable.hg/tools/hotplug'
make -C Linux install
make[2]: Entering directory `/home/niagara/hghome/netchannel2-xen-unstable.hg/tools/hotplug/Linux'
[ -d //etc/init.d ] || ../../..//tools/cross-install -d -m0755 -p //etc/init.d
[ -d //etc/sysconfig ] || ../../..//tools/cross-install -d -m0755 -p //etc/sysconfig
../../..//tools/cross-install -m0755 -p init.d/xend //etc/init.d
../../..//tools/cross-install -m0755 -p init.d/xendomains //etc/init.d
../../..//tools/cross-install -m0755 -p init.d/sysconfig.xendomains //etc/sysconfig/xendomains
[ -d //etc/xen/scripts ] || \
		../../..//tools/cross-install -d -m0755 -p //etc/xen/scripts
set -e; for i in network-bridge vif-bridge network-route vif-route network-nat vif-nat block block-enbd block-nbd vtpm vtpm-delete xen-hotplug-cleanup external-device-migrate vscsi; \
	    do \
	    ../../..//tools/cross-install -m0755 -p $i //etc/xen/scripts; \
	done
set -e; for i in xen-script-common.sh locking.sh logging.sh xen-hotplug-common.sh xen-network-common.sh vif-common.sh block-common.sh vtpm-common.sh vtpm-hotplug-common.sh vtpm-migration.sh vtpm-impl; \
	    do \
	    ../../..//tools/cross-install -m0644 -p $i //etc/xen/scripts; \
	done
[ -d //etc/udev ] || \
		../../..//tools/cross-install -d -m0755 -p //etc/udev/rules.d
set -e; for i in xen-backend.rules xend.rules; \
	    do \
	    ../../..//tools/cross-install -m0644 -p $i //etc/udev; \
	    ln -sf ../$i //etc/udev/rules.d; \
	done
make[2]: Leaving directory `/home/niagara/hghome/netchannel2-xen-unstable.hg/tools/hotplug/Linux'
make[1]: Leaving directory `/home/niagara/hghome/netchannel2-xen-unstable.hg/tools/hotplug'

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Experience with netchannel2

[Steven Smith]

[-- Attachment #1.1: Type: text/plain, Size: 4250 bytes --]

> >> 1. During xen-netchannel2 building, vif2 script
> >> tools/hotplug/Linux/vif2 isn't installed to /etc/xen/scripts.
> >> Is this script needed at all?
> >>     
> > It is needed, and it should be installed in /etc/xen/scripts.  The
> > normal tools makefiles are supposed to do this automatically when you
> > run ``make install'' (and they do on my test box).  Could you post
> > the output of ``cd tools/hotplug; make install'', please?
> >   
> I've attached the log in 'hotplug_makefile' file.
> It seems that tools/hotplug/Makefile doesn't know that it should place
> 'vif2' to /etc/xen/scripts.
Ah, yes, sorry about that.  It looks like I lost the relevant line of
Makefile during one of the merges.  Should be fixed by cset
39b6b4761690.

> >> What variable should be placed in domU config : 'vif' or 'vif2'?
> >>     
> > vif2.  e.g.:
> >
> > vif2 = [ 'bridge=eth1' ]
> >   
> Hmm, there is no key 'bridge' for vif2 in tools/python/xen/xm/create.py
> And domU doesn't start with
> Error: Invalid vif option: bridge
> 
>      for c in vifs2:
>          d = comma_sep_kv_to_dict(c)
>          config_vif = ['vif2']
> 
>          for k in d.keys():
>              if k not in ['front_mac', 'back_mac', 'backend', 'pdev',
>                           'max_bypasses']:
>                  err('Invalid vif option: ' + k)
>              config_vif.append([k, d[k]])
>          config_devs.append(['device', config_vif])
Err... yes, sorry again.  I plumbed the bridge parameter through for
network2-attach, but it looks like I forgot about create.  Should be
fixed by cset c14cc8002561.

> Moreover, the domU is starting without 'bridge' key with
> vif2 =[ '' ]
> but what i have, are
> 1. Virtual network interface is created in dom0 with name 'eth1', while
> in old netchannel1,  virtual network intarface
> has the name 'vifN.M'. It's very strange that i have 'eth1' instead of
> 'vifN.M'.
Is it?  It's easy enough to change the name of the interface, but I've
never really understood why it was desirable.

> 2. Despite the fact that DomU have started, there no connection in domU.
> I suppose it's due to the virtual interface have not been attached to
> the bridge in Dom0.
Yes, that'll have been because the bridge parameter was broken.

> I noticed that there are differencies between /etc/xen/scripts/vif and
> /etc/xen/scritps/vif2. The old vif uses 'brctl show' command to find the
> bridge, but vif2 script doesn't.
Yes, the behaviour differs if you don't specify a bridge.  vif-bridge
picks one at random and attaches the interface to that, whereas vif2
just doesn't attach at all.  Neither is really correct.

I've now modified the vif2 script so that:

-- If you specify a bridge, it uses that.
-- If you specify a bridge of ``-'', it leaves the vif hanging.
-- If you leave the bridge unspecified, and you only have one in dom0,
   it uses the only bridge.
-- Otherwise, it flags an error.

Does that sound sane?  That's still not perfect, because it means that
creating a new bridge can suddenly make all of your existing domains
stop working, but I can't really see any better ways of handling this.

> >> 2. I have managed to start domU with netchannel2 succesfully with single virtual ethernet card.
> >>  But when I added second vif device to the domU configuration, i got black screen in dom0, during
> >> domU starting. It happens on the on network initialisation stage, so i could consider netchannel2
> >> affects this somehow. I didn't dig it yet, but it's critical to me to have more then one NIC.
> >>     
> > What do you mean by ``black screen''?  That dom0 crashes?  Is there
> > any chance you could get a serial console on it, please?
> >   
> I mean my dom0 is rebooting, tripple fault have been occured.
> I'm going to research this problem. Here my test case
> 1. Start domU with two netchannel1 NIC
> vif = [ '', '' ]
> 2. In DomU obtain IP-address manually for  first NIC, then for second one.
> dhclient eth0
> dhclient eth1
> 
> Before second dhclient finished, dom0 goes to reboot.
Got it, thanks.  Should be fixed by cset b1779e71b375.

I'm sorry this has been more trouble than it should have been.

Steven.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Experience with netchannel2

[Sergey Tovpeko]

[-- Attachment #1: Type: text/plain, Size: 859 bytes --]

Hello, Steven!
>> Moreover, the domU is starting without 'bridge' key with
>> vif2 =[ '' ]
>> but what i have, are
>> 1. Virtual network interface is created in dom0 with name 'eth1', while
>> in old netchannel1,  virtual network intarface
>> has the name 'vifN.M'. It's very strange that i have 'eth1' instead of
>> 'vifN.M'.
>>     
> Is it?  It's easy enough to change the name of the interface, but I've
> never really understood why it was desirable.
>   
I'd like to distinguish somehow, that device named 'vif' is virtual 
network interface for domU,
at least for testing purposes.


Thank you very much for your efforts and fixes.
Please, consider some minor changes in gnttab_request_version
I added the return code to it., because HYPERVISOR_grant_table_op could 
return -EFAULT, -EINVAL additional to -ENOSYS.
I attached the file.

Sergey Tovpeko.

[-- Attachment #2: versions.diff --]
[-- Type: text/x-patch, Size: 2140 bytes --]

diff --git a/drivers/xen/core/gnttab.c b/drivers/xen/core/gnttab.c
index 5314932..5f27756 100644
--- a/drivers/xen/core/gnttab.c
+++ b/drivers/xen/core/gnttab.c
@@ -654,28 +654,31 @@ static inline unsigned int max_nr_grant_frames(void)
 	return xen_max;
 }
 
-static void gnttab_request_version(void)
+static int gnttab_request_version(void)
 {
         int rc;
         struct gnttab_set_version gsv;
+	int version = 2;
 
-        gsv.version = 2;
+        gsv.version = version;
         rc = HYPERVISOR_grant_table_op(GNTTABOP_set_version, &gsv, 1);
         if (rc == 0) {
-                grant_table_version = 2;
-		printk("<0>Grant table version 2.\n");
+		grant_table_version = version;
+		printk("<0>Grant table version %d.\n", grant_table_version);
         } else {
-                if (grant_table_version == 2) {
+		if (rc == -ENOSYS && grant_table_version == 2) {
                         /* If we've already used version 2 features,
                            but then suddenly discover that they're not
                            available (e.g. migrating to an older
                            version of Xen), almost unbounded badness
                            can happen. */
                         panic("we need grant tables version 2, but only version 1 is available");
+			grant_table_version = 1;
+			printk("<0>Grant table version 1.\n");
+			rc = 0;
                 }
-                grant_table_version = 1;
-		printk("<0>Grant table version 1.\n");
         }
+	return rc;
 }
 
 #ifdef CONFIG_XEN
@@ -1011,7 +1014,10 @@ EXPORT_SYMBOL(gnttab_post_map_adjust);
 
 int gnttab_resume(void)
 {
-    gnttab_request_version();
+	int rc;
+
+	if (rc = gnttab_request_version())
+		return rc;
 	if (max_nr_grant_frames() < nr_grant_frames)
 		return -ENOSYS;
 	return gnttab_map(0, nr_grant_frames - 1);
@@ -1092,8 +1098,10 @@ int gnttab_resume(void)
 {
 	unsigned int max_nr_gframes, nr_gframes;
 	unsigned int max_nr_sframes, nr_sframes;
+	int rc;
 
-        gnttab_request_version();
+	if (rc = gnttab_request_version())
+		return rc;
 
 	nr_gframes = nr_grant_frames;
 	max_nr_gframes = max_nr_grant_frames();

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Experience with netchannel2

[Steven Smith]

[-- Attachment #1.1: Type: text/plain, Size: 851 bytes --]

> Thank you very much for your efforts and fixes.
> Please, consider some minor changes in gnttab_request_version
> I added the return code to it., because HYPERVISOR_grant_table_op could 
> return -EFAULT, -EINVAL additional to -ENOSYS.
> I attached the file.
Hmm.  I'm not convinced by this, for a couple of reasons:

a) HYPERVISOR_grant_table_op(GNTTABOP_set_version) won't return
   -EFAULT or -EINVAL for this particular call, because the parameters
   to the call are correct.

b) Even if it did, the current behaviour (of just assuming version 1)
   is pretty reasonable.

c) If gnttab_resume() fails then there isn't really anything which the
   rest of the kernel can do to recover, so the guest will almost
   certainly just crash a few instructions later.

So I think I'm going to pass on this one.

Thanks,

Steven.

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel