save / migrate not working on gentoo?

save / migrate not working on gentoo?

[tdc]

[-- Attachment #1: Type: text/plain, Size: 1223 bytes --]

Hi all,
I've tested several versions of Xen (2.0, 2.0.3, 2.0-testing, unstable) 
on different machines (VIA C3, PIII, AMD XP), all running up-to-date 
gentoo system and I'm unable to migrate (or even save) domains on any of 
those machines.
Currently I'm making tests mainly on the Athlon box, on which I've tried 
two GCC versions - 3.3.5 at first, later I've recompiled glibc and then 
Xen 2.0.3 with gcc 3.4.2; still no luck.

xend is running, xfrd is running, all requirements are installed 
(twisted 1.3, ip, brctl), virt-1 domain is running against local 'file' 
based VBDs.

# xm dmesg | grep version
Xen version 2.0.3 (root@here) (gcc version 3.4.2 (Gentoo Linux 3.4.2-r2, 
ssp-3.4.1-1, pie-8.7.6.5
)) Tue Jan 18 21:14:17 CET 2005

# xm list
Name              Id  Mem(MB)  CPU  State  Time(s)  Console
Domain-0           0       91    0  r----  138106.9
virt-1             2      127    0  -b---      7.9    9602

# xm save virt-1 virt-1.save
Error: Error: [Failure instance: Traceback: 
xen.xend.XendError.XendError, save failed

# xm migrate virt-1 localhost
Error: Error: [Failure instance: Traceback: 
xen.xend.XendError.XendError, migrate failed

Logs for the 'save' attempt are attached. Any hints?

TIA
Dave

[-- Attachment #2: xfrd.log --]
[-- Type: text/plain, Size: 1548 bytes --]

25601 [INF] XFRD> Accepted connection from 127.0.0.1:1115 on 2
25667 [INF] XFRD> Xfr service for 127.0.0.1:1115
[DEBUG] Conn_init> flags=1
[DEBUG] Conn_init> write stream...
[DEBUG] stream_init>mode=w flags=1 compress=0
[DEBUG] stream_init> unbuffer...
[DEBUG] stream_init< err=0
[DEBUG] Conn_init> read stream...
[DEBUG] stream_init>mode=r flags=1 compress=0
[DEBUG] stream_init> unbuffer...
[DEBUG] stream_init< err=0
[DEBUG] Conn_sxpr>
(xfr.hello 1 0)[DEBUG] Conn_sxpr< err=0
[DEBUG] Conn_sxpr>
(xfr.save 2 "(domain (id 2) (name virt-1) (memory 127) (maxmem 131072) (state -b---) (cpu 0) (cpu_time 7.66579571) (up_time 45.2437880039) (start_time 1106262818.13) (console (status listening) (id 13) (domain 2) (local_port 13) (remote_port 1) (console_port 9602)) (devices (vif (idx 0) (vif 0) (mac aa:00:00:2a:f6:2a) (evtchn 17 4) (index 0)) (vbd (idx 0) (vdev 2049) (device 1792) (mode w) (dev sda1) (uname file:/xen/virt-1) (node /dev/loop0) (index 0)) (vbd (idx 1) (vdev 2050) (device 1793) (mode w) (dev sda2) (uname file:/xen/virt-1.swap) (node /dev/loop1) (index 1))) (config (vm (name virt-1) (memory 128) (image (linux (kernel /boot/vmlinuz-2.6.10-xenU) (ip 192.168.0.31:1.2.3.4:192.168.0.1:255.255.254.0:gentoo:eth0:off) (root '/dev/sda1 ro'))) (device (vbd (uname file:/xen/virt-1) (dev sda1) (mode w))) (device (vbd (uname file:/xen/virt-1.swap) (dev sda2) (mode w))) (device (vif (mac aa:00:00:2a:f6:2a))))))" /xen/virt-1.save)[DEBUG] Conn_sxpr< err=0
[1106262863.378546] xc_linux_save start 2

xc_linux_save start 2

[-- Attachment #3: xend-debug.log --]
[-- Type: text/plain, Size: 2762 bytes --]

network start bridge=xen-br0 netdev=eth0 antispoof=no
VIRTUAL MEMORY ARRANGEMENT:
 Loaded kernel: c0100000->c0451144
 Init. ramdisk: c0452000->c0452000
 Phys-Mach map: c0452000->c0472000
 Page tables:   c0472000->c0475000
 Start info:    c0475000->c0476000
 Boot stack:    c0476000->c0477000
 TOTAL:         c0000000->c0800000
 ENTRY ADDRESS: c0100000
/dev/loop: Is a directory
/dev/loop: Is a directory
ioctl: LOOP_SET_FD: Device or resource busy
vif-bridge up vif=vif2.0 domain=virt-1 mac=aa:00:00:2a:f6:2a bridge=xen-br0
recv_fe_driver_status> {'status': 1}

recv_fe_driver_status>

recv_fe_interface_connect {'tx_shmem_frame': 17426, 'rx_shmem_frame': 18698, 'handle': 0}
sync_session> <type 'str'> 1 ['save', ['id', '1'], ['state', 'begin'], ['domain', '2'], ['file', '/xen/virt-1.save']]
Started to connect self= <xen.xend.XendMigrate.XfrdClientFactory instance at 0xb76c34cc> connector= <twisted.internet.tcp.Connector instance at 0xb76c34ec>
buildProtocol> IPv4Address(TCP, 'localhost', 8002)
***request> (domain (id 2) (name virt-1) (memory 127) (maxmem 131072) (state -b---) (cpu 0) (cpu_time 7.66579571) (up_time 45.2407670021) (start_time 1106262818.13) (console (status listening) (id 13) (domain 2) (local_port 13) (remote_port 1) (console_port 9602)) (devices (vif (idx 0) (vif 0) (mac aa:00:00:2a:f6:2a) (evtchn 17 4) (index 0)) (vbd (idx 0) (vdev 2049) (device 1792) (mode w) (dev sda1) (uname file:/xen/virt-1) (node /dev/loop0) (index 0)) (vbd (idx 1) (vdev 2050) (device 1793) (mode w) (dev sda2) (uname file:/xen/virt-1.swap) (node /dev/loop1) (index 1))) (config (vm (name virt-1) (memory 128) (image (linux (kernel /boot/vmlinuz-2.6.10-xenU) (ip 192.168.0.31:1.2.3.4:192.168.0.1:255.255.254.0:gentoo:eth0:off) (root '/dev/sda1 ro'))) (device (vbd (uname file:/xen/virt-1) (dev sda1) (mode w))) (device (vbd (uname file:/xen/virt-1.swap) (dev sda2) (mode w))) (device (vif (mac aa:00:00:2a:f6:2a))))))
***request> begin
xfr_err> ['xfr.err', '0']
xfr_err> <type 'str'> 0
Xfrd>connectionLost> [Failure instance: Traceback: twisted.internet.error.ConnectionDone, Connection was closed cleanly.
]
XfrdSaveInfo>connectionLost> [Failure instance: Traceback: twisted.internet.error.ConnectionDone, Connection was closed cleanly.
]
XfrdInfo>connectionLost> [Failure instance: Traceback: twisted.internet.error.ConnectionDone, Connection was closed cleanly.
]
Error> save failed
Error> calling errback
***cbremove> [Failure instance: Traceback: xen.xend.XendError.XendError, save failed
]
***_delete_session> 1
clientConnectionLost> connector= <twisted.internet.tcp.Connector instance at 0xb76c34ec> reason= [Failure instance: Traceback: twisted.internet.error.ConnectionDone, Connection was closed cleanly.
]

[-- Attachment #4: xend.log --]
[-- Type: text/plain, Size: 1862 bytes --]

[2005-01-21 00:13:24 xend] INFO (SrvDaemon:607) Xend Daemon started
[2005-01-21 00:13:24 xend] INFO (XendRoot:91) EVENT> xend.start 0
[2005-01-21 00:13:38 xend] DEBUG (XendDomainInfo:720) init_domain> Created domain=2 name=virt-1 memory=128
[2005-01-21 00:13:38 xend] INFO (console:92) Created console id=13 domain=2 port=9602
[2005-01-21 00:13:38 xend] DEBUG (XendDomainInfo:1130) Creating vbd dom=2 uname=file:/xen/virt-1
[2005-01-21 00:13:38 xend] DEBUG (blkif:146) Connecting blkif <BlkifBackendInterface 2 0>
[2005-01-21 00:13:38 xend] DEBUG (XendDomainInfo:1130) Creating vbd dom=2 uname=file:/xen/virt-1.swap
[2005-01-21 00:13:38 xend] DEBUG (blkif:146) Connecting blkif <BlkifBackendInterface 2 0>
[2005-01-21 00:13:38 xend] DEBUG (XendDomainInfo:1107) Creating vif dom=2 vif=0 mac=aa:00:00:2a:f6:2a
[2005-01-21 00:13:38 xend] INFO (XendRoot:91) EVENT> xend.console.create [13, 2, 9602]
[2005-01-21 00:13:38 xend] INFO (XendRoot:91) EVENT> xend.domain.create ['virt-1', '2']
[2005-01-21 00:13:38 xend] INFO (XendRoot:91) EVENT> xend.domain.unpause ['virt-1', '2']
[2005-01-21 00:13:42 xend] DEBUG (blkif:192) Connecting blkif to event channel <BlkifBackendInterface 2 0> ports=16:3
[2005-01-21 00:14:23 xend] INFO (XendMigrate:370) Save BEGIN: ['save', ['id', '1'], ['state', 'begin'], ['domain', '2'], ['file', '/xen/virt-1.save']]
[2005-01-21 00:14:23 xend] INFO (XendRoot:91) EVENT> xend.domain.save ['virt-1', '2', 'begin', ['save', ['id', '1'], ['state', 'begin'], ['domain', '2'], ['file', '/xen/virt-1.save']]]
[2005-01-21 00:14:23 xend] INFO (XendMigrate:390) Save ERROR: ['save', ['id', '1'], ['state', 'error'], ['domain', '2'], ['file', '/xen/virt-1.save']]
[2005-01-21 00:14:23 xend] INFO (XendRoot:91) EVENT> xend.domain.save ['virt-1', '2', 'error', ['save', ['id', '1'], ['state', 'error'], ['domain', '2'], ['file', '/xen/virt-1.save']]]

RE: save / migrate not working on gentoo?

[Ian Pratt]

 

> # xm save virt-1 virt-1.save
> Error: Error: [Failure instance: Traceback: 
> xen.xend.XendError.XendError, save failed

If save doesn't work something really dumb must be happening.

Is there anything useful in /var/log/xfrd.log or xend.debug?

BTW: I've heard that setting mem= on the domU kernel command line can
cause problems, but haven't investigated yet. 

Obviously the size of the domain shuoldn't make any difference, but have
you tried creating and saving a small domain e.g. 64MB? (I'm just
wandering whether something daft is happening with EINTR or something
during IO).

Ian 


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: save / migrate not working on gentoo?

[tdc]

> Is there anything useful in /var/log/xfrd.log or xend.debug?

both were included in previous mail, not sure if there is anything 
usefull for debugging this

> BTW: I've heard that setting mem= on the domU kernel command line can
> cause problems, but haven't investigated yet. 

my domain setup script is quite simple, no 'mem=' on domU kernel command 
line

--
kernel = "/boot/vmlinuz-2.6.10-xenU"
name = "virt-1"
hostname = "virt"
memory = 128
disk = [ 'file:/xen/virt-1,sda1,w', file:/xen/virt-1.swap,sda2,w' ]
nics=1
ip = "192.168.0.31"
netmask = "255.255.254.0"
gateway = "81.95.106.1"
root = "/dev/sda1 ro"

> Obviously the size of the domain shuoldn't make any difference, but have
> you tried creating and saving a small domain e.g. 64MB? (I'm just
> wandering whether something daft is happening with EINTR or something
> during IO).

I've tried it just now - domU with 48MB, still the same results.

There is some python testing script in tools/xfrd, maybe it could help 
me to find the problem, but I'm not sure how much I can tinker with it. 
I don't know, which commands to use and how the xfrd reply should look like.

Dave



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: save / migrate not working on gentoo?

[tdc]

I suspect some front-end tools are not working correctly and I'd like to 
test to save domain manually. Can somebody post here a sequence of 
commands, which needs to be sent to xfrd and what results should I 
expect? It looks that running xfrdClient.py works (at least it doesn't 
throw errors), but I'm not sure, if it really migrates a domain or not.

Dave




-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: save / migrate not working on gentoo?

[Jan Kundrát]

tdc wrote:
> ip = "192.168.0.31"
> netmask = "255.255.254.0"
> gateway = "81.95.106.1"

Probably completely irrelevant, but how could this work? Isn't gateway 
supposed to be on the same subnet as the host?

-jkt

-- 
cd /local/pub && more beer > /dev/mouth



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: save / migrate not working on gentoo?

[Felipe Alfaro Solana]

On 23 Jan 2005, at 21:20, Jan Kundrát wrote:

> tdc wrote:
>> ip = "192.168.0.31"
>> netmask = "255.255.254.0"
>> gateway = "81.95.106.1"
>
> Probably completely irrelevant, but how could this work? Isn't gateway 
> supposed to be on the same subnet as the host?

Yep! The gateway must be directly accesible on the local subnet. 
However, 81.95.106.1 is not on the same subnet as 192.168.0.31.


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: save / migrate not working on gentoo?

[tdc]

> Probably completely irrelevant, but how could this work? Isn't gateway 
> supposed to be on the same subnet as the host?
> 
> -jkt


Noticed that just after I sent that, but this is not the source of my 
problem - I've just pasted and edited config from different machine as I 
wasn't logged on the home-devel one (except for the different net 
config, which got mixed, these two config files are exactly the same)

Dave



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: save / migrate not working on gentoo?

[Matthieu]

On January 21, 12:54 pm tdc <tdc@phreaker.net> wrote:
> I suspect some front-end tools are not working correctly and I'd like to
> test to save domain manually. Can somebody post here a sequence of
> commands, which needs to be sent to xfrd and what results should I
> expect? It looks that running xfrdClient.py works (at least it doesn't
> throw errors), but I'm not sure, if it really migrates a domain or not.
>
> Dave
I've have the same problems as debribed before in the thread.
I've done a very simple installation of debian (with debootstrap) on a very
simple configuration: one vif and one vbd and i'm able to save the domain. 

But since i wasn't able to save a more complex domain (multiple vif and
vbd) i investigate and i've see that when you have more than  vdb to your
xenU configuration then you can't save it (i got the same message mentioned
before).

HTH
Matthieu


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl

Re: save / migrate not working on gentoo?

[Todd Jones]

I've have the same problems as debribed before in the thread.
I've done a very simple installation of debian (with debootstrap) on a very
simple configuration: one vif and one vbd and i'm able to save the domain. 

But since i wasn't able to save a more complex domain (multiple vif and
vbd) i investigate and i've see that when you have more than  vdb to your
xenU configuration then you can't save it (i got the same message mentioned
before).

HTH
Matthieu

  

This is the same behavior I've seen (Also on Debian).  A domain with 
fewer than 3 file-backed block devices saves/restores fine.  With 3 or 
more the attempt fails immediately.  As I recall the error in the log is 
-12.  I haven't had time to diagnose it further than that.

Todd



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl