From mboxrd@z Thu Jan 1 00:00:00 1970 From: "James Song" Subject: =?UTF-8?Q?RE=EF=BC=9A=20RE:=20=20when=20timer=20go=20?= =?UTF-8?Q?back=20in=20dom0=20save=20and=20restore=20ormigrate, =20PV=20do?= =?UTF-8?Q?main=20hung?= Date: Thu, 27 Nov 2008 03:21:44 -0700 Message-ID: <492F0F690200002000007F61@lucius.provo.novell.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1332052855==" Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: keir.fraser@eu.citrix.com, kevin.tian@intel.com, xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org This is a MIME message. If you are reading this text, you may want to consider changing to a mail reader or gateway that understands how to properly handle MIME multipart messages. --===============1332052855== Content-Type: multipart/alternative; boundary="=__PartD3FBB9A8.2__=" This is a MIME message. If you are reading this text, you may want to consider changing to a mail reader or gateway that understands how to properly handle MIME multipart messages. --=__PartD3FBB9A8.2__= Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi, Ok, now two machine A and B. the system-time of A is ahead of B. So = wc_sec of A is also bigger than B. When PV dom in A migrate to B, we = haven't upate that PV dom's wc_sec to equal with B. Ok, now we see pv = dom's kernel: xen_sched_clock() in arch/86/xen/time.c andxen_clocksource_read() = arch/x86/kernel/time_32-xen.c you will find if state_entry_time of its's vcpu, because the state_entry_= time is initalized in machine A. this time it more big than "now" of = machine B. So no schedule, no system-update in Guest os. I don't whether did I describe it clearly. >>> "Tian, Kevin" 08/11/27 PM 9:18 >>>there's a = clock_was_set called for each settimeofday. In latest kernel, clock_was_set= will adjust CLOCK_REALTIME queue accordingly, while in 2.6.18 it's = defined as a nop. That says, current domU would be unable to handle = wallclock change, but newer kernel with pvops could. ---- yes, it works = for FV, but for a modified PV domain, mybe not.=20 =20 for the issue reported in original thread, I agree that James should dig = into the hang and explain the exact reason first. =20 Thanks Kevin From: Keir Fraser [mailto:keir.fraser@eu.citrix.com]=20 Sent: Wednesday, November 26, 2008 10:58 PM To: Tian, Kevin; 'James Song'; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] when timer go back in dom0 save and restore or = migrate, PV domain hung =20 So what happens if someone changes wallclock using 'date'? That's = basically kind of what will appear to happen when s/r occurs. -- Keir On 26/11/08 14:32, "Tian, Kevin" wrote: hrtimer supports two timer bases: CLOCK_MONOTONIC and CLOCK_REALTIME.= wall_to_monotonic is only added in former case, and for latter = instead TOD is used directly per my reading. I did a quick search, and = it looks that futex and ntp are using CLOCK_REALTIME. Also there's one = vsyscall gate which can pass CLOCK_REALTIME from caller too. Thanks, Kevin =20 =20 mailto:keir.fraser@eu.citrix.com] =20 Sent: Wednesday, November 26, 2008 10:26 PM To: Tian, Kevin; 'James Song'; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] when timer go back in dom0 save and = restore or migrate, PV domain hung =20 hrtimers add wall_to_monotonic to xtime to get a timesource that = doesn't (or shouldn't!) warp. -- Keir On 26/11/08 14:20, "Tian, Kevin" = wrote: =20 how about hrtimers? one mode is CLOCK_REALTIME, which uses = getnstimeofday as expiration. Once system time is changed either = in local or new machine, that expiration can't be adjusted. but = i'm not sure whether it still makes sense to try hrtimers in a = guest. Thanks Kevin =20 =20 =20 =20 mailto:keir.fraser@eu.citrix.com] =20 Sent: Wednesday, November 26, 2008 10:11 PM To: Tian, Kevin; 'James Song'; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] when timer go back in dom0 save and = restore or migrate, PV domain hung =20 The problem hasn't been fully explained, but I can say that PV = guests expect system time to jump across s/r and deal with = that. For example, Linux doesn't use Xen system time internally= , but uses its progress to periodically update jiffies, which = does not warp across s/r. We have had problems corrupting wc_sec/wc_nsec in xc_domain_res= tore.c, but that was fixed some time ago. -- Keir On 26/11/08 14:00, "Tian, Kevin" = wrote: =20 =20 This is not a s/r or lm specific issue. For example, = system time can be changed even when pv guest is running. = Your patch only hacks restore point once, and wc_sec can = still be changed later when system time is changed on-the-fly= again. IIRC, pv guest can catch up wall clock change in timer = interrupt, and time_resume will sync internal processed = system time with new system time after restored. But I'm not = sure whether it's enough. Actually the more interesting is = the uptime difference. For example, timer with expiration = calculated on previous system time may wait nearly infinite = if uptime among two boxes vary a lot. But I think such issue = should have been considered already, e.g. some user tool = assistance. I think Keir can comment better = here. BTW, do you happen to know what exactly dom0 hangs on? In = some busy loop to catch up time, or long delay to some = critical timer expiration? Thanks, Kevin =20 =20 =20 =20 =20 =20 mailto:xen-devel-bounces@lists.xensource.com] = On Behalf Of James Song Sent: Tuesday, November 25, 2008 4:02 PM To: xen-devel@lists.xensource.com Subject: [Xen-devel] when timer go back in dom0 save and = restore or migrate, PV domain hung =20 Hi, I find PV domin hung, When we take those steps = =20 1, save PV domain =20 2, change system time of PV domain back = =20 3, restore a PV domain =20 or =20 1, migrate a PV domain from Machine A to Machine = B 2, the system time of Machine B is slower than = Machine A. the problem is wc_sec will be change when system-time = chanaged in dom0 or restore in a slower-system-time = machine, but when restoring, xen don't restore the wc_sec = of share_info from xenstore and use native one. = So guest os will hang. =20 this patch will work for this issue. Thanks -- Song Wei diff -r a5ed0dbc829f tools/libxc/xc_domain_restore.c --- a/tools/libxc/xc_domain_restore.c = Tue Nov 18 14:34:14 2008 +0800 +++ b/tools/libxc/xc_domain_restore.c Fri Nov 21 = 17:34:15 2008 +0800 @@ -328,6 +328,16 @@ =20 /* For info only */ nr_pfns =3D 0; + //jsong@novell.com, james song + memset(&domctl, 0, sizeof(domctl)); + domctl.domain =3D dom; + domctl.cmd =3D XEN_DOMCTL_restoredo= main; + frc =3D do_domctl(xc_handle, &domctl); + if ( frc !=3D 0 ) + { + ERROR("Unable to set flag of = restore."); + goto out; + } =20 if ( read_exact(io_fd, &p2m_size, = sizeof(unsigned long)) ) { @@ -1120,6 +1130,8 @@ =20 /* restore saved vcpu_info and arch specific info = */ MEMCPY_FIELD(new_shared_info, old_shared_info, = vcpu_info); + MEMCPY_FIELD(new_shared_info, = old_shared_info, wc_nsec); + MEMCPY_FIELD(new_shared_info, = old_shared_info, wc_sec); MEMCPY_FIELD(new_shared_info, old_shared_info, = arch); =20 /* clear any pending events and the selector = */ diff -r a5ed0dbc829f xen/arch/x86/time.c --- a/xen/arch/x86/time.c Tue Nov 18 = 14:34:14 2008 +0800 +++ b/xen/arch/x86/time.c Fri Nov 21 = 17:34:15 2008 +0800 @@ -689,7 +689,6 @@ wmb(); (*version)++; } - void update_vcpu_system_time(struct vcpu = *v) { struct cpu_time *t; @@ -703,7 +702,6 @@ =20 if ( u->tsc_timestamp =3D=3D t->local_tsc_stamp = ) return; - version_update_begin(&u->version); =20 u->tsc_timestamp =3D t->local_tsc_stamp; @@ -713,14 +711,19 @@ =20 version_update_end(&u->version); } - void update_domain_wallclock_time(struct domain = *d) { spin_lock(&wc_lock); + if(d->after_restore ) + { + d->after_restore =3D 0; + goto out; //jsong@novell.com + } version_update_begin(&shared_info(d, wc_version)); shared_info(d, wc_sec) =3D wc_sec + = d->time_offset_seconds; shared_info(d, wc_nsec) =3D wc_nsec; version_update_end(&shared_info(d, wc_version)); +out: spin_unlock(&wc_lock); } =20 @@ -751,7 +754,6 @@ u64 x; u32 y, _wc_sec, _wc_nsec; struct domain *d; - x =3D (secs * 1000000000ULL) + (u64)nsecs - = system_time_base; y =3D do_div(x, 1000000000); =20 @@ -1050,7 +1052,6 @@ struct tm wallclock_time(void) { uint64_t seconds; - if ( !wc_sec ) return (struct tm) { 0 }; =20 diff -r a5ed0dbc829f xen/common/domctl.c --- a/xen/common/domctl.c Tue Nov 18 = 14:34:14 2008 +0800 +++ b/xen/common/domctl.c Fri Nov 21 = 17:34:15 2008 +0800 @@ -24,7 +24,6 @@ #include #include #include - extern long arch_do_domctl( struct xen_domctl *op, XEN_GUEST_HANDLE(xen_domctl_t)= u_domctl); =20 @@ -315,6 +314,16 @@ ret =3D 0; } break; + case XEN_DOMCTL_restoredomain: + { + struct domain *d; + if ( (d =3D rcu_lock_domain_by_id(= op->domain)) =3D=3D NULL ) + break; + =20 + d->after_restore =3D 1; + rcu_unlock_domain(d); + break; + } =20 case XEN_DOMCTL_createdomain: { diff -r a5ed0dbc829f xen/include/public/domc= tl.h --- a/xen/include/public/domctl.h Tue = Nov 18 14:34:14 2008 +0800 +++ b/xen/include/public/domctl.h Fri Nov 21 17:34:15 = 2008 +0800 @@ -61,6 +61,7 @@ #define XEN_DOMCTL_destroydomain 2 #define XEN_DOMCTL_pausedomain 3 #define XEN_DOMCTL_unpausedomain 4 +#define XEN_DOMCTL_restoredomain 51 #define XEN_DOMCTL_resumedomain 27 =20 #define XEN_DOMCTL_getdomaininfo 5 diff -r a5ed0dbc829f xen/include/xen/sched.h --- a/xen/include/xen/sched.h Tue Nov 18 = 14:34:14 2008 +0800 +++ b/xen/include/xen/sched.h Fri Nov 21 = 17:34:15 2008 +0800 @@ -231,6 +231,7 @@ * cause a deadlock. Acquirers don't spin waiting; = they preempt. */ spinlock_t hypercall_deadlock_mutex; + int after_restore; //jsong@novell.com }; =20 struct domain_setup_info ---------------------------------------------------------------------------= ------------------ Thanks --Song wei --=__PartD3FBB9A8.2__= Content-Type: text/html; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi,
     Ok, now two machine A = and B. the system-time of A is ahead of B. So wc_sec of A is also bigger = than B. When PV dom in A migrate to B, we haven't upate that PV dom's = wc_sec to equal with B. Ok, now we see pv dom's kernel:
  &nbs= p; xen_sched_clock() in arch/86/xen/time.c andxen_clocksource_read()  = arch/x86/kernel/time_32-xen.c
  you will find if state_entry_time = of its's vcpu, because the state_entry_time is initalized in machine A. = this time it more big than "now" of machine B. So no schedule, no = system-update in Guest os.
I don't whether did I describe it clearly.
>>> "Tian, Kevin" 08/11/27 PM 9:18 = >>>
there's a clock_was_set called = for each settimeofday. In=20 latest kernel, clock_was_set will adjust CLOCK_REALTIME queue accordingly, = while=20 in 2.6.18 it's defined as a nop. That says, current domU would be = unable to=20 handle wallclock change, but newer kernel with pvops could.  ---- = yes, it works for FV, but for a modified PV domain, mybe not.
 
for the issue reported in = original thread, I agree that=20 James should dig into the hang and explain the exact reason=20 first.
 
Thanks
Kevin

From: Keir Fraser=20 [mailto:keir.fraser@eu.citrix.com]
Sent: Wednesday, November = 26,=20 2008 10:58 PM
To: Tian, Kevin; 'James Song';=20 xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] when = timer go=20 back in dom0 save and restore or migrate, PV domain hung

So what happens if someone changes wallclock using=20 'date'? That's basically kind of what will appear to happen when s/r=20 occurs.

 -- Keir

On 26/11/08 14:32, "Tian, Kevin"=20 <kevin.tian@intel.com> wrote:
hrtimer supports two timer bases: CLOCK_MONOTONIC and=20 CLOCK_REALTIME. wall_to_monotonic is only added in former case, and = for=20 latter instead TOD is used directly per my reading. I did a quick = search,=20 and it looks that futex and ntp are using CLOCK_REALTIME. Also there's = one=20 vsyscall gate which can pass CLOCK_REALTIME from caller=20 too.

Thanks,
Kevin
=

 
From: Keir Fraser  [mailto:keir.fraser@eu.citrix.c= om]=20
Sent: Wednesday, November 26,  2008 10:26 PM
To= :=20 Tian, Kevin; 'James Song';=20  xen-devel@lists.xensource.com
Subject: Re: [Xen-devel= ]=20 when timer go  back in dom0 save and restore or migrate, PV = domain=20 hung

 hrtimers add=20 wall_to_monotonic to xtime to get a  timesource that doesn't = (or=20 shouldn't!) warp.

 -- Keir

On  26/11/08 = 14:20,=20 "Tian, Kevin" <kevin.tian@intel.com>=20  wrote:

 
= how about hrtimers? one mode is CLOCK_REALTIME, which = uses=20  getnstimeofday as expiration. Once system time is changed = either=20 in local or  new machine, that expiration can't be adjusted. = but=20 i'm not sure whether it  still makes sense to try hrtimers in = a=20 guest.

Thanks
Ke= vin

 

 
 
From: Keir Fraser  [mailto:keir.fraser@eu.citrix.c= om]=20  
Sent: Wednesday, November 26,  2008 = 10:11=20 PM
To:  Tian, Kevin; 'James Song';=20   xen-devel@lists.xensource.com
Subject: = Re:=20 [Xen-devel]  when timer go  back in dom0 save and = restore or=20 migrate, PV domain  hung

 
The  problem=20 hasn't been fully explained, but I can say  that PV = guests=20  expect system time to jump across s/r and deal with that. = For=20   example, Linux doesn't use Xen system time internally= , but=20 uses its  progress  to periodically update jiffies, = which=20 does not warp across  s/r.

We have  had = problems=20 corrupting wc_sec/wc_nsec in  xc_domain_restore.c, but that = was=20  fixed some time  ago.

 -- Keir

On= =20 26/11/08 14:00, "Tian,  Kevin"  <kevin.tian@intel.co= m>=20 wrote:

 
 
This is not a s/r or lm specific issue. For = example,=20 system  time  can be changed even when pv guest = is=20 running. Your patch only  hacks restore  point once, = and=20 wc_sec can still be changed later  when system time is=20  changed on-the-fly  again.

IIRC, pv guest can catch up wall=20 clock change in timer  interrupt,  and time_resume = will=20 sync internal processed system  time with new system =  time=20 after restored. But I'm not sure whether  it's enough. = Actually=20 the more  interesting is the uptime  difference. = For=20 example, timer with expiration  calculated on  previo= us=20 system time may wait nearly infinite if uptime among =  two=20  boxes vary a lot. But I think such issue should have = been=20 considered   already, e.g. some user tool assistance.= I=20 think Keir can comment  better=20  here.

BTW, do you = happen to know what=20 exactly dom0 hangs on? In  some  busy loop to catch = up=20 time, or long delay to some critical  timer=20  expiration?

T= hanks,
Kevin

 
 

 
 
 
From:=20   xen-devel-bounces@lists.xensource.com  [mailto:xen-devel-b= ounces@lists.xensource.com]=20   On Behalf Of James  Song
Sent:<= /b>=20 Tuesday,  November 25,  2008 4:02 PM
To:= =20    xen-devel@lists.xensource.com
Subject= :=20  [Xen-devel] when  timer go  back in dom0 = save and=20 restore or  migrate, PV domain  hung

 
Hi,
   = ;I=20   find PV domin hung, When we take those steps=20    
      &nb= sp;  1,=20  save PV  domain=20   
       &nb= sp; 2,=20   change system time of  PV domain back=20   
       &nb= sp; 3,=20  restore   a PV domain=20  
        or= =20    
      &nb= sp;  1,=20  migrate  a PV domain  from Machine A to = Machine=20   B
       &n= bsp; 2,=20  the system   time of Machine B is slower = than=20 Machine  A.
   the  problem = is=20  wc_sec will be  change when system-time chanaged = in=20 dom0  or restore in a   slower-system-time = machine,=20 but when restoring, xen  don't  restore the = wc_sec=20  of share_info from xenstore and use native   = one.=20 So guest os will hang.  
this patch will work for=20  this  issue.

 Thanks
 -- = Song=20   Wei

diff -r  a5ed0dbc829f=20  tools/libxc/xc_domain_restore.c
---=20    a/tools/libxc/xc_domain_restore.c=20    Tue  Nov 18  14:34:14 2008=20  +0800
+++  b/tools/libxc/xc_domain_restore.c=20=     Fri Nov 21   17:34:15 = 2008=20 +0800
@@ -328,6  +328,16=20   @@
 
     /* = For=20 info   only=20  */
     nr_pfns =3D = 0;
+=20       //jsong@novell.com, = james=20 song
+      memset(&domctl, = 0,=20   sizeof(domctl));
+=20     domctl.domain =3D   dom;+=20     domctl.cmd    =3D=20    XEN_DOMCTL_restoredomain;
+=20    frc =3D   do_domctl(xc_handle,=20=  &domctl);
+     if ( frc =  !=3D=20 0 )
+      {
+=20           &= nbsp;   ERROR("Unable=20   to set flag of  restore.");
+=20           &= nbsp;   goto=20   out;
+=20      }
 
  &nbs= p;  if=20  (   read_exact(io_fd, &p2m_size,=20 sizeof(unsigned long))=20    )
     {
@@= =20 -1120,6 +1130,8=20    @@
 
    &nb= sp;/*=20 restore  saved  vcpu_info and arch  specific = info=20   */
     MEMCPY_FIELD(= new_shared_info,=20    old_shared_info, vcpu_info);
+=20       MEMCPY_FIELD(new_shared_i= nfo,=20  old_shared_info,   wc_nsec);
+=20     MEMCPY_FIELD(new_shared_info,=20    old_shared_info,=20   wc_sec);
      M= EMCPY_FIELD(new_shared_info,=20   old_shared_info,=20    arch);
 
    = ; /*=20 clear  any  pending events and  the = selector=20 */
diff -r  a5ed0dbc829f  xen/arch/x86/time.c---=20   a/xen/arch/x86/time.c     Tue= Nov=20 18  14:34:14 2008 +0800
+++=20   b/xen/arch/x86/time.c     Fri= Nov=20 21 17:34:15 2008  +0800
@@   -689,7 = +689,6=20   @@
      wmb();<= br>     (*version)++;
 }
-
 voi= d=20    update_vcpu_system_time(struct vcpu=20   *v)
 {
     &= nbsp;struct=20  cpu_time        *t;<= br>@@=20 -703,7  +702,6=20   @@
 
     if = (=20   u->tsc_timestamp =3D=3D  t->local_tsc_= stamp=20   )
       &n= bsp;  return;
-
      version= _update_begin(&u->version);
 
    &nb= sp; u->tsc_timestamp=20       =3D t->local_tsc_stamp= ;
@@=20   -713,14  +711,19=20   @@
 
     &nb= sp;version_update_end(&u->version);
 }
-
 void=20=    update_domain_wallclock_time(struct = domain=20    *d)
 {
    &= nbsp; spin_lock(&wc_lock);
+=20      if(d->after_restore =  )
+=20      {
+=20           d= ->after_restore=20  =3D  0;
+=20        goto   ou= t;=20  //jsong@novell.com
+=20      }
    &n= bsp; version_update_begin(&shared_info(d,=20    wc_version));
    &n= bsp;shared_info(d,=20   wc_sec)  =3D  wc_sec +=20   d->time_offset_seconds;
   &= nbsp; shared_info(d,=20    wc_nsec) =3D=20   wc_nsec;
      v= ersion_update_end(&shared_info(d,=20    wc_version));
+out:
   = ;   spin_unlock(&wc_lock);
 }
 
@@=20=   -751,7 +754,6=20  @@
     u64=20   x;
     u32 y,=20  _wc_sec,=20   _wc_nsec;
     struct= =20 domain=20    *d;
-
     x= =3D=20 (secs *  1000000000ULL)  + (u64)nsecs -=20   system_time_base;
    &nbs= p;y=20  =3D  do_div(x,  1000000000);
 
@@ = -1050,7=20 +1052,6   @@
 struct tm=20    wallclock_time(void)
 {
 &n= bsp;   uint64_t=20    seconds;
-
    &n= bsp;if=20 (  !wc_sec=20    )
      &n= bsp;  return=20   (struct tm) { 0  };
 
diff -r=20 a5ed0dbc829f   xen/common/domctl.c
---=20  a/xen/common/domctl.c      Tue= Nov=20 18 14:34:14 2008 +0800
+++=20    b/xen/common/domctl.c    Fri= Nov=20 21  17:34:15 2008  +0800
@@  -24,7 = +24,6=20 @@
 #include=20   <asm/current.h>
 #include=20    <public/domctl.h>
 #include= =20    <xsm/xsm.h>
-
 extern = long=20    arch_do_domctl(
    =  struct=20  xen_domctl  *op,  XEN_GUEST_HANDLE(xen_domctl= _t)=20  u_domctl);
 
@@  -315,6 +314,16=20    @@
      &= nbsp;  ret=20  =3D=20    0;
     }
&n= bsp;     break;
+=20     case XEN_DOMCTL_restoredomain:
+= =20     {
+=20          struct= =20 domain   *d;
+=20         if ( (d =  =3D=20   rcu_lock_domain_by_id(op->domain)) =3D=3D = NULL=20   )
+=20           &= nbsp;   break;
+=20           <= br>+=20          d->a= fter_restore=20 =3D    1;
+=20           r= cu_unlock_domain(d);
+=20           b= reak;
+=20     }
 
   &nbs= p; case=20    XEN_DOMCTL_createdomain:
  &nb= sp;  {
diff=20   -r a5ed0dbc829f=20  xen/include/public/domctl.h
---=20    a/xen/include/public/domctl.h=20    Tue Nov 18  14:34:14  2008=20  +0800
+++ b/xen/include/public/domctl.h=20      Fri Nov 21  17:34:15 = 2008=20 +0800
@@  -61,6 +61,7  @@
 #define=20  XEN_DOMCTL_destroydomain=20        2
 #define= =20    XEN_DOMCTL_pausedomain=20           3=
 #define=20   XEN_DOMCTL_unpausedomain=20        4
+#define=20   XEN_DOMCTL_restoredomain=20         51
 #= define=20   XEN_DOMCTL_resumedomain=20         27
 <= br> #define=20    XEN_DOMCTL_getdomaininfo=20       5
diff -r=20   a5ed0dbc829f  xen/include/xen/sched.h
---= =20   a/xen/include/xen/sched.h     = ;Tue=20 Nov 18 14:34:14 2008   +0800
+++=20  b/xen/include/xen/sched.h    Fri Nov = 21=20  17:34:15   2008 +0800
@@ -231,6 +231,7=20   @@
      * = cause a=20   deadlock.  Acquirers don't spin waiting; = they=20    preempt.
     &= nbsp;*/
      spinlock_t=20   hypercall_deadlock_mutex;
+    = int=20  after_restore;=20    //jsong@novell.com
 };
  struct=20    domain_setup_info
----------------------= -----------------------------------------------------------------------
=  Thanks
--Song=20    wei







--=__PartD3FBB9A8.2__=-- --===============1332052855== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --===============1332052855==--