[PATCH] hw/xen: Set suppress-vmdesc for Xen machines

[PATCH] hw/xen: Set suppress-vmdesc for Xen machines

[Jason Andryuk]

xen-save-devices-state doesn't currently generate a vmdesc, so restore
always triggers "Expected vmdescription section, but got 0".  This is
not a problem when restore comes from a file.  However, when QEMU runs
in a linux stubdom and comes over a console, EOF is not received.  This
causes a delay restoring - though it does restore.

Setting suppress-vmdesc skips looking for the vmdesc during restore and
avoids the wait.

The other approach would be generate a vmdesc in qemu_save_device_state.
Since COLO shared that function, and the vmdesc is just discarded on
restore, we choose to skip it.

Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Signed-off-by: Jason Andryuk <jandryuk@gmail.com>
---
 hw/i386/pc_piix.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index 3c2ae0612b..0cf22a57ad 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -987,7 +987,7 @@ static void xenfv_4_2_machine_options(MachineClass *m)
     pc_i440fx_4_2_machine_options(m);
     m->desc = "Xen Fully-virtualized PC";
     m->max_cpus = HVM_MAX_VCPUS;
-    m->default_machine_opts = "accel=xen";
+    m->default_machine_opts = "accel=xen,suppress-vmdesc=on";
 }
 
 DEFINE_PC_MACHINE(xenfv_4_2, "xenfv-4.2", pc_xen_hvm_init,
@@ -999,7 +999,7 @@ static void xenfv_3_1_machine_options(MachineClass *m)
     m->desc = "Xen Fully-virtualized PC";
     m->alias = "xenfv";
     m->max_cpus = HVM_MAX_VCPUS;
-    m->default_machine_opts = "accel=xen";
+    m->default_machine_opts = "accel=xen,suppress-vmdesc=on";
 }
 
 DEFINE_PC_MACHINE(xenfv, "xenfv-3.1", pc_xen_hvm_init,
-- 
2.25.1

Re: [PATCH] hw/xen: Set suppress-vmdesc for Xen machines

[Anthony PERARD via]

On Tue, Oct 13, 2020 at 03:05:06PM -0400, Jason Andryuk wrote:
> xen-save-devices-state doesn't currently generate a vmdesc, so restore
> always triggers "Expected vmdescription section, but got 0".  This is
> not a problem when restore comes from a file.  However, when QEMU runs
> in a linux stubdom and comes over a console, EOF is not received.  This
> causes a delay restoring - though it does restore.
> 
> Setting suppress-vmdesc skips looking for the vmdesc during restore and
> avoids the wait.

suppress-vmdesc is only used during restore, right? So starting a guest
without it, saving the guest and restoring the guest with
suppress-vmdesc=on added will work as intended? (I'm checking that migration
across update of QEMU will work.)

Thanks,

-- 
Anthony PERARD

Re: [PATCH] hw/xen: Set suppress-vmdesc for Xen machines

[Jason Andryuk]

On Fri, Oct 16, 2020 at 11:38 AM Anthony PERARD
<anthony.perard@citrix.com> wrote:
>
> On Tue, Oct 13, 2020 at 03:05:06PM -0400, Jason Andryuk wrote:
> > xen-save-devices-state doesn't currently generate a vmdesc, so restore
> > always triggers "Expected vmdescription section, but got 0".  This is
> > not a problem when restore comes from a file.  However, when QEMU runs
> > in a linux stubdom and comes over a console, EOF is not received.  This
> > causes a delay restoring - though it does restore.
> >
> > Setting suppress-vmdesc skips looking for the vmdesc during restore and
> > avoids the wait.
>
> suppress-vmdesc is only used during restore, right? So starting a guest
> without it, saving the guest and restoring the guest with
> suppress-vmdesc=on added will work as intended? (I'm checking that migration
> across update of QEMU will work.)

vmdesc is a json description of the migration stream that comes after
the QEMU migration stream.  For our purposes, <migration
stream><vmdesc json blob>.  Normal QEMU savevm will generate it,
unless suppress-vmdesc is set.  QEMU restore will read it because:
"Try to read in the VMDESC section as well, so that dumping tools that
intercept our migration stream have the chance to see it."

Xen save does not go through savevm, but instead
xen-save-devices-state, which is a subset of the QEMU savevm.  It
skips RAM since that is read out through Xen interfaces.  Xen uses
xen-load-devices-state to restore device state.  That goes through the
common qemu_loadvm_state which tries to read the vmdesc stream.

For Xen, yes, suppress-vmdesc only matters for the restore case, and
it suppresses the attempt to read the vmdesc.  I think every Xen
restore currently has "Expected vmdescription section, but got -1" in
the -dm.log since the vmdesc is missing.  I have not tested restoring
across this change, but since it just controls reading and discarding
the vmdesc stream, I don't think it will break migration across
update.

Regards,
Jason

Re: [PATCH] hw/xen: Set suppress-vmdesc for Xen machines

[Anthony PERARD via]

On Fri, Oct 16, 2020 at 12:01:47PM -0400, Jason Andryuk wrote:
> On Fri, Oct 16, 2020 at 11:38 AM Anthony PERARD
> <anthony.perard@citrix.com> wrote:
> >
> > On Tue, Oct 13, 2020 at 03:05:06PM -0400, Jason Andryuk wrote:
> > > xen-save-devices-state doesn't currently generate a vmdesc, so restore
> > > always triggers "Expected vmdescription section, but got 0".  This is
> > > not a problem when restore comes from a file.  However, when QEMU runs
> > > in a linux stubdom and comes over a console, EOF is not received.  This
> > > causes a delay restoring - though it does restore.
> > >
> > > Setting suppress-vmdesc skips looking for the vmdesc during restore and
> > > avoids the wait.
> >
> > suppress-vmdesc is only used during restore, right? So starting a guest
> > without it, saving the guest and restoring the guest with
> > suppress-vmdesc=on added will work as intended? (I'm checking that migration
> > across update of QEMU will work.)
> 
> vmdesc is a json description of the migration stream that comes after
> the QEMU migration stream.  For our purposes, <migration
> stream><vmdesc json blob>.  Normal QEMU savevm will generate it,
> unless suppress-vmdesc is set.  QEMU restore will read it because:
> "Try to read in the VMDESC section as well, so that dumping tools that
> intercept our migration stream have the chance to see it."
> 
> Xen save does not go through savevm, but instead
> xen-save-devices-state, which is a subset of the QEMU savevm.  It
> skips RAM since that is read out through Xen interfaces.  Xen uses
> xen-load-devices-state to restore device state.  That goes through the
> common qemu_loadvm_state which tries to read the vmdesc stream.
> 
> For Xen, yes, suppress-vmdesc only matters for the restore case, and
> it suppresses the attempt to read the vmdesc.  I think every Xen
> restore currently has "Expected vmdescription section, but got -1" in
> the -dm.log since the vmdesc is missing.  I have not tested restoring
> across this change, but since it just controls reading and discarding
> the vmdesc stream, I don't think it will break migration across
> update.

Thanks for the explanation.

Acked-by: Anthony PERARD <anthony.perard@citrix.com>

Do you think you could send a patch for libxl as well? Since libxl in
some cases may use the "pc machine instead of "xenfv". I can send the
patch otherwise.

Cheers,

-- 
Anthony PERARD

Re: [PATCH] hw/xen: Set suppress-vmdesc for Xen machines

[Jason Andryuk]

On Fri, Oct 16, 2020 at 12:44 PM Anthony PERARD
<anthony.perard@citrix.com> wrote:
>
> On Fri, Oct 16, 2020 at 12:01:47PM -0400, Jason Andryuk wrote:
> > On Fri, Oct 16, 2020 at 11:38 AM Anthony PERARD
> > <anthony.perard@citrix.com> wrote:
> > >
> > > On Tue, Oct 13, 2020 at 03:05:06PM -0400, Jason Andryuk wrote:
> > > > xen-save-devices-state doesn't currently generate a vmdesc, so restore
> > > > always triggers "Expected vmdescription section, but got 0".  This is
> > > > not a problem when restore comes from a file.  However, when QEMU runs
> > > > in a linux stubdom and comes over a console, EOF is not received.  This
> > > > causes a delay restoring - though it does restore.
> > > >
> > > > Setting suppress-vmdesc skips looking for the vmdesc during restore and
> > > > avoids the wait.
> > >
> > > suppress-vmdesc is only used during restore, right? So starting a guest
> > > without it, saving the guest and restoring the guest with
> > > suppress-vmdesc=on added will work as intended? (I'm checking that migration
> > > across update of QEMU will work.)
> >
> > vmdesc is a json description of the migration stream that comes after
> > the QEMU migration stream.  For our purposes, <migration
> > stream><vmdesc json blob>.  Normal QEMU savevm will generate it,
> > unless suppress-vmdesc is set.  QEMU restore will read it because:
> > "Try to read in the VMDESC section as well, so that dumping tools that
> > intercept our migration stream have the chance to see it."
> >
> > Xen save does not go through savevm, but instead
> > xen-save-devices-state, which is a subset of the QEMU savevm.  It
> > skips RAM since that is read out through Xen interfaces.  Xen uses
> > xen-load-devices-state to restore device state.  That goes through the
> > common qemu_loadvm_state which tries to read the vmdesc stream.
> >
> > For Xen, yes, suppress-vmdesc only matters for the restore case, and
> > it suppresses the attempt to read the vmdesc.  I think every Xen
> > restore currently has "Expected vmdescription section, but got -1" in
> > the -dm.log since the vmdesc is missing.  I have not tested restoring
> > across this change, but since it just controls reading and discarding
> > the vmdesc stream, I don't think it will break migration across
> > update.
>
> Thanks for the explanation.
>
> Acked-by: Anthony PERARD <anthony.perard@citrix.com>
>
> Do you think you could send a patch for libxl as well? Since libxl in
> some cases may use the "pc machine instead of "xenfv". I can send the
> patch otherwise.

I should be able to, yes.

Regards,
Jason