Re: [Qemu-devel] [PATCH v5 for 2.13 3/4] docs: tpm: add VM save/restore example and troubleshooting guide

["Dr. David Alan Gilbert" <dgilbert@redhat.com>]

* Stefan Berger (stefanb@linux.vnet.ibm.com) wrote:
> On 03/28/2018 11:48 AM, Marc-André Lureau wrote:
> > Hi
> > 
> > On Fri, Mar 16, 2018 at 10:46 PM, Stefan Berger
> > <stefanb@linux.vnet.ibm.com> wrote:
> > > Extend the docs related to TPM with specs related to VM save and
> > > restore and a troubleshooting guide for TPM migration.
> > > 
> > Thanks a lot for writing this! some questions below
> > 
> > > Signed-off-by: Stefan Berger <stefanb@linux.vnet.ibm.com>
> > > ---
> > >   docs/specs/tpm.txt | 106 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> > >   1 file changed, 106 insertions(+)
> > > 
> > > diff --git a/docs/specs/tpm.txt b/docs/specs/tpm.txt
> > > index d1d7157..c230c4c 100644
> > > --- a/docs/specs/tpm.txt
> > > +++ b/docs/specs/tpm.txt
> > > @@ -200,3 +200,109 @@ crw-------. 1 root root 10, 224 Jul 11 10:11 /dev/tpm0
> > >   PCR-00: 35 4E 3B CE 23 9F 38 59 ...
> > >   ...
> > >   PCR-23: 00 00 00 00 00 00 00 00 ...
> > > +
> > > +
> > > +=== Migration with the TPM emulator ===
> > > +
> > > +The TPM emulator supports the following types of virtual machine migration:
> > > +
> > > +- VM save / restore (migration into a file)
> > > +- Network migration
> > > +- Snapshotting (migration into storage like QoW2 or QED)
> > > +
> > > +The following command sequences can be used to test VM save / restore.
> > > +
> > > +
> > > +In a 1st terminal start an instance of a swtpm using the following command:
> > > +
> > > +mkdir /tmp/mytpm1
> > > +swtpm socket --tpmstate dir=/tmp/mytpm1 \
> > > +  --ctrl type=unixio,path=/tmp/mytpm1/swtpm-sock \
> > > +  --log level=20 --tpm2
> > > +
> > > +In a 2nd terminal start the VM:
> > > +
> > > +qemu-system-x86_64 -display sdl -enable-kvm \
> > > +  -m 1024 -boot d -bios bios-256k.bin -boot menu=on \
> > > +  -chardev socket,id=chrtpm,path=/tmp/mytpm1/swtpm-sock \
> > > +  -tpmdev emulator,id=tpm0,chardev=chrtpm \
> > > +  -device tpm-tis,tpmdev=tpm0 \
> > > +  -monitor stdio \
> > > +  test.img
> > > +
> > > +Verify that the attached TPM is working as expected using applications inside
> > > +the VM.
> > > +
> > > +To store the state of the VM use the following command in the QEMU monitor in
> > > +the 2nd terminal:
> > > +
> > > +(qemu) migrate "exec:cat > testvm.bin"
> > > +(qemu) quit
> > > +
> > > +At this point a file called 'testvm.bin' should exists and the swtpm and QEMU
> > > +processes should have ended.
> > When is swtpm ending, when qemu leaves? Hopefully you can do several
> > migrate commands.
> 
> Yes, QEMU does not send it the signal to shut down. We can fall back to the
> source if the destination fails.
> 
> > 
> > > +
> > > +To test 'VM restore' you have to start the swtpm with the same parameters
> > > +as before. If previously a TPM 2 [--tpm2] was saved, --tpm2 must now be
> > > +passed again on the command line.
> > > +
> > > +In the 1st terminal restart the swtpm with the same command line as before:
> > > +
> > > +swtpm socket --tpmstate dir=/tmp/mytpm1 \
> > > +  --ctrl type=unixio,path=/tmp/mytpm1/swtpm-sock \
> > > +  --log level=20 --tpm2
> > Does that mean the tpmstate directory content must be the same and
> > thus migrated as well? Can in be empty in the destination? If not,
> > what should be done to initialize it? Could it be empty instead?
> 
> QEMU migrates the state of the TPM with the CMD_GET_STATEBLOB to retrieve
> the state blobs and CMD_SET_STATEBLOB to set them on the destination. The
> destination only needs to have the TPM running but the directory must have
> been created (--tpmstate dir=...).
> 
> One can try this with localhost migration over the network as well, but I
> didn't want to show this more complicated scenario in the doc:
> 
> destination QEMU:
> sudo ./x86_64-softmmu/qemu-system-x86_64 -vnc :11 -enable-kvm -m 1024 -smp 8
> -boot d -L /usr/share/seabios -bios bios-256k.bin -boot menu=on -chardev
> socket,id=chrtpm,path=/tmp/mytpm2/ctrl.sock -tpmdev
> emulator,id=tpm0,chardev=chrtpm -device tpm-tis,tpmdev=tpm0 -monitor stdio
> -chardev file,id=pts2,path=/tmp/seabios.log -device isa-serial,chardev=pts2
> /var/lib/libvirt/images/FC27 -incoming "exec:nc -l 127.0.0.1 12345"
> 
> source QEMU:
> sudo ./x86_64-softmmu/qemu-system-x86_64 -vnc :10 -enable-kvm -m 1024 -smp 8
> -boot d -L /usr/share/seabios -bios bios-256k.bin -boot menu=on -chardev
> socket,id=chrtpm,path=/tmp/mytpm1/ctrl.sock -tpmdev
> emulator,id=tpm0,chardev=chrtpm -device tpm-tis,tpmdev=tpm0 -monitor stdio
> -chardev file,id=pts2,path=/tmp/seabios.log -device isa-serial,chardev=pts2
> /var/lib/libvirt/images/FC27
> 
> (qemu) migrate "exec:nc 127.0.0.1 12345"

Migration has a tcp form that's a bit easier;
-incoming tcp:127.0.0.1:12345
migrate tcp:127.0.0.1:12345

Dave

> Just tested again.
> 
>     Stefan
> 
> 
> > > +
> > > +In the 2nd terminal restore the state of the VM using the additonal
> > > +'-incoming' option.
> > > +
> > > +qemu-system-x86_64 -display sdl -enable-kvm \
> > > +  -m 1024 -boot d -bios bios-256k.bin -boot menu=on \
> > > +  -chardev socket,id=chrtpm,path=/tmp/mytpm1/swtpm-sock \
> > > +  -tpmdev emulator,id=tpm0,chardev=chrtpm \
> > > +  -device tpm-tis,tpmdev=tpm0 \
> > > +  -incoming "exec:cat < testvm.bin" \
> > > +  test.img
> > > +
> > > +
> > > +Troubleshooting migration:
> > > +
> > > +There are several reasons why migration may fail. In case of problems,
> > > +please ensure that the command lines adhere to the following rules and,
> > > +if possible, that identical versions of QEMU and swtpm are used at all
> > > +times.
> > > +
> > > +VM save and restore:
> > > + - QEMU command line parameters should be identical apart from the
> > > +   '-incoming' option on VM restore
> > > + - swtpm command line parameters should be identical
> > > +
> > > +VM migration to 'localhost':
> > > + - QEMU command line parameters should be identical apart from the
> > > +   '-incoming' option on the destination side
> > > + - swtpm command line parameters should point to two different
> > > +   directories on the source and destination swtpm (--tpmstate dir=...)
> > > +   (especially if different versions of libtpms were to be used on the
> > > +   same machine).
> > > +
> > > +VM migration across the network:
> > > + - QEMU command line parameters should be identical apart from the
> > > +   '-incoming' option on the destination side
> > > + - swtpm command line parameters should be identical
> > > +
> > > +VM Snapshotting:
> > > + - QEMU command line parameters should be identical
> > > + - swtpm command line parameters should be identical
> > > +
> > > +
> > > +Besides that, migration failure reasons on the swtpm level may include
> > > +the following:
> > > +
> > > + - the versions of the swtpm on the source and destination sides are
> > > +   incompatible
> > > +   - downgrading of TPM state may not be supported
> > > +   - the source and destination libtpms were compiled with different
> > > +     compile-time options and the destination side refuses to accept the
> > > +     state
> > > + - different migration keys are used on the source and destination side
> > > +   and the destination side cannot decrypt the migrated state
> > > +   (swtpm ... --migration-key ... )
> > > --
> > > 2.5.5
> > > 
> > 
> > 
> > 
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK