[Qemu-devel] [PATCH 00/11 v10] introducing a new, dedicated guest memory dump mechanism

[Qemu-devel] [PATCH 00/11 v10] introducing a new, dedicated guest memory dump mechanism

[Wen Congyang]

Hi, all

'virsh dump' can not work when host pci device is used by guest. We have
discussed this issue here:
http://lists.nongnu.org/archive/html/qemu-devel/2011-10/msg00736.html

The last version is here:
http://lists.nongnu.org/archive/html/qemu-devel/2012-03/msg02536.html

We have determined to introduce a new command dump-guest-memory to dump
guest's memory. The core file's format is elf32 or elf64.

Note:
1. The guest should be x86 or x86_64. The other arch is not supported now.
2. If you use old gdb, gdb may crash. I use gdb-7.3.1, and it does not crash.
3. If the OS is in the second kernel, gdb may not work well, and crash can
   work by specifying '--machdep phys_addr=xxx' in the command line. The
   reason is that the second kernel will update the page table, and we can
   not get the page table for the first kernel.
4. The cpu's state is stored in QEMU note. You neet to modify crash to use
   it to calculate phys_base.
5. If the guest OS is 32 bit and the memory size is larger than 4G, the vmcore
   is elf64 format. You should use the gdb which is built with --enable-64-bit-bfd.
6. This patchset is based on the upstream tree, and apply one patch that is still
   in Luiz Capitulino's tree, because I use the API qemu_get_fd() in this patchset.

Changes from v9 to v10:
1. fix some bug
2. addressed Luiz's and Hatayam's comment
3. remove cancel and query command

Changes from v8 to v9:
1. remove async support(it will be reimplemented after QAPI async commands support
   is finished)
2. fix some typo error

Changes from v7 to v8:
1. addressed Hatayama's comments

Changes from v6 to v7:
1. addressed Jan's comments
2. fix some bugs
3. store cpu's state into the vmcore

Changes from v5 to v6:
1. allow user to dump a fraction of the memory
2. fix some bugs

Changes from v4 to v5:
1. convert the new command dump to QAPI 

Changes from v3 to v4:
1. support it to run asynchronously
2. add API to cancel dumping and query dumping progress
3. add API to control dumping speed
4. auto cancel dumping when the user resumes vm, and the status is failed.

Changes from v2 to v3:
1. address Jan Kiszka's comment

Changes from v1 to v2:
1. fix virt addr in the vmcore.

Wen Congyang (11):
  Add API to create memory mapping list
  Add API to check whether a physical address is I/O address
  implement cpu_get_memory_mapping()
  Add API to check whether paging mode is enabled
  Add API to get memory mapping
  Add API to get memory mapping without do paging
  target-i386: Add API to write elf notes to core file
  target-i386: Add API to write cpu status to core file
  target-i386: add API to get dump info
  make gdb_id() generally avialable
  introduce a new monitor command 'dump-guest-memory' to dump guest's
    memory

 Makefile.target                   |    3 +
 configure                         |    8 +
 cpu-all.h                         |   67 +++
 cpu-common.h                      |    2 +
 dump.c                            |  841 +++++++++++++++++++++++++++++++++++++
 dump.h                            |   23 +
 elf.h                             |    5 +
 exec.c                            |    9 +
 gdbstub.c                         |    9 -
 gdbstub.h                         |    9 +
 hmp-commands.hx                   |   28 ++
 hmp.c                             |   22 +
 hmp.h                             |    1 +
 memory_mapping.c                  |  236 +++++++++++
 memory_mapping.h                  |   68 +++
 qapi-schema.json                  |   18 +
 qmp-commands.hx                   |   38 ++
 target-i386/arch_dump.c           |  425 +++++++++++++++++++
 target-i386/arch_memory_mapping.c |  271 ++++++++++++
 19 files changed, 2074 insertions(+), 9 deletions(-)
 create mode 100644 dump.c
 create mode 100644 dump.h
 create mode 100644 memory_mapping.c
 create mode 100644 memory_mapping.h
 create mode 100644 target-i386/arch_dump.c
 create mode 100644 target-i386/arch_memory_mapping.c

Re: [Qemu-devel] [PATCH 00/11 v10] introducing a new, dedicated guest memory dump mechanism

[Wen Congyang]

Does anyone have time to review this patchset?

Thanks
Wen Congyang

At 03/20/2012 11:28 AM, Wen Congyang Wrote:
> Hi, all
> 
> 'virsh dump' can not work when host pci device is used by guest. We have
> discussed this issue here:
> http://lists.nongnu.org/archive/html/qemu-devel/2011-10/msg00736.html
> 
> The last version is here:
> http://lists.nongnu.org/archive/html/qemu-devel/2012-03/msg02536.html
> 
> We have determined to introduce a new command dump-guest-memory to dump
> guest's memory. The core file's format is elf32 or elf64.
> 
> Note:
> 1. The guest should be x86 or x86_64. The other arch is not supported now.
> 2. If you use old gdb, gdb may crash. I use gdb-7.3.1, and it does not crash.
> 3. If the OS is in the second kernel, gdb may not work well, and crash can
>    work by specifying '--machdep phys_addr=xxx' in the command line. The
>    reason is that the second kernel will update the page table, and we can
>    not get the page table for the first kernel.
> 4. The cpu's state is stored in QEMU note. You neet to modify crash to use
>    it to calculate phys_base.
> 5. If the guest OS is 32 bit and the memory size is larger than 4G, the vmcore
>    is elf64 format. You should use the gdb which is built with --enable-64-bit-bfd.
> 6. This patchset is based on the upstream tree, and apply one patch that is still
>    in Luiz Capitulino's tree, because I use the API qemu_get_fd() in this patchset.
> 
> Changes from v9 to v10:
> 1. fix some bug
> 2. addressed Luiz's and Hatayam's comment
> 3. remove cancel and query command
> 
> Changes from v8 to v9:
> 1. remove async support(it will be reimplemented after QAPI async commands support
>    is finished)
> 2. fix some typo error
> 
> Changes from v7 to v8:
> 1. addressed Hatayama's comments
> 
> Changes from v6 to v7:
> 1. addressed Jan's comments
> 2. fix some bugs
> 3. store cpu's state into the vmcore
> 
> Changes from v5 to v6:
> 1. allow user to dump a fraction of the memory
> 2. fix some bugs
> 
> Changes from v4 to v5:
> 1. convert the new command dump to QAPI 
> 
> Changes from v3 to v4:
> 1. support it to run asynchronously
> 2. add API to cancel dumping and query dumping progress
> 3. add API to control dumping speed
> 4. auto cancel dumping when the user resumes vm, and the status is failed.
> 
> Changes from v2 to v3:
> 1. address Jan Kiszka's comment
> 
> Changes from v1 to v2:
> 1. fix virt addr in the vmcore.
> 
> Wen Congyang (11):
>   Add API to create memory mapping list
>   Add API to check whether a physical address is I/O address
>   implement cpu_get_memory_mapping()
>   Add API to check whether paging mode is enabled
>   Add API to get memory mapping
>   Add API to get memory mapping without do paging
>   target-i386: Add API to write elf notes to core file
>   target-i386: Add API to write cpu status to core file
>   target-i386: add API to get dump info
>   make gdb_id() generally avialable
>   introduce a new monitor command 'dump-guest-memory' to dump guest's
>     memory
> 
>  Makefile.target                   |    3 +
>  configure                         |    8 +
>  cpu-all.h                         |   67 +++
>  cpu-common.h                      |    2 +
>  dump.c                            |  841 +++++++++++++++++++++++++++++++++++++
>  dump.h                            |   23 +
>  elf.h                             |    5 +
>  exec.c                            |    9 +
>  gdbstub.c                         |    9 -
>  gdbstub.h                         |    9 +
>  hmp-commands.hx                   |   28 ++
>  hmp.c                             |   22 +
>  hmp.h                             |    1 +
>  memory_mapping.c                  |  236 +++++++++++
>  memory_mapping.h                  |   68 +++
>  qapi-schema.json                  |   18 +
>  qmp-commands.hx                   |   38 ++
>  target-i386/arch_dump.c           |  425 +++++++++++++++++++
>  target-i386/arch_memory_mapping.c |  271 ++++++++++++
>  19 files changed, 2074 insertions(+), 9 deletions(-)
>  create mode 100644 dump.c
>  create mode 100644 dump.h
>  create mode 100644 memory_mapping.c
>  create mode 100644 memory_mapping.h
>  create mode 100644 target-i386/arch_dump.c
>  create mode 100644 target-i386/arch_memory_mapping.c
> 
> 
> 

Re: [Qemu-devel] [PATCH 00/11 v10] introducing a new, dedicated guest memory dump mechanism

[Luiz Capitulino]

On Tue, 20 Mar 2012 11:28:17 +0800
Wen Congyang <wency@cn.fujitsu.com> wrote:

> Hi, all
> 
> 'virsh dump' can not work when host pci device is used by guest. We have
> discussed this issue here:
> http://lists.nongnu.org/archive/html/qemu-devel/2011-10/msg00736.html

I've reviewed the QMP part of this series and it looks good to me (only
spot small things).

Now, I'd like to get an ACK from Jan and/or Anthony before applying,
specially wrt the fact this is a long running synchronous command (btw,
I think this is ok, as this command is more likely to run when the VM
guest crashes...).

Re: [Qemu-devel] [PATCH 00/11 v10] introducing a new, dedicated guest memory dump mechanism

[Anthony Liguori]

On 03/19/2012 10:28 PM, Wen Congyang wrote:
> Hi, all
>
> 'virsh dump' can not work when host pci device is used by guest. We have
> discussed this issue here:
> http://lists.nongnu.org/archive/html/qemu-devel/2011-10/msg00736.html
>
> The last version is here:
> http://lists.nongnu.org/archive/html/qemu-devel/2012-03/msg02536.html
>
> We have determined to introduce a new command dump-guest-memory to dump
> guest's memory. The core file's format is elf32 or elf64.

These patches aren't properly threaded and I don't see patch 5/11.  Can you 
resubmit with git-send-email and follow the info on the wiki 
http://wiki.qemu.org/Contribute/SubmitAPatch

Regards,

Anthony Liguori