[Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format

[Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format

[qiaonuohan]

From: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>

Hi, all

The last version is here:
http://lists.gnu.org/archive/html/qemu-devel/2013-05/msg00803.html

Command 'dump-guest-memory' was introduced to dump guest's memory. But the
vmcore's format is only elf32 or elf64. The message is here:
http://lists.gnu.org/archive/html/qemu-devel/2012-04/msg03379.html

For migration, 'dump-guest-memory' is supposed to support compression feature.
Because of the regression, the missing of compression feature, we post these
patches to make 'dump-guest-memory' be able to dump guest's in kdump-compressed
format. Then vmcore can be much smaller, and easily be delivered.

The kdump-compressed format is *linux specific* *linux standard* crash dump
format used in kdump framework. The kdump-compressed format is readable only
with the crash utility, and it can be smaller than the ELF format because of
the compression support.

Note, similar to 'dump-guest-memory':
1. The guest should be x86 or x86_64. The other arch is not supported now.
2. If the OS is in the second kernel, gdb may not work well, and crash can
   work by specifying '--machdep phys_addr=xxx' in the command line. The
   reason is that the second kernel will update the page table, and we can
   not get the page table for the first kernel.
3. The cpu's state is stored in QEMU note.
4. The vmcore are able to be compressed with zlib, lzo or snappy. zlib is
   available by default, and option '--enable-lzo' or '--enable-snappy'
   should be used with configure to make lzo or snappy available.

Changelog:
Changes from v1 to v2:
1. Address Eric & Daniel's comment: fix manner of  string copy.
2. Address Eric's comment: replace reinventing new constants by using the
   ready-made ones accoring.
3. Address Andreas's comment: remove useless include.

Qiao Nuohan (9):
  Add API to manipulate dump_bitmap
  Add API to manipulate cache_data
  Move includes and struct definition to dump.h
  Add API to create header of vmcore
  Add API to create data of dump bitmap
  Add API to create page
  Add API to free buf used by creating header, bitmap and page
  Add API to write header, bitmap and page into vmcore
  Make monitor command 'dump-guest-memory' dump in kdump-compressed
    format

 Makefile.target       |    2 +-
 cache_data.c          |  121 +++++++
 configure             |   50 +++
 dump.c                |  877 ++++++++++++++++++++++++++++++++++++++++++++++---
 dump_bitmap.c         |  171 ++++++++++
 hmp-commands.hx       |   12 +-
 hmp.c                 |   23 ++-
 include/cache_data.h  |   56 ++++
 include/dump_bitmap.h |   60 ++++
 include/sysemu/dump.h |  176 ++++++++++
 qapi-schema.json      |   23 ++-
 qmp-commands.hx       |    6 +-
 12 files changed, 1527 insertions(+), 50 deletions(-)
 create mode 100644 cache_data.c
 create mode 100644 dump_bitmap.c
 create mode 100644 include/cache_data.h
 create mode 100644 include/dump_bitmap.h

Re: [Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format

[qiaonuohan]

Sorry, the cc list is wrong, I will resend the mails.
> -----Original Message-----
> From: qiaonuohan@cn.fujitsu.com [mailto:qiaonuohan@cn.fujitsu.com]
> Sent: Wednesday, May 08, 2013 9:54 AM
> To: qemu-devel@nongnu.org
> Cc:
> eblake@redhat.com.berrange@redhat.com.afaerber@suse.de.d.hatayama@jp.fujitsu.c
> om.zhangxh@cn.fujitsu.com; Qiao Nuohan
> Subject: [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in
> kdump-compressed format
> 
> From: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
> 
> Hi, all
> 
> The last version is here:
> http://lists.gnu.org/archive/html/qemu-devel/2013-05/msg00803.html
> 
> Command 'dump-guest-memory' was introduced to dump guest's memory. But the
> vmcore's format is only elf32 or elf64. The message is here:
> http://lists.gnu.org/archive/html/qemu-devel/2012-04/msg03379.html
> 
> For migration, 'dump-guest-memory' is supposed to support compression feature.
> Because of the regression, the missing of compression feature, we post these
> patches to make 'dump-guest-memory' be able to dump guest's in kdump-
> compressed
> format. Then vmcore can be much smaller, and easily be delivered.
> 
> The kdump-compressed format is *linux specific* *linux standard* crash dump
> format used in kdump framework. The kdump-compressed format is readable only
> with the crash utility, and it can be smaller than the ELF format because of
> the compression support.
> 
> Note, similar to 'dump-guest-memory':
> 1. The guest should be x86 or x86_64. The other arch is not supported now.
> 2. If the OS is in the second kernel, gdb may not work well, and crash can
>    work by specifying '--machdep phys_addr=xxx' in the command line. The
>    reason is that the second kernel will update the page table, and we can
>    not get the page table for the first kernel.
> 3. The cpu's state is stored in QEMU note.
> 4. The vmcore are able to be compressed with zlib, lzo or snappy. zlib is
>    available by default, and option '--enable-lzo' or '--enable-snappy'
>    should be used with configure to make lzo or snappy available.
> 
> Changelog:
> Changes from v1 to v2:
> 1. Address Eric & Daniel's comment: fix manner of  string copy.
> 2. Address Eric's comment: replace reinventing new constants by using the
>    ready-made ones accoring.
> 3. Address Andreas's comment: remove useless include.
> 
> Qiao Nuohan (9):
>   Add API to manipulate dump_bitmap
>   Add API to manipulate cache_data
>   Move includes and struct definition to dump.h
>   Add API to create header of vmcore
>   Add API to create data of dump bitmap
>   Add API to create page
>   Add API to free buf used by creating header, bitmap and page
>   Add API to write header, bitmap and page into vmcore
>   Make monitor command 'dump-guest-memory' dump in kdump-compressed
>     format
> 
>  Makefile.target       |    2 +-
>  cache_data.c          |  121 +++++++
>  configure             |   50 +++
>  dump.c                |  877 ++++++++++++++++++++++++++++++++++++++++++++++--
> -
>  dump_bitmap.c         |  171 ++++++++++
>  hmp-commands.hx       |   12 +-
>  hmp.c                 |   23 ++-
>  include/cache_data.h  |   56 ++++
>  include/dump_bitmap.h |   60 ++++
>  include/sysemu/dump.h |  176 ++++++++++
>  qapi-schema.json      |   23 ++-
>  qmp-commands.hx       |    6 +-
>  12 files changed, 1527 insertions(+), 50 deletions(-)
>  create mode 100644 cache_data.c
>  create mode 100644 dump_bitmap.c
>  create mode 100644 include/cache_data.h
>  create mode 100644 include/dump_bitmap.h
> 

[Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format

[Qiao Nuohan]

Hi, all

The last version is here:
http://lists.gnu.org/archive/html/qemu-devel/2013-05/msg00803.html

Command 'dump-guest-memory' was introduced to dump guest's memory. But the
vmcore's format is only elf32 or elf64. The message is here:
http://lists.gnu.org/archive/html/qemu-devel/2012-04/msg03379.html

For migration, 'dump-guest-memory' is supposed to support compression feature.
Because of the regression, the missing of compression feature, we post these
patches to make 'dump-guest-memory' be able to dump guest's in kdump-compressed
format. Then vmcore can be much smaller, and easily be delivered.

The kdump-compressed format is *linux specific* *linux standard* crash dump
format used in kdump framework. The kdump-compressed format is readable only
with the crash utility, and it can be smaller than the ELF format because of
the compression support.

Note, similar to 'dump-guest-memory':
1. The guest should be x86 or x86_64. The other arch is not supported now.
2. If the OS is in the second kernel, gdb may not work well, and crash can
   work by specifying '--machdep phys_addr=xxx' in the command line. The
   reason is that the second kernel will update the page table, and we can
   not get the page table for the first kernel.
3. The cpu's state is stored in QEMU note.
4. The vmcore are able to be compressed with zlib, lzo or snappy. zlib is
   available by default, and option '--enable-lzo' or '--enable-snappy'
   should be used with configure to make lzo or snappy available.

Changelog:
Changes from v1 to v2:
1. Address Eric & Daniel's comment: fix manner of  string copy.
2. Address Eric's comment: replace reinventing new constants by using the
   ready-made ones accoring.
3. Address Andreas's comment: remove useless include.

Qiao Nuohan (9):
  Add API to manipulate dump_bitmap
  Add API to manipulate cache_data
  Move includes and struct definition to dump.h
  Add API to create header of vmcore
  Add API to create data of dump bitmap
  Add API to create page
  Add API to free buf used by creating header, bitmap and page
  Add API to write header, bitmap and page into vmcore
  Make monitor command 'dump-guest-memory' dump in kdump-compressed
    format

 Makefile.target       |    2 +-
 cache_data.c          |  121 +++++++
 configure             |   50 +++
 dump.c                |  877 ++++++++++++++++++++++++++++++++++++++++++++++---
 dump_bitmap.c         |  171 ++++++++++
 hmp-commands.hx       |   12 +-
 hmp.c                 |   23 ++-
 include/cache_data.h  |   56 ++++
 include/dump_bitmap.h |   60 ++++
 include/sysemu/dump.h |  176 ++++++++++
 qapi-schema.json      |   23 ++-
 qmp-commands.hx       |    6 +-
 12 files changed, 1527 insertions(+), 50 deletions(-)
 create mode 100644 cache_data.c
 create mode 100644 dump_bitmap.c
 create mode 100644 include/cache_data.h
 create mode 100644 include/dump_bitmap.h