[Qemu-devel] [PULL v3 00/60] Misc patches for soft freeze

[Qemu-devel] [PULL v3 00/60] Misc patches for soft freeze

[Paolo Bonzini]

The following changes since commit 4a83bf2f339d4b63ecd5ef48b9816c3b7ee24553:

  Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20180627' into staging (2018-06-28 15:31:42 +0100)

are available in the git repository at:


  git://github.com/bonzini/qemu.git tags/for-upstream

for you to fetch changes up to 28a3cfc10b2e1a34985797357b4aa7558a63d08f:

  tests/boot-serial: Do not delete the output file in case of errors (2018-06-29 13:02:50 +0200)

----------------------------------------------------------------
* "info mtree" improvements (Alexey)
* fake VPD block limits for SCSI passthrough (Daniel Barboza)
* chardev and main loop fixes (Daniel Berrangé, Sergio, Stefan)
* help fixes (Eduardo)
* pc-dimm refactoring (David)
* tests improvements and fixes (Emilio, Thomas)
* SVM emulation fixes (Jan)
* MemoryRegionCache fix (Eric)
* WHPX improvements (Justin)
* ESP cleanup (Mark)
* -overcommit option (Michael)
* qemu-pr-helper fixes (me)
* "info pic" improvements for x86 (Peter)
* x86 TCG emulation fixes (Richard)
* KVM slot handling fix (Shannon)
* Next round of deprecation (Thomas)
* Windows dump format support (Viktor)

----------------------------------------------------------------
Alexey Kardashevskiy (1):
      memory/hmp: Print owners/parents in "info mtree"

Daniel Henrique Barboza (3):
      hw/scsi: cleanups before VPD BL emulation
      hw/scsi: centralize SG_IO calls into single function
      hw/scsi: add VPD Block Limits emulation

Daniel P. Berrangé (1):
      chardev: don't splatter terminal settings on exit if not previously set

David Hildenbrand (15):
      memory-device: turn alignment assert into check
      exec: check that alignment is a power of two
      pc-dimm: remove leftover "struct pc_dimms_capacity"
      pc: rename pc_dimm_(plug|unplug|...)* into pc_memory_(plug|unplug|...)*
      pc-dimm: rename pc_dimm_memory_* to pc_dimm_*
      pc-dimm: remove pc_dimm_get_free_slot() from header
      pc: factor out pc specific dimm checks into pc_memory_pre_plug()
      nvdimm: no need to overwrite get_vmstate_memory_region()
      hostmem: drop error variable from host_memory_backend_get_memory()
      pc-dimm: merge get_(vmstate_)memory_region()
      nvdimm: convert "unarmed" into a static property
      nvdimm: convert nvdimm_mr into a pointer
      nvdimm: make get_memory_region() perform checks and initialization
      pc-dimm: get_memory_region() will not fail after realize
      numa: report all DIMM/NVDIMMs as plugged memory

Eduardo Habkost (1):
      qemu-options: Add missing newline to -accel help text

Emilio G. Cota (1):
      tests/atomic_add-bench: add -m option to use mutexes

Eric Auger (1):
      exec: Fix MAP_RAM for cached access

Jan Kiszka (3):
      target-i386: Add NMI interception to SVM
      target-i386: Allow interrupt injection after STGI
      target-i386: Mark cpu_vmexit noreturn

Justin Terry (VM) (2):
      WHPX workaround bug in OSVW handling
      WHPX: register for unrecognized MSR exits

Mark Cave-Ayland (2):
      hw/mips/jazz: create ESP device directly via qdev
      esp: remove legacy esp_init() function

Michael S. Tsirkin (2):
      kvm: support -overcommit cpu-pm=on|off
      i386/cpu: make -cpu host support monitor/mwait

Paolo Bonzini (10):
      move public invalidate APIs out of translate-all.{c,h}, clean up
      whpx: commit missing file
      QemuMutex: support --enable-debug-mutex
      osdep: work around Coverity parsing errors
      pr-helper: fix --socket-path default in help
      pr-helper: fix assertion failure on failed multipath PERSISTENT RESERVE IN
      pr-manager-helper: avoid SIGSEGV when writing to the socket fail
      pr-manager: put stubs in .c file
      pr-manager: add query-pr-managers QMP command
      pr-manager-helper: report event on connection/disconnection

Peter Xu (7):
      qemu-thread: introduce qemu-thread-common.h
      configure: enable debug-mutex if debug enabled
      doc: another fix to "info pic"
      ioapic: support "info pic"
      ioapic: some proper indents when dump info
      ioapic: support "info irq"
      hmp: obsolete "info ioapic"

Richard Henderson (1):
      target/i386: Fix BLSR and BLSI

Sergio Lopez (1):
      hw/char/serial: Only retry if qemu_chr_fe_write returns 0

Shannon Zhao (1):
      kvm: Delete the slot if and only if the KVM_MEM_READONLY flag is changed

Stefan Hajnoczi (1):
      main-loop: document IOCanReadHandler

Thomas Huth (3):
      Replace '-enable-kvm' with '-accel kvm' in docs and help texts
      Deprecate the -enable-hax option
      tests/boot-serial: Do not delete the output file in case of errors

Viktor Prutyanov (4):
      dump: add Windows dump format to dump-guest-memory
      dump: use system context in Windows dump
      dump: add fallback KDBG using in Windows dump
      dump: add Windows live system dump

 Makefile.target                   |   1 +
 accel/kvm/kvm-all.c               |  17 +-
 accel/tcg/translate-all.c         |  28 +--
 accel/tcg/translate-all.h         |   1 -
 accel/tcg/user-exec-stub.c        |   3 +
 backends/hostmem.c                |   3 +-
 chardev/char-stdio.c              |   6 +-
 configure                         |  11 +
 docs/COLO-FT.txt                  |   8 +-
 docs/can.txt                      |   4 +-
 docs/multi-thread-compression.txt |   2 +-
 docs/multiseat.txt                |   2 +-
 docs/specs/tpm.txt                |   8 +-
 dump.c                            |  24 ++-
 exec.c                            |  39 +++-
 hmp-commands-info.hx              |   9 +-
 hmp-commands.hx                   |  13 +-
 hmp.c                             |   9 +-
 hw/block/dataplane/virtio-blk.c   |   4 +-
 hw/char/serial.c                  |   2 +-
 hw/i386/kvm/ioapic.c              |  11 +-
 hw/i386/pc.c                      |  73 +++----
 hw/intc/ioapic.c                  |  12 +-
 hw/intc/ioapic_common.c           |  47 ++++-
 hw/mem/memory-device.c            |   8 +-
 hw/mem/nvdimm.c                   |  93 +++++----
 hw/mem/pc-dimm.c                  |  35 ++--
 hw/mips/mips_jazz.c               |  19 +-
 hw/misc/ivshmem.c                 |   3 +-
 hw/ppc/spapr.c                    |  18 +-
 hw/scsi/esp.c                     |  30 ---
 hw/scsi/scsi-disk.c               | 427 +++++++++++++++++++-------------------
 hw/scsi/scsi-generic.c            | 246 ++++++++++++++++------
 hw/scsi/virtio-scsi-dataplane.c   |   4 +-
 include/exec/exec-all.h           |   8 +-
 include/exec/memory.h             |   2 +-
 include/exec/ram_addr.h           |   2 +
 include/hw/i386/ioapic_internal.h |   3 +
 include/hw/i386/pc.h              |   3 -
 include/hw/mem/nvdimm.h           |   2 +-
 include/hw/mem/pc-dimm.h          |  17 +-
 include/hw/scsi/esp.h             |   5 -
 include/hw/scsi/scsi.h            |   6 +-
 include/qemu/main-loop.h          |  14 ++
 include/qemu/osdep.h              |  15 ++
 include/qemu/thread-posix.h       |   4 +
 include/qemu/thread-win32.h       |   4 +
 include/scsi/pr-manager.h         |  11 +-
 include/sysemu/hostmem.h          |   3 +-
 include/sysemu/kvm_int.h          |   1 +
 include/sysemu/sysemu.h           |   1 +
 linux-user/mmap.c                 |   1 -
 memory.c                          |  72 ++++++-
 memory_ldst.inc.c                 |  47 ++---
 monitor.c                         |   4 +-
 numa.c                            |   9 +-
 qapi/block.json                   |  52 +++++
 qapi/misc.json                    |   5 +-
 qemu-doc.texi                     |   5 +
 qemu-options.hx                   |  28 ++-
 scsi/Makefile.objs                |   1 +
 scsi/pr-manager-helper.c          |  28 +++
 scsi/pr-manager-stub.c            |  30 +++
 scsi/pr-manager.c                 |  45 ++++
 scsi/qemu-pr-helper.c             |  21 +-
 target/i386/cpu.c                 |  32 ++-
 target/i386/cpu.h                 |  13 +-
 target/i386/kvm.c                 |  32 +++
 target/i386/monitor.c             |   8 +-
 target/i386/seg_helper.c          |   1 +
 target/i386/svm_helper.c          |   1 +
 target/i386/translate.c           |  29 +--
 target/i386/whp-dispatch.h        |  56 +++++
 target/i386/whpx-all.c            |  54 ++++-
 target/xtensa/op_helper.c         |   9 +-
 tests/atomic_add-bench.c          |  19 +-
 tests/boot-serial-test.c          |  17 +-
 trace/control-target.c            |   1 -
 util/qemu-thread-common.h         |  55 +++++
 util/qemu-thread-posix.c          |  19 +-
 util/qemu-thread-win32.c          |  17 +-
 vl.c                              |  33 ++-
 win_dump.c                        | 385 ++++++++++++++++++++++++++++++++++
 win_dump.h                        | 176 ++++++++++++++++
 84 files changed, 1934 insertions(+), 692 deletions(-)
 create mode 100644 scsi/pr-manager-stub.c
 create mode 100644 target/i386/whp-dispatch.h
 create mode 100644 util/qemu-thread-common.h
 create mode 100644 win_dump.c
 create mode 100644 win_dump.h
-- 
1.8.3.1

[Qemu-devel] [PULL 52/60] i386/cpu: make -cpu host support monitor/mwait

[Paolo Bonzini]

From: "Michael S. Tsirkin" <mst@redhat.com>

When guest CPU PM is enabled, and with -cpu host, expose the host CPU
MWAIT leaf in the CPUID so guest can make good PM decisions.

Note: the result is 100% CPU utilization reported by host as host
no longer knows that the CPU is halted.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180622192148.178309-3-mst@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 accel/tcg/user-exec-stub.c |  3 +++
 target/i386/cpu.c          | 32 ++++++++++++++++++++++----------
 target/i386/cpu.h          |  9 +++++++++
 target/i386/kvm.c          |  9 +++++++++
 4 files changed, 43 insertions(+), 10 deletions(-)

diff --git a/accel/tcg/user-exec-stub.c b/accel/tcg/user-exec-stub.c
index dbcf1ad..a32b449 100644
--- a/accel/tcg/user-exec-stub.c
+++ b/accel/tcg/user-exec-stub.c
@@ -2,6 +2,9 @@
 #include "qemu-common.h"
 #include "qom/cpu.h"
 #include "sysemu/replay.h"
+#include "sysemu/sysemu.h"
+
+bool enable_cpu_pm = false;
 
 void cpu_resume(CPUState *cpu)
 {
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index e6c2f8a..1e6a7d0 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -3959,11 +3959,11 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         }
         break;
     case 5:
-        /* mwait info: needed for Core compatibility */
-        *eax = 0; /* Smallest monitor-line size in bytes */
-        *ebx = 0; /* Largest monitor-line size in bytes */
-        *ecx = CPUID_MWAIT_EMX | CPUID_MWAIT_IBE;
-        *edx = 0;
+        /* MONITOR/MWAIT Leaf */
+        *eax = cpu->mwait.eax; /* Smallest monitor-line size in bytes */
+        *ebx = cpu->mwait.ebx; /* Largest monitor-line size in bytes */
+        *ecx = cpu->mwait.ecx; /* flags */
+        *edx = cpu->mwait.edx; /* mwait substates */
         break;
     case 6:
         /* Thermal and Power Leaf */
@@ -4804,13 +4804,25 @@ static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
     Error *local_err = NULL;
     static bool ht_warned;
 
-    if (xcc->host_cpuid_required && !accel_uses_host_cpuid()) {
-        char *name = x86_cpu_class_get_model_name(xcc);
-        error_setg(&local_err, "CPU model '%s' requires KVM", name);
-        g_free(name);
-        goto out;
+    if (xcc->host_cpuid_required) {
+        if (!accel_uses_host_cpuid()) {
+            char *name = x86_cpu_class_get_model_name(xcc);
+            error_setg(&local_err, "CPU model '%s' requires KVM", name);
+            g_free(name);
+            goto out;
+        }
+
+        if (enable_cpu_pm) {
+            host_cpuid(5, 0, &cpu->mwait.eax, &cpu->mwait.ebx,
+                       &cpu->mwait.ecx, &cpu->mwait.edx);
+            env->features[FEAT_1_ECX] |= CPUID_EXT_MONITOR;
+        }
     }
 
+    /* mwait extended info: needed for Core compatibility */
+    /* We always wake on interrupt even if host does not have the capability */
+    cpu->mwait.ecx |= CPUID_MWAIT_EMX | CPUID_MWAIT_IBE;
+
     if (cpu->apic_id == UNASSIGNED_APIC_ID) {
         error_setg(errp, "apic-id property was not initialized properly");
         return;
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 16c59b7..8eaefee 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1382,6 +1382,15 @@ struct X86CPU {
     /* if true the CPUID code directly forward host cache leaves to the guest */
     bool cache_info_passthrough;
 
+    /* if true the CPUID code directly forwards
+     * host monitor/mwait leaves to the guest */
+    struct {
+        uint32_t eax;
+        uint32_t ebx;
+        uint32_t ecx;
+        uint32_t edx;
+    } mwait;
+
     /* Features that were filtered out because of missing host capabilities */
     uint32_t filtered_features[FEATURE_WORDS];
 
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index dc991f6..c5f72d6 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -366,6 +366,15 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function,
         if (!kvm_irqchip_in_kernel()) {
             ret &= ~CPUID_EXT_X2APIC;
         }
+
+        if (enable_cpu_pm) {
+            int disable_exits = kvm_check_extension(s,
+                                                    KVM_CAP_X86_DISABLE_EXITS);
+
+            if (disable_exits & KVM_X86_DISABLE_EXITS_MWAIT) {
+                ret |= CPUID_EXT_MONITOR;
+            }
+        }
     } else if (function == 6 && reg == R_EAX) {
         ret |= CPUID_6_EAX_ARAT; /* safe to allow because of emulated APIC */
     } else if (function == 7 && index == 0 && reg == R_EBX) {
-- 
1.8.3.1

Re: [Qemu-devel] [PULL v3 00/60] Misc patches for soft freeze

[Peter Maydell]

On 29 June 2018 at 12:05, Paolo Bonzini <pbonzini@redhat.com> wrote:
> The following changes since commit 4a83bf2f339d4b63ecd5ef48b9816c3b7ee24553:
>
>   Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20180627' into staging (2018-06-28 15:31:42 +0100)
>
> are available in the git repository at:
>
>
>   git://github.com/bonzini/qemu.git tags/for-upstream
>
> for you to fetch changes up to 28a3cfc10b2e1a34985797357b4aa7558a63d08f:
>
>   tests/boot-serial: Do not delete the output file in case of errors (2018-06-29 13:02:50 +0200)
>
> ----------------------------------------------------------------
> * "info mtree" improvements (Alexey)
> * fake VPD block limits for SCSI passthrough (Daniel Barboza)
> * chardev and main loop fixes (Daniel Berrangé, Sergio, Stefan)
> * help fixes (Eduardo)
> * pc-dimm refactoring (David)
> * tests improvements and fixes (Emilio, Thomas)
> * SVM emulation fixes (Jan)
> * MemoryRegionCache fix (Eric)
> * WHPX improvements (Justin)
> * ESP cleanup (Mark)
> * -overcommit option (Michael)
> * qemu-pr-helper fixes (me)
> * "info pic" improvements for x86 (Peter)
> * x86 TCG emulation fixes (Richard)
> * KVM slot handling fix (Shannon)
> * Next round of deprecation (Thomas)
> * Windows dump format support (Viktor)
>

Applied, thanks.

-- PMM

Re: [Qemu-devel] [PULL v3 00/60] Misc patches for soft freeze

[Peter Maydell]

On 29 June 2018 at 12:05, Paolo Bonzini <pbonzini@redhat.com> wrote:
> The following changes since commit 4a83bf2f339d4b63ecd5ef48b9816c3b7ee24553:
>
>   Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20180627' into staging (2018-06-28 15:31:42 +0100)
>
> are available in the git repository at:
>
>
>   git://github.com/bonzini/qemu.git tags/for-upstream
>
> for you to fetch changes up to 28a3cfc10b2e1a34985797357b4aa7558a63d08f:
>
>   tests/boot-serial: Do not delete the output file in case of errors (2018-06-29 13:02:50 +0200)
>
> ----------------------------------------------------------------
> * "info mtree" improvements (Alexey)
> * fake VPD block limits for SCSI passthrough (Daniel Barboza)
> * chardev and main loop fixes (Daniel Berrangé, Sergio, Stefan)
> * help fixes (Eduardo)
> * pc-dimm refactoring (David)
> * tests improvements and fixes (Emilio, Thomas)
> * SVM emulation fixes (Jan)
> * MemoryRegionCache fix (Eric)
> * WHPX improvements (Justin)
> * ESP cleanup (Mark)
> * -overcommit option (Michael)
> * qemu-pr-helper fixes (me)
> * "info pic" improvements for x86 (Peter)
> * x86 TCG emulation fixes (Richard)
> * KVM slot handling fix (Shannon)
> * Next round of deprecation (Thomas)
> * Windows dump format support (Viktor)
>

Hi; looks like this broke the Travis --disable-tcg build:
https://travis-ci.org/qemu/qemu/jobs/398262491

In file included from /home/travis/build/qemu/qemu/exec.c:62:0:
/home/travis/build/qemu/qemu/include/exec/ram_addr.h:96:6: error:
conflicting types for ‘tb_invalidate_phys_range’
 void tb_invalidate_phys_range(ram_addr_t start, ram_addr_t end);
      ^
In file included from /home/travis/build/qemu/qemu/exec.c:24:0:
/home/travis/build/qemu/qemu/include/exec/exec-all.h:309:6: note:
previous declaration of ‘tb_invalidate_phys_range’ was here
 void tb_invalidate_phys_range(target_ulong start, target_ulong end);
      ^
/home/travis/build/qemu/qemu/exec.c:1043:6: error: conflicting types
for ‘tb_invalidate_phys_addr’
 void tb_invalidate_phys_addr(AddressSpace *as, hwaddr addr, MemTxAttrs attrs)
      ^
In file included from /home/travis/build/qemu/qemu/exec.c:24:0:
/home/travis/build/qemu/qemu/include/exec/exec-all.h:308:6: note:
previous declaration of ‘tb_invalidate_phys_addr’ was here
 void tb_invalidate_phys_addr(target_ulong addr);
      ^

(with apologies for the smartquote damage).

Could you have a look, please?

thanks
-- PMM

Re: [Qemu-devel] [PULL v3 00/60] Misc patches for soft freeze

[Philippe Mathieu-Daudé]

On 06/29/2018 03:03 PM, Peter Maydell wrote:
> On 29 June 2018 at 12:05, Paolo Bonzini <pbonzini@redhat.com> wrote:
>> The following changes since commit 4a83bf2f339d4b63ecd5ef48b9816c3b7ee24553:
>>
>>   Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20180627' into staging (2018-06-28 15:31:42 +0100)
>>
>> are available in the git repository at:
>>
>>
>>   git://github.com/bonzini/qemu.git tags/for-upstream
>>
>> for you to fetch changes up to 28a3cfc10b2e1a34985797357b4aa7558a63d08f:
>>
>>   tests/boot-serial: Do not delete the output file in case of errors (2018-06-29 13:02:50 +0200)
>>
>> ----------------------------------------------------------------
>> * "info mtree" improvements (Alexey)
>> * fake VPD block limits for SCSI passthrough (Daniel Barboza)
>> * chardev and main loop fixes (Daniel Berrangé, Sergio, Stefan)
>> * help fixes (Eduardo)
>> * pc-dimm refactoring (David)
>> * tests improvements and fixes (Emilio, Thomas)
>> * SVM emulation fixes (Jan)
>> * MemoryRegionCache fix (Eric)
>> * WHPX improvements (Justin)
>> * ESP cleanup (Mark)
>> * -overcommit option (Michael)
>> * qemu-pr-helper fixes (me)
>> * "info pic" improvements for x86 (Peter)
>> * x86 TCG emulation fixes (Richard)
>> * KVM slot handling fix (Shannon)
>> * Next round of deprecation (Thomas)
>> * Windows dump format support (Viktor)
>>
> 
> Hi; looks like this broke the Travis --disable-tcg build:
> https://travis-ci.org/qemu/qemu/jobs/398262491
> 
> In file included from /home/travis/build/qemu/qemu/exec.c:62:0:
> /home/travis/build/qemu/qemu/include/exec/ram_addr.h:96:6: error:
> conflicting types for ‘tb_invalidate_phys_range’
>  void tb_invalidate_phys_range(ram_addr_t start, ram_addr_t end);
>       ^
> In file included from /home/travis/build/qemu/qemu/exec.c:24:0:
> /home/travis/build/qemu/qemu/include/exec/exec-all.h:309:6: note:
> previous declaration of ‘tb_invalidate_phys_range’ was here
>  void tb_invalidate_phys_range(target_ulong start, target_ulong end);
>       ^
> /home/travis/build/qemu/qemu/exec.c:1043:6: error: conflicting types
> for ‘tb_invalidate_phys_addr’
>  void tb_invalidate_phys_addr(AddressSpace *as, hwaddr addr, MemTxAttrs attrs)
>       ^
> In file included from /home/travis/build/qemu/qemu/exec.c:24:0:
> /home/travis/build/qemu/qemu/include/exec/exec-all.h:308:6: note:
> previous declaration of ‘tb_invalidate_phys_addr’ was here
>  void tb_invalidate_phys_addr(target_ulong addr);
>       ^
> 
> (with apologies for the smartquote damage).
> 
> Could you have a look, please?

I just sent a patch for this:
http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg09072.html