* [Qemu-devel] [PATCH 01/13] target-ppc kvm: save cr register
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
@ 2013-06-04 12:20 ` Alexey Kardashevskiy
2013-06-04 12:20 ` [Qemu-devel] [PATCH 02/13] savevm: Implement VMS_DIVIDE flag Alexey Kardashevskiy
` (12 subsequent siblings)
13 siblings, 0 replies; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-04 12:20 UTC (permalink / raw)
To: qemu-devel
Cc: Alexey Kardashevskiy, Alexander Graf, qemu-ppc, Paul Mackerras,
David Gibson
This adds a missing code to save CR (condition register) via
kvm_arch_put_registers(). kvm_arch_get_registers() already has it.
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
target-ppc/kvm.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/target-ppc/kvm.c b/target-ppc/kvm.c
index 2bbc3b8..c89dd58 100644
--- a/target-ppc/kvm.c
+++ b/target-ppc/kvm.c
@@ -791,6 +791,11 @@ int kvm_arch_put_registers(CPUState *cs, int level)
for (i = 0;i < 32; i++)
regs.gpr[i] = env->gpr[i];
+ regs.cr = 0;
+ for (i = 0; i < 8; i++) {
+ regs.cr |= (env->crf[i] & 15) << (4 * (7 - i));
+ }
+
ret = kvm_vcpu_ioctl(cs, KVM_SET_REGS, ®s);
if (ret < 0)
return ret;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 26+ messages in thread* [Qemu-devel] [PATCH 02/13] savevm: Implement VMS_DIVIDE flag
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
2013-06-04 12:20 ` [Qemu-devel] [PATCH 01/13] target-ppc kvm: save cr register Alexey Kardashevskiy
@ 2013-06-04 12:20 ` Alexey Kardashevskiy
2013-06-04 12:20 ` [Qemu-devel] [PATCH 03/13] target-ppc: Convert ppc cpu savevm to VMStateDescription Alexey Kardashevskiy
` (11 subsequent siblings)
13 siblings, 0 replies; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-04 12:20 UTC (permalink / raw)
To: qemu-devel; +Cc: aik, Alexander Graf, qemu-ppc, Paul Mackerras, David Gibson
From: David Gibson <david@gibson.dropbear.id.au>
The vmstate infrastructure includes a VMS_MULTIPY flag, and associated
VMSTATE_VBUFFER_MULTIPLY helper macro. These can be used to save a
variably sized buffer where the size in bytes of the buffer isn't directly
accessible as a structure field, but an element count from which the size
can be derived is.
This patch adds an analogous VMS_DIVIDE option, which handles a variably
sized buffer whose size is a submultiple of a field, rather than a
multiple. For example a buffer containing per-page structures whose size
is derived from a field storing the total address space described by the
structures could use this construct.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
include/migration/vmstate.h | 13 +++++++++++++
savevm.c | 8 ++++++++
2 files changed, 21 insertions(+)
diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index ebc4d09..787f1cb 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -98,6 +98,7 @@ enum VMStateFlags {
VMS_MULTIPLY = 0x200, /* multiply "size" field by field_size */
VMS_VARRAY_UINT8 = 0x400, /* Array with size in uint8_t field*/
VMS_VARRAY_UINT32 = 0x800, /* Array with size in uint32_t field*/
+ VMS_DIVIDE = 0x1000, /* divide "size" field by field_size */
};
typedef struct {
@@ -420,6 +421,18 @@ extern const VMStateInfo vmstate_info_bitmap;
.start = (_start), \
}
+#define VMSTATE_VBUFFER_DIVIDE(_field, _state, _version, _test, _start, _field_size, _divide) { \
+ .name = (stringify(_field)), \
+ .version_id = (_version), \
+ .field_exists = (_test), \
+ .size_offset = vmstate_offset_value(_state, _field_size, uint32_t),\
+ .size = (_divide), \
+ .info = &vmstate_info_buffer, \
+ .flags = VMS_VBUFFER|VMS_POINTER|VMS_DIVIDE, \
+ .offset = offsetof(_state, _field), \
+ .start = (_start), \
+}
+
#define VMSTATE_VBUFFER(_field, _state, _version, _test, _start, _field_size) { \
.name = (stringify(_field)), \
.version_id = (_version), \
diff --git a/savevm.c b/savevm.c
index 31dcce9..750b1cb 100644
--- a/savevm.c
+++ b/savevm.c
@@ -1655,6 +1655,10 @@ int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd,
if (field->flags & VMS_MULTIPLY) {
size *= field->size;
}
+ if (field->flags & VMS_DIVIDE) {
+ assert((size % field->size) == 0);
+ size /= field->size;
+ }
}
if (field->flags & VMS_ARRAY) {
n_elems = field->num;
@@ -1719,6 +1723,10 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
if (field->flags & VMS_MULTIPLY) {
size *= field->size;
}
+ if (field->flags & VMS_DIVIDE) {
+ assert((size % field->size) == 0);
+ size /= field->size;
+ }
}
if (field->flags & VMS_ARRAY) {
n_elems = field->num;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 26+ messages in thread* [Qemu-devel] [PATCH 03/13] target-ppc: Convert ppc cpu savevm to VMStateDescription
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
2013-06-04 12:20 ` [Qemu-devel] [PATCH 01/13] target-ppc kvm: save cr register Alexey Kardashevskiy
2013-06-04 12:20 ` [Qemu-devel] [PATCH 02/13] savevm: Implement VMS_DIVIDE flag Alexey Kardashevskiy
@ 2013-06-04 12:20 ` Alexey Kardashevskiy
2013-06-08 10:31 ` Andreas Färber
2013-06-04 12:20 ` [Qemu-devel] [PATCH 04/13] pseries: savevm support for XICS interrupt controller Alexey Kardashevskiy
` (10 subsequent siblings)
13 siblings, 1 reply; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-04 12:20 UTC (permalink / raw)
To: qemu-devel
Cc: Alexey Kardashevskiy, Alexander Graf, qemu-ppc, Paul Mackerras,
David Gibson
Author: David Gibson <david@gibson.dropbear.id.au>
The savevm code for the powerpc cpu emulation is currently based around
the old register_savevm() rather than register_vmstate() method. It's also
rather broken, missing some important state on some CPU models.
This patch completely rewrites the savevm for target-ppc, using the new
VMStateDescription approach. Exactly what needs to be saved in what
configurations has been more carefully examined, too. This introduces a
new version (5) of the cpu save format. The old load function is retained
to support version 4 images.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
target-ppc/cpu-qom.h | 4 +
target-ppc/cpu.h | 8 +-
target-ppc/machine.c | 539 ++++++++++++++++++++++++++++++++++++-------
target-ppc/translate_init.c | 2 +
4 files changed, 460 insertions(+), 93 deletions(-)
diff --git a/target-ppc/cpu-qom.h b/target-ppc/cpu-qom.h
index eb03a00..2b96b04 100644
--- a/target-ppc/cpu-qom.h
+++ b/target-ppc/cpu-qom.h
@@ -102,4 +102,8 @@ PowerPCCPUClass *ppc_cpu_class_by_pvr(uint32_t pvr);
void ppc_cpu_do_interrupt(CPUState *cpu);
+#ifndef CONFIG_USER_ONLY
+extern const struct VMStateDescription vmstate_ppc_cpu;
+#endif
+
#endif
diff --git a/target-ppc/cpu.h b/target-ppc/cpu.h
index aa1d013..e47853a 100644
--- a/target-ppc/cpu.h
+++ b/target-ppc/cpu.h
@@ -946,7 +946,7 @@ struct CPUPPCState {
#if defined(TARGET_PPC64)
/* PowerPC 64 SLB area */
ppc_slb_t slb[64];
- int slb_nr;
+ int32_t slb_nr;
#endif
/* segment registers */
hwaddr htab_base;
@@ -955,11 +955,11 @@ struct CPUPPCState {
/* externally stored hash table */
uint8_t *external_htab;
/* BATs */
- int nb_BATs;
+ uint32_t nb_BATs;
target_ulong DBAT[2][8];
target_ulong IBAT[2][8];
/* PowerPC TLB registers (for 4xx, e500 and 60x software driven TLBs) */
- int nb_tlb; /* Total number of TLB */
+ int32_t nb_tlb; /* Total number of TLB */
int tlb_per_way; /* Speed-up helper: used to avoid divisions at run time */
int nb_ways; /* Number of ways in the TLB set */
int last_way; /* Last used way used to allocate TLB in a LRU way */
@@ -1174,8 +1174,6 @@ static inline CPUPPCState *cpu_init(const char *cpu_model)
#define cpu_signal_handler cpu_ppc_signal_handler
#define cpu_list ppc_cpu_list
-#define CPU_SAVE_VERSION 4
-
/* MMU modes definitions */
#define MMU_MODE0_SUFFIX _user
#define MMU_MODE1_SUFFIX _kernel
diff --git a/target-ppc/machine.c b/target-ppc/machine.c
index 2d10adb..d58e652 100644
--- a/target-ppc/machine.c
+++ b/target-ppc/machine.c
@@ -1,96 +1,12 @@
#include "hw/hw.h"
#include "hw/boards.h"
#include "sysemu/kvm.h"
+#include "helper_regs.h"
-void cpu_save(QEMUFile *f, void *opaque)
+static int cpu_load_old(QEMUFile *f, void *opaque, int version_id)
{
- CPUPPCState *env = (CPUPPCState *)opaque;
- unsigned int i, j;
- uint32_t fpscr;
- target_ulong xer;
-
- for (i = 0; i < 32; i++)
- qemu_put_betls(f, &env->gpr[i]);
-#if !defined(TARGET_PPC64)
- for (i = 0; i < 32; i++)
- qemu_put_betls(f, &env->gprh[i]);
-#endif
- qemu_put_betls(f, &env->lr);
- qemu_put_betls(f, &env->ctr);
- for (i = 0; i < 8; i++)
- qemu_put_be32s(f, &env->crf[i]);
- xer = cpu_read_xer(env);
- qemu_put_betls(f, &xer);
- qemu_put_betls(f, &env->reserve_addr);
- qemu_put_betls(f, &env->msr);
- for (i = 0; i < 4; i++)
- qemu_put_betls(f, &env->tgpr[i]);
- for (i = 0; i < 32; i++) {
- union {
- float64 d;
- uint64_t l;
- } u;
- u.d = env->fpr[i];
- qemu_put_be64(f, u.l);
- }
- fpscr = env->fpscr;
- qemu_put_be32s(f, &fpscr);
- qemu_put_sbe32s(f, &env->access_type);
-#if defined(TARGET_PPC64)
- qemu_put_betls(f, &env->spr[SPR_ASR]);
- qemu_put_sbe32s(f, &env->slb_nr);
-#endif
- qemu_put_betls(f, &env->spr[SPR_SDR1]);
- for (i = 0; i < 32; i++)
- qemu_put_betls(f, &env->sr[i]);
- for (i = 0; i < 2; i++)
- for (j = 0; j < 8; j++)
- qemu_put_betls(f, &env->DBAT[i][j]);
- for (i = 0; i < 2; i++)
- for (j = 0; j < 8; j++)
- qemu_put_betls(f, &env->IBAT[i][j]);
- qemu_put_sbe32s(f, &env->nb_tlb);
- qemu_put_sbe32s(f, &env->tlb_per_way);
- qemu_put_sbe32s(f, &env->nb_ways);
- qemu_put_sbe32s(f, &env->last_way);
- qemu_put_sbe32s(f, &env->id_tlbs);
- qemu_put_sbe32s(f, &env->nb_pids);
- if (env->tlb.tlb6) {
- // XXX assumes 6xx
- for (i = 0; i < env->nb_tlb; i++) {
- qemu_put_betls(f, &env->tlb.tlb6[i].pte0);
- qemu_put_betls(f, &env->tlb.tlb6[i].pte1);
- qemu_put_betls(f, &env->tlb.tlb6[i].EPN);
- }
- }
- for (i = 0; i < 4; i++)
- qemu_put_betls(f, &env->pb[i]);
- for (i = 0; i < 1024; i++)
- qemu_put_betls(f, &env->spr[i]);
- qemu_put_be32s(f, &env->vscr);
- qemu_put_be64s(f, &env->spe_acc);
- qemu_put_be32s(f, &env->spe_fscr);
- qemu_put_betls(f, &env->msr_mask);
- qemu_put_be32s(f, &env->flags);
- qemu_put_sbe32s(f, &env->error_code);
- qemu_put_be32s(f, &env->pending_interrupts);
- qemu_put_be32s(f, &env->irq_input_state);
- for (i = 0; i < POWERPC_EXCP_NB; i++)
- qemu_put_betls(f, &env->excp_vectors[i]);
- qemu_put_betls(f, &env->excp_prefix);
- qemu_put_betls(f, &env->ivor_mask);
- qemu_put_betls(f, &env->ivpr_mask);
- qemu_put_betls(f, &env->hreset_vector);
- qemu_put_betls(f, &env->nip);
- qemu_put_betls(f, &env->hflags);
- qemu_put_betls(f, &env->hflags_nmsr);
- qemu_put_sbe32s(f, &env->mmu_idx);
- qemu_put_sbe32(f, 0);
-}
-
-int cpu_load(QEMUFile *f, void *opaque, int version_id)
-{
- CPUPPCState *env = (CPUPPCState *)opaque;
+ PowerPCCPU *cpu = opaque;
+ CPUPPCState *env = &cpu->env;
unsigned int i, j;
target_ulong sdr1;
uint32_t fpscr;
@@ -177,3 +93,450 @@ int cpu_load(QEMUFile *f, void *opaque, int version_id)
return 0;
}
+
+static int get_avr(QEMUFile *f, void *pv, size_t size)
+{
+ ppc_avr_t *v = pv;
+
+ v->u64[0] = qemu_get_be64(f);
+ v->u64[1] = qemu_get_be64(f);
+
+ return 0;
+}
+
+static void put_avr(QEMUFile *f, void *pv, size_t size)
+{
+ ppc_avr_t *v = pv;
+
+ qemu_put_be64(f, v->u64[0]);
+ qemu_put_be64(f, v->u64[1]);
+}
+
+const VMStateInfo vmstate_info_avr = {
+ .name = "avr",
+ .get = get_avr,
+ .put = put_avr,
+};
+
+#define VMSTATE_AVR_ARRAY_V(_f, _s, _n, _v) \
+ VMSTATE_ARRAY(_f, _s, _n, _v, vmstate_info_avr, ppc_avr_t)
+
+#define VMSTATE_AVR_ARRAY(_f, _s, _n) \
+ VMSTATE_AVR_ARRAY_V(_f, _s, _n, 0)
+
+static void cpu_pre_save(void *opaque)
+{
+ PowerPCCPU *cpu = opaque;
+ CPUPPCState *env = &cpu->env;
+ int i;
+
+ env->spr[SPR_LR] = env->lr;
+ env->spr[SPR_CTR] = env->ctr;
+ env->spr[SPR_XER] = env->xer;
+#if defined(TARGET_PPC64)
+ env->spr[SPR_CFAR] = env->cfar;
+#endif
+ env->spr[SPR_BOOKE_SPEFSCR] = env->spe_fscr;
+
+ for (i = 0; (i < 4) && (i < env->nb_BATs); i++) {
+ env->spr[SPR_DBAT0U + 2*i] = env->DBAT[0][i];
+ env->spr[SPR_DBAT0U + 2*i + 1] = env->DBAT[1][i];
+ env->spr[SPR_IBAT0U + 2*i] = env->IBAT[0][i];
+ env->spr[SPR_IBAT0U + 2*i + 1] = env->IBAT[1][i];
+ }
+ for (i = 0; (i < 4) && ((i+4) < env->nb_BATs); i++) {
+ env->spr[SPR_DBAT4U + 2*i] = env->DBAT[0][i+4];
+ env->spr[SPR_DBAT4U + 2*i + 1] = env->DBAT[1][i+4];
+ env->spr[SPR_IBAT4U + 2*i] = env->IBAT[0][i+4];
+ env->spr[SPR_IBAT4U + 2*i + 1] = env->IBAT[1][i+4];
+ }
+}
+
+static int cpu_post_load(void *opaque, int version_id)
+{
+ PowerPCCPU *cpu = opaque;
+ CPUPPCState *env = &cpu->env;
+ int i;
+
+ env->lr = env->spr[SPR_LR];
+ env->ctr = env->spr[SPR_CTR];
+ env->xer = env->spr[SPR_XER];
+#if defined(TARGET_PPC64)
+ env->cfar = env->spr[SPR_CFAR];
+#endif
+ env->spe_fscr = env->spr[SPR_BOOKE_SPEFSCR];
+
+ for (i = 0; (i < 4) && (i < env->nb_BATs); i++) {
+ env->DBAT[0][i] = env->spr[SPR_DBAT0U + 2*i];
+ env->DBAT[1][i] = env->spr[SPR_DBAT0U + 2*i + 1];
+ env->IBAT[0][i] = env->spr[SPR_IBAT0U + 2*i];
+ env->IBAT[1][i] = env->spr[SPR_IBAT0U + 2*i + 1];
+ }
+ for (i = 0; (i < 4) && ((i+4) < env->nb_BATs); i++) {
+ env->DBAT[0][i+4] = env->spr[SPR_DBAT4U + 2*i];
+ env->DBAT[1][i+4] = env->spr[SPR_DBAT4U + 2*i + 1];
+ env->IBAT[0][i+4] = env->spr[SPR_IBAT4U + 2*i];
+ env->IBAT[1][i+4] = env->spr[SPR_IBAT4U + 2*i + 1];
+ }
+
+ /* Restore htab_base and htab_mask variables */
+ ppc_store_sdr1(env, env->spr[SPR_SDR1]);
+
+ hreg_compute_hflags(env);
+ hreg_compute_mem_idx(env);
+
+ return 0;
+}
+
+static bool fpu_needed(void *opaque)
+{
+ PowerPCCPU *cpu = opaque;
+ CPUPPCState *env = &cpu->env;
+
+ return (env->insns_flags & PPC_FLOAT);
+}
+
+static const VMStateDescription vmstate_fpu = {
+ .name = "cpu/fpu",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_FLOAT64_ARRAY(env.fpr, PowerPCCPU, 32),
+ VMSTATE_UINTTL(env.fpscr, PowerPCCPU),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static bool altivec_needed(void *opaque)
+{
+ PowerPCCPU *cpu = opaque;
+ CPUPPCState *env = &cpu->env;
+
+ return (env->insns_flags & PPC_ALTIVEC);
+}
+
+static const VMStateDescription vmstate_altivec = {
+ .name = "cpu/altivec",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_AVR_ARRAY(env.avr, PowerPCCPU, 32),
+ VMSTATE_UINT32(env.vscr, PowerPCCPU),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static bool vsx_needed(void *opaque)
+{
+ PowerPCCPU *cpu = opaque;
+ CPUPPCState *env = &cpu->env;
+
+ return (env->insns_flags2 & PPC2_VSX);
+}
+
+static const VMStateDescription vmstate_vsx = {
+ .name = "cpu/vsx",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_UINT64_ARRAY(env.vsr, PowerPCCPU, 32),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static bool sr_needed(void *opaque)
+{
+#ifdef TARGET_PPC64
+ PowerPCCPU *cpu = opaque;
+ CPUPPCState *env = &cpu->env;
+
+ return !(env->mmu_model & POWERPC_MMU_64);
+#else
+ return true;
+#endif
+}
+
+static const VMStateDescription vmstate_sr = {
+ .name = "cpu/sr",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_UINTTL_ARRAY(env.sr, PowerPCCPU, 32),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+#ifdef TARGET_PPC64
+static int get_slbe(QEMUFile *f, void *pv, size_t size)
+{
+ ppc_slb_t *v = pv;
+
+ v->esid = qemu_get_be64(f);
+ v->vsid = qemu_get_be64(f);
+
+ return 0;
+}
+
+static void put_slbe(QEMUFile *f, void *pv, size_t size)
+{
+ ppc_slb_t *v = pv;
+
+ qemu_put_be64(f, v->esid);
+ qemu_put_be64(f, v->vsid);
+}
+
+const VMStateInfo vmstate_info_slbe = {
+ .name = "slbe",
+ .get = get_slbe,
+ .put = put_slbe,
+};
+
+#define VMSTATE_SLB_ARRAY_V(_f, _s, _n, _v) \
+ VMSTATE_ARRAY(_f, _s, _n, _v, vmstate_info_slbe, ppc_slb_t)
+
+#define VMSTATE_SLB_ARRAY(_f, _s, _n) \
+ VMSTATE_SLB_ARRAY_V(_f, _s, _n, 0)
+
+static bool slb_needed(void *opaque)
+{
+ PowerPCCPU *cpu = opaque;
+ CPUPPCState *env = &cpu->env;
+
+ /* We don't support any of the old segment table based 64-bit CPUs */
+ return (env->mmu_model & POWERPC_MMU_64);
+}
+
+static const VMStateDescription vmstate_slb = {
+ .name = "cpu/slb",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_INT32_EQUAL(env.slb_nr, PowerPCCPU),
+ VMSTATE_SLB_ARRAY(env.slb, PowerPCCPU, 64),
+ VMSTATE_END_OF_LIST()
+ }
+};
+#endif /* TARGET_PPC64 */
+
+static const VMStateDescription vmstate_tlb6xx_entry = {
+ .name = "cpu/tlb6xx_entry",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_UINTTL(pte0, ppc6xx_tlb_t),
+ VMSTATE_UINTTL(pte1, ppc6xx_tlb_t),
+ VMSTATE_UINTTL(EPN, ppc6xx_tlb_t),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static bool tlb6xx_needed(void *opaque)
+{
+ PowerPCCPU *cpu = opaque;
+ CPUPPCState *env = &cpu->env;
+
+ return env->nb_tlb && (env->tlb_type == TLB_6XX);
+}
+
+static const VMStateDescription vmstate_tlb6xx = {
+ .name = "cpu/tlb6xx",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_INT32_EQUAL(env.nb_tlb, PowerPCCPU),
+ VMSTATE_STRUCT_VARRAY_POINTER_INT32(env.tlb.tlb6, PowerPCCPU,
+ env.nb_tlb,
+ vmstate_tlb6xx_entry,
+ ppc6xx_tlb_t),
+ VMSTATE_UINTTL_ARRAY(env.tgpr, PowerPCCPU, 4),
+ VMSTATE_END_OF_LIST()
+ }
+};
+
+static const VMStateDescription vmstate_tlbemb_entry = {
+ .name = "cpu/tlbemb_entry",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_UINT64(RPN, ppcemb_tlb_t),
+ VMSTATE_UINTTL(EPN, ppcemb_tlb_t),
+ VMSTATE_UINTTL(PID, ppcemb_tlb_t),
+ VMSTATE_UINTTL(size, ppcemb_tlb_t),
+ VMSTATE_UINT32(prot, ppcemb_tlb_t),
+ VMSTATE_UINT32(attr, ppcemb_tlb_t),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static bool tlbemb_needed(void *opaque)
+{
+ PowerPCCPU *cpu = opaque;
+ CPUPPCState *env = &cpu->env;
+
+ return env->nb_tlb && (env->tlb_type == TLB_EMB);
+}
+
+static bool pbr403_needed(void *opaque)
+{
+ PowerPCCPU *cpu = opaque;
+ CPUPPCState *env = &cpu->env;
+ uint32_t pvr = env->spr[SPR_PVR];
+
+ return (pvr & 0xffff0000) == 0x00200000;
+}
+
+static const VMStateDescription vmstate_pbr403 = {
+ .name = "cpu/pbr403",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_UINTTL_ARRAY(env.pb, PowerPCCPU, 4),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static const VMStateDescription vmstate_tlbemb = {
+ .name = "cpu/tlb6xx",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_INT32_EQUAL(env.nb_tlb, PowerPCCPU),
+ VMSTATE_STRUCT_VARRAY_POINTER_INT32(env.tlb.tlbe, PowerPCCPU,
+ env.nb_tlb,
+ vmstate_tlbemb_entry,
+ ppcemb_tlb_t),
+ /* 403 protection registers */
+ VMSTATE_END_OF_LIST()
+ },
+ .subsections = (VMStateSubsection []) {
+ {
+ .vmsd = &vmstate_pbr403,
+ .needed = pbr403_needed,
+ } , {
+ /* empty */
+ }
+ }
+};
+
+static const VMStateDescription vmstate_tlbmas_entry = {
+ .name = "cpu/tlbmas_entry",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_UINT32(mas8, ppcmas_tlb_t),
+ VMSTATE_UINT32(mas1, ppcmas_tlb_t),
+ VMSTATE_UINT64(mas2, ppcmas_tlb_t),
+ VMSTATE_UINT64(mas7_3, ppcmas_tlb_t),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static bool tlbmas_needed(void *opaque)
+{
+ PowerPCCPU *cpu = opaque;
+ CPUPPCState *env = &cpu->env;
+
+ return env->nb_tlb && (env->tlb_type == TLB_MAS);
+}
+
+static const VMStateDescription vmstate_tlbmas = {
+ .name = "cpu/tlbmas",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_INT32_EQUAL(env.nb_tlb, PowerPCCPU),
+ VMSTATE_STRUCT_VARRAY_POINTER_INT32(env.tlb.tlbm, PowerPCCPU,
+ env.nb_tlb,
+ vmstate_tlbmas_entry,
+ ppcmas_tlb_t),
+ VMSTATE_END_OF_LIST()
+ }
+};
+
+const VMStateDescription vmstate_ppc_cpu = {
+ .name = "cpu",
+ .version_id = 5,
+ .minimum_version_id = 5,
+ .minimum_version_id_old = 4,
+ .load_state_old = cpu_load_old,
+ .pre_save = cpu_pre_save,
+ .post_load = cpu_post_load,
+ .fields = (VMStateField []) {
+ /* Verify we haven't changed the pvr */
+ VMSTATE_UINTTL_EQUAL(env.spr[SPR_PVR], PowerPCCPU),
+
+ /* User mode architected state */
+ VMSTATE_UINTTL_ARRAY(env.gpr, PowerPCCPU, 32),
+#if !defined(TARGET_PPC64)
+ VMSTATE_UINTTL_ARRAY(env.gprh, PowerPCCPU, 32),
+#endif
+ VMSTATE_UINT32_ARRAY(env.crf, PowerPCCPU, 8),
+ VMSTATE_UINTTL(env.nip, PowerPCCPU),
+
+ /* SPRs */
+ VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024),
+ VMSTATE_UINT64(env.spe_acc, PowerPCCPU),
+
+ /* Reservation */
+ VMSTATE_UINTTL(env.reserve_addr, PowerPCCPU),
+
+ /* Supervisor mode architected state */
+ VMSTATE_UINTTL(env.msr, PowerPCCPU),
+
+ /* Internal state */
+ VMSTATE_UINTTL(env.hflags_nmsr, PowerPCCPU),
+ /* FIXME: access_type? */
+
+ /* Sanity checking */
+ VMSTATE_UINTTL_EQUAL(env.msr_mask, PowerPCCPU),
+ VMSTATE_UINT64_EQUAL(env.insns_flags, PowerPCCPU),
+ VMSTATE_UINT64_EQUAL(env.insns_flags2, PowerPCCPU),
+ VMSTATE_UINT32_EQUAL(env.nb_BATs, PowerPCCPU),
+ VMSTATE_END_OF_LIST()
+ },
+ .subsections = (VMStateSubsection []) {
+ {
+ .vmsd = &vmstate_fpu,
+ .needed = fpu_needed,
+ } , {
+ .vmsd = &vmstate_altivec,
+ .needed = altivec_needed,
+ } , {
+ .vmsd = &vmstate_vsx,
+ .needed = vsx_needed,
+ } , {
+ .vmsd = &vmstate_sr,
+ .needed = sr_needed,
+ } , {
+#ifdef TARGET_PPC64
+ .vmsd = &vmstate_slb,
+ .needed = slb_needed,
+ } , {
+#endif /* TARGET_PPC64 */
+ .vmsd = &vmstate_tlb6xx,
+ .needed = tlb6xx_needed,
+ } , {
+ .vmsd = &vmstate_tlbemb,
+ .needed = tlbemb_needed,
+ } , {
+ .vmsd = &vmstate_tlbmas,
+ .needed = tlbmas_needed,
+ } , {
+ /* FIXME: DCRs? */
+ /* FIXME: timebase? */
+ /* empty */
+ }
+ }
+};
diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c
index 021a31e..4f89b14 100644
--- a/target-ppc/translate_init.c
+++ b/target-ppc/translate_init.c
@@ -8309,6 +8309,8 @@ static void ppc_cpu_class_init(ObjectClass *oc, void *data)
cc->class_by_name = ppc_cpu_class_by_name;
cc->do_interrupt = ppc_cpu_do_interrupt;
+
+ cpu_class_set_vmsd(cc, &vmstate_ppc_cpu);
}
static const TypeInfo ppc_cpu_type_info = {
--
1.7.10.4
^ permalink raw reply related [flat|nested] 26+ messages in thread* Re: [Qemu-devel] [PATCH 03/13] target-ppc: Convert ppc cpu savevm to VMStateDescription
2013-06-04 12:20 ` [Qemu-devel] [PATCH 03/13] target-ppc: Convert ppc cpu savevm to VMStateDescription Alexey Kardashevskiy
@ 2013-06-08 10:31 ` Andreas Färber
0 siblings, 0 replies; 26+ messages in thread
From: Andreas Färber @ 2013-06-08 10:31 UTC (permalink / raw)
To: Alexey Kardashevskiy
Cc: Paul Mackerras, David Gibson, qemu-ppc, qemu-devel,
Alexander Graf
Am 04.06.2013 14:20, schrieb Alexey Kardashevskiy:
> Author: David Gibson <david@gibson.dropbear.id.au>
From:
>
> The savevm code for the powerpc cpu emulation is currently based around
> the old register_savevm() rather than register_vmstate() method. It's also
> rather broken, missing some important state on some CPU models.
>
> This patch completely rewrites the savevm for target-ppc, using the new
> VMStateDescription approach. Exactly what needs to be saved in what
> configurations has been more carefully examined, too. This introduces a
> new version (5) of the cpu save format. The old load function is retained
> to support version 4 images.
>
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> ---
> target-ppc/cpu-qom.h | 4 +
> target-ppc/cpu.h | 8 +-
> target-ppc/machine.c | 539 ++++++++++++++++++++++++++++++++++++-------
> target-ppc/translate_init.c | 2 +
> 4 files changed, 460 insertions(+), 93 deletions(-)
>
> diff --git a/target-ppc/cpu-qom.h b/target-ppc/cpu-qom.h
> index eb03a00..2b96b04 100644
> --- a/target-ppc/cpu-qom.h
> +++ b/target-ppc/cpu-qom.h
> @@ -102,4 +102,8 @@ PowerPCCPUClass *ppc_cpu_class_by_pvr(uint32_t pvr);
>
> void ppc_cpu_do_interrupt(CPUState *cpu);
>
> +#ifndef CONFIG_USER_ONLY
> +extern const struct VMStateDescription vmstate_ppc_cpu;
> +#endif
> +
> #endif
> diff --git a/target-ppc/cpu.h b/target-ppc/cpu.h
> index aa1d013..e47853a 100644
> --- a/target-ppc/cpu.h
> +++ b/target-ppc/cpu.h
> @@ -946,7 +946,7 @@ struct CPUPPCState {
> #if defined(TARGET_PPC64)
> /* PowerPC 64 SLB area */
> ppc_slb_t slb[64];
> - int slb_nr;
> + int32_t slb_nr;
> #endif
> /* segment registers */
> hwaddr htab_base;
> @@ -955,11 +955,11 @@ struct CPUPPCState {
> /* externally stored hash table */
> uint8_t *external_htab;
> /* BATs */
> - int nb_BATs;
> + uint32_t nb_BATs;
Here you changed to uint32_t but above and below to int32_t?
> target_ulong DBAT[2][8];
> target_ulong IBAT[2][8];
> /* PowerPC TLB registers (for 4xx, e500 and 60x software driven TLBs) */
> - int nb_tlb; /* Total number of TLB */
> + int32_t nb_tlb; /* Total number of TLB */
> int tlb_per_way; /* Speed-up helper: used to avoid divisions at run time */
> int nb_ways; /* Number of ways in the TLB set */
> int last_way; /* Last used way used to allocate TLB in a LRU way */
> @@ -1174,8 +1174,6 @@ static inline CPUPPCState *cpu_init(const char *cpu_model)
> #define cpu_signal_handler cpu_ppc_signal_handler
> #define cpu_list ppc_cpu_list
>
> -#define CPU_SAVE_VERSION 4
> -
> /* MMU modes definitions */
> #define MMU_MODE0_SUFFIX _user
> #define MMU_MODE1_SUFFIX _kernel
> diff --git a/target-ppc/machine.c b/target-ppc/machine.c
> index 2d10adb..d58e652 100644
> --- a/target-ppc/machine.c
> +++ b/target-ppc/machine.c
[...]
> +static bool fpu_needed(void *opaque)
> +{
> + PowerPCCPU *cpu = opaque;
> + CPUPPCState *env = &cpu->env;
> +
> + return (env->insns_flags & PPC_FLOAT);
If you only need env once, you could just access cpu->env.foo directly,
saving a line. Applies to most such helpers below except for those that
access env two times.
> +}
[...]
> +static bool altivec_needed(void *opaque)
> +{
> + PowerPCCPU *cpu = opaque;
> + CPUPPCState *env = &cpu->env;
> +
> + return (env->insns_flags & PPC_ALTIVEC);
> +}
[...]
> +static bool vsx_needed(void *opaque)
> +{
> + PowerPCCPU *cpu = opaque;
> + CPUPPCState *env = &cpu->env;
> +
> + return (env->insns_flags2 & PPC2_VSX);
> +}
[...]
> +static bool sr_needed(void *opaque)
> +{
> +#ifdef TARGET_PPC64
> + PowerPCCPU *cpu = opaque;
> + CPUPPCState *env = &cpu->env;
> +
> + return !(env->mmu_model & POWERPC_MMU_64);
> +#else
> + return true;
> +#endif
> +}
[...]
> +static bool slb_needed(void *opaque)
> +{
> + PowerPCCPU *cpu = opaque;
> + CPUPPCState *env = &cpu->env;
> +
> + /* We don't support any of the old segment table based 64-bit CPUs */
> + return (env->mmu_model & POWERPC_MMU_64);
> +}
[...]
> +static bool tlb6xx_needed(void *opaque)
> +{
> + PowerPCCPU *cpu = opaque;
> + CPUPPCState *env = &cpu->env;
> +
> + return env->nb_tlb && (env->tlb_type == TLB_6XX);
> +}
[...]
> +static bool tlbemb_needed(void *opaque)
> +{
> + PowerPCCPU *cpu = opaque;
> + CPUPPCState *env = &cpu->env;
> +
> + return env->nb_tlb && (env->tlb_type == TLB_EMB);
> +}
> +
> +static bool pbr403_needed(void *opaque)
> +{
> + PowerPCCPU *cpu = opaque;
> + CPUPPCState *env = &cpu->env;
> + uint32_t pvr = env->spr[SPR_PVR];
> +
> + return (pvr & 0xffff0000) == 0x00200000;
> +}
[...]
> +static bool tlbmas_needed(void *opaque)
> +{
> + PowerPCCPU *cpu = opaque;
> + CPUPPCState *env = &cpu->env;
> +
> + return env->nb_tlb && (env->tlb_type == TLB_MAS);
> +}
[...]
> +const VMStateDescription vmstate_ppc_cpu = {
> + .name = "cpu",
> + .version_id = 5,
> + .minimum_version_id = 5,
> + .minimum_version_id_old = 4,
> + .load_state_old = cpu_load_old,
> + .pre_save = cpu_pre_save,
> + .post_load = cpu_post_load,
> + .fields = (VMStateField []) {
[...]
> +};
> diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c
> index 021a31e..4f89b14 100644
> --- a/target-ppc/translate_init.c
> +++ b/target-ppc/translate_init.c
> @@ -8309,6 +8309,8 @@ static void ppc_cpu_class_init(ObjectClass *oc, void *data)
>
> cc->class_by_name = ppc_cpu_class_by_name;
> cc->do_interrupt = ppc_cpu_do_interrupt;
> +
> + cpu_class_set_vmsd(cc, &vmstate_ppc_cpu);
> }
>
> static const TypeInfo ppc_cpu_type_info = {
The way to hook it up looks fine now. Device-based CPU VMState is still
not ready and would be incompatible from what I understand, so with or
without my minor comments addressed this looks good to go to me now!
Regards,
Andreas
--
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 04/13] pseries: savevm support for XICS interrupt controller
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
` (2 preceding siblings ...)
2013-06-04 12:20 ` [Qemu-devel] [PATCH 03/13] target-ppc: Convert ppc cpu savevm to VMStateDescription Alexey Kardashevskiy
@ 2013-06-04 12:20 ` Alexey Kardashevskiy
2013-06-05 3:22 ` Alexey Kardashevskiy
2013-06-04 12:20 ` [Qemu-devel] [PATCH 05/13] pseries: savevm support for VIO devices Alexey Kardashevskiy
` (9 subsequent siblings)
13 siblings, 1 reply; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-04 12:20 UTC (permalink / raw)
To: qemu-devel; +Cc: aik, Alexander Graf, qemu-ppc, Paul Mackerras, David Gibson
From: David Gibson <david@gibson.dropbear.id.au>
From: David Gibson <david@gibson.dropbear.id.au>
This patch adds the necessary VMStateDescription information to support
savevm/loadvm for the XICS interrupt controller used on the pseries
machine.
[aik: added ics_resend() on post_load]
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
hw/ppc/xics.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++++----
1 file changed, 66 insertions(+), 4 deletions(-)
diff --git a/hw/ppc/xics.c b/hw/ppc/xics.c
index 1b25075..8a17175 100644
--- a/hw/ppc/xics.c
+++ b/hw/ppc/xics.c
@@ -50,7 +50,7 @@ struct icp_server_state {
struct ics_state;
struct icp_state {
- long nr_servers;
+ uint32_t nr_servers;
struct icp_server_state *ss;
struct ics_state *ics;
};
@@ -173,7 +173,7 @@ static void icp_irq(struct icp_state *icp, int server, int nr, uint8_t priority)
*/
struct ics_irq_state {
- int server;
+ uint32_t server;
uint8_t priority;
uint8_t saved_priority;
#define XICS_STATUS_ASSERTED 0x1
@@ -184,8 +184,8 @@ struct ics_irq_state {
};
struct ics_state {
- int nr_irqs;
- int offset;
+ uint32_t nr_irqs;
+ uint32_t offset;
qemu_irq *qirqs;
bool *islsi;
struct ics_irq_state *irqs;
@@ -523,6 +523,61 @@ static void xics_reset(void *opaque)
}
}
+static int ics_post_load(void *opaque, int version_id)
+{
+ int i;
+ struct ics_state *ics = opaque;
+
+ for (i = 0; i < ics->nr_irqs; i++) {
+ icp_resend(ics->icp, i);
+ }
+
+ return 0;
+}
+
+static const VMStateDescription vmstate_icp_server = {
+ .name = "icp/server",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ /* Sanity check */
+ VMSTATE_UINT32(xirr, struct icp_server_state),
+ VMSTATE_UINT8(pending_priority, struct icp_server_state),
+ VMSTATE_UINT8(mfrr, struct icp_server_state),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static const VMStateDescription vmstate_ics_irq = {
+ .name = "ics/irq",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_UINT32(server, struct ics_irq_state),
+ VMSTATE_UINT8(priority, struct ics_irq_state),
+ VMSTATE_UINT8(saved_priority, struct ics_irq_state),
+ VMSTATE_UINT8(status, struct ics_irq_state),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static const VMStateDescription vmstate_ics = {
+ .name = "ics",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .post_load = ics_post_load,
+ .fields = (VMStateField []) {
+ /* Sanity check */
+ VMSTATE_UINT32_EQUAL(nr_irqs, struct ics_state),
+
+ VMSTATE_STRUCT_VARRAY_POINTER_UINT32(irqs, struct ics_state, nr_irqs, vmstate_ics_irq, struct ics_irq_state),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
void xics_cpu_setup(struct icp_state *icp, PowerPCCPU *cpu)
{
CPUState *cs = CPU(cpu);
@@ -545,6 +600,8 @@ void xics_cpu_setup(struct icp_state *icp, PowerPCCPU *cpu)
"bus model\n");
abort();
}
+
+ vmstate_register(NULL, cs->cpu_index, &vmstate_icp_server, ss);
}
struct icp_state *xics_system_init(int nr_servers, int nr_irqs)
@@ -579,5 +636,10 @@ struct icp_state *xics_system_init(int nr_servers, int nr_irqs)
qemu_register_reset(xics_reset, icp);
+ /* We use each the ICS's offset into the global irq number space
+ * as an instance id. This means we can extend to multiple ICS
+ * instances without needing to change the savevm format */
+ vmstate_register(NULL, ics->offset, &vmstate_ics, ics);
+
return icp;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 26+ messages in thread* Re: [Qemu-devel] [PATCH 04/13] pseries: savevm support for XICS interrupt controller
2013-06-04 12:20 ` [Qemu-devel] [PATCH 04/13] pseries: savevm support for XICS interrupt controller Alexey Kardashevskiy
@ 2013-06-05 3:22 ` Alexey Kardashevskiy
2013-06-05 9:55 ` David Gibson
0 siblings, 1 reply; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-05 3:22 UTC (permalink / raw)
To: Alexey Kardashevskiy
Cc: Alexander Graf, qemu-devel, qemu-ppc, Paul Mackerras,
David Gibson
On 06/04/2013 10:20 PM, Alexey Kardashevskiy wrote:
> From: David Gibson <david@gibson.dropbear.id.au>
>
> From: David Gibson <david@gibson.dropbear.id.au>
>
> This patch adds the necessary VMStateDescription information to support
> savevm/loadvm for the XICS interrupt controller used on the pseries
> machine.
>
> [aik: added ics_resend() on post_load]
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> ---
> hw/ppc/xics.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++++----
> 1 file changed, 66 insertions(+), 4 deletions(-)
>
> diff --git a/hw/ppc/xics.c b/hw/ppc/xics.c
> index 1b25075..8a17175 100644
> --- a/hw/ppc/xics.c
> +++ b/hw/ppc/xics.c
> @@ -50,7 +50,7 @@ struct icp_server_state {
> struct ics_state;
>
> struct icp_state {
> - long nr_servers;
> + uint32_t nr_servers;
> struct icp_server_state *ss;
> struct ics_state *ics;
> };
> @@ -173,7 +173,7 @@ static void icp_irq(struct icp_state *icp, int server, int nr, uint8_t priority)
> */
>
> struct ics_irq_state {
> - int server;
> + uint32_t server;
> uint8_t priority;
> uint8_t saved_priority;
> #define XICS_STATUS_ASSERTED 0x1
> @@ -184,8 +184,8 @@ struct ics_irq_state {
> };
>
> struct ics_state {
> - int nr_irqs;
> - int offset;
> + uint32_t nr_irqs;
> + uint32_t offset;
> qemu_irq *qirqs;
> bool *islsi;
> struct ics_irq_state *irqs;
> @@ -523,6 +523,61 @@ static void xics_reset(void *opaque)
> }
> }
>
> +static int ics_post_load(void *opaque, int version_id)
> +{
> + int i;
> + struct ics_state *ics = opaque;
> +
> + for (i = 0; i < ics->nr_irqs; i++) {
Obviously here is a bug...
- for (i = 0; i < ics->nr_irqs; i++) {
+ for (i = 0; i < ics->icp->nr_servers; i++) {
> + icp_resend(ics->icp, i);
> + }
> +
> + return 0;
> +}
> +
> +static const VMStateDescription vmstate_icp_server = {
> + .name = "icp/server",
> + .version_id = 1,
> + .minimum_version_id = 1,
> + .minimum_version_id_old = 1,
> + .fields = (VMStateField []) {
> + /* Sanity check */
> + VMSTATE_UINT32(xirr, struct icp_server_state),
> + VMSTATE_UINT8(pending_priority, struct icp_server_state),
> + VMSTATE_UINT8(mfrr, struct icp_server_state),
> + VMSTATE_END_OF_LIST()
> + },
> +};
> +
> +static const VMStateDescription vmstate_ics_irq = {
> + .name = "ics/irq",
> + .version_id = 1,
> + .minimum_version_id = 1,
> + .minimum_version_id_old = 1,
> + .fields = (VMStateField []) {
> + VMSTATE_UINT32(server, struct ics_irq_state),
> + VMSTATE_UINT8(priority, struct ics_irq_state),
> + VMSTATE_UINT8(saved_priority, struct ics_irq_state),
> + VMSTATE_UINT8(status, struct ics_irq_state),
> + VMSTATE_END_OF_LIST()
> + },
> +};
> +
> +static const VMStateDescription vmstate_ics = {
> + .name = "ics",
> + .version_id = 1,
> + .minimum_version_id = 1,
> + .minimum_version_id_old = 1,
> + .post_load = ics_post_load,
> + .fields = (VMStateField []) {
> + /* Sanity check */
> + VMSTATE_UINT32_EQUAL(nr_irqs, struct ics_state),
> +
> + VMSTATE_STRUCT_VARRAY_POINTER_UINT32(irqs, struct ics_state, nr_irqs, vmstate_ics_irq, struct ics_irq_state),
> + VMSTATE_END_OF_LIST()
> + },
> +};
> +
> void xics_cpu_setup(struct icp_state *icp, PowerPCCPU *cpu)
> {
> CPUState *cs = CPU(cpu);
> @@ -545,6 +600,8 @@ void xics_cpu_setup(struct icp_state *icp, PowerPCCPU *cpu)
> "bus model\n");
> abort();
> }
> +
> + vmstate_register(NULL, cs->cpu_index, &vmstate_icp_server, ss);
> }
>
> struct icp_state *xics_system_init(int nr_servers, int nr_irqs)
> @@ -579,5 +636,10 @@ struct icp_state *xics_system_init(int nr_servers, int nr_irqs)
>
> qemu_register_reset(xics_reset, icp);
>
> + /* We use each the ICS's offset into the global irq number space
> + * as an instance id. This means we can extend to multiple ICS
> + * instances without needing to change the savevm format */
> + vmstate_register(NULL, ics->offset, &vmstate_ics, ics);
> +
> return icp;
> }
>
--
Alexey
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Qemu-devel] [PATCH 04/13] pseries: savevm support for XICS interrupt controller
2013-06-05 3:22 ` Alexey Kardashevskiy
@ 2013-06-05 9:55 ` David Gibson
2013-06-05 11:17 ` Alexey Kardashevskiy
0 siblings, 1 reply; 26+ messages in thread
From: David Gibson @ 2013-06-05 9:55 UTC (permalink / raw)
To: Alexey Kardashevskiy; +Cc: qemu-ppc, qemu-devel, Paul Mackerras, Alexander Graf
[-- Attachment #1: Type: text/plain, Size: 847 bytes --]
On Wed, Jun 05, 2013 at 01:22:02PM +1000, Alexey Kardashevskiy wrote:
> On 06/04/2013 10:20 PM, Alexey Kardashevskiy wrote:
[snip]
> > +static int ics_post_load(void *opaque, int version_id)
> > +{
> > + int i;
> > + struct ics_state *ics = opaque;
> > +
> > + for (i = 0; i < ics->nr_irqs; i++) {
>
> Obviously here is a bug...
>
> - for (i = 0; i < ics->nr_irqs; i++) {
> + for (i = 0; i < ics->icp->nr_servers; i++) {
Ah.. yes. Although it would be a little nicer to avoid the duplicate
calls to ics_resend() that will cause. Still the simple version
should be correct for now. Please resend with this fixed.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Qemu-devel] [PATCH 04/13] pseries: savevm support for XICS interrupt controller
2013-06-05 9:55 ` David Gibson
@ 2013-06-05 11:17 ` Alexey Kardashevskiy
0 siblings, 0 replies; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-05 11:17 UTC (permalink / raw)
To: David Gibson; +Cc: qemu-ppc, qemu-devel, Paul Mackerras, Alexander Graf
On 05.06.2013 19:55, David Gibson wrote:
> On Wed, Jun 05, 2013 at 01:22:02PM +1000, Alexey Kardashevskiy wrote:
>> On 06/04/2013 10:20 PM, Alexey Kardashevskiy wrote:
> [snip]
>>> +static int ics_post_load(void *opaque, int version_id)
>>> +{
>>> + int i;
>>> + struct ics_state *ics = opaque;
>>> +
>>> + for (i = 0; i < ics->nr_irqs; i++) {
>>
>> Obviously here is a bug...
>>
>> - for (i = 0; i < ics->nr_irqs; i++) {
>> + for (i = 0; i < ics->icp->nr_servers; i++) {
>
> Ah.. yes. Although it would be a little nicer to avoid the duplicate
> calls to ics_resend() that will cause. Still the simple version
> should be correct for now. Please resend with this fixed.
I'll wait till Friday (do we expect anyone to comment, besides you?) and
repost the whole series with fixed "from" :)
--
With best regards
Alexey Kardashevskiy -- icq: 52150396
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 05/13] pseries: savevm support for VIO devices
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
` (3 preceding siblings ...)
2013-06-04 12:20 ` [Qemu-devel] [PATCH 04/13] pseries: savevm support for XICS interrupt controller Alexey Kardashevskiy
@ 2013-06-04 12:20 ` Alexey Kardashevskiy
2013-06-04 12:20 ` [Qemu-devel] [PATCH 06/13] pseries: savevm support for PAPR VIO logical lan Alexey Kardashevskiy
` (8 subsequent siblings)
13 siblings, 0 replies; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-04 12:20 UTC (permalink / raw)
To: qemu-devel; +Cc: aik, Alexander Graf, qemu-ppc, Paul Mackerras, David Gibson
From: David Gibson <david@gibson.dropbear.id.au>
This patch adds helpers to allow PAPR VIO devices to save state common
to all VIO devices during savevm.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
hw/ppc/spapr_vio.c | 20 ++++++++++++++++++++
include/hw/ppc/spapr_vio.h | 5 +++++
2 files changed, 25 insertions(+)
diff --git a/hw/ppc/spapr_vio.c b/hw/ppc/spapr_vio.c
index 3cfa326..7ad1dd4 100644
--- a/hw/ppc/spapr_vio.c
+++ b/hw/ppc/spapr_vio.c
@@ -544,6 +544,26 @@ static const TypeInfo spapr_vio_bridge_info = {
.class_init = spapr_vio_bridge_class_init,
};
+const VMStateDescription vmstate_spapr_vio = {
+ .name = "spapr_vio",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ /* Sanity check */
+ VMSTATE_UINT32_EQUAL(reg, VIOsPAPRDevice),
+ VMSTATE_UINT32_EQUAL(irq, VIOsPAPRDevice),
+
+ /* General VIO device state */
+ VMSTATE_UINTTL(signal_state, VIOsPAPRDevice),
+ VMSTATE_UINT64(crq.qladdr, VIOsPAPRDevice),
+ VMSTATE_UINT32(crq.qsize, VIOsPAPRDevice),
+ VMSTATE_UINT32(crq.qnext, VIOsPAPRDevice),
+
+ VMSTATE_END_OF_LIST()
+ },
+};
+
static void vio_spapr_device_class_init(ObjectClass *klass, void *data)
{
DeviceClass *k = DEVICE_CLASS(klass);
diff --git a/include/hw/ppc/spapr_vio.h b/include/hw/ppc/spapr_vio.h
index 3609327..46edc2a 100644
--- a/include/hw/ppc/spapr_vio.h
+++ b/include/hw/ppc/spapr_vio.h
@@ -134,4 +134,9 @@ VIOsPAPRDevice *spapr_vty_get_default(VIOsPAPRBus *bus);
void spapr_vio_quiesce(void);
+extern const VMStateDescription vmstate_spapr_vio;
+
+#define VMSTATE_SPAPR_VIO(_f, _s) \
+ VMSTATE_STRUCT(_f, _s, 0, vmstate_spapr_vio, VIOsPAPRDevice)
+
#endif /* _HW_SPAPR_VIO_H */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 26+ messages in thread* [Qemu-devel] [PATCH 06/13] pseries: savevm support for PAPR VIO logical lan
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
` (4 preceding siblings ...)
2013-06-04 12:20 ` [Qemu-devel] [PATCH 05/13] pseries: savevm support for VIO devices Alexey Kardashevskiy
@ 2013-06-04 12:20 ` Alexey Kardashevskiy
2013-06-04 12:20 ` [Qemu-devel] [PATCH 07/13] pseries: savevm support for PAPR TCE tables Alexey Kardashevskiy
` (7 subsequent siblings)
13 siblings, 0 replies; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-04 12:20 UTC (permalink / raw)
To: qemu-devel; +Cc: aik, Alexander Graf, qemu-ppc, Paul Mackerras, David Gibson
From: David Gibson <david@gibson.dropbear.id.au>
This patch adds the necessary VMStateDescription information to support
savevm/loadvm for the spapr_llan (PAPR logical lan) device.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
hw/char/spapr_vty.c | 16 ++++++++++++++++
hw/net/spapr_llan.c | 24 ++++++++++++++++++++++--
2 files changed, 38 insertions(+), 2 deletions(-)
diff --git a/hw/char/spapr_vty.c b/hw/char/spapr_vty.c
index 2993848..a799721 100644
--- a/hw/char/spapr_vty.c
+++ b/hw/char/spapr_vty.c
@@ -142,6 +142,21 @@ static Property spapr_vty_properties[] = {
DEFINE_PROP_END_OF_LIST(),
};
+static const VMStateDescription vmstate_spapr_vty = {
+ .name = "spapr_vty",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_SPAPR_VIO(sdev, VIOsPAPRVTYDevice),
+
+ VMSTATE_UINT32(in, VIOsPAPRVTYDevice),
+ VMSTATE_UINT32(out, VIOsPAPRVTYDevice),
+ VMSTATE_BUFFER(buf, VIOsPAPRVTYDevice),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
static void spapr_vty_class_init(ObjectClass *klass, void *data)
{
DeviceClass *dc = DEVICE_CLASS(klass);
@@ -152,6 +167,7 @@ static void spapr_vty_class_init(ObjectClass *klass, void *data)
k->dt_type = "serial";
k->dt_compatible = "hvterm1";
dc->props = spapr_vty_properties;
+ dc->vmsd = &vmstate_spapr_vty;
}
static const TypeInfo spapr_vty_info = {
diff --git a/hw/net/spapr_llan.c b/hw/net/spapr_llan.c
index 03a09f2..46f7d5f 100644
--- a/hw/net/spapr_llan.c
+++ b/hw/net/spapr_llan.c
@@ -81,9 +81,9 @@ typedef struct VIOsPAPRVLANDevice {
VIOsPAPRDevice sdev;
NICConf nicconf;
NICState *nic;
- int isopen;
+ bool isopen;
target_ulong buf_list;
- int add_buf_ptr, use_buf_ptr, rx_bufs;
+ uint32_t add_buf_ptr, use_buf_ptr, rx_bufs;
target_ulong rxq_ptr;
} VIOsPAPRVLANDevice;
@@ -500,6 +500,25 @@ static Property spapr_vlan_properties[] = {
DEFINE_PROP_END_OF_LIST(),
};
+static const VMStateDescription vmstate_spapr_llan = {
+ .name = "spapr_llan",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_SPAPR_VIO(sdev, VIOsPAPRVLANDevice),
+ /* LLAN state */
+ VMSTATE_BOOL(isopen, VIOsPAPRVLANDevice),
+ VMSTATE_UINTTL(buf_list, VIOsPAPRVLANDevice),
+ VMSTATE_UINT32(add_buf_ptr, VIOsPAPRVLANDevice),
+ VMSTATE_UINT32(use_buf_ptr, VIOsPAPRVLANDevice),
+ VMSTATE_UINT32(rx_bufs, VIOsPAPRVLANDevice),
+ VMSTATE_UINTTL(rxq_ptr, VIOsPAPRVLANDevice),
+
+ VMSTATE_END_OF_LIST()
+ },
+};
+
static void spapr_vlan_class_init(ObjectClass *klass, void *data)
{
DeviceClass *dc = DEVICE_CLASS(klass);
@@ -514,6 +533,7 @@ static void spapr_vlan_class_init(ObjectClass *klass, void *data)
k->signal_mask = 0x1;
dc->props = spapr_vlan_properties;
k->rtce_window_size = 0x10000000;
+ dc->vmsd = &vmstate_spapr_llan;
}
static const TypeInfo spapr_vlan_info = {
--
1.7.10.4
^ permalink raw reply related [flat|nested] 26+ messages in thread* [Qemu-devel] [PATCH 07/13] pseries: savevm support for PAPR TCE tables
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
` (5 preceding siblings ...)
2013-06-04 12:20 ` [Qemu-devel] [PATCH 06/13] pseries: savevm support for PAPR VIO logical lan Alexey Kardashevskiy
@ 2013-06-04 12:20 ` Alexey Kardashevskiy
2013-06-04 12:21 ` [Qemu-devel] [PATCH 08/13] pseries: rework PAPR virtual SCSI Alexey Kardashevskiy
` (6 subsequent siblings)
13 siblings, 0 replies; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-04 12:20 UTC (permalink / raw)
To: qemu-devel; +Cc: aik, Alexander Graf, qemu-ppc, Paul Mackerras, David Gibson
From: David Gibson <david@gibson.dropbear.id.au>
This patch adds the necessary VMStateDescription information to save the
state of PAPR TCE tables (that is, the PAPR specified IOMMU).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
hw/ppc/spapr_iommu.c | 30 ++++++++++++++++++++++++++++--
1 file changed, 28 insertions(+), 2 deletions(-)
diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c
index 1d11e28..2fdf728 100644
--- a/hw/ppc/spapr_iommu.c
+++ b/hw/ppc/spapr_iommu.c
@@ -110,6 +110,26 @@ static IOMMUTLBEntry spapr_tce_translate_iommu(MemoryRegion *iommu, hwaddr addr)
};
}
+static const VMStateDescription vmstate_spapr_tce_table = {
+ .name = "spapr_iommu",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ /* Sanity check */
+ VMSTATE_UINT32_EQUAL(liobn, sPAPRTCETable),
+ VMSTATE_UINT32_EQUAL(window_size, sPAPRTCETable),
+
+ /* IOMMU state */
+ VMSTATE_BOOL(bypass, sPAPRTCETable),
+ VMSTATE_VBUFFER_DIVIDE(table, sPAPRTCETable, 0, NULL, 0, window_size,
+ SPAPR_TCE_PAGE_SIZE / sizeof(sPAPRTCE)),
+
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+
static MemoryRegionIOMMUOps spapr_iommu_ops = {
.translate = spapr_tce_translate_iommu,
};
@@ -155,6 +175,8 @@ sPAPRTCETable *spapr_tce_new_table(uint32_t liobn, size_t window_size)
QLIST_INSERT_HEAD(&spapr_tce_tables, tcet, list);
+ vmstate_register(NULL, tcet->liobn, &vmstate_spapr_tce_table, tcet);
+
return tcet;
}
@@ -162,9 +184,13 @@ void spapr_tce_free(sPAPRTCETable *tcet)
{
QLIST_REMOVE(tcet, list);
+ vmstate_unregister(NULL, &vmstate_spapr_tce_table, tcet);
+
+ QLIST_REMOVE(tcet, list);
+
if (!kvm_enabled() ||
- (kvmppc_remove_spapr_tce(tcet->table, tcet->fd,
- tcet->window_size) != 0)) {
+ (kvmppc_remove_spapr_tce(tcet->table, tcet->fd,
+ tcet->window_size) != 0)) {
g_free(tcet->table);
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 26+ messages in thread* [Qemu-devel] [PATCH 08/13] pseries: rework PAPR virtual SCSI
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
` (6 preceding siblings ...)
2013-06-04 12:20 ` [Qemu-devel] [PATCH 07/13] pseries: savevm support for PAPR TCE tables Alexey Kardashevskiy
@ 2013-06-04 12:21 ` Alexey Kardashevskiy
2013-06-04 12:21 ` [Qemu-devel] [PATCH 09/13] pseries: savevm support for " Alexey Kardashevskiy
` (5 subsequent siblings)
13 siblings, 0 replies; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-04 12:21 UTC (permalink / raw)
To: qemu-devel
Cc: Alexey Kardashevskiy, Alexander Graf, qemu-ppc, Paul Mackerras,
David Gibson
The patch reimplements handling of indirect requests in order to
simplify upcoming live migration support.
- all pointers (except SCSIRequest*) were replaces with integer
indexes and offsets;
- DMA'ed srp_direct_buf kept untouched (ie. BE format);
- vscsi_fetch_desc() is added, now it is the only place where
descriptors are fetched and byteswapped;
- vscsi_req struct fields converted to migration-friendly types;
- many dprintf()'s fixed.
This also removed an unused field 'lun' from the spapr_vscsi device
which is assigned, but never used. So, remove it.
[David Gibson: removed unused 'lun']
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Cc: David Gibson <david@gibson.dropbear.id.au>
---
hw/scsi/spapr_vscsi.c | 224 +++++++++++++++++++++++++++++--------------------
1 file changed, 131 insertions(+), 93 deletions(-)
diff --git a/hw/scsi/spapr_vscsi.c b/hw/scsi/spapr_vscsi.c
index e8978bf..1e93102 100644
--- a/hw/scsi/spapr_vscsi.c
+++ b/hw/scsi/spapr_vscsi.c
@@ -75,20 +75,19 @@ typedef struct vscsi_req {
/* SCSI request tracking */
SCSIRequest *sreq;
uint32_t qtag; /* qemu tag != srp tag */
- int lun;
- int active;
- long data_len;
- int writing;
- int senselen;
+ bool active;
+ uint32_t data_len;
+ bool writing;
+ uint32_t senselen;
uint8_t sense[SCSI_SENSE_BUF_SIZE];
/* RDMA related bits */
uint8_t dma_fmt;
- struct srp_direct_buf ext_desc;
- struct srp_direct_buf *cur_desc;
- struct srp_indirect_buf *ind_desc;
- int local_desc;
- int total_desc;
+ uint16_t local_desc;
+ uint16_t total_desc;
+ uint16_t cdb_offset;
+ uint16_t cur_desc_num;
+ uint16_t cur_desc_offset;
} vscsi_req;
#define TYPE_VIO_SPAPR_VSCSI_DEVICE "spapr-vscsi"
@@ -264,93 +263,139 @@ static int vscsi_send_rsp(VSCSIState *s, vscsi_req *req,
return 0;
}
-static inline void vscsi_swap_desc(struct srp_direct_buf *desc)
+static inline struct srp_direct_buf vscsi_swap_desc(struct srp_direct_buf desc)
{
- desc->va = be64_to_cpu(desc->va);
- desc->len = be32_to_cpu(desc->len);
+ desc.va = be64_to_cpu(desc.va);
+ desc.len = be32_to_cpu(desc.len);
+ return desc;
+}
+
+static int vscsi_fetch_desc(VSCSIState *s, struct vscsi_req *req,
+ unsigned n, unsigned buf_offset,
+ struct srp_direct_buf *ret)
+{
+ struct srp_cmd *cmd = &req->iu.srp.cmd;
+
+ switch (req->dma_fmt) {
+ case SRP_NO_DATA_DESC: {
+ dprintf("VSCSI: no data descriptor\n");
+ return 0;
+ }
+ case SRP_DATA_DESC_DIRECT: {
+ *ret = *(struct srp_direct_buf *)(cmd->add_data + req->cdb_offset);
+ assert(req->cur_desc_num == 0);
+ dprintf("VSCSI: direct segment");
+ break;
+ }
+ case SRP_DATA_DESC_INDIRECT: {
+ struct srp_indirect_buf *tmp = (struct srp_indirect_buf *)
+ (cmd->add_data + req->cdb_offset);
+ if (n < req->local_desc) {
+ *ret = tmp->desc_list[n];
+ dprintf("VSCSI: indirect segment local tag=0x%x desc#%d/%d",
+ req->qtag, n, req->local_desc);
+
+ } else if (n < req->total_desc) {
+ int rc;
+ struct srp_direct_buf tbl_desc = vscsi_swap_desc(tmp->table_desc);
+ unsigned desc_offset = (n - req->local_desc) *
+ sizeof(struct srp_direct_buf);
+
+ if (desc_offset > tbl_desc.len) {
+ dprintf("VSCSI: #%d is ouf of range (%d bytes)\n",
+ n, desc_offset);
+ return -1;
+ }
+ rc = spapr_vio_dma_read(&s->vdev, tbl_desc.va + desc_offset,
+ ret, sizeof(struct srp_direct_buf));
+ if (rc) {
+ dprintf("VSCSI: spapr_vio_dma_read -> %d reading ext_desc\n",
+ rc);
+ return rc;
+ }
+ dprintf("VSCSI: indirect segment ext. tag=0x%x desc#%d/%d { va=%"PRIx64" len=%x }",
+ req->qtag, n, req->total_desc, tbl_desc.va, tbl_desc.len);
+ } else {
+ dprintf("VSCSI: Out of descriptors !\n");
+ return 0;
+ }
+ break;
+ }
+ default:
+ fprintf(stderr, "VSCSI: Unknown format %x\n", req->dma_fmt);
+ return -1;
+ }
+
+ *ret = vscsi_swap_desc(*ret);
+ if (buf_offset > ret->len) {
+ dprintf(" offset=%x is out of a descriptor #%d boundary=%x\n",
+ buf_offset, req->cur_desc_num, ret->len);
+ return -1;
+ }
+ ret->va += buf_offset;
+ ret->len -= buf_offset;
+
+ dprintf(" cur=%d offs=%x ret { va=%"PRIx64" len=%x }\n",
+ req->cur_desc_num, req->cur_desc_offset, ret->va, ret->len);
+
+ return ret->len ? 1 : 0;
}
static int vscsi_srp_direct_data(VSCSIState *s, vscsi_req *req,
uint8_t *buf, uint32_t len)
{
- struct srp_direct_buf *md = req->cur_desc;
+ struct srp_direct_buf md;
uint32_t llen;
int rc = 0;
- dprintf("VSCSI: direct segment 0x%x bytes, va=0x%llx desc len=0x%x\n",
- len, (unsigned long long)md->va, md->len);
+ rc = vscsi_fetch_desc(s, req, req->cur_desc_num, req->cur_desc_offset, &md);
+ if (rc < 0) {
+ return -1;
+ } else if (rc == 0) {
+ return 0;
+ }
- llen = MIN(len, md->len);
+ llen = MIN(len, md.len);
if (llen) {
if (req->writing) { /* writing = to device = reading from memory */
- rc = spapr_vio_dma_read(&s->vdev, md->va, buf, llen);
+ rc = spapr_vio_dma_read(&s->vdev, md.va, buf, llen);
} else {
- rc = spapr_vio_dma_write(&s->vdev, md->va, buf, llen);
+ rc = spapr_vio_dma_write(&s->vdev, md.va, buf, llen);
}
}
- md->len -= llen;
- md->va += llen;
if (rc) {
return -1;
}
+ req->cur_desc_offset += llen;
+
return llen;
}
static int vscsi_srp_indirect_data(VSCSIState *s, vscsi_req *req,
uint8_t *buf, uint32_t len)
{
- struct srp_direct_buf *td = &req->ind_desc->table_desc;
- struct srp_direct_buf *md = req->cur_desc;
+ struct srp_direct_buf md;
int rc = 0;
uint32_t llen, total = 0;
- dprintf("VSCSI: indirect segment 0x%x bytes, td va=0x%llx len=0x%x\n",
- len, (unsigned long long)td->va, td->len);
+ dprintf("VSCSI: indirect segment 0x%x bytes\n", len);
/* While we have data ... */
while (len) {
- /* If we have a descriptor but it's empty, go fetch a new one */
- if (md && md->len == 0) {
- /* More local available, use one */
- if (req->local_desc) {
- md = ++req->cur_desc;
- --req->local_desc;
- --req->total_desc;
- td->va += sizeof(struct srp_direct_buf);
- } else {
- md = req->cur_desc = NULL;
- }
- }
- /* No descriptor at hand, fetch one */
- if (!md) {
- if (!req->total_desc) {
- dprintf("VSCSI: Out of descriptors !\n");
- break;
- }
- md = req->cur_desc = &req->ext_desc;
- dprintf("VSCSI: Reading desc from 0x%llx\n",
- (unsigned long long)td->va);
- rc = spapr_vio_dma_read(&s->vdev, td->va, md,
- sizeof(struct srp_direct_buf));
- if (rc) {
- dprintf("VSCSI: spapr_vio_dma_read -> %d reading ext_desc\n",
- rc);
- break;
- }
- vscsi_swap_desc(md);
- td->va += sizeof(struct srp_direct_buf);
- --req->total_desc;
+ rc = vscsi_fetch_desc(s, req, req->cur_desc_num, req->cur_desc_offset, &md);
+ if (rc < 0) {
+ return -1;
+ } else if (rc == 0) {
+ break;
}
- dprintf("VSCSI: [desc va=0x%llx,len=0x%x] remaining=0x%x\n",
- (unsigned long long)md->va, md->len, len);
/* Perform transfer */
- llen = MIN(len, md->len);
+ llen = MIN(len, md.len);
if (req->writing) { /* writing = to device = reading from memory */
- rc = spapr_vio_dma_read(&s->vdev, md->va, buf, llen);
+ rc = spapr_vio_dma_read(&s->vdev, md.va, buf, llen);
} else {
- rc = spapr_vio_dma_write(&s->vdev, md->va, buf, llen);
+ rc = spapr_vio_dma_write(&s->vdev, md.va, buf, llen);
}
if (rc) {
dprintf("VSCSI: spapr_vio_dma_r/w(%d) -> %d\n", req->writing, rc);
@@ -361,10 +406,18 @@ static int vscsi_srp_indirect_data(VSCSIState *s, vscsi_req *req,
len -= llen;
buf += llen;
+
total += llen;
- md->va += llen;
- md->len -= llen;
+
+ /* Update current position in the current descriptor */
+ req->cur_desc_offset += llen;
+ if (md.len == llen) {
+ /* Go to the next descriptor if the current one finished */
+ ++req->cur_desc_num;
+ req->cur_desc_offset = 0;
+ }
}
+
return rc ? -1 : total;
}
@@ -412,14 +465,13 @@ static int data_out_desc_size(struct srp_cmd *cmd)
static int vscsi_preprocess_desc(vscsi_req *req)
{
struct srp_cmd *cmd = &req->iu.srp.cmd;
- int offset, i;
- offset = cmd->add_cdb_len & ~3;
+ req->cdb_offset = cmd->add_cdb_len & ~3;
if (req->writing) {
req->dma_fmt = cmd->buf_fmt >> 4;
} else {
- offset += data_out_desc_size(cmd);
+ req->cdb_offset += data_out_desc_size(cmd);
req->dma_fmt = cmd->buf_fmt & ((1U << 4) - 1);
}
@@ -427,31 +479,18 @@ static int vscsi_preprocess_desc(vscsi_req *req)
case SRP_NO_DATA_DESC:
break;
case SRP_DATA_DESC_DIRECT:
- req->cur_desc = (struct srp_direct_buf *)(cmd->add_data + offset);
req->total_desc = req->local_desc = 1;
- vscsi_swap_desc(req->cur_desc);
- dprintf("VSCSI: using direct RDMA %s, 0x%x bytes MD: 0x%llx\n",
- req->writing ? "write" : "read",
- req->cur_desc->len, (unsigned long long)req->cur_desc->va);
break;
- case SRP_DATA_DESC_INDIRECT:
- req->ind_desc = (struct srp_indirect_buf *)(cmd->add_data + offset);
- vscsi_swap_desc(&req->ind_desc->table_desc);
- req->total_desc = req->ind_desc->table_desc.len /
- sizeof(struct srp_direct_buf);
+ case SRP_DATA_DESC_INDIRECT: {
+ struct srp_indirect_buf *ind_tmp = (struct srp_indirect_buf *)
+ (cmd->add_data + req->cdb_offset);
+
+ req->total_desc = be32_to_cpu(ind_tmp->table_desc.len) /
+ sizeof(struct srp_direct_buf);
req->local_desc = req->writing ? cmd->data_out_desc_cnt :
- cmd->data_in_desc_cnt;
- for (i = 0; i < req->local_desc; i++) {
- vscsi_swap_desc(&req->ind_desc->desc_list[i]);
- }
- req->cur_desc = req->local_desc ? &req->ind_desc->desc_list[0] : NULL;
- dprintf("VSCSI: using indirect RDMA %s, 0x%x bytes %d descs "
- "(%d local) VA: 0x%llx\n",
- req->writing ? "read" : "write",
- be32_to_cpu(req->ind_desc->len),
- req->total_desc, req->local_desc,
- (unsigned long long)req->ind_desc->table_desc.va);
+ cmd->data_in_desc_cnt;
break;
+ }
default:
fprintf(stderr,
"vscsi_preprocess_desc: Unknown format %x\n", req->dma_fmt);
@@ -499,8 +538,8 @@ static void vscsi_command_complete(SCSIRequest *sreq, uint32_t status, size_t re
vscsi_req *req = sreq->hba_private;
int32_t res_in = 0, res_out = 0;
- dprintf("VSCSI: SCSI cmd complete, r=0x%x tag=0x%x status=0x%x, req=%p\n",
- reason, sreq->tag, status, req);
+ dprintf("VSCSI: SCSI cmd complete, tag=0x%x status=0x%x, req=%p\n",
+ sreq->tag, status, req);
if (req == NULL) {
fprintf(stderr, "VSCSI: Can't find request for tag 0x%x\n", sreq->tag);
return;
@@ -509,7 +548,7 @@ static void vscsi_command_complete(SCSIRequest *sreq, uint32_t status, size_t re
if (status == CHECK_CONDITION) {
req->senselen = scsi_req_get_sense(req->sreq, req->sense,
sizeof(req->sense));
- dprintf("VSCSI: Sense data, %d bytes:\n", len);
+ dprintf("VSCSI: Sense data, %d bytes:\n", req->senselen);
dprintf(" %02x %02x %02x %02x %02x %02x %02x %02x\n",
req->sense[0], req->sense[1], req->sense[2], req->sense[3],
req->sense[4], req->sense[5], req->sense[6], req->sense[7]);
@@ -621,12 +660,11 @@ static int vscsi_queue_cmd(VSCSIState *s, vscsi_req *req)
} return 1;
}
- req->lun = lun;
req->sreq = scsi_req_new(sdev, req->qtag, lun, srp->cmd.cdb, req);
n = scsi_req_enqueue(req->sreq);
- dprintf("VSCSI: Queued command tag 0x%x CMD 0x%x ID %d LUN %d ret: %d\n",
- req->qtag, srp->cmd.cdb[0], id, lun, n);
+ dprintf("VSCSI: Queued command tag 0x%x CMD 0x%x LUN %d ret: %d\n",
+ req->qtag, srp->cmd.cdb[0], lun, n);
if (n) {
/* Transfer direction must be set before preprocessing the
--
1.7.10.4
^ permalink raw reply related [flat|nested] 26+ messages in thread* [Qemu-devel] [PATCH 09/13] pseries: savevm support for PAPR virtual SCSI
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
` (7 preceding siblings ...)
2013-06-04 12:21 ` [Qemu-devel] [PATCH 08/13] pseries: rework PAPR virtual SCSI Alexey Kardashevskiy
@ 2013-06-04 12:21 ` Alexey Kardashevskiy
2013-06-04 12:21 ` [Qemu-devel] [PATCH 10/13] pseries: savevm support for pseries machine Alexey Kardashevskiy
` (4 subsequent siblings)
13 siblings, 0 replies; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-04 12:21 UTC (permalink / raw)
To: qemu-devel
Cc: Alexey Kardashevskiy, Alexander Graf, qemu-ppc, Paul Mackerras,
David Gibson
Author: David Gibson <david@gibson.dropbear.id.au>
This patch adds the necessary support for saving the state of the PAPR VIO
virtual SCSI device. This also saves and restores active SCSI requests.
[David Gibson: implemented vscsi device vmsd]
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Cc: David Gibson <david@gibson.dropbear.id.au>
---
hw/scsi/spapr_vscsi.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 81 insertions(+), 1 deletion(-)
diff --git a/hw/scsi/spapr_vscsi.c b/hw/scsi/spapr_vscsi.c
index 1e93102..4db3a47 100644
--- a/hw/scsi/spapr_vscsi.c
+++ b/hw/scsi/spapr_vscsi.c
@@ -579,6 +579,69 @@ static void vscsi_request_cancelled(SCSIRequest *sreq)
vscsi_put_req(req);
}
+static const VMStateDescription vmstate_spapr_vscsi_req = {
+ .name = "spapr_vscsi_req",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_BUFFER(crq.raw, vscsi_req),
+ VMSTATE_BUFFER(iu.srp.reserved, vscsi_req),
+ VMSTATE_UINT32(qtag, vscsi_req),
+ VMSTATE_BOOL(active, vscsi_req),
+ VMSTATE_UINT32(data_len, vscsi_req),
+ VMSTATE_BOOL(writing, vscsi_req),
+ VMSTATE_UINT32(senselen, vscsi_req),
+ VMSTATE_BUFFER(sense, vscsi_req),
+ VMSTATE_UINT8(dma_fmt, vscsi_req),
+ VMSTATE_UINT16(local_desc, vscsi_req),
+ VMSTATE_UINT16(total_desc, vscsi_req),
+ VMSTATE_UINT16(cdb_offset, vscsi_req),
+ /*Restart SCSI request from the beginning for now */
+ /*VMSTATE_UINT16(cur_desc_num, vscsi_req),
+ VMSTATE_UINT16(cur_desc_offset, vscsi_req),*/
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static void vscsi_save_request(QEMUFile *f, SCSIRequest *sreq)
+{
+ vscsi_req *req = sreq->hba_private;
+ assert(req->active);
+
+ vmstate_save_state(f, &vmstate_spapr_vscsi_req, req);
+
+ dprintf("VSCSI: saving tag=%u, current desc#%d, offset=%x\n",
+ req->qtag, req->cur_desc_num, req->cur_desc_offset);
+}
+
+static void *vscsi_load_request(QEMUFile *f, SCSIRequest *sreq)
+{
+ SCSIBus *bus = sreq->bus;
+ VSCSIState *s = VIO_SPAPR_VSCSI_DEVICE(bus->qbus.parent);
+ vscsi_req *req;
+ int rc;
+
+ assert(sreq->tag < VSCSI_REQ_LIMIT);
+ req = &s->reqs[sreq->tag];
+ assert(!req->active);
+
+ memset(req, 0, sizeof(*req));
+ rc = vmstate_load_state(f, &vmstate_spapr_vscsi_req, req, 1);
+ if (rc) {
+ fprintf(stderr, "VSCSI: failed loading request tag#%u\n", sreq->tag);
+ return NULL;
+ }
+ assert(req->active);
+
+ req->sreq = scsi_req_ref(sreq);
+
+ dprintf("VSCSI: restoring tag=%u, current desc#%d, offset=%x\n",
+ req->qtag, req->cur_desc_num, req->cur_desc_offset);
+
+ return req;
+}
+
static void vscsi_process_login(VSCSIState *s, vscsi_req *req)
{
union viosrp_iu *iu = &req->iu;
@@ -933,7 +996,9 @@ static const struct SCSIBusInfo vscsi_scsi_info = {
.transfer_data = vscsi_transfer_data,
.complete = vscsi_command_complete,
- .cancel = vscsi_request_cancelled
+ .cancel = vscsi_request_cancelled,
+ .save_request = vscsi_save_request,
+ .load_request = vscsi_load_request,
};
static void spapr_vscsi_reset(VIOsPAPRDevice *dev)
@@ -992,6 +1057,20 @@ static Property spapr_vscsi_properties[] = {
DEFINE_PROP_END_OF_LIST(),
};
+static const VMStateDescription vmstate_spapr_vscsi = {
+ .name = "spapr_vscsi",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_SPAPR_VIO(vdev, VSCSIState),
+ /* VSCSI state */
+ /* ???? */
+
+ VMSTATE_END_OF_LIST()
+ },
+};
+
static void spapr_vscsi_class_init(ObjectClass *klass, void *data)
{
DeviceClass *dc = DEVICE_CLASS(klass);
@@ -1006,6 +1085,7 @@ static void spapr_vscsi_class_init(ObjectClass *klass, void *data)
k->signal_mask = 0x00000001;
dc->props = spapr_vscsi_properties;
k->rtce_window_size = 0x10000000;
+ dc->vmsd = &vmstate_spapr_vscsi;
}
static const TypeInfo spapr_vscsi_info = {
--
1.7.10.4
^ permalink raw reply related [flat|nested] 26+ messages in thread* [Qemu-devel] [PATCH 10/13] pseries: savevm support for pseries machine
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
` (8 preceding siblings ...)
2013-06-04 12:21 ` [Qemu-devel] [PATCH 09/13] pseries: savevm support for " Alexey Kardashevskiy
@ 2013-06-04 12:21 ` Alexey Kardashevskiy
2013-06-04 12:21 ` [Qemu-devel] [PATCH 11/13] pseries: savevm support for PCI host bridge Alexey Kardashevskiy
` (3 subsequent siblings)
13 siblings, 0 replies; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-04 12:21 UTC (permalink / raw)
To: qemu-devel; +Cc: aik, Alexander Graf, qemu-ppc, Paul Mackerras, David Gibson
From: David Gibson <david@gibson.dropbear.id.au>
This adds the necessary pieces to implement savevm / migration for the
pseries machine. The most complex part here is migrating the hash
table - for the paravirtualized pseries machine the guest's hash page
table is not stored within guest memory, but externally and the guest
accesses it via hypercalls.
This patch uses a hypervisor reserved bit of the HPTE as a dirty bit
(tracking changes to the HPTE itself, not the page it references).
This is used to implement a live migration style incremental save and
restore of the hash table contents.
In addition it adds VMStateDescription information to save and restore
the (few) remaining pieces of state information needed by the pseries
machine.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
hw/ppc/spapr.c | 269 +++++++++++++++++++++++++++++++++++++++++++++++-
hw/ppc/spapr_hcall.c | 8 +-
include/hw/ppc/spapr.h | 12 ++-
3 files changed, 281 insertions(+), 8 deletions(-)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 218ea23..8916142 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -32,6 +32,7 @@
#include "sysemu/cpus.h"
#include "sysemu/kvm.h"
#include "kvm_ppc.h"
+#include "mmu-hash64.h"
#include "hw/boards.h"
#include "hw/ppc/ppc.h"
@@ -667,7 +668,7 @@ static void spapr_cpu_reset(void *opaque)
env->spr[SPR_HIOR] = 0;
- env->external_htab = spapr->htab;
+ env->external_htab = (uint8_t *)spapr->htab;
env->htab_base = -1;
env->htab_mask = HTAB_SIZE(spapr) - 1;
env->spr[SPR_SDR1] = (unsigned long)spapr->htab |
@@ -719,6 +720,268 @@ static int spapr_vga_init(PCIBus *pci_bus)
}
}
+static const VMStateDescription vmstate_spapr = {
+ .name = "spapr",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_UINT32(next_irq, sPAPREnvironment),
+
+ /* RTC offset */
+ VMSTATE_UINT64(rtc_offset, sPAPREnvironment),
+
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+#define HPTE(_table, _i) (void *)(((uint64_t *)(_table)) + ((_i) * 2))
+#define HPTE_VALID(_hpte) (tswap64(*((uint64_t *)(_hpte))) & HPTE64_V_VALID)
+#define HPTE_DIRTY(_hpte) (tswap64(*((uint64_t *)(_hpte))) & HPTE64_V_HPTE_DIRTY)
+#define CLEAN_HPTE(_hpte) ((*(uint64_t *)(_hpte)) &= tswap64(~HPTE64_V_HPTE_DIRTY))
+
+static int htab_save_setup(QEMUFile *f, void *opaque)
+{
+ sPAPREnvironment *spapr = opaque;
+
+ spapr->htab_save_index = 0;
+ spapr->htab_first_pass = true;
+
+ /* "Iteration" header */
+ qemu_put_be32(f, spapr->htab_shift);
+
+ return 0;
+}
+
+#define MAX_ITERATION_NS 5000000 /* 5 ms */
+
+static void htab_save_first_pass(QEMUFile *f, sPAPREnvironment *spapr,
+ int64_t max_ns)
+{
+ int htabslots = HTAB_SIZE(spapr) / HASH_PTE_SIZE_64;
+ int index = spapr->htab_save_index;
+ int64_t starttime = qemu_get_clock_ns(rt_clock);
+
+ assert(spapr->htab_first_pass);
+
+ do {
+ int chunkstart;
+
+ /* Consume invalid HPTEs */
+ while ((index < htabslots)
+ && !HPTE_VALID(HPTE(spapr->htab, index))) {
+ index++;
+ CLEAN_HPTE(HPTE(spapr->htab, index));
+ }
+
+ /* Consume valid HPTEs */
+ chunkstart = index;
+ while ((index < htabslots)
+ && HPTE_VALID(HPTE(spapr->htab, index))) {
+ index++;
+ CLEAN_HPTE(HPTE(spapr->htab, index));
+ }
+
+ if (index > chunkstart) {
+ int n_valid = index - chunkstart;
+
+ qemu_put_be32(f, chunkstart);
+ qemu_put_be16(f, n_valid);
+ qemu_put_be16(f, 0);
+ qemu_put_buffer(f, HPTE(spapr->htab, chunkstart),
+ HASH_PTE_SIZE_64 * n_valid);
+
+ if ((qemu_get_clock_ns(rt_clock) - starttime) > max_ns) {
+ break;
+ }
+ }
+ } while ((index < htabslots) && !qemu_file_rate_limit(f));
+
+ if (index >= htabslots) {
+ assert(index == htabslots);
+ index = 0;
+ spapr->htab_first_pass = false;
+ }
+ spapr->htab_save_index = index;
+}
+
+static bool htab_save_later_pass(QEMUFile *f, sPAPREnvironment *spapr,
+ int64_t max_ns)
+{
+ bool final = max_ns < 0;
+ int htabslots = HTAB_SIZE(spapr) / HASH_PTE_SIZE_64;
+ int examined = 0, sent = 0;
+ int index = spapr->htab_save_index;
+ int64_t starttime = qemu_get_clock_ns(rt_clock);
+
+ assert(!spapr->htab_first_pass);
+
+ do {
+ int chunkstart, invalidstart;
+
+ /* Consume non-dirty HPTEs */
+ while ((index < htabslots)
+ && !HPTE_DIRTY(HPTE(spapr->htab, index))) {
+ index++;
+ examined++;
+ }
+
+ chunkstart = index;
+ /* Consume valid dirty HPTEs */
+ while ((index < htabslots)
+ && HPTE_DIRTY(HPTE(spapr->htab, index))
+ && HPTE_VALID(HPTE(spapr->htab, index))) {
+ CLEAN_HPTE(HPTE(spapr->htab, index));
+ index++;
+ examined++;
+ }
+
+ invalidstart = index;
+ /* Consume invalid dirty HPTEs */
+ while ((index < htabslots)
+ && HPTE_DIRTY(HPTE(spapr->htab, index))
+ && !HPTE_VALID(HPTE(spapr->htab, index))) {
+ CLEAN_HPTE(HPTE(spapr->htab, index));
+ index++;
+ examined++;
+ }
+
+ if (index > chunkstart) {
+ int n_valid = invalidstart - chunkstart;
+ int n_invalid = index - invalidstart;
+
+ qemu_put_be32(f, chunkstart);
+ qemu_put_be16(f, n_valid);
+ qemu_put_be16(f, n_invalid);
+ qemu_put_buffer(f, HPTE(spapr->htab, chunkstart),
+ HASH_PTE_SIZE_64 * n_valid);
+ sent += index - chunkstart;
+
+ if (!final && (qemu_get_clock_ns(rt_clock) - starttime) > max_ns) {
+ break;
+ }
+ }
+
+ if (examined >= htabslots) {
+ break;
+ }
+
+ if (index >= htabslots) {
+ assert(index == htabslots);
+ index = 0;
+ }
+ } while ((examined < htabslots) && (!qemu_file_rate_limit(f) || final));
+
+ if (index >= htabslots) {
+ assert(index == htabslots);
+ index = 0;
+ }
+
+ spapr->htab_save_index = index;
+
+ return (examined >= htabslots) && (sent == 0);
+}
+
+static int htab_save_iterate(QEMUFile *f, void *opaque)
+{
+ sPAPREnvironment *spapr = opaque;
+ bool nothingleft = false;;
+
+ /* Iteration header */
+ qemu_put_be32(f, 0);
+
+ if (spapr->htab_first_pass) {
+ htab_save_first_pass(f, spapr, MAX_ITERATION_NS);
+ } else {
+ nothingleft = htab_save_later_pass(f, spapr, MAX_ITERATION_NS);
+ }
+
+ /* End marker */
+ qemu_put_be32(f, 0);
+ qemu_put_be16(f, 0);
+ qemu_put_be16(f, 0);
+
+ return nothingleft ? 1 : 0;
+}
+
+static int htab_save_complete(QEMUFile *f, void *opaque)
+{
+ sPAPREnvironment *spapr = opaque;
+
+ /* Iteration header */
+ qemu_put_be32(f, 0);
+
+ htab_save_later_pass(f, spapr, -1);
+
+ /* End marker */
+ qemu_put_be32(f, 0);
+ qemu_put_be16(f, 0);
+ qemu_put_be16(f, 0);
+
+ return 0;
+}
+
+static int htab_load(QEMUFile *f, void *opaque, int version_id)
+{
+ sPAPREnvironment *spapr = opaque;
+ uint32_t section_hdr;
+
+ if (version_id < 1 || version_id > 1) {
+ fprintf(stderr, "htab_load() bad version\n");
+ return -EINVAL;
+ }
+
+ section_hdr = qemu_get_be32(f);
+
+ if (section_hdr) {
+ /* First section, just the hash shift */
+ if (spapr->htab_shift != section_hdr) {
+ return -EINVAL;
+ }
+ return 0;
+ }
+
+ while (true) {
+ uint32_t index;
+ uint16_t n_valid, n_invalid;
+
+ index = qemu_get_be32(f);
+ n_valid = qemu_get_be16(f);
+ n_invalid = qemu_get_be16(f);
+
+ if ((index == 0) && (n_valid == 0) && (n_invalid == 0)) {
+ /* End of Stream */
+ break;
+ }
+
+ if ((index + n_valid + n_invalid) >=
+ (HTAB_SIZE(spapr) / HASH_PTE_SIZE_64)) {
+ /* Bad index in stream */
+ fprintf(stderr, "htab_load() bad index %d (%hd+%hd entries) "
+ "in htab stream\n", index, n_valid, n_invalid);
+ return -EINVAL;
+ }
+
+ if (n_valid) {
+ qemu_get_buffer(f, HPTE(spapr->htab, index),
+ HASH_PTE_SIZE_64 * n_valid);
+ }
+ if (n_invalid) {
+ memset(HPTE(spapr->htab, index + n_valid), 0,
+ HASH_PTE_SIZE_64 * n_invalid);
+ }
+ }
+
+ return 0;
+}
+
+static SaveVMHandlers savevm_htab_handlers = {
+ .save_live_setup = htab_save_setup,
+ .save_live_iterate = htab_save_iterate,
+ .save_live_complete = htab_save_complete,
+ .load_state = htab_load,
+};
+
/* pSeries LPAR / sPAPR hardware init */
static void ppc_spapr_init(QEMUMachineInitArgs *args)
{
@@ -959,6 +1222,10 @@ static void ppc_spapr_init(QEMUMachineInitArgs *args)
spapr->entry_point = 0x100;
+ vmstate_register(NULL, 0, &vmstate_spapr, spapr);
+ register_savevm_live(NULL, "spapr/htab", -1, 1,
+ &savevm_htab_handlers, spapr);
+
/* Prepare the device tree */
spapr->fdt_skel = spapr_create_fdt_skel(cpu_model,
initrd_base, initrd_size,
diff --git a/hw/ppc/spapr_hcall.c b/hw/ppc/spapr_hcall.c
index 8f0b7e8..8ceae3c 100644
--- a/hw/ppc/spapr_hcall.c
+++ b/hw/ppc/spapr_hcall.c
@@ -115,7 +115,7 @@ static target_ulong h_enter(PowerPCCPU *cpu, sPAPREnvironment *spapr,
}
ppc_hash64_store_hpte1(env, hpte, ptel);
/* eieio(); FIXME: need some sort of barrier for smp? */
- ppc_hash64_store_hpte0(env, hpte, pteh);
+ ppc_hash64_store_hpte0(env, hpte, pteh | HPTE64_V_HPTE_DIRTY);
args[0] = pte_index + i;
return H_SUCCESS;
@@ -152,7 +152,7 @@ static target_ulong remove_hpte(CPUPPCState *env, target_ulong ptex,
}
*vp = v;
*rp = r;
- ppc_hash64_store_hpte0(env, hpte, 0);
+ ppc_hash64_store_hpte0(env, hpte, HPTE64_V_HPTE_DIRTY);
rb = compute_tlbie_rb(v, r, ptex);
ppc_tlb_invalidate_one(env, rb);
return REMOVE_SUCCESS;
@@ -282,11 +282,11 @@ static target_ulong h_protect(PowerPCCPU *cpu, sPAPREnvironment *spapr,
r |= (flags << 48) & HPTE64_R_KEY_HI;
r |= flags & (HPTE64_R_PP | HPTE64_R_N | HPTE64_R_KEY_LO);
rb = compute_tlbie_rb(v, r, pte_index);
- ppc_hash64_store_hpte0(env, hpte, v & ~HPTE64_V_VALID);
+ ppc_hash64_store_hpte0(env, hpte, (v & ~HPTE64_V_VALID) | HPTE64_V_HPTE_DIRTY);
ppc_tlb_invalidate_one(env, rb);
ppc_hash64_store_hpte1(env, hpte, r);
/* Don't need a memory barrier, due to qemu's global lock */
- ppc_hash64_store_hpte0(env, hpte, v);
+ ppc_hash64_store_hpte0(env, hpte, v | HPTE64_V_HPTE_DIRTY);
return H_SUCCESS;
}
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index a83720e..f59a581 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -9,6 +9,8 @@ struct sPAPRPHBState;
struct sPAPRNVRAM;
struct icp_state;
+#define HPTE64_V_HPTE_DIRTY 0x0000000000000040ULL
+
typedef struct sPAPREnvironment {
struct VIOsPAPRBus *vio_bus;
QLIST_HEAD(, sPAPRPHBState) phbs;
@@ -17,20 +19,24 @@ typedef struct sPAPREnvironment {
hwaddr ram_limit;
void *htab;
- long htab_shift;
+ uint32_t htab_shift;
hwaddr rma_size;
int vrma_adjust;
hwaddr fdt_addr, rtas_addr;
long rtas_size;
void *fdt_skel;
target_ulong entry_point;
- int next_irq;
- int rtc_offset;
+ uint32_t next_irq;
+ uint64_t rtc_offset;
char *cpu_model;
bool has_graphics;
uint32_t epow_irq;
Notifier epow_notifier;
+
+ /* Migration state */
+ int htab_save_index;
+ bool htab_first_pass;
} sPAPREnvironment;
#define H_SUCCESS 0
--
1.7.10.4
^ permalink raw reply related [flat|nested] 26+ messages in thread* [Qemu-devel] [PATCH 11/13] pseries: savevm support for PCI host bridge
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
` (9 preceding siblings ...)
2013-06-04 12:21 ` [Qemu-devel] [PATCH 10/13] pseries: savevm support for pseries machine Alexey Kardashevskiy
@ 2013-06-04 12:21 ` Alexey Kardashevskiy
2013-06-05 10:00 ` David Gibson
2013-06-04 12:21 ` [Qemu-devel] [PATCH 12/13] target-ppc: Add helper for KVM_PPC_RTAS_DEFINE_TOKEN Alexey Kardashevskiy
` (2 subsequent siblings)
13 siblings, 1 reply; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-04 12:21 UTC (permalink / raw)
To: qemu-devel
Cc: Alexey Kardashevskiy, Alexander Graf, qemu-ppc, Paul Mackerras,
David Gibson
Author: David Gibson <david@gibson.dropbear.id.au>
This adds the necessary support for saving the state of the PAPR virtual
PCI host bridge (or host bridges).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
hw/ppc/spapr_pci.c | 49 +++++++++++++++++++++++++++++++++++++++++++
include/hw/pci-host/spapr.h | 6 +++---
2 files changed, 52 insertions(+), 3 deletions(-)
diff --git a/hw/ppc/spapr_pci.c b/hw/ppc/spapr_pci.c
index 762db62..a50a45a 100644
--- a/hw/ppc/spapr_pci.c
+++ b/hw/ppc/spapr_pci.c
@@ -694,6 +694,54 @@ static Property spapr_phb_properties[] = {
DEFINE_PROP_END_OF_LIST(),
};
+static const VMStateDescription vmstate_spapr_pci_lsi = {
+ .name = "spapr_pci/lsi",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_UINT32_EQUAL(irq, struct spapr_pci_lsi),
+
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static const VMStateDescription vmstate_spapr_pci_msi = {
+ .name = "spapr_pci/lsi",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_UINT32(config_addr, struct spapr_pci_msi),
+ VMSTATE_UINT32(irq, struct spapr_pci_msi),
+ VMSTATE_UINT32(nvec, struct spapr_pci_msi),
+
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static const VMStateDescription vmstate_spapr_pci = {
+ .name = "spapr_pci",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .minimum_version_id_old = 1,
+ .fields = (VMStateField []) {
+ VMSTATE_UINT64_EQUAL(buid, sPAPRPHBState),
+ VMSTATE_UINT32_EQUAL(dma_liobn, sPAPRPHBState),
+ VMSTATE_UINT64_EQUAL(mem_win_addr, sPAPRPHBState),
+ VMSTATE_UINT64_EQUAL(mem_win_size, sPAPRPHBState),
+ VMSTATE_UINT64_EQUAL(io_win_addr, sPAPRPHBState),
+ VMSTATE_UINT64_EQUAL(io_win_size, sPAPRPHBState),
+ VMSTATE_UINT64_EQUAL(msi_win_addr, sPAPRPHBState),
+ VMSTATE_STRUCT_ARRAY(lsi_table, sPAPRPHBState, PCI_NUM_PINS, 0,
+ vmstate_spapr_pci_lsi, struct spapr_pci_lsi),
+ VMSTATE_STRUCT_ARRAY(msi_table, sPAPRPHBState, SPAPR_MSIX_MAX_DEVS, 0,
+ vmstate_spapr_pci_msi, struct spapr_pci_msi),
+
+ VMSTATE_END_OF_LIST()
+ },
+};
+
static void spapr_phb_class_init(ObjectClass *klass, void *data)
{
SysBusDeviceClass *sdc = SYS_BUS_DEVICE_CLASS(klass);
@@ -702,6 +750,7 @@ static void spapr_phb_class_init(ObjectClass *klass, void *data)
sdc->init = spapr_phb_init;
dc->props = spapr_phb_properties;
dc->reset = spapr_phb_reset;
+ dc->vmsd = &vmstate_spapr_pci;
}
static const TypeInfo spapr_phb_info = {
diff --git a/include/hw/pci-host/spapr.h b/include/hw/pci-host/spapr.h
index 1e23dbf..93f9511 100644
--- a/include/hw/pci-host/spapr.h
+++ b/include/hw/pci-host/spapr.h
@@ -52,14 +52,14 @@ typedef struct sPAPRPHBState {
sPAPRTCETable *tcet;
AddressSpace iommu_as;
- struct {
+ struct spapr_pci_lsi {
uint32_t irq;
} lsi_table[PCI_NUM_PINS];
- struct {
+ struct spapr_pci_msi {
uint32_t config_addr;
uint32_t irq;
- int nvec;
+ uint32_t nvec;
} msi_table[SPAPR_MSIX_MAX_DEVS];
QLIST_ENTRY(sPAPRPHBState) list;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 26+ messages in thread* Re: [Qemu-devel] [PATCH 11/13] pseries: savevm support for PCI host bridge
2013-06-04 12:21 ` [Qemu-devel] [PATCH 11/13] pseries: savevm support for PCI host bridge Alexey Kardashevskiy
@ 2013-06-05 10:00 ` David Gibson
2013-06-05 10:57 ` Alexey Kardashevskiy
0 siblings, 1 reply; 26+ messages in thread
From: David Gibson @ 2013-06-05 10:00 UTC (permalink / raw)
To: Alexey Kardashevskiy; +Cc: qemu-ppc, qemu-devel, Paul Mackerras, Alexander Graf
[-- Attachment #1: Type: text/plain, Size: 605 bytes --]
On Tue, Jun 04, 2013 at 10:21:03PM +1000, Alexey Kardashevskiy wrote:
> Author: David Gibson <david@gibson.dropbear.id.au>
>
> This adds the necessary support for saving the state of the PAPR virtual
> PCI host bridge (or host bridges).
I think the PCI cleanup patches I was working on need to go in before
this, to make sure the migration stream pieces for PCI have same paths
from the beginning.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH 11/13] pseries: savevm support for PCI host bridge
2013-06-05 10:00 ` David Gibson
@ 2013-06-05 10:57 ` Alexey Kardashevskiy
2013-06-05 23:22 ` David Gibson
0 siblings, 1 reply; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-05 10:57 UTC (permalink / raw)
To: David Gibson; +Cc: qemu-ppc, qemu-devel, Paul Mackerras, Alexander Graf
On 05.06.2013 20:00, David Gibson wrote:
> On Tue, Jun 04, 2013 at 10:21:03PM +1000, Alexey Kardashevskiy wrote:
>> Author: David Gibson <david@gibson.dropbear.id.au>
>>
>> This adds the necessary support for saving the state of the PAPR virtual
>> PCI host bridge (or host bridges).
>
> I think the PCI cleanup patches I was working on need to go in before
> this, to make sure the migration stream pieces for PCI have same paths
> from the beginning.
We really want migration to go upstream... Are any close to get PCI
stuff upstream? I did not see updates for a while and could not find
anything new in your git on github.
--
With best regards
Alexey Kardashevskiy -- icq: 52150396
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH 11/13] pseries: savevm support for PCI host bridge
2013-06-05 10:57 ` Alexey Kardashevskiy
@ 2013-06-05 23:22 ` David Gibson
0 siblings, 0 replies; 26+ messages in thread
From: David Gibson @ 2013-06-05 23:22 UTC (permalink / raw)
To: Alexey Kardashevskiy; +Cc: qemu-ppc, qemu-devel, Paul Mackerras, Alexander Graf
[-- Attachment #1: Type: text/plain, Size: 1090 bytes --]
On Wed, Jun 05, 2013 at 08:57:29PM +1000, Alexey Kardashevskiy wrote:
> On 05.06.2013 20:00, David Gibson wrote:
> > On Tue, Jun 04, 2013 at 10:21:03PM +1000, Alexey Kardashevskiy wrote:
> >> Author: David Gibson <david@gibson.dropbear.id.au>
> >>
> >> This adds the necessary support for saving the state of the PAPR virtual
> >> PCI host bridge (or host bridges).
> >
> > I think the PCI cleanup patches I was working on need to go in before
> > this, to make sure the migration stream pieces for PCI have same paths
> > from the beginning.
>
> We really want migration to go upstream... Are any close to get PCI
> stuff upstream? I did not see updates for a while and could not find
> anything new in your git on github.
I've been trying to find time to address Michael Tsirkin's remaining
comments. I got a fair ways last night, so I hope I can repost these
shortly.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 12/13] target-ppc: Add helper for KVM_PPC_RTAS_DEFINE_TOKEN
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
` (10 preceding siblings ...)
2013-06-04 12:21 ` [Qemu-devel] [PATCH 11/13] pseries: savevm support for PCI host bridge Alexey Kardashevskiy
@ 2013-06-04 12:21 ` Alexey Kardashevskiy
2013-06-05 10:02 ` David Gibson
2013-06-04 12:21 ` [Qemu-devel] [PATCH 13/13] pseries: savevm support with KVM Alexey Kardashevskiy
2013-06-04 12:42 ` [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Andreas Färber
13 siblings, 1 reply; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-04 12:21 UTC (permalink / raw)
To: qemu-devel; +Cc: aik, Alexander Graf, qemu-ppc, Paul Mackerras, David Gibson
From: David Gibson <david@gibson.dropbear.id.au>
Recent PowerKVM allows the kernel to intercept some RTAS calls from the
guest directly. This is used to implement the more efficient in-kernel
XICS for example. qemu is still responsible for assigning the RTAS token
numbers however, and needs to tell the kernel which RTAS function name is
assigned to a given token value. This patch adds a convenience wrapper for
the KVM_PPC_RTAS_DEFINE_TOKEN ioctl() which is used for this purpose.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
target-ppc/kvm.c | 14 ++++++++++++++
target-ppc/kvm_ppc.h | 7 +++++++
2 files changed, 21 insertions(+)
diff --git a/target-ppc/kvm.c b/target-ppc/kvm.c
index c89dd58..33ddf63 100644
--- a/target-ppc/kvm.c
+++ b/target-ppc/kvm.c
@@ -1787,6 +1787,20 @@ static int kvm_ppc_register_host_cpu_type(void)
return 0;
}
+int kvmppc_define_rtas_token(uint32_t token, const char *function)
+{
+ struct kvm_rtas_token_args args = {
+ .token = token,
+ };
+
+ if (!kvm_check_extension(kvm_state, KVM_CAP_PPC_RTAS)) {
+ return -ENOENT;
+ }
+
+ strncpy(args.name, function, sizeof(args.name));
+
+ return kvm_vm_ioctl(kvm_state, KVM_PPC_RTAS_DEFINE_TOKEN, &args);
+}
bool kvm_arch_stop_on_emulation_error(CPUState *cpu)
{
diff --git a/target-ppc/kvm_ppc.h b/target-ppc/kvm_ppc.h
index 771cfbe..21939a8 100644
--- a/target-ppc/kvm_ppc.h
+++ b/target-ppc/kvm_ppc.h
@@ -38,6 +38,7 @@ uint64_t kvmppc_rma_size(uint64_t current_size, unsigned int hash_shift);
#endif /* !CONFIG_USER_ONLY */
int kvmppc_fixup_cpu(PowerPCCPU *cpu);
bool kvmppc_has_cap_epr(void);
+int kvmppc_define_rtas_token(uint32_t token, const char *function);
#else
@@ -159,6 +160,12 @@ static inline bool kvmppc_has_cap_epr(void)
{
return false;
}
+
+static inline int kvmppc_define_rtas_token(uint32_t token,
+ const char *function)
+{
+ return -1;
+}
#endif
#ifndef CONFIG_KVM
--
1.7.10.4
^ permalink raw reply related [flat|nested] 26+ messages in thread* Re: [Qemu-devel] [PATCH 12/13] target-ppc: Add helper for KVM_PPC_RTAS_DEFINE_TOKEN
2013-06-04 12:21 ` [Qemu-devel] [PATCH 12/13] target-ppc: Add helper for KVM_PPC_RTAS_DEFINE_TOKEN Alexey Kardashevskiy
@ 2013-06-05 10:02 ` David Gibson
2013-06-05 10:59 ` Alexey Kardashevskiy
0 siblings, 1 reply; 26+ messages in thread
From: David Gibson @ 2013-06-05 10:02 UTC (permalink / raw)
To: Alexey Kardashevskiy; +Cc: qemu-ppc, qemu-devel, Paul Mackerras, Alexander Graf
[-- Attachment #1: Type: text/plain, Size: 914 bytes --]
On Tue, Jun 04, 2013 at 10:21:04PM +1000, Alexey Kardashevskiy wrote:
> From: David Gibson <david@gibson.dropbear.id.au>
>
> Recent PowerKVM allows the kernel to intercept some RTAS calls from the
> guest directly. This is used to implement the more efficient in-kernel
> XICS for example. qemu is still responsible for assigning the RTAS token
> numbers however, and needs to tell the kernel which RTAS function name is
> assigned to a given token value. This patch adds a convenience wrapper for
> the KVM_PPC_RTAS_DEFINE_TOKEN ioctl() which is used for this purpose.
Um.. have you actually compiled this on ppc. AFAICT, the necessary
ioctl constant isn't yet imported into the mainline qemu tree.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH 12/13] target-ppc: Add helper for KVM_PPC_RTAS_DEFINE_TOKEN
2013-06-05 10:02 ` David Gibson
@ 2013-06-05 10:59 ` Alexey Kardashevskiy
0 siblings, 0 replies; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-05 10:59 UTC (permalink / raw)
To: David Gibson; +Cc: qemu-ppc, qemu-devel, Paul Mackerras, Alexander Graf
On 05.06.2013 20:02, David Gibson wrote:
> On Tue, Jun 04, 2013 at 10:21:04PM +1000, Alexey Kardashevskiy wrote:
>> From: David Gibson <david@gibson.dropbear.id.au>
>>
>> Recent PowerKVM allows the kernel to intercept some RTAS calls from the
>> guest directly. This is used to implement the more efficient in-kernel
>> XICS for example. qemu is still responsible for assigning the RTAS token
>> numbers however, and needs to tell the kernel which RTAS function name is
>> assigned to a given token value. This patch adds a convenience wrapper for
>> the KVM_PPC_RTAS_DEFINE_TOKEN ioctl() which is used for this purpose.
>
> Um.. have you actually compiled this on ppc. AFAICT, the necessary
> ioctl constant isn't yet imported into the mainline qemu tree.
Agrh. Yes, it compiles but headers need an update I suppose. I use the
ones from v3.10-rc2. I'll post headers update request tomorrow.
--
With best regards
Alexey Kardashevskiy -- icq: 52150396
^ permalink raw reply [flat|nested] 26+ messages in thread
* [Qemu-devel] [PATCH 13/13] pseries: savevm support with KVM
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
` (11 preceding siblings ...)
2013-06-04 12:21 ` [Qemu-devel] [PATCH 12/13] target-ppc: Add helper for KVM_PPC_RTAS_DEFINE_TOKEN Alexey Kardashevskiy
@ 2013-06-04 12:21 ` Alexey Kardashevskiy
2013-06-04 12:42 ` [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Andreas Färber
13 siblings, 0 replies; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-04 12:21 UTC (permalink / raw)
To: qemu-devel; +Cc: aik, Alexander Graf, qemu-ppc, Paul Mackerras, David Gibson
From: David Gibson <david@gibson.dropbear.id.au>
At present, the savevm / migration support for the pseries machine will not
work when KVM is enabled. That's because KVM manages the guest's hash page
table in the host kernel, so qemu has no visibility of it. This patch
fixes this by using new kernel interfaces to extract and reinsert the
guest's hash table during the migration process.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
hw/ppc/spapr.c | 108 +++++++++++++++++++++++++++++++++++++-----------
include/hw/ppc/spapr.h | 1 +
target-ppc/kvm.c | 69 +++++++++++++++++++++++++++++++
target-ppc/kvm_ppc.h | 22 ++++++++++
4 files changed, 177 insertions(+), 23 deletions(-)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 8916142..71da11b 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -744,17 +744,27 @@ static int htab_save_setup(QEMUFile *f, void *opaque)
{
sPAPREnvironment *spapr = opaque;
- spapr->htab_save_index = 0;
- spapr->htab_first_pass = true;
-
/* "Iteration" header */
qemu_put_be32(f, spapr->htab_shift);
+ if (spapr->htab) {
+ spapr->htab_save_index = 0;
+ spapr->htab_first_pass = true;
+ } else {
+ assert(kvm_enabled());
+
+ spapr->htab_fd = kvmppc_get_htab_fd(false);
+ if (spapr->htab_fd < 0) {
+ fprintf(stderr, "Unable to open fd for reading hash table from KVM: %s\n",
+ strerror(errno));
+ return -1;
+ }
+ }
+
+
return 0;
}
-#define MAX_ITERATION_NS 5000000 /* 5 ms */
-
static void htab_save_first_pass(QEMUFile *f, sPAPREnvironment *spapr,
int64_t max_ns)
{
@@ -805,8 +815,8 @@ static void htab_save_first_pass(QEMUFile *f, sPAPREnvironment *spapr,
spapr->htab_save_index = index;
}
-static bool htab_save_later_pass(QEMUFile *f, sPAPREnvironment *spapr,
- int64_t max_ns)
+static int htab_save_later_pass(QEMUFile *f, sPAPREnvironment *spapr,
+ int64_t max_ns)
{
bool final = max_ns < 0;
int htabslots = HTAB_SIZE(spapr) / HASH_PTE_SIZE_64;
@@ -879,21 +889,32 @@ static bool htab_save_later_pass(QEMUFile *f, sPAPREnvironment *spapr,
spapr->htab_save_index = index;
- return (examined >= htabslots) && (sent == 0);
+ return (examined >= htabslots) && (sent == 0) ? 1 : 0;
}
-
+
+#define MAX_ITERATION_NS 5000000 /* 5 ms */
+#define MAX_KVM_BUF_SIZE 2048
+
static int htab_save_iterate(QEMUFile *f, void *opaque)
{
sPAPREnvironment *spapr = opaque;
- bool nothingleft = false;;
+ int rc = 0;
/* Iteration header */
qemu_put_be32(f, 0);
- if (spapr->htab_first_pass) {
+ if (!spapr->htab) {
+ assert(kvm_enabled());
+
+ rc = kvmppc_save_htab(f, spapr->htab_fd,
+ MAX_KVM_BUF_SIZE, MAX_ITERATION_NS);
+ if (rc < 0) {
+ return rc;
+ }
+ } else if (spapr->htab_first_pass) {
htab_save_first_pass(f, spapr, MAX_ITERATION_NS);
} else {
- nothingleft = htab_save_later_pass(f, spapr, MAX_ITERATION_NS);
+ rc = htab_save_later_pass(f, spapr, MAX_ITERATION_NS);
}
/* End marker */
@@ -901,7 +922,7 @@ static int htab_save_iterate(QEMUFile *f, void *opaque)
qemu_put_be16(f, 0);
qemu_put_be16(f, 0);
- return nothingleft ? 1 : 0;
+ return rc;
}
static int htab_save_complete(QEMUFile *f, void *opaque)
@@ -911,7 +932,20 @@ static int htab_save_complete(QEMUFile *f, void *opaque)
/* Iteration header */
qemu_put_be32(f, 0);
- htab_save_later_pass(f, spapr, -1);
+ if (!spapr->htab) {
+ int rc;
+
+ assert(kvm_enabled());
+
+ rc = kvmppc_save_htab(f, spapr->htab_fd, MAX_KVM_BUF_SIZE, -1);
+ if (rc < 0) {
+ return rc;
+ }
+ close(spapr->htab_fd);
+ spapr->htab_fd = -1;
+ } else {
+ htab_save_later_pass(f, spapr, -1);
+ }
/* End marker */
qemu_put_be32(f, 0);
@@ -925,6 +959,7 @@ static int htab_load(QEMUFile *f, void *opaque, int version_id)
{
sPAPREnvironment *spapr = opaque;
uint32_t section_hdr;
+ int fd = -1;
if (version_id < 1 || version_id > 1) {
fprintf(stderr, "htab_load() bad version\n");
@@ -941,6 +976,16 @@ static int htab_load(QEMUFile *f, void *opaque, int version_id)
return 0;
}
+ if (!spapr->htab) {
+ assert(kvm_enabled());
+
+ fd = kvmppc_get_htab_fd(true);
+ if (fd < 0) {
+ fprintf(stderr, "Unable to open fd to restore KVM hash table: %s\n",
+ strerror(errno));
+ }
+ }
+
while (true) {
uint32_t index;
uint16_t n_valid, n_invalid;
@@ -954,24 +999,41 @@ static int htab_load(QEMUFile *f, void *opaque, int version_id)
break;
}
- if ((index + n_valid + n_invalid) >=
+ if ((index + n_valid + n_invalid) >
(HTAB_SIZE(spapr) / HASH_PTE_SIZE_64)) {
/* Bad index in stream */
fprintf(stderr, "htab_load() bad index %d (%hd+%hd entries) "
- "in htab stream\n", index, n_valid, n_invalid);
+ "in htab stream (htab_shift=%d)\n", index, n_valid, n_invalid,
+ spapr->htab_shift);
return -EINVAL;
}
- if (n_valid) {
- qemu_get_buffer(f, HPTE(spapr->htab, index),
- HASH_PTE_SIZE_64 * n_valid);
- }
- if (n_invalid) {
- memset(HPTE(spapr->htab, index + n_valid), 0,
- HASH_PTE_SIZE_64 * n_invalid);
+ if (spapr->htab) {
+ if (n_valid) {
+ qemu_get_buffer(f, HPTE(spapr->htab, index),
+ HASH_PTE_SIZE_64 * n_valid);
+ }
+ if (n_invalid) {
+ memset(HPTE(spapr->htab, index + n_valid), 0,
+ HASH_PTE_SIZE_64 * n_invalid);
+ }
+ } else {
+ int rc;
+
+ assert(fd >= 0);
+
+ rc = kvmppc_load_htab_chunk(f, fd, index, n_valid, n_invalid);
+ if (rc < 0) {
+ return rc;
+ }
}
}
+ if (!spapr->htab) {
+ assert(fd >= 0);
+ close(fd);
+ }
+
return 0;
}
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index f59a581..7dab05a 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -37,6 +37,7 @@ typedef struct sPAPREnvironment {
/* Migration state */
int htab_save_index;
bool htab_first_pass;
+ int htab_fd;
} sPAPREnvironment;
#define H_SUCCESS 0
diff --git a/target-ppc/kvm.c b/target-ppc/kvm.c
index 33ddf63..ff85c19 100644
--- a/target-ppc/kvm.c
+++ b/target-ppc/kvm.c
@@ -65,6 +65,7 @@ static int cap_one_reg;
static int cap_epr;
static int cap_ppc_watchdog;
static int cap_papr;
+static int cap_htab_fd;
/* XXX We have a race condition where we actually have a level triggered
* interrupt, but the infrastructure can't expose that yet, so the guest
@@ -101,6 +102,7 @@ int kvm_arch_init(KVMState *s)
cap_ppc_watchdog = kvm_check_extension(s, KVM_CAP_PPC_BOOKE_WATCHDOG);
/* Note: we don't set cap_papr here, because this capability is
* only activated after this by kvmppc_set_papr() */
+ cap_htab_fd = kvm_check_extension(s, KVM_CAP_PPC_HTAB_FD);
if (!cap_interrupt_level) {
fprintf(stderr, "KVM: Couldn't find level irq capability. Expect the "
@@ -1802,6 +1804,73 @@ int kvmppc_define_rtas_token(uint32_t token, const char *function)
return kvm_vm_ioctl(kvm_state, KVM_PPC_RTAS_DEFINE_TOKEN, &args);
}
+int kvmppc_get_htab_fd(bool write)
+{
+ struct kvm_get_htab_fd s = {
+ .flags = write ? KVM_GET_HTAB_WRITE : 0,
+ .start_index = 0,
+ };
+
+ if (!cap_htab_fd) {
+ fprintf(stderr, "KVM version doesn't support saving the hash table\n");
+ return -1;
+ }
+
+ return kvm_vm_ioctl(kvm_state, KVM_PPC_GET_HTAB_FD, &s);
+}
+
+int kvmppc_save_htab(QEMUFile *f, int fd, size_t bufsize, int64_t max_ns)
+{
+ int64_t starttime = qemu_get_clock_ns(rt_clock);
+ uint8_t buf[bufsize];
+ ssize_t rc;
+
+ do {
+ rc = read(fd, buf, bufsize);
+ if (rc < 0) {
+ fprintf(stderr, "Error reading data from KVM HTAB fd: %s\n",
+ strerror(errno));
+ return rc;
+ } else if (rc) {
+ /* Kernel already retuns data in BE format for the file */
+ qemu_put_buffer(f, buf, rc);
+ }
+ } while ((rc != 0)
+ && ((max_ns < 0)
+ || ((qemu_get_clock_ns(rt_clock) - starttime) < max_ns)));
+
+ return (rc == 0) ? 1 : 0;
+}
+
+int kvmppc_load_htab_chunk(QEMUFile *f, int fd, uint32_t index,
+ uint16_t n_valid, uint16_t n_invalid)
+{
+ struct kvm_get_htab_header *buf;
+ size_t chunksize = sizeof(*buf) + n_valid*HASH_PTE_SIZE_64;
+ ssize_t rc;
+
+ buf = alloca(chunksize);
+ /* This is KVM on ppc, so this is all big-endian */
+ buf->index = index;
+ buf->n_valid = n_valid;
+ buf->n_invalid = n_invalid;
+
+ qemu_get_buffer(f, (void *)(buf + 1), HASH_PTE_SIZE_64*n_valid);
+
+ rc = write(fd, buf, chunksize);
+ if (rc < 0) {
+ fprintf(stderr, "Error writing KVM hash table: %s\n",
+ strerror(errno));
+ return rc;
+ }
+ if (rc != chunksize) {
+ /* We should never get a short write on a single chunk */
+ fprintf(stderr, "Short write, restoring KVM hash table\n");
+ return -1;
+ }
+ return 0;
+}
+
bool kvm_arch_stop_on_emulation_error(CPUState *cpu)
{
return true;
diff --git a/target-ppc/kvm_ppc.h b/target-ppc/kvm_ppc.h
index 21939a8..12564ef 100644
--- a/target-ppc/kvm_ppc.h
+++ b/target-ppc/kvm_ppc.h
@@ -39,6 +39,10 @@ uint64_t kvmppc_rma_size(uint64_t current_size, unsigned int hash_shift);
int kvmppc_fixup_cpu(PowerPCCPU *cpu);
bool kvmppc_has_cap_epr(void);
int kvmppc_define_rtas_token(uint32_t token, const char *function);
+int kvmppc_get_htab_fd(bool write);
+int kvmppc_save_htab(QEMUFile *f, int fd, size_t bufsize, int64_t max_ns);
+int kvmppc_load_htab_chunk(QEMUFile *f, int fd, uint32_t index,
+ uint16_t n_valid, uint16_t n_invalid);
#else
@@ -166,6 +170,24 @@ static inline int kvmppc_define_rtas_token(uint32_t token,
{
return -1;
}
+
+static inline int kvmppc_get_htab_fd(bool write)
+{
+ return -1;
+}
+
+static inline int kvmppc_save_htab(QEMUFile *f, int fd, size_t bufsize,
+ int64_t max_ns)
+{
+ abort();
+}
+
+static inline int kvmppc_load_htab_chunk(QEMUFile *f, int fd, uint32_t index,
+ uint16_t n_valid, uint16_t n_invalid)
+{
+ abort();
+}
+
#endif
#ifndef CONFIG_KVM
--
1.7.10.4
^ permalink raw reply related [flat|nested] 26+ messages in thread* Re: [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support
2013-06-04 12:20 [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Alexey Kardashevskiy
` (12 preceding siblings ...)
2013-06-04 12:21 ` [Qemu-devel] [PATCH 13/13] pseries: savevm support with KVM Alexey Kardashevskiy
@ 2013-06-04 12:42 ` Andreas Färber
2013-06-05 1:59 ` Alexey Kardashevskiy
13 siblings, 1 reply; 26+ messages in thread
From: Andreas Färber @ 2013-06-04 12:42 UTC (permalink / raw)
To: Alexey Kardashevskiy
Cc: Paul Mackerras, David Gibson, qemu-ppc, qemu-devel,
Alexander Graf
Am 04.06.2013 14:20, schrieb Alexey Kardashevskiy:
> I still have to learn how to force git not to overwrite the original commit author :-/
> Is there any way to tell the original creator in the commit message so it could live
> in a git forever and survive commit changes?
To fix it up:
git commit --amend --author="Foo Bar <foo@bar>"
If you cherry-pick patches in the first place, the author gets preserved
and you can update them as part of git-rebase or do your modifications
using just git commit --amend.
Note that at least one patch was Author: instead of From: and at least
one two From:s.
Andreas
--
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
^ permalink raw reply [flat|nested] 26+ messages in thread* Re: [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support
2013-06-04 12:42 ` [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support Andreas Färber
@ 2013-06-05 1:59 ` Alexey Kardashevskiy
2013-06-05 7:46 ` Alexey Kardashevskiy
0 siblings, 1 reply; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-05 1:59 UTC (permalink / raw)
To: Andreas Färber
Cc: Paul Mackerras, David Gibson, qemu-ppc, qemu-devel,
Alexander Graf
On 06/04/2013 10:42 PM, Andreas Färber wrote:
> Am 04.06.2013 14:20, schrieb Alexey Kardashevskiy:
>> I still have to learn how to force git not to overwrite the original commit author :-/
>> Is there any way to tell the original creator in the commit message so it could live
>> in a git forever and survive commit changes?
>
> To fix it up:
>
> git commit --amend --author="Foo Bar <foo@bar>"
Then I have to recall about every patch whether it is mine or not. Annoying :(
> If you cherry-pick patches in the first place, the author gets preserved
> and you can update them as part of git-rebase or do your modifications
> using just git commit --amend.
>
> Note that at least one patch was Author: instead of From: and at least
> one two From:s.
git removed "From" from some patch commit messages and left in others :-/
--
Alexey
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [Qemu-devel] [PATCH 00/13 v2] pseries: savevm / migration support
2013-06-05 1:59 ` Alexey Kardashevskiy
@ 2013-06-05 7:46 ` Alexey Kardashevskiy
0 siblings, 0 replies; 26+ messages in thread
From: Alexey Kardashevskiy @ 2013-06-05 7:46 UTC (permalink / raw)
To: Andreas Färber
Cc: Paul Mackerras, David Gibson, qemu-ppc, qemu-devel,
Alexander Graf
On 06/05/2013 11:59 AM, Alexey Kardashevskiy wrote:
> On 06/04/2013 10:42 PM, Andreas Färber wrote:
>> Am 04.06.2013 14:20, schrieb Alexey Kardashevskiy:
>>> I still have to learn how to force git not to overwrite the original commit author :-/
>>> Is there any way to tell the original creator in the commit message so it could live
>>> in a git forever and survive commit changes?
>>
>> To fix it up:
>>
>> git commit --amend --author="Foo Bar <foo@bar>"
>
> Then I have to recall about every patch whether it is mine or not. Annoying :(
Sorry for my ignorance, figured it all out now. git rather rules :)
--
Alexey
^ permalink raw reply [flat|nested] 26+ messages in thread