* [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format
@ 2014-03-25 20:17 Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 01/16] Visitor: Add methods for migration format use Dr. David Alan Gilbert (git)
` (16 more replies)
0 siblings, 17 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
This is a work in progress cut of my visitor+BER format migration
world; there's lots to do but it's starting to get there.
I'd appreciate if anyone who has more experience with either the
visitor code, or ASN.1/BER in general has a look to see if I'm
doing anything particularly odd.
I've cherry picked bits of Michael Roth's visitor set from 2011 and
Stefan Berger's BER code from this time last year, but not necessarily
used things in the same way.
The good:
It can perform a migration using a BER format file that's described
and verifiable against a schema.
It can also perform a migration using the old binary format.
A lot of the nasty detail of the old format is hidden in the compatibility
visitor (e.g. the subsection end detection).
The ugly:
1) The encoding for zero pages is bloated, it can be simplified a
lot (see previous discussion with Michael from a week or two back
that I've not yet implemented)
2) There are a few shims that get passed visitors but need files
or the other way that should be removable. The shim to get from
a qemu file back to a visitor is patch 3 and marked as 'tmp' and
the main uses are in piix4, pci.c, spapr_vscsi.c.
3) There are places where the visitor interface is too tied to
the old file format, but most of those can be removed with effort.
4) I've made extensions to the Visitor type, many of these are very
migration specific; and use things like get_next_type and the list
visitors in quite a different way from the current uses.
Should I really have a separate type?
(see patch 1)
5) At the moment you select BER output format by setting an environment
variable ( export QEMUMIGFORMAT=BER ) , I need to put more thought
in to the right way to do this, there are some harder questions like
what happens to devices that are still using pre-vmstate encodings
(that are currently sent as blobs) when they eventually convert over
and thus how to keep compatibility with earlier BER output versions
where they were blobs.
6) At the moment for blobs (i.e. pre-vmstate) the whole blob gets loaded
into memory and then processed; that's not necessary and can be done
with a QEMUFile shim.
7) I need to sort out my Error** handling (probably in a lot of places)
8) The visitors should be able to share much more code.
The not-yet-done:
a) XBZRLE
b) block-migration
c) SPAPR (it's an iterative migration like RAM and block)
d) RDMA (I'm guessing I've broken it, but haven't tried)
e) Floats are currently sent as blobs.
What I've done:
There are separate output visitors (binary compatible, debug and BER) and
they're selected by setting QEMUMIGFORMAT to one of 'debug', 'BER' or just
leaving it unset.
The Input visitor is selected automatically from the first 4 bytes of the
file.
In general most types are BER sequences of indefinite length, with some
types that I've allocated an Application specific type tag to. There is a hook
to give any VMState it's own type tag (which I've shown using one example
for cpu-common). Integers and strings are standard 'universal' types.
Objects with .get/.put methods or register_savevm are saved as an 'octet string'.
There are a few places where a device registered with .get/.put calls back into
vmstate_save_state for part of their state (typically for devices built on PCI)
and when they do that, even when in BER mode, those components get stored inside
the octet string in the old format.
I've used the 'asn1c' tool to validate the schema (which is
in docs/specs/migration.schema) and also to verify streams that I've produced
match the schema.
I've tested it with virt-test (hacked to have different source/dest qemu's)
and tried bin-bin, ber-ber, and pre-visitor-qemu -> this and
this -> pre-visitor-qemu just on the standard migration.default.tcp migration
test, but I've not tried a large combination of configurations.
My fix for qemu_peek_buffer that's on the list is needed to stabilise
the binary format input visitor.
I'll keep chipping away at that list, and would expect to pop another
version out in a month or so.
Dave
Dr. David Alan Gilbert (16):
Visitor: Add methods for migration format use
QEMUSizedBuffer/QEMUFile
qemu-file: Add set/get tmp_visitor
Header/constant/types fixes for visitors
Visitor: Binary compatible output visitor
Visitor: Debug output visitor
Visitor: Binary compatible input visitor
Visitor: Output path
Visitor: Load path
Visitor: Common types to use visitors
Choose output visitor based on env variable
BER Visitor: Create output visitor
BER Visitor: Create input visitor
Start some BER format docs
ASN.1 schema for new migration format
Wire in BER visitors
arch_init.c | 231 +++--
block-migration.c | 13 +-
docs/migration.txt | 34 +-
docs/specs/migration.schema | 113 +++
exec.c | 2 +
hw/acpi/piix4.c | 5 +-
hw/pci/pci.c | 5 +-
hw/ppc/spapr.c | 9 +-
hw/scsi/spapr_vscsi.c | 6 +-
include/migration/migration.h | 17 +
include/migration/qemu-file.h | 31 +
include/migration/vmstate.h | 28 +-
include/qapi/ber.h | 108 +++
include/qapi/qemu-file-ber-input-visitor.h | 26 +
include/qapi/qemu-file-ber-output-visitor.h | 26 +
include/qapi/qemu-file-binary-input-visitor.h | 27 +
include/qapi/qemu-file-binary-output-visitor.h | 26 +
include/qapi/qemu-file-debug-output-visitor.h | 26 +
include/qapi/visitor-impl.h | 23 +
include/qapi/visitor.h | 51 ++
include/qemu/typedefs.h | 4 +-
qapi/Makefile.objs | 4 +-
qapi/qapi-visit-core.c | 80 ++
qapi/qemu-file-ber-input-visitor.c | 1163 ++++++++++++++++++++++++
qapi/qemu-file-ber-output-visitor.c | 916 +++++++++++++++++++
qapi/qemu-file-binary-input-visitor.c | 688 ++++++++++++++
qapi/qemu-file-binary-output-visitor.c | 564 ++++++++++++
qapi/qemu-file-debug-output-visitor.c | 471 ++++++++++
qemu-file.c | 423 +++++++++
savevm.c | 412 +++++++--
vmstate.c | 619 ++++++-------
31 files changed, 5593 insertions(+), 558 deletions(-)
create mode 100644 docs/specs/migration.schema
create mode 100644 include/qapi/ber.h
create mode 100644 include/qapi/qemu-file-ber-input-visitor.h
create mode 100644 include/qapi/qemu-file-ber-output-visitor.h
create mode 100644 include/qapi/qemu-file-binary-input-visitor.h
create mode 100644 include/qapi/qemu-file-binary-output-visitor.h
create mode 100644 include/qapi/qemu-file-debug-output-visitor.h
create mode 100644 qapi/qemu-file-ber-input-visitor.c
create mode 100644 qapi/qemu-file-ber-output-visitor.c
create mode 100644 qapi/qemu-file-binary-input-visitor.c
create mode 100644 qapi/qemu-file-binary-output-visitor.c
create mode 100644 qapi/qemu-file-debug-output-visitor.c
--
1.8.5.3
^ permalink raw reply [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 01/16] Visitor: Add methods for migration format use
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-06-05 17:00 ` Markus Armbruster
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 02/16] QEMUSizedBuffer/QEMUFile Dr. David Alan Gilbert (git)
` (15 subsequent siblings)
16 siblings, 1 reply; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
array types
From https://lists.gnu.org/archive/html/qemu-devel/2011-09/msg02465.html
str256 type
For the upto 256byte strings QEMU commonly uses for IDs
buffer type
For a blob of data that the caller wants to deliver whole (e.g.
a page of RAM or block of disk)
Load/save flags to let a user perform pre-save/post-load checking
An accessor to get the underlying QEMUFile* (for compatibility)
compat-sequences
Provide enough information for a visitor providing compatibility
with the old format to generate it's byte stream, while allowing a new
visitor to do something sensible.
destroy
Allows the caller to destroy a visitor without knowing what type of
visitor it is.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
include/qapi/visitor-impl.h | 23 +++++++++++++
include/qapi/visitor.h | 51 +++++++++++++++++++++++++++++
qapi/qapi-visit-core.c | 80 +++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 154 insertions(+)
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index f3fa420..10cdbf7 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -15,6 +15,9 @@
#include "qapi/error.h"
#include "qapi/visitor.h"
+#define VISITOR_SAVING (1<<0)
+#define VISITOR_LOADING (1<<1)
+
struct Visitor
{
/* Must be set */
@@ -29,6 +32,10 @@ struct Visitor
void (*start_list)(Visitor *v, const char *name, Error **errp);
GenericList *(*next_list)(Visitor *v, GenericList **list, Error **errp);
void (*end_list)(Visitor *v, Error **errp);
+ void (*start_array)(Visitor *v, void **obj, const char *name,
+ size_t elem_count, size_t elem_size, Error **errp);
+ void (*next_array)(Visitor *v, Error **errp);
+ void (*end_array)(Visitor *v, Error **errp);
void (*type_enum)(Visitor *v, int *obj, const char *strings[],
const char *kind, const char *name, Error **errp);
@@ -38,6 +45,7 @@ struct Visitor
void (*type_int)(Visitor *v, int64_t *obj, const char *name, Error **errp);
void (*type_bool)(Visitor *v, bool *obj, const char *name, Error **errp);
void (*type_str)(Visitor *v, char **obj, const char *name, Error **errp);
+ void (*type_str256)(Visitor *v, char *obj, const char *name, Error **errp);
void (*type_number)(Visitor *v, double *obj, const char *name,
Error **errp);
@@ -49,6 +57,8 @@ struct Visitor
void (*start_handle)(Visitor *v, void **obj, const char *kind,
const char *name, Error **errp);
void (*end_handle)(Visitor *v, Error **errp);
+ void (*type_buffer)(Visitor *v, void *data, size_t len, bool async,
+ const char *name, Error **errp);
void (*type_uint8)(Visitor *v, uint8_t *obj, const char *name, Error **errp);
void (*type_uint16)(Visitor *v, uint16_t *obj, const char *name, Error **errp);
void (*type_uint32)(Visitor *v, uint32_t *obj, const char *name, Error **errp);
@@ -59,6 +69,19 @@ struct Visitor
void (*type_int64)(Visitor *v, int64_t *obj, const char *name, Error **errp);
/* visit_type_size() falls back to (*type_uint64)() if type_size is unset */
void (*type_size)(Visitor *v, uint64_t *obj, const char *name, Error **errp);
+
+ void (*destroy)(Visitor *v, Error **errp);
+
+ void (*start_sequence_compat)(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ void *opaque, Error **errp);
+ void (*end_sequence_compat)(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ Error **errp);
+
+ QEMUFile* (*get_qemufile)(Visitor *v);
+
+ uint64_t flags;
};
void input_type_enum(Visitor *v, int *obj, const char *strings[],
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 29da211..70c20df 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -39,11 +39,22 @@ void visit_end_implicit_struct(Visitor *v, Error **errp);
void visit_start_list(Visitor *v, const char *name, Error **errp);
GenericList *visit_next_list(Visitor *v, GenericList **list, Error **errp);
void visit_end_list(Visitor *v, Error **errp);
+void visit_start_array(Visitor *v, void **obj, const char *name,
+ size_t elem_count, size_t elem_size, Error **errp);
+void visit_next_array(Visitor *v, Error **errp);
+void visit_end_array(Visitor *v, Error **errp);
+
void visit_start_optional(Visitor *v, bool *present, const char *name,
Error **errp);
void visit_end_optional(Visitor *v, Error **errp);
void visit_get_next_type(Visitor *v, int *obj, const int *qtypes,
const char *name, Error **errp);
+/* Blocks of guest memory,disk or otherwise opaque data that there is a lot
+ * of and must be handled efficiently. 'async' true if the write can happen
+ * 'later'
+ */
+void visit_type_buffer(Visitor *v, void *data, size_t len, bool async,
+ const char *name, Error **errp);
void visit_type_enum(Visitor *v, int *obj, const char *strings[],
const char *kind, const char *name, Error **errp);
void visit_type_int(Visitor *v, int64_t *obj, const char *name, Error **errp);
@@ -58,6 +69,46 @@ void visit_type_int64(Visitor *v, int64_t *obj, const char *name, Error **errp);
void visit_type_size(Visitor *v, uint64_t *obj, const char *name, Error **errp);
void visit_type_bool(Visitor *v, bool *obj, const char *name, Error **errp);
void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp);
+/* A string no more than 256 (including term) characters in length */
+void visit_type_str256(Visitor *v, char *obj, const char *name, Error **errp);
void visit_type_number(Visitor *v, double *obj, const char *name, Error **errp);
+void visit_destroy(Visitor *v, Error **errp);
+
+/* Type of visitor, used where actions have to be taken when (de)serializing */
+bool visitor_loading(Visitor *v);
+bool visitor_saving(Visitor *v);
+
+typedef enum {
+ VISIT_SEQ_COMPAT_BYTE0TERM, /* list terminated with a 0 byte */
+ VISIT_SEQ_COMPAT_FILE, /* The top level file object */
+ VISIT_SEQ_COMPAT_SUBSECLIST, /* list terminated by
+ historical complexity */
+ VISIT_SEQ_COMPAT_SUBSECTION, /* One subsection */
+ VISIT_SEQ_COMPAT_SECTION_HEADER, /* SECTION_FULL/START, header + name */
+ VISIT_SEQ_COMPAT_SECTION_MIN, /* SECTION_PART/END - minimal header */
+ VISIT_SEQ_COMPAT_RAMSECLIST, /* list terminated by int64 bit
+ RAM_SAVE_FLAG_EOS */
+ VISIT_SEQ_COMPAT_RAMSECENTRY, /* Entry in RAM Sec list */
+ VISIT_SEQ_COMPAT_VMSTATE, /* One VMState */
+ VISIT_SEQ_COMPAT_BLOB /* A binary old-format blob */
+} Visit_seq_compat_mode;
+
+/* Start a sequence of items (which may be of unknown length and unknown
+ * mix of some subset of types), specify a compatibility mode that's only
+ * used by an implementation trying to match the existing binary migration
+ * format.
+ * opaque is compat_mode specific
+ */
+void visit_start_sequence_compat(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ void *opaque,
+ Error **errp);
+/* Use visit_get_next_type for each entry including the first */
+void visit_end_sequence_compat(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ Error **errp);
+
+/* Don't Use! - lets us move forward until we can get rid of all file uses */
+QEMUFile *visitor_get_qemufile(Visitor *v);
#endif
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 6451a21..2d20fde 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -84,6 +84,28 @@ void visit_end_list(Visitor *v, Error **errp)
v->end_list(v, errp);
}
+void visit_start_array(Visitor *v, void **obj, const char *name,
+ size_t elem_count, size_t elem_size, Error **errp)
+{
+ if (!error_is_set(errp)) {
+ v->start_array(v, obj, name, elem_count, elem_size, errp);
+ }
+}
+
+void visit_next_array(Visitor *v, Error **errp)
+{
+ if (!error_is_set(errp)) {
+ v->next_array(v, errp);
+ }
+}
+
+void visit_end_array(Visitor *v, Error **errp)
+{
+ if (!error_is_set(errp)) {
+ v->end_array(v, errp);
+ }
+}
+
void visit_start_optional(Visitor *v, bool *present, const char *name,
Error **errp)
{
@@ -107,6 +129,14 @@ void visit_get_next_type(Visitor *v, int *obj, const int *qtypes,
}
}
+void visit_type_buffer(Visitor *v, void *data, size_t len, bool async,
+ const char *name, Error **errp)
+{
+ if (!error_is_set(errp)) {
+ v->type_buffer(v, data, len, async, name, errp);
+ }
+}
+
void visit_type_enum(Visitor *v, int *obj, const char *strings[],
const char *kind, const char *name, Error **errp)
{
@@ -291,6 +321,13 @@ void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp)
}
}
+void visit_type_str256(Visitor *v, char *obj, const char *name, Error **errp)
+{
+ if (!error_is_set(errp)) {
+ v->type_str256(v, obj, name, errp);
+ }
+}
+
void visit_type_number(Visitor *v, double *obj, const char *name, Error **errp)
{
if (!error_is_set(errp)) {
@@ -347,3 +384,46 @@ void input_type_enum(Visitor *v, int *obj, const char *strings[],
g_free(enum_str);
*obj = value;
}
+
+void visit_destroy(Visitor *v, Error **errp)
+{
+ v->destroy(v, errp);
+}
+
+void visit_start_sequence_compat(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ void *opaque, Error **errp)
+{
+ if (error_is_set(errp)) {
+ return;
+ }
+
+ v->start_sequence_compat(v, name, compat_mode, opaque, errp);
+}
+
+void visit_end_sequence_compat(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ Error **errp)
+{
+ if (error_is_set(errp)) {
+ return;
+ }
+
+ v->end_sequence_compat(v, name, compat_mode, errp);
+}
+
+QEMUFile *visitor_get_qemufile(Visitor *v)
+{
+ return v->get_qemufile(v);
+}
+
+bool visitor_loading(Visitor *v)
+{
+ return v->flags & VISITOR_LOADING;
+}
+
+bool visitor_saving(Visitor *v)
+{
+ return v->flags & VISITOR_SAVING;
+}
+
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 02/16] QEMUSizedBuffer/QEMUFile
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 01/16] Visitor: Add methods for migration format use Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 03/16] qemu-file: Add set/get tmp_visitor Dr. David Alan Gilbert (git)
` (14 subsequent siblings)
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Stefan Berger's to create a QEMUFile that goes to a memory buffer;
from:
http://lists.gnu.org/archive/html/qemu-devel/2013-03/msg05036.html
Using the QEMUFile interface, this patch adds support functions for
operating
on in-memory sized buffers that can be written to or read from.
Signed-off-by: Stefan Berger <stefanb@linux.vnet.ibm.com>
Signed-off-by: Joel Schopp <jschopp@linux.vnet.ibm.com>
---
include/migration/qemu-file.h | 27 +++
qemu-file.c | 411 ++++++++++++++++++++++++++++++++++++++++++
2 files changed, 438 insertions(+)
diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index a191fb6..f066801 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -25,6 +25,8 @@
#define QEMU_FILE_H 1
#include "exec/cpu-common.h"
+#include <stdint.h>
+
/* This function writes a chunk of data to a file at the given position.
* The pos argument can be ignored if the file is only being used for
* streaming. The handler should try to write all of the data it can.
@@ -94,11 +96,31 @@ typedef struct QEMUFileOps {
QEMURamSaveFunc *save_page;
} QEMUFileOps;
+struct QEMUSizedBuffer {
+ struct iovec *iov;
+ size_t n_iov;
+ size_t size; /* total allocated size in all iov's */
+ size_t used; /* number of used bytes */
+};
+
+typedef struct QEMUSizedBuffer QEMUSizedBuffer;
+QEMUSizedBuffer *qsb_create(const uint8_t *buffer, size_t len);
+QEMUSizedBuffer *qsb_clone(const QEMUSizedBuffer *);
+void qsb_free(QEMUSizedBuffer *);
+size_t qsb_set_length(QEMUSizedBuffer *qsb, size_t length);
+size_t qsb_get_length(const QEMUSizedBuffer *qsb);
+ssize_t qsb_get_buffer(const QEMUSizedBuffer *, off_t start, size_t count,
+ uint8_t **buf);
+ssize_t qsb_write_at(QEMUSizedBuffer *qsb, const uint8_t *buf,
+ off_t pos, size_t count);
+
+
QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops);
QEMUFile *qemu_fopen(const char *filename, const char *mode);
QEMUFile *qemu_fdopen(int fd, const char *mode);
QEMUFile *qemu_fopen_socket(int fd, const char *mode);
QEMUFile *qemu_popen_cmd(const char *command, const char *mode);
+QEMUFile *qemu_bufopen(const char *mode, QEMUSizedBuffer *input);
int qemu_get_fd(QEMUFile *f);
int qemu_fclose(QEMUFile *f);
int64_t qemu_ftell(QEMUFile *f);
@@ -111,6 +133,11 @@ void qemu_put_byte(QEMUFile *f, int v);
void qemu_put_buffer_async(QEMUFile *f, const uint8_t *buf, int size);
bool qemu_file_mode_is_not_valid(const char *mode);
+/*
+ * For use on files opened with qemu_bufopen
+ */
+const QEMUSizedBuffer *qemu_buf_get(QEMUFile *f);
+
static inline void qemu_put_ubyte(QEMUFile *f, unsigned int v)
{
qemu_put_byte(f, (int)v);
diff --git a/qemu-file.c b/qemu-file.c
index e5ec798..b1b1a5b 100644
--- a/qemu-file.c
+++ b/qemu-file.c
@@ -831,3 +831,414 @@ uint64_t qemu_get_be64(QEMUFile *f)
v |= qemu_get_be32(f);
return v;
}
+
+
+#define QSB_CHUNK_SIZE (1 << 10)
+#define QSB_MAX_CHUNK_SIZE (10 * QSB_CHUNK_SIZE)
+
+/**
+ * Create a QEMUSizedBuffer
+ * This type of buffer uses scatter-gather lists internally and
+ * can grow to any size. Any data array in the scatter-gather list
+ * can hold different amount of bytes.
+ *
+ * @buffer: Optional buffer to copy into the QSB
+ * @len: size of initial buffer; if @buffer is given, buffer must
+ * hold at least len bytes
+ *
+ * Returns a pointer to a QEMUSizedBuffer
+ */
+QEMUSizedBuffer *qsb_create(const uint8_t *buffer, size_t len)
+{
+ QEMUSizedBuffer *qsb;
+ size_t alloc_len, num_chunks, i, to_copy;
+ size_t chunk_size = (len > QSB_MAX_CHUNK_SIZE)
+ ? QSB_MAX_CHUNK_SIZE
+ : QSB_CHUNK_SIZE;
+
+ if (len == 0) {
+ /* we want to allocate at least one chunk */
+ len = QSB_CHUNK_SIZE;
+ }
+
+ num_chunks = DIV_ROUND_UP(len, chunk_size);
+ alloc_len = num_chunks * chunk_size;
+
+ qsb = g_new0(QEMUSizedBuffer, 1);
+ qsb->iov = g_new0(struct iovec, num_chunks);
+ qsb->n_iov = num_chunks;
+
+ for (i = 0; i < num_chunks; i++) {
+ qsb->iov[i].iov_base = g_malloc0(chunk_size);
+ qsb->iov[i].iov_len = chunk_size;
+ if (buffer) {
+ to_copy = (len - qsb->used) > chunk_size
+ ? chunk_size : (len - qsb->used);
+ memcpy(qsb->iov[i].iov_base, &buffer[qsb->used], to_copy);
+ qsb->used += to_copy;
+ }
+ }
+
+ qsb->size = alloc_len;
+
+ return qsb;
+}
+
+/**
+ * Free the QEMUSizedBuffer
+ *
+ * @qsb: The QEMUSizedBuffer to free
+ */
+void qsb_free(QEMUSizedBuffer *qsb)
+{
+ size_t i;
+
+ if (!qsb) {
+ return;
+ }
+
+ for (i = 0; i < qsb->n_iov; i++) {
+ g_free(qsb->iov[i].iov_base);
+ }
+ g_free(qsb->iov);
+ g_free(qsb);
+}
+
+/**
+ * Get the number of of used bytes in the QEMUSizedBuffer
+ *
+ * @qsb: A QEMUSizedBuffer
+ *
+ * Returns the number of bytes currently used in this buffer
+ */
+size_t qsb_get_length(const QEMUSizedBuffer *qsb)
+{
+ return qsb->used;
+}
+
+/**
+ * Set the length of the buffer; The primary usage of this
+ * function is to truncate the number of used bytes in the buffer.
+ * The size will not be extended beyond the current number of
+ * allocated bytes in the QEMUSizedBuffer.
+ *
+ * @qsb: A QEMUSizedBuffer
+ * @new_len : The new length of bytes in the buffer
+ *
+ * Returns the number of bytes the buffer was trucated or extended
+ * to.
+ */
+size_t qsb_set_length(QEMUSizedBuffer *qsb, size_t new_len)
+{
+ if (new_len <= qsb->size) {
+ qsb->used = new_len;
+ } else {
+ qsb->used = qsb->size;
+ }
+ return qsb->used;
+}
+
+/**
+ * Get the iovec that holds the data for a given position @pos.
+ *
+ * @qsb: A QEMUSizedBuffer
+ * @pos: The index of a byte in the buffer
+ * @d_off: Pointer to an offset that this function will indicate
+ * at what position within the returned iovec the byte
+ * is to be found
+ *
+ * Returns the index of the iovec that holds the byte at the given
+ * index @pos in the byte stream; a negative number if the iovec
+ * for the given position @pos does not exist.
+ */
+static ssize_t qsb_get_iovec(const QEMUSizedBuffer *qsb,
+ off_t pos, off_t *d_off)
+{
+ ssize_t i;
+ off_t curr = 0;
+
+ if (pos > qsb->used) {
+ return -1;
+ }
+
+ for (i = 0; i < qsb->n_iov; i++) {
+ if (curr + qsb->iov[i].iov_len > pos) {
+ *d_off = pos - curr;
+ return i;
+ }
+ curr += qsb->iov[i].iov_len;
+ }
+ return -1;
+}
+
+/*
+ * Convert the QEMUSizedBuffer into a flat buffer.
+ *
+ * Note: If at all possible, try to avoid this function since it
+ * may unnecessarily copy memory around.
+ *
+ * @qsb: pointer to QEMUSizedBuffer
+ * @start : offset to start at
+ * @count: number of bytes to copy
+ * @buf: a pointer to an optional buffer to write into; the pointer may
+ * point to NULL in which case the buffer will be allocated;
+ * if buffer is provided, it must be large enough to hold @count bytes
+ *
+ * Returns the number of bytes copied into the output buffer
+ */
+ssize_t qsb_get_buffer(const QEMUSizedBuffer *qsb, off_t start,
+ size_t count, uint8_t **buf)
+{
+ uint8_t *buffer;
+ const struct iovec *iov;
+ size_t to_copy, all_copy;
+ ssize_t index;
+ off_t s_off;
+ off_t d_off = 0;
+ char *s;
+
+ if (start > qsb->used) {
+ return 0;
+ }
+
+ all_copy = qsb->used - start;
+ if (all_copy > count) {
+ all_copy = count;
+ } else {
+ count = all_copy;
+ }
+
+ if (*buf == NULL) {
+ *buf = g_malloc(all_copy);
+ }
+ buffer = *buf;
+
+ index = qsb_get_iovec(qsb, start, &s_off);
+ if (index < 0) {
+ return 0;
+ }
+
+ while (all_copy > 0) {
+ iov = &qsb->iov[index];
+
+ s = iov->iov_base;
+
+ to_copy = iov->iov_len - s_off;
+ if (to_copy > all_copy) {
+ to_copy = all_copy;
+ }
+ memcpy(&buffer[d_off], &s[s_off], to_copy);
+
+ d_off += to_copy;
+ all_copy -= to_copy;
+
+ s_off = 0;
+ index++;
+ }
+
+ return count;
+}
+
+/**
+ * Grow the QEMUSizedBuffer to the given size and allocated
+ * memory for it.
+ *
+ * @qsb: A QEMUSizedBuffer
+ * @new_size: The new size of the buffer
+ *
+ * Returns an error code in case of memory allocation failure
+ * or the new size of the buffer otherwise. The returned size
+ * may be greater or equal to @new_size.
+ */
+static ssize_t qsb_grow(QEMUSizedBuffer *qsb, size_t new_size)
+{
+ size_t needed_chunks, i;
+ size_t chunk_size = QSB_CHUNK_SIZE;
+
+ if (qsb->size < new_size) {
+ needed_chunks = DIV_ROUND_UP(new_size - qsb->size,
+ chunk_size);
+
+ qsb->iov = g_realloc_n(qsb->iov, qsb->n_iov + needed_chunks,
+ sizeof(struct iovec));
+ if (qsb->iov == NULL) {
+ return -ENOMEM;
+ }
+
+ for (i = qsb->n_iov; i < qsb->n_iov + needed_chunks; i++) {
+ qsb->iov[i].iov_base = g_malloc0(chunk_size);
+ qsb->iov[i].iov_len = chunk_size;
+ }
+
+ qsb->n_iov += needed_chunks;
+ qsb->size += (needed_chunks * chunk_size);
+ }
+
+ return qsb->size;
+}
+
+/**
+ * Write into the QEMUSizedBuffer at a given position and a given
+ * number of bytes. This function will automatically grow the
+ * QEMUSizedBuffer.
+ *
+ * @qsb: A QEMUSizedBuffer
+ * @source: A byte array to copy data from
+ * @pos: The position withing the @qsb to write data to
+ * @size: The number of bytes to copy into the @qsb
+ *
+ * Returns an error code in case of memory allocation failure,
+ * @size otherwise.
+ */
+ssize_t qsb_write_at(QEMUSizedBuffer *qsb, const uint8_t *source,
+ off_t pos, size_t count)
+{
+ ssize_t rc = qsb_grow(qsb, pos + count);
+ size_t to_copy;
+ size_t all_copy = count;
+ const struct iovec *iov;
+ ssize_t index;
+ char *dest;
+ off_t d_off, s_off = 0;
+
+ if (rc < 0) {
+ return rc;
+ }
+
+ if (pos + count > qsb->used) {
+ qsb->used = pos + count;
+ }
+
+ index = qsb_get_iovec(qsb, pos, &d_off);
+ if (index < 0) {
+ return 0;
+ }
+
+ while (all_copy > 0) {
+ iov = &qsb->iov[index];
+
+ dest = iov->iov_base;
+
+ to_copy = iov->iov_len - d_off;
+ if (to_copy > all_copy) {
+ to_copy = all_copy;
+ }
+
+ memcpy(&dest[d_off], &source[s_off], to_copy);
+
+ s_off += to_copy;
+ all_copy -= to_copy;
+
+ d_off = 0;
+ index++;
+ }
+
+ return count;
+}
+
+/**
+ * Create an exact copy of the given QEMUSizedBuffer.
+ *
+ * @qsb : A QEMUSizedBuffer
+ *
+ * Returns a clone of @qsb
+ */
+QEMUSizedBuffer *qsb_clone(const QEMUSizedBuffer *qsb)
+{
+ QEMUSizedBuffer *out = qsb_create(NULL, qsb_get_length(qsb));
+ size_t i;
+ off_t pos = 0;
+
+ for (i = 0; i < qsb->n_iov; i++) {
+ pos += qsb_write_at(out, qsb->iov[i].iov_base,
+ pos, qsb->iov[i].iov_len);
+ }
+
+ return out;
+}
+
+typedef struct QEMUBuffer {
+ QEMUSizedBuffer *qsb;
+ QEMUFile *file;
+} QEMUBuffer;
+
+static int buf_get_buffer(void *opaque, uint8_t *buf, int64_t pos, int size)
+{
+ QEMUBuffer *s = opaque;
+ ssize_t len = qsb_get_length(s->qsb) - pos;
+
+ if (len <= 0) {
+ return 0;
+ }
+
+ if (len > size) {
+ len = size;
+ }
+ return qsb_get_buffer(s->qsb, pos, len, &buf);
+}
+
+static int buf_put_buffer(void *opaque, const uint8_t *buf,
+ int64_t pos, int size)
+{
+ QEMUBuffer *s = opaque;
+
+ return qsb_write_at(s->qsb, buf, pos, size);
+}
+
+static int buf_close(void *opaque)
+{
+ QEMUBuffer *s = opaque;
+
+ qsb_free(s->qsb);
+
+ g_free(s);
+
+ return 0;
+}
+
+const QEMUSizedBuffer *qemu_buf_get(QEMUFile *f)
+{
+ QEMUBuffer *p;
+
+ qemu_fflush(f);
+
+ p = (QEMUBuffer *)f->opaque;
+
+ return p->qsb;
+}
+
+static const QEMUFileOps buf_read_ops = {
+ .get_buffer = buf_get_buffer,
+ .close = buf_close
+};
+
+static const QEMUFileOps buf_write_ops = {
+ .put_buffer = buf_put_buffer,
+ .close = buf_close
+};
+
+QEMUFile *qemu_bufopen(const char *mode, QEMUSizedBuffer *input)
+{
+ QEMUBuffer *s;
+
+ if (mode == NULL || (mode[0] != 'r' && mode[0] != 'w') || mode[1] != 0) {
+ fprintf(stderr, "qemu_bufopen: Argument validity check failed\n");
+ return NULL;
+ }
+
+ s = g_malloc0(sizeof(QEMUBuffer));
+ if (mode[0] == 'r') {
+ s->qsb = input;
+ }
+
+ if (s->qsb == NULL) {
+ s->qsb = qsb_create(NULL, 0);
+ }
+
+ if (mode[0] == 'r') {
+ s->file = qemu_fopen_ops(s, &buf_read_ops);
+ } else {
+ s->file = qemu_fopen_ops(s, &buf_write_ops);
+ }
+ return s->file;
+}
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 03/16] qemu-file: Add set/get tmp_visitor
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 01/16] Visitor: Add methods for migration format use Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 02/16] QEMUSizedBuffer/QEMUFile Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 04/16] Header/constant/types fixes for visitors Dr. David Alan Gilbert (git)
` (13 subsequent siblings)
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
include/migration/qemu-file.h | 4 ++++
qemu-file.c | 12 ++++++++++++
2 files changed, 16 insertions(+)
diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index f066801..e36c03f 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -174,6 +174,10 @@ int qemu_file_get_error(QEMUFile *f);
void qemu_file_set_error(QEMUFile *f, int ret);
void qemu_fflush(QEMUFile *f);
+/* TRANSITION ONLY - DON'T USE! */
+Visitor *qemu_file_get_tmp_visitor(QEMUFile *f);
+void qemu_file_set_tmp_visitor(QEMUFile *f, Visitor *v);
+
static inline void qemu_put_be64s(QEMUFile *f, const uint64_t *pv)
{
qemu_put_be64(f, *pv);
diff --git a/qemu-file.c b/qemu-file.c
index b1b1a5b..feb3e68 100644
--- a/qemu-file.c
+++ b/qemu-file.c
@@ -25,6 +25,8 @@ struct QEMUFile {
unsigned int iovcnt;
int last_error;
+
+ Visitor *tmp_visitor;
};
typedef struct QEMUFileStdio {
@@ -37,6 +39,16 @@ typedef struct QEMUFileSocket {
QEMUFile *file;
} QEMUFileSocket;
+Visitor *qemu_file_get_tmp_visitor(QEMUFile *f)
+{
+ return f->tmp_visitor;
+}
+
+void qemu_file_set_tmp_visitor(QEMUFile *f, Visitor *v)
+{
+ f->tmp_visitor = v;
+}
+
static ssize_t socket_writev_buffer(void *opaque, struct iovec *iov, int iovcnt,
int64_t pos)
{
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 04/16] Header/constant/types fixes for visitors
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (2 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 03/16] qemu-file: Add set/get tmp_visitor Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 05/16] Visitor: Binary compatible output visitor Dr. David Alan Gilbert (git)
` (12 subsequent siblings)
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Move constants around and add types to allow file structure to move into
visitors.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
arch_init.c | 12 ------------
include/migration/migration.h | 15 +++++++++++++++
include/migration/vmstate.h | 20 +++++++++++++++++---
include/qemu/typedefs.h | 4 ++--
4 files changed, 34 insertions(+), 17 deletions(-)
diff --git a/arch_init.c b/arch_init.c
index 60c975d..73b9303 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -110,18 +110,6 @@ static bool mig_throttle_on;
static int dirty_rate_high_cnt;
static void check_guest_throttling(void);
-/***********************************************************/
-/* ram save/restore */
-
-#define RAM_SAVE_FLAG_FULL 0x01 /* Obsolete, not used anymore */
-#define RAM_SAVE_FLAG_COMPRESS 0x02
-#define RAM_SAVE_FLAG_MEM_SIZE 0x04
-#define RAM_SAVE_FLAG_PAGE 0x08
-#define RAM_SAVE_FLAG_EOS 0x10
-#define RAM_SAVE_FLAG_CONTINUE 0x20
-#define RAM_SAVE_FLAG_XBZRLE 0x40
-/* 0x80 is reserved in migration.h start with 0x100 next */
-
static struct defconfig_file {
const char *filename;
/* Indicates it is an user config file (disabled by -no-user-config) */
diff --git a/include/migration/migration.h b/include/migration/migration.h
index 3e1e6c7..1987e79 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -165,7 +165,16 @@ void ram_control_load_hook(QEMUFile *f, uint64_t flags);
* side. This lets before_ram_iterate/after_ram_iterate add
* transport-specific sections to the RAM migration data.
*/
+/* ram save/restore */
+#define RAM_SAVE_FLAG_FULL 0x01 /* Obsolete, not used anymore */
+#define RAM_SAVE_FLAG_COMPRESS 0x02
+#define RAM_SAVE_FLAG_MEM_SIZE 0x04
+#define RAM_SAVE_FLAG_PAGE 0x08
+#define RAM_SAVE_FLAG_EOS 0x10
+#define RAM_SAVE_FLAG_CONTINUE 0x20
+#define RAM_SAVE_FLAG_XBZRLE 0x40
#define RAM_SAVE_FLAG_HOOK 0x80
+#define RAM_SAVE_FLAG_MASK 0x1ff
#define RAM_SAVE_CONTROL_NOT_SUPP -1000
#define RAM_SAVE_CONTROL_DELAYED -2000
@@ -174,4 +183,10 @@ size_t ram_control_save_page(QEMUFile *f, ram_addr_t block_offset,
ram_addr_t offset, size_t size,
int *bytes_sent);
+typedef struct {
+ uint64_t addr;
+ uint16_t flags;
+ char idstr[256];
+} ramsecentry_header;
+
#endif
diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index e7e1705..a5e4b0b 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -26,6 +26,7 @@
#ifndef QEMU_VMSTATE_H
#define QEMU_VMSTATE_H 1
+#include "qemu/typedefs.h"
#ifndef CONFIG_USER_ONLY
#include <migration/qemu-file.h>
#endif
@@ -49,15 +50,27 @@ typedef struct SaveVMHandlers {
* use data that is local to the migration thread or protected
* by other locks.
*/
- int (*save_live_iterate)(QEMUFile *f, void *opaque);
+ int (*save_live_iterate)(Visitor *v, void *opaque);
/* This runs outside the iothread lock! */
- int (*save_live_setup)(QEMUFile *f, void *opaque);
- uint64_t (*save_live_pending)(QEMUFile *f, void *opaque, uint64_t max_size);
+ int (*save_live_setup)(Visitor *v, void *opaque);
+ uint64_t (*save_live_pending)(void *opaque, uint64_t max_size);
LoadStateHandler *load_state;
} SaveVMHandlers;
+/* This is the data used to identify a section as passed
+ * into the section version of the compat sequence visitor
+ * (TODO: Probably want to move the whole name lookup into there
+ * and keep the section_id wrapped inside the binary visitor)
+ */
+typedef struct SectionHeader {
+ uint32_t section_id;
+ uint32_t instance_id; /* Below only used for full version */
+ uint32_t version_id;
+ char idstr[256];
+} SectionHeader;
+
int register_savevm(DeviceState *dev,
const char *idstr,
int instance_id,
@@ -134,6 +147,7 @@ struct VMStateDescription {
void (*pre_save)(void *opaque);
VMStateField *fields;
const VMStateSubsection *subsections;
+ uint32_t ber_tag;
};
#ifdef CONFIG_USER_ONLY
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index bf8daac..3fea88e 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -10,8 +10,6 @@ typedef struct QEMUBH QEMUBH;
typedef struct AioContext AioContext;
-typedef struct Visitor Visitor;
-
struct Monitor;
typedef struct Monitor Monitor;
typedef struct MigrationParams MigrationParams;
@@ -39,6 +37,7 @@ typedef struct DriveInfo DriveInfo;
typedef struct DisplayState DisplayState;
typedef struct DisplayChangeListener DisplayChangeListener;
typedef struct DisplaySurface DisplaySurface;
+typedef struct Error Error;
typedef struct PixelFormat PixelFormat;
typedef struct QemuConsole QemuConsole;
typedef struct CharDriverState CharDriverState;
@@ -73,5 +72,6 @@ typedef struct SHPCDevice SHPCDevice;
typedef struct FWCfgState FWCfgState;
typedef struct PcGuestInfo PcGuestInfo;
typedef struct Range Range;
+typedef struct Visitor Visitor;
#endif /* QEMU_TYPEDEFS_H */
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 05/16] Visitor: Binary compatible output visitor
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (3 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 04/16] Header/constant/types fixes for visitors Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 06/16] Visitor: Debug " Dr. David Alan Gilbert (git)
` (11 subsequent siblings)
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Binary compatible output visitor to write to a QEMUFile in current migration
format.
This is based on Michael Roth's series :
https://lists.gnu.org/archive/html/qemu-devel/2011-09/msg02466.html
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
include/qapi/qemu-file-binary-output-visitor.h | 26 ++
qapi/Makefile.objs | 2 +-
qapi/qemu-file-binary-output-visitor.c | 564 +++++++++++++++++++++++++
3 files changed, 591 insertions(+), 1 deletion(-)
create mode 100644 include/qapi/qemu-file-binary-output-visitor.h
create mode 100644 qapi/qemu-file-binary-output-visitor.c
diff --git a/include/qapi/qemu-file-binary-output-visitor.h b/include/qapi/qemu-file-binary-output-visitor.h
new file mode 100644
index 0000000..f563407
--- /dev/null
+++ b/include/qapi/qemu-file-binary-output-visitor.h
@@ -0,0 +1,26 @@
+/*
+ * QEMUFile Visitor
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ * Michael Roth <mdroth@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef QEMU_FILE_BINARY_OUTPUT_VISITOR_H
+#define QEMU_FILE_BINARY_OUTPUT_VISITOR_H
+
+#include "visitor.h"
+
+typedef struct QemuFileBinOutputVisitor QemuFileBinOutputVisitor;
+
+QemuFileBinOutputVisitor *qemu_file_bin_output_visitor_new(QEMUFile *f);
+void qemu_file_bin_output_visitor_cleanup(QemuFileBinOutputVisitor *d);
+
+Visitor *qemu_file_bin_output_get_visitor(QemuFileBinOutputVisitor *v);
+
+#endif
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index 1f9c973..a054d52 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -1,5 +1,5 @@
util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qmp-input-visitor.o
util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
util-obj-y += string-input-visitor.o string-output-visitor.o
-
+util-obj-y += qemu-file-binary-output-visitor.o
util-obj-y += opts-visitor.o
diff --git a/qapi/qemu-file-binary-output-visitor.c b/qapi/qemu-file-binary-output-visitor.c
new file mode 100644
index 0000000..bc3f419
--- /dev/null
+++ b/qapi/qemu-file-binary-output-visitor.c
@@ -0,0 +1,564 @@
+/*
+ * QEMUFile Output Visitor
+ *
+ * Copyright IBM, Corp. 2011
+ * Copyright 2014 Red Hat, Inc. and/or its affiliates
+ *
+ * Authors:
+ * Michael Roth <mdroth@linux.vnet.ibm.com>
+ * David Gilbert <dgilbert@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include "qapi/qemu-file-binary-output-visitor.h"
+#include "qapi/visitor-impl.h"
+#include "qapi/qmp/qerror.h"
+#include "qemu/queue.h"
+#include "qemu-common.h"
+#include "hw/hw.h"
+#include "migration/migration.h"
+
+#if 0
+#define DPRINTF(fmt, ...) \
+ do { \
+ fprintf(stderr, "qfbov/%s/%d: " fmt "\n", __func__, \
+ __LINE__, ## __VA_ARGS__); \
+ } while (0)
+#else
+#define DPRINTF(fmt, ...) \
+ do { } while (0)
+#endif
+
+/* TODO: This needs sharing between input/output binary visitors */
+typedef struct {
+ size_t elem_count;
+ size_t elem_size;
+ size_t pos;
+} ArrayInfo;
+
+typedef struct {
+ Visit_seq_compat_mode mode;
+ const void *data;
+} SeqCompatInfo;
+
+typedef struct StackEntry {
+ enum {
+ QFOV_ARRAY,
+ QFOV_LIST,
+ QFOV_STRUCT,
+ QFOV_SEQCOMPAT,
+ } type;
+ ArrayInfo array_info;
+ SeqCompatInfo seqcompat_info;
+ bool is_list_head;
+ QTAILQ_ENTRY(StackEntry) node;
+} StackEntry;
+
+struct QemuFileBinOutputVisitor {
+ Visitor visitor;
+ QTAILQ_HEAD(, StackEntry) stack;
+ QEMUFile *file;
+};
+
+static QemuFileBinOutputVisitor *to_ov(Visitor *v)
+{
+ return container_of(v, QemuFileBinOutputVisitor, visitor);
+}
+
+static void qfbo_push(QemuFileBinOutputVisitor *ov, StackEntry *e)
+{
+ QTAILQ_INSERT_HEAD(&ov->stack, e, node);
+}
+
+static void qfbo_push_array(QemuFileBinOutputVisitor *ov, ArrayInfo ai)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFOV_ARRAY;
+ e->array_info = ai;
+ qfbo_push(ov, e);
+}
+
+static void qfbo_push_list(QemuFileBinOutputVisitor *ov)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFOV_LIST;
+ e->is_list_head = true;
+ qfbo_push(ov, e);
+}
+
+static void qfbo_push_seqcompat(QemuFileBinOutputVisitor *ov,
+ SeqCompatInfo sci)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFOV_SEQCOMPAT;
+ e->seqcompat_info = sci;
+ qfbo_push(ov, e);
+}
+
+static void qfbo_push_struct(QemuFileBinOutputVisitor *ov)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFOV_STRUCT;
+ qfbo_push(ov, e);
+}
+
+static StackEntry *qfbo_pop(QemuFileBinOutputVisitor *ov)
+{
+ StackEntry *e = QTAILQ_FIRST(&ov->stack);
+ QTAILQ_REMOVE(&ov->stack, e, node);
+ return e;
+}
+
+static bool qfbo_is_array(QemuFileBinOutputVisitor *ov)
+{
+ StackEntry *e = QTAILQ_FIRST(&ov->stack);
+ return e && e->type == QFOV_ARRAY;
+}
+
+static bool qfbo_is_list(QemuFileBinOutputVisitor *ov)
+{
+ StackEntry *e = QTAILQ_FIRST(&ov->stack);
+ return e && e->type == QFOV_LIST;
+}
+
+/* If we are in a seqcompat list return true and fill in
+ * sci with the compat mode
+ */
+static bool qfbo_is_seqcompat(QemuFileBinOutputVisitor *ov,
+ SeqCompatInfo *sci)
+{
+ StackEntry *e = QTAILQ_FIRST(&ov->stack);
+ if (e && e->type == QFOV_SEQCOMPAT) {
+ *sci = e->seqcompat_info;
+ return true;
+ }
+ return false;
+}
+
+static void qfbo_start_struct(Visitor *v, void **obj,
+ const char *kind, const char *name,
+ size_t unused, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+
+ qfbo_push_struct(ov);
+}
+
+static void qfbo_end_struct(Visitor *v, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ StackEntry *e = qfbo_pop(ov);
+
+ if (!e || e->type != QFOV_STRUCT) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+ g_free(e);
+}
+
+static void qfbo_start_list(Visitor *v, const char *name,
+ Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ qfbo_push_list(ov);
+}
+
+static GenericList *qfbo_next_list(Visitor *v, GenericList **list,
+ Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ GenericList *entry;
+ StackEntry *e = QTAILQ_FIRST(&ov->stack);
+
+ /* Some users don't actually have a list */
+ if (!list) {
+ return NULL;
+ }
+ entry = *list;
+
+ if (!entry || !qfbo_is_list(ov)) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ }
+
+ /* The way the list iterator is currently used unfortunately clobbers
+ * **list by subseqently assigning our return value to the same container.
+ * This can cause an infinite loop, but we can get around this by tracking
+ * a bit of state to note when we should pass back the next entry rather
+ * than the current one.
+ */
+ if (e->is_list_head) {
+ e->is_list_head = false;
+ return entry;
+ }
+
+ *list = entry->next;
+ return entry->next;
+}
+
+static void qfbo_end_list(Visitor *v, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ StackEntry *e = qfbo_pop(ov);
+ if (!e || e->type != QFOV_LIST) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ }
+ g_free(e);
+}
+
+static void qfbo_start_array(Visitor *v, void **obj,
+ const char *name,
+ size_t elem_count,
+ size_t elem_size, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ ArrayInfo ai = {
+ .elem_count = elem_count,
+ .elem_size = elem_size,
+ .pos = 0
+ };
+ qfbo_push_array(ov, ai);
+}
+
+static void qfbo_next_array(Visitor *v, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ StackEntry *e = QTAILQ_FIRST(&ov->stack);
+ if (!qfbo_is_array(ov) ||
+ e->array_info.pos >= e->array_info.elem_count) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ }
+
+ e->array_info.pos++;
+}
+
+static void qfbo_end_array(Visitor *v, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ StackEntry *e = qfbo_pop(ov);
+ if (!e || e->type != QFOV_ARRAY) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+ g_free(e);
+}
+
+static void qfbo_type_str(Visitor *v, char **obj, const char *name,
+ Error **errp)
+{
+ if (obj) {
+ g_free(*obj);
+ }
+}
+
+/* A string upto 256 bytes in length (including terminator)
+ * output as length byte (not including term) followed by text
+ * (also not including term)
+ */
+static void qfbo_type_str256(Visitor *v, char *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ size_t len = strlen(obj);
+
+ DPRINTF("qfbo_type_str256: len=%d str=%s", (int)len, obj);
+
+ if (len > 255) {
+ error_set(errp, QERR_BUFFER_OVERRUN);
+ return;
+ }
+ qemu_put_byte(ov->file, (uint8_t)len);
+ qemu_put_buffer(ov->file, (uint8_t *)obj, len);
+}
+
+static void qfbo_type_buffer(Visitor *v, void *data, size_t len, bool async,
+ const char *name, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ if (async) {
+ qemu_put_buffer_async(ov->file, data, len);
+ } else {
+ qemu_put_buffer(ov->file, data, len);
+ }
+}
+
+static void qfbo_type_uint8(Visitor *v, uint8_t *obj,
+ const char *name,
+ Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ qemu_put_byte(ov->file, *obj);
+}
+
+static void qfbo_type_uint16(Visitor *v, uint16_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ qemu_put_be16(ov->file, *obj);
+}
+
+static void qfbo_type_uint32(Visitor *v, uint32_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ qemu_put_be32(ov->file, *obj);
+}
+
+static void qfbo_type_uint64(Visitor *v, uint64_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ qemu_put_be64(ov->file, *obj);
+}
+
+static void qfbo_type_int8(Visitor *v, int8_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ qemu_put_byte(ov->file, *obj);
+}
+
+static void qfbo_type_int16(Visitor *v, int16_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ qemu_put_sbe16(ov->file, *obj);
+}
+
+static void qfbo_type_int32(Visitor *v, int32_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ qemu_put_sbe32(ov->file, *obj);
+}
+
+static void qfbo_type_int64(Visitor *v, int64_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ qemu_put_sbe64(ov->file, *obj);
+}
+
+static void qfbo_type_bool(Visitor *v, bool *obj, const char *name,
+ Error **errp)
+{
+ uint8_t val = *obj;
+ qfbo_type_uint8(v, &val, name, errp);
+}
+
+static QEMUFile *qfbo_get_qemufile(Visitor *v)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+
+ return ov->file;
+}
+
+/* this is a bit odd - I'm using 'get_next_type' to plant the compatibility
+ * bytes in lists.
+ */
+static void qfbo_get_next_type(Visitor *v, int *kind, const int *qobjects,
+ const char *name, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ SeqCompatInfo sci;
+
+ if (qfbo_is_seqcompat(ov, &sci)) {
+ /* The binary formats used here have a byte on each entry
+ * identifying the type, or a terminator (that varies
+ * in different lists).
+ */
+ uint8_t tmpbyte;
+ tmpbyte = *kind & 0xff;
+
+ DPRINTF("qfbo_get_next_type for %s", name);
+
+ switch (sci.mode) {
+ case VISIT_SEQ_COMPAT_BYTE0TERM:
+ qemu_put_byte(ov->file, tmpbyte);
+ break;
+
+ default:
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+ return;
+ }
+
+ /* Only dealing with SeqCompat's for the moment */
+ error_set(errp, QERR_UNDEFINED_ERROR);
+}
+
+static void qfbo_start_sequence_compat(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ void *opaque, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ SeqCompatInfo sci = {
+ .mode = compat_mode,
+ .data = opaque
+ };
+ SectionHeader *sh;
+ ramsecentry_header *rse_hdr;
+
+ switch (compat_mode) {
+ case VISIT_SEQ_COMPAT_FILE:
+ qemu_put_be32(ov->file, QEMU_VM_FILE_MAGIC);
+ qemu_put_be32(ov->file, QEMU_VM_FILE_VERSION);
+ break;
+
+ case VISIT_SEQ_COMPAT_SECTION_HEADER:
+ /*
+ * for VM_SECTION_FULL and VM_SECTION_START
+ * 'opaque' points to a struct Sectionheader
+ */
+ sh = opaque;
+ qemu_put_be32(ov->file, sh->section_id);
+ qemu_put_byte(ov->file, strlen(sh->idstr));
+ qemu_put_buffer(ov->file, (uint8_t *)sh->idstr, strlen(sh->idstr));
+ qemu_put_be32(ov->file, sh->instance_id);
+ qemu_put_be32(ov->file, sh->version_id);
+ break;
+
+ case VISIT_SEQ_COMPAT_SECTION_MIN:
+ /* for VM_SECTION_PART/END, the section name->ID is already known */
+ sh = opaque;
+ qemu_put_be32(ov->file, sh->section_id);
+ break;
+
+ case VISIT_SEQ_COMPAT_SUBSECTION: /* An element in a subsection list */
+ sh = opaque;
+ qemu_put_byte(ov->file, QEMU_VM_SUBSECTION);
+ qemu_put_byte(ov->file, strlen(sh->idstr));
+ qemu_put_buffer(ov->file, (uint8_t *)sh->idstr, strlen(sh->idstr));
+ qemu_put_be32(ov->file, sh->version_id);
+ break;
+
+ case VISIT_SEQ_COMPAT_BYTE0TERM:
+ case VISIT_SEQ_COMPAT_SUBSECLIST:
+ case VISIT_SEQ_COMPAT_RAMSECLIST:
+ case VISIT_SEQ_COMPAT_VMSTATE:
+ /* These don't need anything in the header on the compatibility side */
+ break;
+
+ case VISIT_SEQ_COMPAT_RAMSECENTRY:
+ rse_hdr = opaque;
+ qemu_put_be64(ov->file, rse_hdr->addr | rse_hdr->flags);
+ if ((rse_hdr->flags &
+ (RAM_SAVE_FLAG_CONTINUE | RAM_SAVE_FLAG_MEM_SIZE |
+ RAM_SAVE_FLAG_HOOK)) == 0) {
+ qemu_put_byte(ov->file, strlen(rse_hdr->idstr));
+ qemu_put_buffer(ov->file, (uint8_t *)rse_hdr->idstr,
+ strlen(rse_hdr->idstr));
+ }
+ break;
+
+ case VISIT_SEQ_COMPAT_BLOB:
+ /*
+ * Nothing in the header, but opaque gets a copy of our QEMUFile -
+ * other implementations might give a different QEMUFile
+ */
+ *(QEMUFile **)opaque = ov->file;
+ break;
+
+ }
+
+ DPRINTF("qfbo_start_sequence_compat for %s\n", name);
+ qfbo_push_seqcompat(ov, sci);
+ /* We don't need to read anything at this point */
+}
+
+static void qfbo_end_sequence_compat(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+ StackEntry *e = qfbo_pop(ov);
+ if (!e || e->type != QFOV_SEQCOMPAT) {
+ error_setg(errp, "bad struct stack %d", e ? e->type : -1);
+ if (e) {
+ g_free(e);
+ }
+ return;
+ }
+ switch (e->seqcompat_info.mode) {
+ case VISIT_SEQ_COMPAT_BYTE0TERM:
+ qemu_put_byte(ov->file, 0);
+ break;
+
+ case VISIT_SEQ_COMPAT_RAMSECLIST:
+ qemu_put_be64(ov->file, RAM_SAVE_FLAG_EOS);
+ break;
+
+ default:
+ break;
+ }
+
+ if (e->seqcompat_info.mode != compat_mode) {
+ error_setg(errp, "mismatched seqcompat mode %d/%d", compat_mode,
+ e->seqcompat_info.mode);
+ }
+
+ DPRINTF("qfbo_end_sequence_compat %s\n", name);
+ g_free(e);
+}
+
+static void qfbo_destroy(Visitor *v, Error **errp)
+{
+ QemuFileBinOutputVisitor *ov = to_ov(v);
+
+ qemu_file_bin_output_visitor_cleanup(ov);
+}
+
+Visitor *qemu_file_bin_output_get_visitor(QemuFileBinOutputVisitor *v)
+{
+ return &v->visitor;
+}
+
+void qemu_file_bin_output_visitor_cleanup(QemuFileBinOutputVisitor *ov)
+{
+ g_free(ov);
+}
+
+QemuFileBinOutputVisitor *qemu_file_bin_output_visitor_new(QEMUFile *f)
+{
+ QemuFileBinOutputVisitor *v;
+
+ v = g_malloc0(sizeof(*v));
+
+ v->file = f;
+
+ v->visitor.start_struct = qfbo_start_struct;
+ v->visitor.end_struct = qfbo_end_struct;
+ v->visitor.start_list = qfbo_start_list;
+ v->visitor.next_list = qfbo_next_list;
+ v->visitor.end_list = qfbo_end_list;
+ v->visitor.start_array = qfbo_start_array;
+ v->visitor.next_array = qfbo_next_array;
+ v->visitor.end_array = qfbo_end_array;
+ v->visitor.type_buffer = qfbo_type_buffer;
+ v->visitor.type_int = qfbo_type_int64;
+ v->visitor.type_uint8 = qfbo_type_uint8;
+ v->visitor.type_uint16 = qfbo_type_uint16;
+ v->visitor.type_uint32 = qfbo_type_uint32;
+ v->visitor.type_uint64 = qfbo_type_uint64;
+ v->visitor.type_int8 = qfbo_type_int8;
+ v->visitor.type_int16 = qfbo_type_int16;
+ v->visitor.type_int32 = qfbo_type_int32;
+ v->visitor.type_int64 = qfbo_type_int64;
+ v->visitor.type_bool = qfbo_type_bool;
+ v->visitor.type_str = qfbo_type_str;
+ v->visitor.type_str256 = qfbo_type_str256;
+ v->visitor.destroy = qfbo_destroy;
+ v->visitor.start_sequence_compat = qfbo_start_sequence_compat;
+ v->visitor.get_next_type = qfbo_get_next_type;
+ v->visitor.end_sequence_compat = qfbo_end_sequence_compat;
+ v->visitor.get_qemufile = qfbo_get_qemufile;
+
+ v->visitor.flags = VISITOR_SAVING;
+
+ QTAILQ_INIT(&v->stack);
+
+ return v;
+}
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 06/16] Visitor: Debug output visitor
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (4 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 05/16] Visitor: Binary compatible output visitor Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 07/16] Visitor: Binary compatible input visitor Dr. David Alan Gilbert (git)
` (10 subsequent siblings)
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
A migration visitor whose output is textual and purely for human
consumption for debug.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
include/qapi/qemu-file-debug-output-visitor.h | 26 ++
qapi/Makefile.objs | 1 +
qapi/qemu-file-debug-output-visitor.c | 471 ++++++++++++++++++++++++++
3 files changed, 498 insertions(+)
create mode 100644 include/qapi/qemu-file-debug-output-visitor.h
create mode 100644 qapi/qemu-file-debug-output-visitor.c
diff --git a/include/qapi/qemu-file-debug-output-visitor.h b/include/qapi/qemu-file-debug-output-visitor.h
new file mode 100644
index 0000000..427d5b2
--- /dev/null
+++ b/include/qapi/qemu-file-debug-output-visitor.h
@@ -0,0 +1,26 @@
+/*
+ * QEMUFile Visitor
+ *
+ * Copyright Red Hat, Corp. 2014
+ *
+ * Authors:
+ * David Gilbert <dgilbert@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef QEMU_FILE_DEBUG_OUTPUT_VISITOR_H
+#define QEMU_FILE_DEBUG_OUTPUT_VISITOR_H
+
+#include "visitor.h"
+
+typedef struct QemuFileDebugOutputVisitor QemuFileDebugOutputVisitor;
+
+QemuFileDebugOutputVisitor *qemu_file_debug_output_visitor_new(QEMUFile *f);
+void qemu_file_debug_output_visitor_cleanup(QemuFileDebugOutputVisitor *d);
+
+Visitor *qemu_file_debug_output_get_visitor(QemuFileDebugOutputVisitor *v);
+
+#endif
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index a054d52..06c69e6 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -2,4 +2,5 @@ util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qmp-input-visitor.o
util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
util-obj-y += string-input-visitor.o string-output-visitor.o
util-obj-y += qemu-file-binary-output-visitor.o
+util-obj-y += qemu-file-debug-output-visitor.o
util-obj-y += opts-visitor.o
diff --git a/qapi/qemu-file-debug-output-visitor.c b/qapi/qemu-file-debug-output-visitor.c
new file mode 100644
index 0000000..8694212
--- /dev/null
+++ b/qapi/qemu-file-debug-output-visitor.c
@@ -0,0 +1,471 @@
+/*
+ * QEMUFile Output Visitor
+ *
+ * Copyright Red Hat, 2014
+ *
+ * Authors:
+ * David Gilbert <dgilbert@redhat.com>
+ *
+ * Based on the binary file output visitor
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ * Produce a textual output designed purely for ease of reading
+ */
+
+#include "qapi/qemu-file-binary-output-visitor.h"
+#include "qapi/qemu-file-debug-output-visitor.h"
+#include "qapi/visitor-impl.h"
+#include "qapi/qmp/qerror.h"
+#include "qemu/queue.h"
+#include "qemu-common.h"
+#include "hw/hw.h"
+#include "migration/migration.h"
+
+struct QemuFileDebugOutputVisitor {
+ Visitor visitor;
+ int depth;
+ QEMUFile *file;
+
+ bool in_binwrapper; /* When true we're in a compat wrapper where everything
+ should be going through the QEMUFile we provide it */
+ QEMUFile *binfile; /* Passed to the caller in a compat wrapper */
+};
+
+static void qemufile_printf(QEMUFile *qf, int depth, const char *fmt, ...)
+{
+ char *tmp;
+ va_list va;
+ va_start(va, fmt);
+ assert(vasprintf(&tmp, fmt, va) != -1);
+ va_end(va);
+
+ while (depth--) {
+ qemu_put_buffer(qf, (uint8_t *)" ", 2);
+ }
+
+ qemu_put_buffer(qf, (uint8_t *)tmp, strlen(tmp));
+ qemu_fflush(qf);
+ free(tmp);
+}
+
+#define DPRINTF(fmt, ...) \
+ qemufile_printf(ov->file, ov->depth*2, fmt, __VA_ARGS__)
+
+/*
+ * TDO: This should be shared somewhere, util/hexdump.c has one to output to
+ * FILE*, and util/iov.c uses that to output an iov to FILE*, neither includes
+ * the printables.
+ */
+/* We print the iov upto 'len' bytes because the iov is allocated in chunks */
+static void hexdump_to_qemufile(QEMUFile *qf, size_t len, size_t n_iov,
+ struct iovec *iov, int depth)
+{
+ const unsigned int bytes_per_line = 16;
+ /*
+ * Of the form: 41 42 43 44 45 46 47 48 ABCDEFGH\0
+ * 3 byte> < 1 byte> <
+ */
+ const unsigned int line_buf_len = (bytes_per_line * (3+1)) + 2;
+ char linebuf[line_buf_len];
+ size_t cur_iov, cur_offset;
+ unsigned int line_index = 0;
+
+ memset(linebuf, ' ', bytes_per_line * (3+1)+2);
+ linebuf[line_buf_len-1] = '\0';
+
+ for (cur_iov = 0; len && (cur_iov < n_iov); cur_iov++) {
+ for (cur_offset = 0;
+ len && (cur_offset < iov[cur_iov].iov_len);
+ cur_offset++, len--) {
+ uint8_t cur_byte = ((uint8_t *)iov[cur_iov].iov_base)[cur_offset];
+ const char *hexstring = "0123456789abcdef";
+
+ linebuf[line_index*3] = hexstring[(cur_byte >> 4) & 0xf];
+ linebuf[line_index*3+1] = hexstring[cur_byte & 0xf];
+ linebuf[bytes_per_line*3+1+line_index] =
+ isprint(cur_byte) ? cur_byte : '.';
+ line_index++;
+
+ if (line_index == bytes_per_line) {
+ qemufile_printf(qf, depth, "%s\n", linebuf);
+ line_index = 0;
+ }
+ }
+ }
+ if (line_index) {
+ /* Still some bytes left */
+ if (line_index != bytes_per_line) {
+ memset(linebuf + line_index*3, '#', 3*(bytes_per_line-line_index));
+ memset(linebuf + bytes_per_line*3+1+line_index, '#',
+ bytes_per_line-line_index);
+ qemufile_printf(qf, depth, "%s\n", linebuf);
+ }
+ }
+}
+
+static QemuFileDebugOutputVisitor *to_ov(Visitor *v)
+{
+ return container_of(v, QemuFileDebugOutputVisitor, visitor);
+}
+
+static void qfdo_push(QemuFileDebugOutputVisitor *ov)
+{
+ ov->depth++;
+}
+
+static void qfdo_pop(QemuFileDebugOutputVisitor *ov)
+{
+ ov->depth--;
+ assert(ov >= 0);
+ return;
+}
+
+static void qfdo_start_struct(Visitor *v, void **obj, const char *kind,
+ const char *name, size_t unused,
+ Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+
+ DPRINTF("<struct '%s' of '%s'\n", name, kind);
+ qfdo_push(ov);
+}
+
+static void qfdo_end_struct(Visitor *v, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+
+ qfdo_pop(ov);
+ DPRINTF("struct %s>\n", "");
+}
+
+static void qfdo_start_list(Visitor *v, const char *name,
+ Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ DPRINTF("<list '%s'\n", name);
+ qfdo_push(ov);
+}
+
+static GenericList *qfdo_next_list(Visitor *v, GenericList **list,
+ Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+
+ DPRINTF("|list %s\n", "");
+
+ return NULL; /* Not generally valid! */
+}
+
+static void qfdo_end_list(Visitor *v, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+
+ qfdo_pop(ov);
+ DPRINTF("list %s>\n", "");
+}
+
+static void qfdo_start_array(Visitor *v, void **obj,
+ const char *name,
+ size_t elem_count,
+ size_t elem_size, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ DPRINTF("<array '%s' [%zd] of %zd\n", name, elem_count, elem_size);
+
+ qfdo_push(ov);
+}
+
+static void qfdo_next_array(Visitor *v, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ DPRINTF("|array %s\n", "");
+}
+
+static void qfdo_end_array(Visitor *v, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ qfdo_pop(ov);
+
+ DPRINTF("array %s>\n", "");
+}
+
+static void qfdo_type_str(Visitor *v, char **obj, const char *name,
+ Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+
+ DPRINTF("string: '%s'=%s\n", name, *obj);
+}
+
+/* A string upto 256 bytes in length (including terminator)
+ * output as length byte (not including term) followed by text
+ * (also not including term)
+ */
+static void qfdo_type_str256(Visitor *v, char *obj, const char *name,
+ Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+
+ DPRINTF("str256: '%s'=%s\n", name, obj);
+}
+
+static void qfdo_type_buffer(Visitor *v, void *data, size_t len, bool async,
+ const char *name, Error **errp)
+{
+ struct iovec tmpiov;
+ tmpiov.iov_base = data;
+ tmpiov.iov_len = len;
+
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ DPRINTF("Buffer of '%s' len=%zd async=%d\n", name, len, async);
+ hexdump_to_qemufile(ov->file, len, 1, &tmpiov, ov->depth * 2);
+}
+
+static void qfdo_type_uint8(Visitor *v, uint8_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ DPRINTF("uint8_t %s: 0x%2x\n", name, *obj);
+}
+
+static void qfdo_type_uint16(Visitor *v, uint16_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ DPRINTF("uint16_t %s: 0x%4x\n", name, *obj);
+}
+
+static void qfdo_type_uint32(Visitor *v, uint32_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ DPRINTF("uint32_t %s: 0x%8x\n", name, *obj);
+}
+
+static void qfdo_type_uint64(Visitor *v, uint64_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ DPRINTF("uint64_t %s: 0x%16lx\n", name, *obj);
+}
+
+static void qfdo_type_int8(Visitor *v, int8_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ DPRINTF("int8_t %s: 0x%2x\n", name, *obj);
+}
+
+static void qfdo_type_int16(Visitor *v, int16_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ DPRINTF("int16_t %s: 0x%4x\n", name, *obj);
+}
+
+static void qfdo_type_int32(Visitor *v, int32_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ DPRINTF("int32_t %s: 0x%8x\n", name, *obj);
+}
+
+static void qfdo_type_int64(Visitor *v, int64_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ DPRINTF("int64_t %s: 0x%16lx\n", name, *obj);
+}
+
+static void qfdo_type_bool(Visitor *v, bool *obj, const char *name,
+ Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ DPRINTF("bool %s: %d\n", name, *obj);
+}
+
+static QEMUFile *qfdo_get_qemufile(Visitor *v)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+
+ return ov->file;
+}
+
+static void qfdo_get_next_type(Visitor *v, int *kind, const int *qobjects,
+ const char *name, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+
+ DPRINTF("|next_type %s\n", name);
+}
+
+static void qfdo_start_sequence_compat(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ void *opaque, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+ SectionHeader *sh;
+ ramsecentry_header *rse_hdr;
+ const char *tmps;
+
+ switch (compat_mode) {
+ case VISIT_SEQ_COMPAT_FILE:
+ DPRINTF("<SEQCOMPAT SoF: %s\n", name);
+ break;
+
+ case VISIT_SEQ_COMPAT_SECTION_HEADER:
+ /*
+ * for VM_SECTION_FULL and VM_SECTION_START
+ * 'opaque' points to a struct Sectionheader
+ */
+ sh = opaque;
+ DPRINTF("<SEQCOMPAT Section (%s): %s (%d / %d version %d)\n", name,
+ sh->idstr, sh->section_id, sh->instance_id, sh->version_id);
+ break;
+
+ case VISIT_SEQ_COMPAT_SECTION_MIN:
+ /* VM_SECTION_PART/END where the section name->ID is already known */
+ sh = opaque;
+ DPRINTF("<SEQCOMPAT Section (%s): %d\n", name, sh->section_id);
+ break;
+
+
+ case VISIT_SEQ_COMPAT_BYTE0TERM:
+ DPRINTF("<SEQCOMPAT 0 term list (%s)\n", name);
+ break;
+
+ case VISIT_SEQ_COMPAT_SUBSECLIST:
+ DPRINTF("<SEQCOMPAT Subsection list (%s)\n", name);
+ break;
+
+ case VISIT_SEQ_COMPAT_SUBSECTION:
+ DPRINTF("<SEQCOMPAT Subsection (%s)\n", name);
+ break;
+
+ case VISIT_SEQ_COMPAT_RAMSECLIST:
+ DPRINTF("<SEQCOMPAT RAMseclist (%s)\n", name);
+ break;
+
+ case VISIT_SEQ_COMPAT_VMSTATE:
+ DPRINTF("<SEQCOMPAT VMState (%s)\n", name);
+ break;
+
+ case VISIT_SEQ_COMPAT_RAMSECENTRY:
+ rse_hdr = opaque;
+ if ((rse_hdr->flags &
+ (RAM_SAVE_FLAG_CONTINUE | RAM_SAVE_FLAG_MEM_SIZE |
+ RAM_SAVE_FLAG_HOOK)) == 0) {
+ tmps = rse_hdr->idstr;
+ } else {
+ tmps = "(cont)";
+ }
+ DPRINTF("<SEQCOMPAT RAMsecentry %s for 0x%lx flags=0x%x id=%s\n",
+ name, rse_hdr->addr, rse_hdr->flags, tmps);
+ break;
+
+ case VISIT_SEQ_COMPAT_BLOB:
+ DPRINTF("<SEQCOMPAT Blob (%s)\n", name);
+
+ /* Opaque is given a QEMUFile into which it writes the binary data */
+ ov->in_binwrapper = true;
+ ov->binfile = qemu_bufopen("w", NULL);
+ /* and give that wrapper a binary output visitor so that it keeps
+ * substructures in compatibility mode
+ */
+ QemuFileBinOutputVisitor *qfbov =
+ qemu_file_bin_output_visitor_new(ov->binfile);
+ Visitor *v = qemu_file_bin_output_get_visitor(qfbov);
+ qemu_file_set_tmp_visitor(ov->binfile, v);
+
+ *(QEMUFile **)opaque = ov->binfile;
+ break;
+ }
+
+ qfdo_push(ov);
+}
+
+static void qfdo_end_sequence_compat(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+
+ /* bin wrappers can't nest - or at least if they did they'd have a new
+ * visitor instance
+ */
+ if (ov->in_binwrapper) {
+ Error *local_err = NULL;
+ Visitor *bv = qemu_file_get_tmp_visitor(ov->binfile);
+
+ visit_destroy(bv, &local_err);
+ const QEMUSizedBuffer *qsb = qemu_buf_get(ov->binfile);
+ size_t len = qsb_get_length(qsb);
+
+ hexdump_to_qemufile(ov->file, len, qsb->n_iov, qsb->iov, ov->depth*2);
+
+ qemu_fclose(ov->binfile);
+ ov->in_binwrapper = false;
+ }
+
+ qfdo_pop(ov);
+ DPRINTF("SEQCOMPAT (%d) %s>\n", compat_mode, name);
+}
+
+static void qfdo_destroy(Visitor *v, Error **errp)
+{
+ QemuFileDebugOutputVisitor *ov = to_ov(v);
+
+ qemu_file_debug_output_visitor_cleanup(ov);
+}
+
+Visitor *qemu_file_debug_output_get_visitor(QemuFileDebugOutputVisitor *v)
+{
+ return &v->visitor;
+}
+
+void qemu_file_debug_output_visitor_cleanup(QemuFileDebugOutputVisitor *ov)
+{
+ g_free(ov);
+}
+
+QemuFileDebugOutputVisitor *qemu_file_debug_output_visitor_new(QEMUFile *f)
+{
+ QemuFileDebugOutputVisitor *v;
+
+ v = g_malloc0(sizeof(*v));
+
+ v->file = f;
+
+ v->visitor.start_struct = qfdo_start_struct;
+ v->visitor.end_struct = qfdo_end_struct;
+ v->visitor.start_list = qfdo_start_list;
+ v->visitor.next_list = qfdo_next_list;
+ v->visitor.end_list = qfdo_end_list;
+ v->visitor.start_array = qfdo_start_array;
+ v->visitor.next_array = qfdo_next_array;
+ v->visitor.end_array = qfdo_end_array;
+ v->visitor.type_buffer = qfdo_type_buffer;
+ v->visitor.type_int = qfdo_type_int64;
+ v->visitor.type_uint8 = qfdo_type_uint8;
+ v->visitor.type_uint16 = qfdo_type_uint16;
+ v->visitor.type_uint32 = qfdo_type_uint32;
+ v->visitor.type_uint64 = qfdo_type_uint64;
+ v->visitor.type_int8 = qfdo_type_int8;
+ v->visitor.type_int16 = qfdo_type_int16;
+ v->visitor.type_int32 = qfdo_type_int32;
+ v->visitor.type_int64 = qfdo_type_int64;
+ v->visitor.type_bool = qfdo_type_bool;
+ v->visitor.type_str = qfdo_type_str;
+ v->visitor.type_str256 = qfdo_type_str256;
+ v->visitor.destroy = qfdo_destroy;
+ v->visitor.start_sequence_compat = qfdo_start_sequence_compat;
+ v->visitor.get_next_type = qfdo_get_next_type;
+ v->visitor.end_sequence_compat = qfdo_end_sequence_compat;
+ v->visitor.get_qemufile = qfdo_get_qemufile;
+
+ v->visitor.flags = VISITOR_SAVING;
+
+ return v;
+}
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 07/16] Visitor: Binary compatible input visitor
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (5 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 06/16] Visitor: Debug " Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 08/16] Visitor: Output path Dr. David Alan Gilbert (git)
` (9 subsequent siblings)
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
based on Michael Roth's
https://lists.gnu.org/archive/html/qemu-devel/2011-09/msg02470.html
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
include/qapi/qemu-file-binary-input-visitor.h | 27 +
qapi/Makefile.objs | 2 +-
qapi/qemu-file-binary-input-visitor.c | 688 ++++++++++++++++++++++++++
3 files changed, 716 insertions(+), 1 deletion(-)
create mode 100644 include/qapi/qemu-file-binary-input-visitor.h
create mode 100644 qapi/qemu-file-binary-input-visitor.c
diff --git a/include/qapi/qemu-file-binary-input-visitor.h b/include/qapi/qemu-file-binary-input-visitor.h
new file mode 100644
index 0000000..dd7e40e
--- /dev/null
+++ b/include/qapi/qemu-file-binary-input-visitor.h
@@ -0,0 +1,27 @@
+/*
+ * QEMUFile Visitor
+ *
+ * Copyright IBM, Corp. 2011
+ *
+ * Authors:
+ * Michael Roth <mdroth@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef QEMU_FILE_BINARY_INPUT_VISITOR_H
+#define QEMU_FILE_BINARY_INPUT_VISITOR_H
+
+#include "visitor.h"
+
+typedef struct QemuFileBinInputVisitor QemuFileBinInputVisitor;
+
+QemuFileBinInputVisitor *qemu_file_bin_input_visitor_new(QEMUFile *f);
+void qemu_file_bin_input_visitor_cleanup(QemuFileBinInputVisitor *d);
+
+Visitor *qemu_file_bin_input_get_visitor(QemuFileBinInputVisitor *v);
+
+#endif
+
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index 06c69e6..3d9d47a 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -1,6 +1,6 @@
util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qmp-input-visitor.o
util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
util-obj-y += string-input-visitor.o string-output-visitor.o
-util-obj-y += qemu-file-binary-output-visitor.o
+util-obj-y += qemu-file-binary-output-visitor.o qemu-file-binary-input-visitor.o
util-obj-y += qemu-file-debug-output-visitor.o
util-obj-y += opts-visitor.o
diff --git a/qapi/qemu-file-binary-input-visitor.c b/qapi/qemu-file-binary-input-visitor.c
new file mode 100644
index 0000000..162765e
--- /dev/null
+++ b/qapi/qemu-file-binary-input-visitor.c
@@ -0,0 +1,688 @@
+/*
+ * QEMUFile Output Visitor
+ *
+ * Copyright IBM, Corp. 2011
+ * Copyright Red Hat, Corp. 2014
+ *
+ * Authors:
+ * Michael Roth <mdroth@linux.vnet.ibm.com>
+ * David Gilbert <dgilbert@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include "qapi/qemu-file-binary-input-visitor.h"
+#include "qapi/visitor-impl.h"
+#include "qapi/qmp/qerror.h"
+#include "qemu/queue.h"
+#include "qemu-common.h"
+#include "hw/hw.h"
+#include "migration/migration.h"
+
+/* Note that this can generate so much debug virt-test times out */
+#if 0
+#define DPRINTF(v, fmt, ...) \
+ do { \
+ fprintf(stderr, "%*s qfbiv/%s/%d: " fmt "\n", v->depth, "", __func__, \
+ __LINE__, ## __VA_ARGS__); \
+ } while (0)
+#else
+#define DPRINTF(v, fmt, ...) \
+ do { } while (0)
+#endif
+
+typedef struct {
+ size_t elem_count;
+ size_t elem_size;
+ size_t pos;
+} ArrayInfo;
+
+typedef struct {
+ Visit_seq_compat_mode mode;
+ const void *data;
+ bool hit_end;
+} SeqCompatInfo;
+
+typedef struct StackEntry {
+ enum {
+ QFIV_ARRAY,
+ QFIV_LIST,
+ QFIV_STRUCT,
+ QFIV_SEQCOMPAT,
+ } type;
+ ArrayInfo array_info;
+ SeqCompatInfo seqcompat_info;
+ QTAILQ_ENTRY(StackEntry) node;
+} StackEntry;
+
+struct QemuFileBinInputVisitor {
+ Visitor visitor;
+ QTAILQ_HEAD(, StackEntry) stack;
+ QEMUFile *file;
+ unsigned int depth;
+};
+
+static QemuFileBinInputVisitor *to_iv(Visitor *v)
+{
+ return container_of(v, QemuFileBinInputVisitor, visitor);
+}
+
+static void qfbi_push(QemuFileBinInputVisitor *iv, StackEntry *e)
+{
+ QTAILQ_INSERT_HEAD(&iv->stack, e, node);
+ iv->depth++;
+}
+
+static void qfbi_push_array(QemuFileBinInputVisitor *iv,
+ ArrayInfo ai)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFIV_ARRAY;
+ e->array_info = ai;
+ qfbi_push(iv, e);
+}
+
+static void qfbi_push_list(QemuFileBinInputVisitor *iv)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFIV_LIST;
+ qfbi_push(iv, e);
+}
+
+static void qfbi_push_seqcompat(QemuFileBinInputVisitor *iv,
+ SeqCompatInfo sci)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFIV_SEQCOMPAT;
+ e->seqcompat_info = sci;
+ qfbi_push(iv, e);
+}
+
+static void qfbi_push_struct(QemuFileBinInputVisitor *iv)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFIV_STRUCT;
+ qfbi_push(iv, e);
+}
+
+static void *qfbi_pop(QemuFileBinInputVisitor *iv)
+{
+ StackEntry *e = QTAILQ_FIRST(&iv->stack);
+ QTAILQ_REMOVE(&iv->stack, e, node);
+ iv->depth--;
+ return e;
+}
+
+static bool qfbi_is_array(QemuFileBinInputVisitor *iv)
+{
+ StackEntry *e = QTAILQ_FIRST(&iv->stack);
+ return e->type == QFIV_ARRAY;
+}
+
+static bool qfbi_is_list(QemuFileBinInputVisitor *iv)
+{
+ StackEntry *e = QTAILQ_FIRST(&iv->stack);
+ return e && e->type == QFIV_LIST;
+}
+
+/* If we are in a seqcompat list return true and fill in
+ * sci with the compat mode
+ */
+static bool qfbi_is_seqcompat(QemuFileBinInputVisitor *iv,
+ SeqCompatInfo **sci)
+{
+ StackEntry *e = QTAILQ_FIRST(&iv->stack);
+ if (e && e->type == QFIV_SEQCOMPAT) {
+ *sci = &e->seqcompat_info;
+ return true;
+ }
+ return false;
+}
+
+static void qfbi_start_struct(Visitor *v, void **obj,
+ const char *kind,
+ const char *name, size_t size,
+ Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ DPRINTF(iv, "for '%s' of '%s'", name, kind);
+
+ qfbi_push_struct(iv);
+}
+
+static void qfbi_end_struct(Visitor *v, Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ StackEntry *e = qfbi_pop(iv);
+
+ DPRINTF(iv, "<");
+ if (!e || e->type != QFIV_STRUCT) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+ g_free(e);
+}
+
+static void qfbi_start_list(Visitor *v, const char *name, Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ qfbi_push_list(iv);
+}
+
+static GenericList *qfbi_next_list(Visitor *v, GenericList **list, Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ GenericList *entry;
+
+ if (!qfbi_is_list(iv)) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ }
+
+ /* Some users maintain their own list structure */
+ if (!list) {
+ return NULL;
+ }
+
+ entry = g_malloc0(sizeof(*entry));
+ if (*list) {
+ (*list)->next = entry;
+ }
+
+ *list = entry;
+ return entry;
+}
+
+static void qfbi_end_list(Visitor *v, Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ StackEntry *e = qfbi_pop(iv);
+ if (!e || e->type != QFIV_LIST) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+ g_free(e);
+}
+
+static void qfbi_start_array(Visitor *v, void **obj, const char *name,
+ size_t elem_count, size_t elem_size, Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ ArrayInfo ai = {
+ .elem_count = elem_count,
+ .elem_size = elem_size,
+ .pos = 0
+ };
+ if (obj && (*obj == NULL) && elem_size) {
+ *obj = g_malloc0(elem_count * elem_size);
+ }
+ qfbi_push_array(iv, ai);
+}
+
+static void qfbi_next_array(Visitor *v, Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ StackEntry *e = QTAILQ_FIRST(&iv->stack);
+
+ if (!qfbi_is_array(iv) ||
+ e->array_info.pos >= e->array_info.elem_count) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ }
+
+ e->array_info.pos++;
+}
+
+static void qfbi_end_array(Visitor *v, Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ StackEntry *e = qfbi_pop(iv);
+ if (!e || e->type != QFIV_ARRAY) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+ g_free(e);
+}
+
+static void qfbi_type_str(Visitor *v, char **obj, const char *name,
+ Error **errp)
+{
+ if (obj) {
+ g_free(*obj);
+ }
+}
+
+/* Read in a byte+buffer -> giving a string. obj must be a buffer of
+ * at least 256 chars in length
+ */
+static void qfbi_type_str256(Visitor *v, char *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ unsigned int len = qemu_get_byte(iv->file);
+
+ qemu_get_buffer(iv->file, (uint8_t *)obj, len);
+ obj[len] = 0;
+ DPRINTF(iv, "for '%s' len=%d str=%s", name, len, obj);
+}
+
+static void qfbi_type_buffer(Visitor *v, void *data, size_t len, bool async,
+ const char *name, Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ qemu_get_buffer(iv->file, data, len);
+}
+
+static void qfbi_type_uint8(Visitor *v, uint8_t *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ *obj = qemu_get_byte(iv->file);
+ DPRINTF(iv, "for '%s' / %u", name, *obj);
+}
+
+static void qfbi_type_uint16(Visitor *v, uint16_t *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ qemu_get_be16s(iv->file, obj);
+ DPRINTF(iv, "for '%s' / %u", name, *obj);
+}
+
+static void qfbi_type_uint32(Visitor *v, uint32_t *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ qemu_get_be32s(iv->file, obj);
+ DPRINTF(iv, "for '%s' / %u", name, *obj);
+}
+
+static void qfbi_type_uint64(Visitor *v, uint64_t *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ qemu_get_be64s(iv->file, obj);
+ DPRINTF(iv, "for '%s' / %lu", name, *obj);
+}
+
+static void qfbi_type_int8(Visitor *v, int8_t *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ *obj = qemu_get_sbyte(iv->file);
+ DPRINTF(iv, "for '%s' / %d", name, *obj);
+}
+
+static void qfbi_type_int16(Visitor *v, int16_t *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ qemu_get_sbe16s(iv->file, obj);
+ DPRINTF(iv, "for '%s' / %d", name, *obj);
+}
+
+static void qfbi_type_int32(Visitor *v, int32_t *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ qemu_get_sbe32s(iv->file, obj);
+ DPRINTF(iv, "for '%s' / %d", name, *obj);
+}
+
+static void qfbi_type_int64(Visitor *v, int64_t *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ qemu_get_sbe64s(iv->file, obj);
+ DPRINTF(iv, "for '%s' / %ld", name, *obj);
+}
+
+static void qfbi_type_bool(Visitor *v, bool *obj, const char *name,
+ Error **errp)
+{
+ uint8_t val;
+ qfbi_type_uint8(v, &val, name, errp);
+ *obj = val;
+}
+
+static QEMUFile *qfbi_get_qemufile(Visitor *v)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+
+ return iv->file;
+}
+
+static void qfbi_get_next_type(Visitor *v, int *kind, const int *qobjects,
+ const char *name, Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ SeqCompatInfo *sci;
+ uint64_t tmp64;
+ uint8_t tmpbyte;
+
+ DPRINTF(iv, "for '%s'", name);
+ if (qfbi_is_seqcompat(iv, &sci)) {
+ DPRINTF(iv, "/seqcompat for '%s'", name);
+ if (sci->hit_end) {
+ error_setg(errp, "Attempted to read beyond the end of list '%s'",
+ name);
+ *kind = -1;
+ return;
+ }
+
+ DPRINTF(iv, "/seqcompat for '%s' mode=%d tmpbyte=%d", name, sci->mode,
+ tmpbyte);
+ switch (sci->mode) {
+ case VISIT_SEQ_COMPAT_BYTE0TERM:
+ /*
+ * End of the list is marked by a 0 byte,
+ * we'll consume the byte read
+ */
+ tmpbyte = qemu_get_byte(iv->file);
+ if (tmpbyte == 0) {
+ sci->hit_end = true;
+ }
+ *kind = tmpbyte;
+ break;
+
+ case VISIT_SEQ_COMPAT_RAMSECLIST:
+ /*
+ * the RAM sections are lists of entries each that start with a
+ * 64 bit word that has an address or length, OR'd with a set of
+ * the RAM_SAVE_FLAG_ bits; RAM_SAVE_FLAG_EOS indicates the end
+ * of the list.
+ * We don't consume the word since it'll be consumed by
+ * the SEQ_COMPAT_RAMSECENTRY; except for the EOS
+ */
+
+ if (qemu_peek_buffer(iv->file, (uint8_t *)&tmp64, 8, 0) != 8) {
+ error_setg(errp, "Failed to read flag word for '%s'", name);
+ *kind = -1;
+ return;
+ }
+
+ tmp64 = be64_to_cpu(tmp64);
+
+ *kind = tmp64 & RAM_SAVE_FLAG_MASK;
+ DPRINTF(iv, "tmp64=0x%lx *kind=%x\n", tmp64, *kind);
+ if (tmp64 & RAM_SAVE_FLAG_EOS) {
+ tmp64 = qemu_get_be64(iv->file); /* Consume terminator */
+ sci->hit_end = true;
+ }
+ break;
+
+ case VISIT_SEQ_COMPAT_SUBSECLIST:
+ /* In theory subsections are terminated by a
+ * byte != QEMU_VM_SUBSECTION
+ * and that byte is part of whatever comes next, so we mustn't
+ * consume the terminator if it's not valid.
+ * However, it's worse than that, historical layouts weren't
+ * really that careful about what came next, so you can get
+ * false sections, so we check everything in the header and if
+ * it's not valid we declare it end-of-section.
+ * For this case the 'opaque' passed in is the parent string
+ * name.
+ */
+ if (qemu_peek_buffer(iv->file, &tmpbyte, 1, 0) != 1) {
+ error_setg(errp, "Failed to read type byte for '%s'", name);
+ *kind = -1;
+ return;
+ }
+
+ if (tmpbyte == QEMU_VM_SUBSECTION) {
+ uint8_t len, size;
+ char idstr[256];
+ char *parent_name = (char *)sci->data;
+ size_t pn_len = strlen(parent_name);
+ DPRINTF(iv, "/subsection for '%s'/%s", name, parent_name);
+
+ len = qemu_peek_byte(iv->file, 1);
+ if (len < (pn_len + 1)) {
+ DPRINTF(iv, "/subsection for '%s'/%s - len too short",
+ name, parent_name);
+ sci->hit_end = true;
+ *kind = 0xff;
+ break;
+ }
+ size = qemu_peek_buffer(iv->file, (uint8_t *)idstr,
+ len, 2);
+ if (size != len) {
+ DPRINTF(iv, "/subsection for '%s'/%s - size!=len",
+ name, parent_name);
+ sci->hit_end = true;
+ *kind = 0xff;
+ break;
+ }
+ idstr[size] = 0;
+
+ if (strncmp(parent_name, idstr, pn_len) != 0) {
+ DPRINTF(iv, "/subsection for n='%s' / pn='%s' "
+ "idstr='%s' - invalid subsection name",
+ name, parent_name, idstr);
+ /* it don't have a valid subsection name */
+ sci->hit_end = true;
+ *kind = 0xff;
+ break;
+ }
+ qemu_file_skip(iv->file, 1); /* subsection byte */
+
+ /* We're not going to consume the name or idstr here
+ * since they're logically part of the item not the
+ * list, so they're going to get re-read.
+ */
+ DPRINTF(iv, "/subsection for '%s' got %s from %s "
+ "tmpbyte=%d", name, idstr, parent_name,
+ tmpbyte);
+ } else {
+ sci->hit_end = true;
+ }
+ *kind = tmpbyte;
+ break;
+
+ default:
+ *kind = -1;
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+ return;
+ }
+
+ /* Only dealing with SeqCompat's for the moment */
+ error_set(errp, QERR_UNDEFINED_ERROR);
+}
+
+static void qfbi_start_sequence_compat(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ void *opaque, Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ SeqCompatInfo sci = {
+ .mode = compat_mode,
+ .data = opaque
+ };
+ SectionHeader *sh;
+ ramsecentry_header *rse_hdr;
+ unsigned int len;
+ uint32_t tmp;
+ uint64_t tmp64;
+
+ switch (compat_mode) {
+ case VISIT_SEQ_COMPAT_FILE:
+ tmp = qemu_get_be32(iv->file);
+ if (tmp != QEMU_VM_FILE_MAGIC) {
+ error_setg(errp, "Incorrect SaveVM file header (read)");
+ return;
+ }
+
+ tmp = qemu_get_be32(iv->file);
+ if (tmp == QEMU_VM_FILE_VERSION_COMPAT) {
+ error_setg(errp, "SaveVM v2 format is obsolete and will not load");
+ return;
+ }
+ if (tmp != QEMU_VM_FILE_VERSION) {
+ error_setg(errp, "Unsupported SaveVM format (%d)", tmp);
+ return;
+ }
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+ case VISIT_SEQ_COMPAT_SECTION_HEADER:
+ /*
+ * for VM_SECTION_FULL and VM_SECTION_START
+ * 'opaque' points to a struct Sectionheader
+ */
+ sh = opaque;
+ sh->section_id = qemu_get_be32(iv->file);
+ len = qemu_get_byte(iv->file);
+ qemu_get_buffer(iv->file, (uint8_t *)sh->idstr, len);
+ sh->idstr[len] = 0;
+ sh->instance_id = qemu_get_be32(iv->file);
+ sh->version_id = qemu_get_be32(iv->file);
+ DPRINTF(iv, "for '%s'/%s v %d", name, sh->idstr, sh->version_id);
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+ case VISIT_SEQ_COMPAT_SECTION_MIN:
+ /* for VM_SECTION_PART?END where the section ID is already known */
+ sh = opaque;
+ sh->section_id = qemu_get_be32(iv->file);
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+
+ case VISIT_SEQ_COMPAT_VMSTATE:
+ /* These don't need anything in the header on the compatibility side */
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+ case VISIT_SEQ_COMPAT_BYTE0TERM:
+ case VISIT_SEQ_COMPAT_SUBSECLIST:
+ case VISIT_SEQ_COMPAT_RAMSECLIST:
+ /* These don't need anything in the header on the compatibility side */
+ break;
+
+ case VISIT_SEQ_COMPAT_RAMSECENTRY:
+ rse_hdr = opaque;
+ tmp64 = qemu_get_be64(iv->file);
+ rse_hdr->flags = tmp64 & RAM_SAVE_FLAG_MASK;
+ rse_hdr->addr = tmp64 - rse_hdr->flags;
+ if ((rse_hdr->flags &
+ (RAM_SAVE_FLAG_CONTINUE | RAM_SAVE_FLAG_MEM_SIZE |
+ RAM_SAVE_FLAG_HOOK)) == 0) {
+ len = qemu_get_byte(iv->file);
+ qemu_get_buffer(iv->file, (uint8_t *)rse_hdr->idstr, len);
+ rse_hdr->idstr[len] = 0;
+ }
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+ case VISIT_SEQ_COMPAT_SUBSECTION: /* An element in a subsection list */
+ sh = opaque;
+ len = qemu_get_byte(iv->file);
+ qemu_get_buffer(iv->file, (uint8_t *)sh->idstr, len);
+ sh->idstr[len] = 0;
+ sh->version_id = qemu_get_be32(iv->file);
+ DPRINTF(iv, "for '%s'/%s v %d", name, sh->idstr, sh->version_id);
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+ case VISIT_SEQ_COMPAT_BLOB:
+ /* Nothing in the header, but opaque gets a copy of our QEMUFile -
+ * other implementations might give a different QEMUFile
+ */
+ *(QEMUFile **)opaque = iv->file;
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+ }
+
+ DPRINTF(iv, "for '%s'", name);
+ qfbi_push_seqcompat(iv, sci);
+
+
+ /* We don't need to read anything at this point */
+}
+
+static void qfbi_end_sequence_compat(Visitor *v, const char* name,
+ Visit_seq_compat_mode compat_mode,
+ Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+ StackEntry *e = qfbi_pop(iv);
+ DPRINTF(iv, "> for '%s'", name);
+ if (!e || e->type != QFIV_SEQCOMPAT) {
+ error_setg(errp, "bad struct stack %d", e ? e->type : -1);
+ if (e) {
+ g_free(e);
+ }
+ return;
+ }
+ if (e->seqcompat_info.mode != compat_mode) {
+ error_setg(errp, "mismatched seqcompat mode %d/%d", compat_mode,
+ e->seqcompat_info.mode);
+ }
+ if (!*errp && !e->seqcompat_info.hit_end) {
+ error_setg(errp, "Didn't read the whole of list for '%s'", name);
+ }
+ g_free(e);
+}
+
+static void qfbi_destroy(Visitor *v, Error **errp)
+{
+ QemuFileBinInputVisitor *iv = to_iv(v);
+
+ qemu_file_bin_input_visitor_cleanup(iv);
+}
+
+Visitor *qemu_file_bin_input_get_visitor(QemuFileBinInputVisitor *iv)
+{
+ return &iv->visitor;
+}
+
+void qemu_file_bin_input_visitor_cleanup(QemuFileBinInputVisitor *iv)
+{
+ g_free(iv);
+}
+
+QemuFileBinInputVisitor *qemu_file_bin_input_visitor_new(QEMUFile *f)
+{
+ QemuFileBinInputVisitor *v;
+
+ v = g_malloc0(sizeof(*v));
+
+ v->file = f;
+
+ v->visitor.start_struct = qfbi_start_struct;
+ v->visitor.end_struct = qfbi_end_struct;
+ v->visitor.start_list = qfbi_start_list;
+ v->visitor.next_list = qfbi_next_list;
+ v->visitor.end_list = qfbi_end_list;
+ v->visitor.start_array = qfbi_start_array;
+ v->visitor.next_array = qfbi_next_array;
+ v->visitor.end_array = qfbi_end_array;
+ v->visitor.type_int = qfbi_type_int64;
+ v->visitor.type_buffer = qfbi_type_buffer;
+ v->visitor.type_uint8 = qfbi_type_uint8;
+ v->visitor.type_uint16 = qfbi_type_uint16;
+ v->visitor.type_uint32 = qfbi_type_uint32;
+ v->visitor.type_uint64 = qfbi_type_uint64;
+ v->visitor.type_int8 = qfbi_type_int8;
+ v->visitor.type_int16 = qfbi_type_int16;
+ v->visitor.type_int32 = qfbi_type_int32;
+ v->visitor.type_int64 = qfbi_type_int64;
+ v->visitor.type_bool = qfbi_type_bool;
+ v->visitor.type_str = qfbi_type_str;
+ v->visitor.type_str256 = qfbi_type_str256;
+ v->visitor.destroy = qfbi_destroy;
+ v->visitor.start_sequence_compat = qfbi_start_sequence_compat;
+ v->visitor.get_next_type = qfbi_get_next_type;
+ v->visitor.end_sequence_compat = qfbi_end_sequence_compat;
+ v->visitor.get_qemufile = qfbi_get_qemufile;
+
+ v->visitor.flags = VISITOR_LOADING;
+
+ QTAILQ_INIT(&v->stack);
+ v->depth = 0;
+
+ return v;
+}
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 08/16] Visitor: Output path
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (6 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 07/16] Visitor: Binary compatible input visitor Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 09/16] Visitor: Load path Dr. David Alan Gilbert (git)
` (8 subsequent siblings)
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Replace QEMUFile by Visitor in most of the output path.
There are still a few places that want a QEMUFile* and
those are TODOs at the moment.
Hacks in pci.c, spapr, spapr_vscsi.c for now where new
visitor/qemufile boundaries have been added.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
arch_init.c | 147 ++++++++++++++++++++++---------
block-migration.c | 13 ++-
hw/pci/pci.c | 2 +-
hw/ppc/spapr.c | 9 +-
hw/scsi/spapr_vscsi.c | 3 +-
include/migration/vmstate.h | 6 +-
savevm.c | 208 +++++++++++++++++++++++++++++++-------------
vmstate.c | 102 ++++++++++++++++++----
8 files changed, 365 insertions(+), 125 deletions(-)
diff --git a/arch_init.c b/arch_init.c
index 73b9303..02bf78a 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -31,6 +31,7 @@
#include "config.h"
#include "monitor/monitor.h"
#include "sysemu/sysemu.h"
+#include "qapi/visitor.h"
#include "qemu/bitops.h"
#include "qemu/bitmap.h"
#include "sysemu/arch_init.h"
@@ -60,6 +61,13 @@
do { } while (0)
#endif
+#define LOCAL_ERR_REPORT(fallout) \
+ if (local_err) { \
+ error_report("%s:%d %s", __func__, __LINE__, \
+ error_get_pretty(local_err)); \
+ fallout \
+ }
+
#ifdef TARGET_SPARC
int graphic_width = 1024;
int graphic_height = 768;
@@ -70,7 +78,6 @@ int graphic_height = 600;
int graphic_depth = 32;
#endif
-
#if defined(TARGET_ALPHA)
#define QEMU_ARCH QEMU_ARCH_ALPHA
#elif defined(TARGET_ARM)
@@ -281,20 +288,22 @@ uint64_t xbzrle_mig_pages_overflow(void)
return acct_info.xbzrle_overflows;
}
-static size_t save_block_hdr(QEMUFile *f, RAMBlock *block, ram_addr_t offset,
+static size_t save_block_hdr(Visitor *v, RAMBlock *block, ram_addr_t offset,
int cont, int flag)
{
- size_t size;
-
- qemu_put_be64(f, offset | cont | flag);
- size = 8;
+ size_t size = 8; /* Note bogus with visitor */
+ Error *local_err = NULL;
+ ramsecentry_header rse_hdr;
+ rse_hdr.addr = offset;
+ rse_hdr.flags = cont | flag;
if (!cont) {
- qemu_put_byte(f, strlen(block->idstr));
- qemu_put_buffer(f, (uint8_t *)block->idstr,
- strlen(block->idstr));
+ strcpy(rse_hdr.idstr, block->idstr);
size += 1 + strlen(block->idstr);
}
+ visit_start_sequence_compat(v, "ramsecentry", VISIT_SEQ_COMPAT_RAMSECENTRY,
+ &rse_hdr, &local_err);
+
return size;
}
@@ -328,10 +337,13 @@ static void xbzrle_cache_zero_page(ram_addr_t current_addr)
#define ENCODING_FLAG_XBZRLE 0x1
-static int save_xbzrle_page(QEMUFile *f, uint8_t *current_data,
+static int save_xbzrle_page(Visitor *v, uint8_t *current_data,
ram_addr_t current_addr, RAMBlock *block,
ram_addr_t offset, int cont, bool last_stage)
{
+ QEMUFile *f = visitor_get_qemufile(v); // TODO
+ Error *local_err = NULL;
+
int encoded_len = 0, bytes_sent = -1;
uint8_t *prev_cached_page;
@@ -371,7 +383,7 @@ static int save_xbzrle_page(QEMUFile *f, uint8_t *current_data,
}
/* Send XBZRLE based compressed page */
- bytes_sent = save_block_hdr(f, block, offset, cont, RAM_SAVE_FLAG_XBZRLE);
+ bytes_sent = save_block_hdr(v, block, offset, cont, RAM_SAVE_FLAG_XBZRLE);
qemu_put_byte(f, ENCODING_FLAG_XBZRLE);
qemu_put_be16(f, encoded_len);
qemu_put_buffer(f, XBZRLE.encoded_buf, encoded_len);
@@ -379,6 +391,9 @@ static int save_xbzrle_page(QEMUFile *f, uint8_t *current_data,
acct_info.xbzrle_pages++;
acct_info.xbzrle_bytes += bytes_sent;
+ visit_end_sequence_compat(v, "ramsecentry", VISIT_SEQ_COMPAT_RAMSECENTRY,
+ &local_err);
+ LOCAL_ERR_REPORT(return -1;);
return bytes_sent;
}
@@ -523,8 +538,11 @@ static void migration_bitmap_sync(void)
* 0 means no dirty pages
*/
-static int ram_save_block(QEMUFile *f, bool last_stage)
+static int ram_save_block(Visitor *v, bool last_stage)
{
+ QEMUFile *f = visitor_get_qemufile(v); // TODO
+ Error *local_err = NULL;
+
RAMBlock *block = last_seen_block;
ram_addr_t offset = last_offset;
bool complete_round = false;
@@ -576,17 +594,23 @@ static int ram_save_block(QEMUFile *f, bool last_stage)
}
}
} else if (is_zero_range(p, TARGET_PAGE_SIZE)) {
+ uint8_t tmpbyte;
acct_info.dup_pages++;
- bytes_sent = save_block_hdr(f, block, offset, cont,
+ bytes_sent = save_block_hdr(v, block, offset, cont,
RAM_SAVE_FLAG_COMPRESS);
- qemu_put_byte(f, 0);
+ tmpbyte = 0;
+ visit_type_uint8(v, &tmpbyte, "pagefill", &local_err);
+ visit_end_sequence_compat(v, "ramsecentry",
+ VISIT_SEQ_COMPAT_RAMSECENTRY,
+ &local_err);
+ LOCAL_ERR_REPORT(return -1;);
bytes_sent++;
/* Must let xbzrle know, otherwise a previous (now 0'd) cached
* page would be stale
*/
xbzrle_cache_zero_page(current_addr);
} else if (!ram_bulk_stage && migrate_use_xbzrle()) {
- bytes_sent = save_xbzrle_page(f, p, current_addr, block,
+ bytes_sent = save_xbzrle_page(v, p, current_addr, block,
offset, cont, last_stage);
if (!last_stage) {
/* We must send exactly what's in the xbzrle cache
@@ -604,12 +628,14 @@ static int ram_save_block(QEMUFile *f, bool last_stage)
/* XBZRLE overflow or normal page */
if (bytes_sent == -1) {
- bytes_sent = save_block_hdr(f, block, offset, cont, RAM_SAVE_FLAG_PAGE);
- if (send_async) {
- qemu_put_buffer_async(f, p, TARGET_PAGE_SIZE);
- } else {
- qemu_put_buffer(f, p, TARGET_PAGE_SIZE);
- }
+ bytes_sent = save_block_hdr(v, block, offset, cont,
+ RAM_SAVE_FLAG_PAGE);
+ visit_type_buffer(v, p, TARGET_PAGE_SIZE, send_async, "page",
+ &local_err);
+ visit_end_sequence_compat(v, "ramsecentry",
+ VISIT_SEQ_COMPAT_RAMSECENTRY,
+ &local_err);
+ LOCAL_ERR_REPORT(return -1;);
bytes_sent += TARGET_PAGE_SIZE;
acct_info.norm_pages++;
}
@@ -711,10 +737,12 @@ static void reset_ram_globals(void)
#define MAX_WAIT 50 /* ms, half buffered_file limit */
-static int ram_save_setup(QEMUFile *f, void *opaque)
+static int ram_save_setup(Visitor *v, void *opaque)
{
RAMBlock *block;
int64_t ram_pages = last_ram_offset() >> TARGET_PAGE_BITS;
+ Error *local_err = NULL;
+ ramsecentry_header rse_hdr;
migration_bitmap = bitmap_new(ram_pages);
bitmap_set(migration_bitmap, 0, ram_pages);
@@ -762,31 +790,65 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
migration_bitmap_sync();
qemu_mutex_unlock_iothread();
- qemu_put_be64(f, ram_bytes_total() | RAM_SAVE_FLAG_MEM_SIZE);
+ visit_start_sequence_compat(v, "ramseclist", VISIT_SEQ_COMPAT_RAMSECLIST,
+ NULL, &local_err);
+ rse_hdr.addr = ram_bytes_total();
+ rse_hdr.flags = RAM_SAVE_FLAG_MEM_SIZE;
+ visit_start_sequence_compat(v, "ramsecentry", VISIT_SEQ_COMPAT_RAMSECENTRY,
+ &rse_hdr, &local_err);
+ LOCAL_ERR_REPORT(
+ qemu_mutex_unlock_ramlist();
+ return -1;
+ );
+ /* Note in the binary file the list is ended when it hits ram_bytes_total
+ * in length rather than any terminator
+ */
+ visit_start_list(v, "blocklist", &local_err);
QTAILQ_FOREACH(block, &ram_list.blocks, next) {
- qemu_put_byte(f, strlen(block->idstr));
- qemu_put_buffer(f, (uint8_t *)block->idstr, strlen(block->idstr));
- qemu_put_be64(f, block->length);
+ visit_next_list(v, NULL, &local_err);
+
+ visit_start_struct(v, NULL, "blockid", "blockid", 0, &local_err);
+ visit_type_str256(v, block->idstr, "blockname", &local_err);
+ visit_type_uint64(v, &block->length, "blocklen", &local_err);
+ LOCAL_ERR_REPORT(
+ qemu_mutex_unlock_ramlist();
+ return -1;
+ );
+ visit_end_struct(v, &local_err);
}
+ LOCAL_ERR_REPORT(
+ qemu_mutex_unlock_ramlist();
+ return -1;
+ );
+
+ visit_end_list(v, &local_err);
qemu_mutex_unlock_ramlist();
- ram_control_before_iterate(f, RAM_CONTROL_SETUP);
- ram_control_after_iterate(f, RAM_CONTROL_SETUP);
+ visit_end_sequence_compat(v, "ramsecentry", VISIT_SEQ_COMPAT_RAMSECENTRY,
+ &local_err);
+ LOCAL_ERR_REPORT(return -1;);
+ ram_control_before_iterate(visitor_get_qemufile(v), RAM_CONTROL_SETUP);
+ ram_control_after_iterate(visitor_get_qemufile(v), RAM_CONTROL_SETUP);
- qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
+ visit_end_sequence_compat(v, "ramseclist", VISIT_SEQ_COMPAT_RAMSECLIST,
+ &local_err);
return 0;
}
-static int ram_save_iterate(QEMUFile *f, void *opaque)
+static int ram_save_iterate(Visitor *v, void *opaque)
{
+ QEMUFile *f = visitor_get_qemufile(v);
+ Error *local_err = NULL;
int ret;
int i;
int64_t t0;
int total_sent = 0;
+ visit_start_sequence_compat(v, "ramseclist", VISIT_SEQ_COMPAT_RAMSECLIST,
+ NULL, &local_err);
qemu_mutex_lock_ramlist();
if (ram_list.version != last_version) {
@@ -800,7 +862,7 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
while ((ret = qemu_file_rate_limit(f)) == 0) {
int bytes_sent;
- bytes_sent = ram_save_block(f, false);
+ bytes_sent = ram_save_block(v, false);
/* no more blocks to sent */
if (bytes_sent == 0) {
break;
@@ -838,7 +900,8 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
* Do not count these 8 bytes into total_sent, so that we can
* return 0 if no page had been dirtied.
*/
- qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
+ visit_end_sequence_compat(v, "ramseclist", VISIT_SEQ_COMPAT_RAMSECLIST,
+ &local_err);
bytes_transferred += 8;
ret = qemu_file_get_error(f);
@@ -846,15 +909,19 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
return ret;
}
+ LOCAL_ERR_REPORT(return -EINVAL;);
return total_sent;
}
-static int ram_save_complete(QEMUFile *f, void *opaque)
+static int ram_save_complete(Visitor *v, void *opaque)
{
+ Error *local_err = NULL;
qemu_mutex_lock_ramlist();
migration_bitmap_sync();
- ram_control_before_iterate(f, RAM_CONTROL_FINISH);
+ visit_start_sequence_compat(v, "ramseclist", VISIT_SEQ_COMPAT_RAMSECLIST,
+ NULL, &local_err);
+ ram_control_before_iterate(visitor_get_qemufile(v), RAM_CONTROL_FINISH);
/* try transferring iterative blocks of memory */
@@ -862,24 +929,26 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
while (true) {
int bytes_sent;
- bytes_sent = ram_save_block(f, true);
- /* no more blocks to sent */
+ bytes_sent = ram_save_block(v, true);
+ /* no more blocks to send */
if (bytes_sent == 0) {
break;
}
bytes_transferred += bytes_sent;
}
- ram_control_after_iterate(f, RAM_CONTROL_FINISH);
+ ram_control_after_iterate(visitor_get_qemufile(v), RAM_CONTROL_FINISH);
+ visit_end_sequence_compat(v, "ramseclist", VISIT_SEQ_COMPAT_RAMSECLIST,
+ &local_err);
migration_end();
qemu_mutex_unlock_ramlist();
- qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
+ LOCAL_ERR_REPORT(return -EINVAL;);
return 0;
}
-static uint64_t ram_save_pending(QEMUFile *f, void *opaque, uint64_t max_size)
+static uint64_t ram_save_pending(void *opaque, uint64_t max_size)
{
uint64_t remaining_size;
diff --git a/block-migration.c b/block-migration.c
index 897fdba..3475d76 100644
--- a/block-migration.c
+++ b/block-migration.c
@@ -16,6 +16,7 @@
#include "qemu-common.h"
#include "block/block_int.h"
#include "hw/hw.h"
+#include "qapi/visitor.h"
#include "qemu/queue.h"
#include "qemu/timer.h"
#include "migration/block.h"
@@ -603,9 +604,10 @@ static void block_migration_cancel(void *opaque)
blk_mig_cleanup();
}
-static int block_save_setup(QEMUFile *f, void *opaque)
+static int block_save_setup(Visitor *v, void *opaque)
{
int ret;
+ QEMUFile *f = visitor_get_qemufile(v); // TODO
DPRINTF("Enter save live setup submitted %d transferred %d\n",
block_mig_state.submitted, block_mig_state.transferred);
@@ -624,8 +626,10 @@ static int block_save_setup(QEMUFile *f, void *opaque)
return ret;
}
-static int block_save_iterate(QEMUFile *f, void *opaque)
+static int block_save_iterate(Visitor *v, void *opaque)
{
+ QEMUFile *f = visitor_get_qemufile(v); // TODO
+
int ret;
int64_t last_ftell = qemu_ftell(f);
@@ -682,8 +686,9 @@ static int block_save_iterate(QEMUFile *f, void *opaque)
/* Called with iothread lock taken. */
-static int block_save_complete(QEMUFile *f, void *opaque)
+static int block_save_complete(Visitor *v, void *opaque)
{
+ QEMUFile *f = visitor_get_qemufile(v); // TODO
int ret;
DPRINTF("Enter save live complete submitted %d transferred %d\n",
@@ -720,7 +725,7 @@ static int block_save_complete(QEMUFile *f, void *opaque)
return 0;
}
-static uint64_t block_save_pending(QEMUFile *f, void *opaque, uint64_t max_size)
+static uint64_t block_save_pending(void *opaque, uint64_t max_size)
{
/* Estimate pending number of bytes to send */
uint64_t pending;
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index 8f722dd..e91f729 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -515,7 +515,7 @@ void pci_device_save(PCIDevice *s, QEMUFile *f)
* This makes us compatible with old devices
* which never set or clear this bit. */
s->config[PCI_STATUS] &= ~PCI_STATUS_INTERRUPT;
- vmstate_save_state(f, pci_get_vmstate(s), s);
+ vmstate_save_state(qemu_file_get_tmp_visitor(f), pci_get_vmstate(s), s);
/* Restore the interrupt status bit. */
pci_update_irq_status(s);
}
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index a11e121..34443c1 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -835,9 +835,10 @@ static const VMStateDescription vmstate_spapr = {
#define HPTE_DIRTY(_hpte) (tswap64(*((uint64_t *)(_hpte))) & HPTE64_V_HPTE_DIRTY)
#define CLEAN_HPTE(_hpte) ((*(uint64_t *)(_hpte)) &= tswap64(~HPTE64_V_HPTE_DIRTY))
-static int htab_save_setup(QEMUFile *f, void *opaque)
+static int htab_save_setup(Visitor *v, void *opaque)
{
sPAPREnvironment *spapr = opaque;
+ QEMUFile *f = visitor_get_qemufile(v); // TODO
/* "Iteration" header */
qemu_put_be32(f, spapr->htab_shift);
@@ -990,8 +991,9 @@ static int htab_save_later_pass(QEMUFile *f, sPAPREnvironment *spapr,
#define MAX_ITERATION_NS 5000000 /* 5 ms */
#define MAX_KVM_BUF_SIZE 2048
-static int htab_save_iterate(QEMUFile *f, void *opaque)
+static int htab_save_iterate(Visitor *v, void *opaque)
{
+ QEMUFile *f = visitor_get_qemufile(v); // TODO
sPAPREnvironment *spapr = opaque;
int rc = 0;
@@ -1020,8 +1022,9 @@ static int htab_save_iterate(QEMUFile *f, void *opaque)
return rc;
}
-static int htab_save_complete(QEMUFile *f, void *opaque)
+static int htab_save_complete(Visitor *v, void *opaque)
{
+ QEMUFile *f = visitor_get_qemufile(v); // TODO
sPAPREnvironment *spapr = opaque;
/* Iteration header */
diff --git a/hw/scsi/spapr_vscsi.c b/hw/scsi/spapr_vscsi.c
index 34478f0..8e799e2 100644
--- a/hw/scsi/spapr_vscsi.c
+++ b/hw/scsi/spapr_vscsi.c
@@ -624,7 +624,8 @@ static void vscsi_save_request(QEMUFile *f, SCSIRequest *sreq)
vscsi_req *req = sreq->hba_private;
assert(req->active);
- vmstate_save_state(f, &vmstate_spapr_vscsi_req, req);
+ vmstate_save_state(qemu_file_get_tmp_visitor(f), &vmstate_spapr_vscsi_req,
+ req);
DPRINTF("VSCSI: saving tag=%u, current desc#%d, offset=%x\n",
req->qtag, req->cur_desc_num, req->cur_desc_offset);
diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index a5e4b0b..b66342b 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -40,7 +40,7 @@ typedef struct SaveVMHandlers {
SaveStateHandler *save_state;
void (*cancel)(void *opaque);
- int (*save_live_complete)(QEMUFile *f, void *opaque);
+ int (*save_live_complete)(Visitor *v, void *opaque);
/* This runs both outside and inside the iothread lock. */
bool (*is_active)(void *opaque);
@@ -98,6 +98,8 @@ struct VMStateInfo {
const char *name;
int (*get)(QEMUFile *f, void *pv, size_t size);
void (*put)(QEMUFile *f, void *pv, size_t size);
+ int (*visit)(Visitor *v, void *pdata, const char* name, size_t size,
+ Error **errp);
};
enum VMStateFlags {
@@ -759,7 +761,7 @@ extern const VMStateInfo vmstate_info_bitmap;
int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd,
void *opaque, int version_id);
-void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
+void vmstate_save_state(Visitor *v, const VMStateDescription *vmsd,
void *opaque);
int vmstate_register_with_alias_id(DeviceState *dev, int instance_id,
diff --git a/savevm.c b/savevm.c
index d094fbb..51efd4a 100644
--- a/savevm.c
+++ b/savevm.c
@@ -24,6 +24,8 @@
#include "config-host.h"
#include "qemu-common.h"
+#include "qapi/qemu-file-binary-input-visitor.h"
+#include "qapi/qemu-file-binary-output-visitor.h"
#include "hw/hw.h"
#include "hw/qdev.h"
#include "net/net.h"
@@ -51,6 +53,13 @@
#define ARP_PTYPE_IP 0x0800
#define ARP_OP_REQUEST_REV 0x3
+#define LOCAL_ERR_REPORT(fallout) \
+ if (local_err) { \
+ error_report("%s:%d %s", __func__, __LINE__, \
+ error_get_pretty(local_err)); \
+ fallout \
+ }
+
static int announce_self_create(uint8_t *buf,
uint8_t *mac_addr)
{
@@ -435,13 +444,42 @@ static int vmstate_load(QEMUFile *f, SaveStateEntry *se, int version_id)
return vmstate_load_state(f, se->vmsd, se->opaque, version_id);
}
-static void vmstate_save(QEMUFile *f, SaveStateEntry *se)
+static int vmstate_save(Visitor *v, SaveStateEntry *se, int version_id)
{
+ int ret = 0;
+ Error *local_err = NULL;
+
+ if (local_err) {
+ error_report("%s", error_get_pretty(local_err));
+ return -1;
+ }
+
if (!se->vmsd) { /* Old style */
- se->ops->save_state(f, se->opaque);
- return;
+ QEMUFile *wrapperf;
+
+ /* Some visitors put old format things down separate QEMUFile's */
+ visit_start_sequence_compat(v, "oldstate", VISIT_SEQ_COMPAT_BLOB,
+ &wrapperf, &local_err);
+ if (local_err) {
+ goto out_lerr;
+ }
+
+ se->ops->save_state(wrapperf, se->opaque);
+
+ visit_end_sequence_compat(v, "oldstate", VISIT_SEQ_COMPAT_BLOB,
+ &local_err);
+ if (local_err) {
+ goto out_lerr;
+ }
+ return ret;
}
- vmstate_save_state(f, se->vmsd, se->opaque);
+ vmstate_save_state(v, se->vmsd, se->opaque);
+
+ return ret;
+
+out_lerr:
+ error_report("%s", error_get_pretty(local_err));
+ return -EINVAL;
}
bool qemu_savevm_state_blocked(Error **errp)
@@ -462,6 +500,12 @@ void qemu_savevm_state_begin(QEMUFile *f,
{
SaveStateEntry *se;
int ret;
+ Error *local_err = NULL;
+ SectionHeader sh;
+
+ QemuFileBinOutputVisitor *qfbov = qemu_file_bin_output_visitor_new(f);
+ Visitor *v = qemu_file_bin_output_get_visitor(qfbov);
+ qemu_file_set_tmp_visitor(f, v);
QTAILQ_FOREACH(se, &savevm_handlers, entry) {
if (!se->ops || !se->ops->set_params) {
@@ -470,11 +514,14 @@ void qemu_savevm_state_begin(QEMUFile *f,
se->ops->set_params(params, se->opaque);
}
- qemu_put_be32(f, QEMU_VM_FILE_MAGIC);
- qemu_put_be32(f, QEMU_VM_FILE_VERSION);
+ /* Maybe we should squash these two together? */
+ visit_start_sequence_compat(v, "file", VISIT_SEQ_COMPAT_FILE, NULL,
+ &local_err);
+ visit_start_sequence_compat(v, "top", VISIT_SEQ_COMPAT_BYTE0TERM, NULL,
+ &local_err);
QTAILQ_FOREACH(se, &savevm_handlers, entry) {
- int len;
+ int32_t section_type;
if (!se->ops || !se->ops->save_live_setup) {
continue;
@@ -485,18 +532,19 @@ void qemu_savevm_state_begin(QEMUFile *f,
}
}
/* Section type */
- qemu_put_byte(f, QEMU_VM_SECTION_START);
- qemu_put_be32(f, se->section_id);
-
- /* ID string */
- len = strlen(se->idstr);
- qemu_put_byte(f, len);
- qemu_put_buffer(f, (uint8_t *)se->idstr, len);
-
- qemu_put_be32(f, se->instance_id);
- qemu_put_be32(f, se->version_id);
-
- ret = se->ops->save_live_setup(f, se->opaque);
+ section_type = QEMU_VM_SECTION_START;
+ visit_get_next_type(v, §ion_type, NULL, "secstart", &local_err);
+
+ sh.section_id = se->section_id;
+ strcpy(sh.idstr, se->idstr);
+ sh.instance_id = se->instance_id;
+ sh.version_id = se->version_id;
+ visit_start_sequence_compat(v, "secstart",
+ VISIT_SEQ_COMPAT_SECTION_HEADER, &sh, &local_err);
+
+ ret = se->ops->save_live_setup(v, se->opaque);
+ visit_end_sequence_compat(v, "secstart",
+ VISIT_SEQ_COMPAT_SECTION_HEADER, &local_err);
if (ret < 0) {
qemu_file_set_error(f, ret);
break;
@@ -512,8 +560,12 @@ void qemu_savevm_state_begin(QEMUFile *f,
*/
int qemu_savevm_state_iterate(QEMUFile *f)
{
+ Visitor *v = qemu_file_get_tmp_visitor(f);
SaveStateEntry *se;
int ret = 1;
+ Error *local_err = NULL;
+ SectionHeader sh;
+ int32_t section_type;
QTAILQ_FOREACH(se, &savevm_handlers, entry) {
if (!se->ops || !se->ops->save_live_iterate) {
@@ -529,12 +581,21 @@ int qemu_savevm_state_iterate(QEMUFile *f)
}
trace_savevm_section_start(se->idstr, se->section_id);
/* Section type */
- qemu_put_byte(f, QEMU_VM_SECTION_PART);
- qemu_put_be32(f, se->section_id);
+ section_type = QEMU_VM_SECTION_PART;
+ visit_get_next_type(v, §ion_type, NULL, "secpart", &local_err);
+
+ sh.section_id = se->section_id;
+ visit_start_sequence_compat(v, "secpart", VISIT_SEQ_COMPAT_SECTION_MIN,
+ &sh, &local_err);
+
+ ret = se->ops->save_live_iterate(v, se->opaque);
+ visit_end_sequence_compat(v, "secpart", VISIT_SEQ_COMPAT_SECTION_MIN,
+ &local_err);
- ret = se->ops->save_live_iterate(f, se->opaque);
trace_savevm_section_end(se->idstr, se->section_id);
+ LOCAL_ERR_REPORT(return -EINVAL;);
+
if (ret < 0) {
qemu_file_set_error(f, ret);
}
@@ -552,10 +613,18 @@ int qemu_savevm_state_iterate(QEMUFile *f)
void qemu_savevm_state_complete(QEMUFile *f)
{
SaveStateEntry *se;
+ SectionHeader sh;
int ret;
cpu_synchronize_all_states();
+ Visitor *v = qemu_file_get_tmp_visitor(f);
+ Error *local_err = NULL;
+ int32_t section_type;
+
+ /* The visitor is in the middle of the 'top' sequence_compat structure
+ * started in begin.
+ */
QTAILQ_FOREACH(se, &savevm_handlers, entry) {
if (!se->ops || !se->ops->save_live_complete) {
continue;
@@ -567,41 +636,54 @@ void qemu_savevm_state_complete(QEMUFile *f)
}
trace_savevm_section_start(se->idstr, se->section_id);
/* Section type */
- qemu_put_byte(f, QEMU_VM_SECTION_END);
- qemu_put_be32(f, se->section_id);
-
- ret = se->ops->save_live_complete(f, se->opaque);
+ section_type = QEMU_VM_SECTION_END;
+ visit_get_next_type(v, §ion_type, NULL, "secend", &local_err);
+ LOCAL_ERR_REPORT()
+ sh.section_id = se->section_id;
+ visit_start_sequence_compat(v, "secend", VISIT_SEQ_COMPAT_SECTION_MIN,
+ &sh, &local_err);
+ LOCAL_ERR_REPORT()
+
+ ret = se->ops->save_live_complete(v, se->opaque);
trace_savevm_section_end(se->idstr, se->section_id);
- if (ret < 0) {
+ visit_end_sequence_compat(v, "secend", VISIT_SEQ_COMPAT_SECTION_MIN,
+ &local_err);
+ if ((ret < 0) || local_err) {
+ LOCAL_ERR_REPORT()
qemu_file_set_error(f, ret);
+ visit_destroy(v, &local_err);
return;
}
}
QTAILQ_FOREACH(se, &savevm_handlers, entry) {
- int len;
-
if ((!se->ops || !se->ops->save_state) && !se->vmsd) {
continue;
}
trace_savevm_section_start(se->idstr, se->section_id);
/* Section type */
- qemu_put_byte(f, QEMU_VM_SECTION_FULL);
- qemu_put_be32(f, se->section_id);
-
- /* ID string */
- len = strlen(se->idstr);
- qemu_put_byte(f, len);
- qemu_put_buffer(f, (uint8_t *)se->idstr, len);
-
- qemu_put_be32(f, se->instance_id);
- qemu_put_be32(f, se->version_id);
-
- vmstate_save(f, se);
+ /* Note: the get_next_type plants the section type in compat mode */
+ section_type = QEMU_VM_SECTION_FULL;
+ visit_get_next_type(v, §ion_type, NULL, "secfull", &local_err);
+
+ sh.section_id = se->section_id;
+ strcpy(sh.idstr, se->idstr);
+ sh.instance_id = se->instance_id;
+ sh.version_id = se->version_id;
+ visit_start_sequence_compat(v, "secfull",
+ VISIT_SEQ_COMPAT_SECTION_HEADER, &sh, &local_err);
+
+ vmstate_save(v, se, se->version_id);
+ visit_end_sequence_compat(v, "secfull",
+ VISIT_SEQ_COMPAT_SECTION_HEADER, &local_err);
trace_savevm_section_end(se->idstr, se->section_id);
}
+ visit_end_sequence_compat(v, "top", VISIT_SEQ_COMPAT_BYTE0TERM, &local_err);
+ visit_end_sequence_compat(v, "file", VISIT_SEQ_COMPAT_FILE, &local_err);
+ visit_destroy(v, &local_err);
+
+ LOCAL_ERR_REPORT(ret = -EINVAL;)
- qemu_put_byte(f, QEMU_VM_EOF);
qemu_fflush(f);
}
@@ -619,7 +701,7 @@ uint64_t qemu_savevm_state_pending(QEMUFile *f, uint64_t max_size)
continue;
}
}
- ret += se->ops->save_live_pending(f, se->opaque, max_size);
+ ret += se->ops->save_live_pending(se->opaque, max_size);
}
return ret;
}
@@ -671,38 +753,44 @@ static int qemu_savevm_state(QEMUFile *f)
static int qemu_save_device_state(QEMUFile *f)
{
SaveStateEntry *se;
+ Error *local_err;
+ SectionHeader sh;
- qemu_put_be32(f, QEMU_VM_FILE_MAGIC);
- qemu_put_be32(f, QEMU_VM_FILE_VERSION);
+ QemuFileBinOutputVisitor *qfbov = qemu_file_bin_output_visitor_new(f);
+ qemu_file_set_tmp_visitor(f, qemu_file_bin_output_get_visitor(qfbov));
+ Visitor *v = qemu_file_bin_output_get_visitor(qfbov);
+ int32_t section_type;
cpu_synchronize_all_states();
-
+ visit_start_sequence_compat(v, "file", VISIT_SEQ_COMPAT_FILE, NULL,
+ &local_err);
+ visit_start_sequence_compat(v, "top", VISIT_SEQ_COMPAT_BYTE0TERM,
+ NULL, &local_err);
QTAILQ_FOREACH(se, &savevm_handlers, entry) {
- int len;
-
if (se->is_ram) {
continue;
}
- if ((!se->ops || !se->ops->save_state) && !se->vmsd) {
+ if ((!se->ops || (!se->ops->save_state)) && !se->vmsd) {
continue;
}
/* Section type */
- qemu_put_byte(f, QEMU_VM_SECTION_FULL);
- qemu_put_be32(f, se->section_id);
-
- /* ID string */
- len = strlen(se->idstr);
- qemu_put_byte(f, len);
- qemu_put_buffer(f, (uint8_t *)se->idstr, len);
+ section_type = QEMU_VM_SECTION_FULL;
+ visit_get_next_type(v, §ion_type, NULL, "secfull", &local_err);
- qemu_put_be32(f, se->instance_id);
- qemu_put_be32(f, se->version_id);
+ sh.section_id = se->section_id;
+ strcpy(sh.idstr, se->idstr);
+ sh.instance_id = se->instance_id;
+ sh.version_id = se->version_id;
+ visit_start_sequence_compat(v, "secfull",
+ VISIT_SEQ_COMPAT_SECTION_HEADER, &sh, &local_err);
- vmstate_save(f, se);
+ vmstate_save(v, se, se->version_id);
}
- qemu_put_byte(f, QEMU_VM_EOF);
+ visit_end_sequence_compat(v, "top", VISIT_SEQ_COMPAT_BYTE0TERM, &local_err);
+ visit_end_sequence_compat(v, "file", VISIT_SEQ_COMPAT_FILE, &local_err);
+ visit_destroy(v, &local_err);
return qemu_file_get_error(f);
}
diff --git a/vmstate.c b/vmstate.c
index d1f5eb0..46bf7b9 100644
--- a/vmstate.c
+++ b/vmstate.c
@@ -3,12 +3,22 @@
#include "migration/qemu-file.h"
#include "migration/vmstate.h"
#include "qemu/bitops.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "qapi/visitor.h"
-static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd,
+static void vmstate_subsection_save(Visitor *v, const VMStateDescription *vmsd,
void *opaque);
static int vmstate_subsection_load(QEMUFile *f, const VMStateDescription *vmsd,
void *opaque);
+#define LOCAL_ERR_REPORT(fallout) \
+ if (local_err) { \
+ error_report("%s:%d %s", __func__, __LINE__, \
+ error_get_pretty(local_err)); \
+ fallout \
+ }
+
int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd,
void *opaque, int version_id)
{
@@ -93,6 +103,14 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
void *opaque)
{
VMStateField *field = vmsd->fields;
+ Error *local_err = NULL;
+ uint32_t tmp32;
+
+ /* BER type comes from the vmsd if it's set */
+ tmp32 = vmsd->ber_tag;
+ visit_start_sequence_compat(v, vmsd->name, VISIT_SEQ_COMPAT_VMSTATE,
+ &tmp32, &local_err);
+ LOCAL_ERR_REPORT(return;);
if (vmsd->pre_save) {
vmsd->pre_save(opaque);
@@ -124,6 +142,11 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
if (field->flags & VMS_POINTER) {
base_addr = *(void **)base_addr + field->start;
}
+
+ if (n_elems > 1) {
+ visit_start_array(v, NULL, field->name, n_elems, 0, &local_err);
+ }
+
for (i = 0; i < n_elems; i++) {
void *addr = base_addr + size * i;
@@ -131,19 +154,50 @@ void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
addr = *(void **)addr;
}
if (field->flags & VMS_STRUCT) {
- vmstate_save_state(f, field->vmsd, addr);
+ vmstate_save_state(v, field->vmsd, addr);
} else {
- field->info->put(f, addr, size);
+ if (field->info->visit) {
+ field->info->visit(v, addr, field->name, size,
+ &local_err);
+ } else {
+ QEMUFile *wrapperf;
+
+ /*
+ * Some visitors put old format things down separate
+ * QEMUFile's
+ */
+ visit_start_sequence_compat(v, field->name,
+ VISIT_SEQ_COMPAT_BLOB,
+ &wrapperf, &local_err);
+ LOCAL_ERR_REPORT(return;);
+
+ field->info->put(wrapperf, addr, size);
+
+ visit_end_sequence_compat(v, field->name,
+ VISIT_SEQ_COMPAT_BLOB,
+ &local_err);
+ LOCAL_ERR_REPORT(return;);
+ }
+ }
+ if ((i+1) != n_elems) {
+ visit_next_array(v, &local_err);
}
}
+ if (n_elems > 1) {
+ visit_end_array(v, &local_err);
+ }
}
field++;
}
- vmstate_subsection_save(f, vmsd, opaque);
+ vmstate_subsection_save(v, vmsd, opaque);
+
+ visit_end_sequence_compat(v, vmsd->name, VISIT_SEQ_COMPAT_VMSTATE,
+ &local_err);
+ LOCAL_ERR_REPORT(return;);
}
static const VMStateDescription *
- vmstate_get_subsection(const VMStateSubsection *sub, char *idstr)
+ vmstate_subsection_name_lookup(const VMStateSubsection *sub, char *idstr)
{
while (sub && sub->needed) {
if (strcmp(idstr, sub->vmsd->name) == 0) {
@@ -195,25 +249,43 @@ static int vmstate_subsection_load(QEMUFile *f, const VMStateDescription *vmsd,
return 0;
}
-static void vmstate_subsection_save(QEMUFile *f, const VMStateDescription *vmsd,
+static void vmstate_subsection_save(Visitor *v, const VMStateDescription *vmsd,
void *opaque)
{
const VMStateSubsection *sub = vmsd->subsections;
+ Error *local_err = NULL;
+
+ if (!sub) {
+ /*
+ * If the type has no subsection defined at all then skip completely
+ * Note that this means if we have conditional subsections we might
+ * plant an empty 'subseclist' if none of them are in use
+ */
+ return;
+ }
+ visit_start_sequence_compat(v, "subseclist", VISIT_SEQ_COMPAT_SUBSECLIST,
+ (char *)vmsd->name, &local_err);
while (sub && sub->needed) {
if (sub->needed(opaque)) {
- const VMStateDescription *vmsd = sub->vmsd;
- uint8_t len;
-
- qemu_put_byte(f, QEMU_VM_SUBSECTION);
- len = strlen(vmsd->name);
- qemu_put_byte(f, len);
- qemu_put_buffer(f, (uint8_t *)vmsd->name, len);
- qemu_put_be32(f, vmsd->version_id);
- vmstate_save_state(f, vmsd, opaque);
+ const VMStateDescription *sub_vmsd = sub->vmsd;
+ SectionHeader sh;
+
+ sh.version_id = sub_vmsd->version_id;
+ strcpy(sh.idstr, sub_vmsd->name);
+ visit_start_sequence_compat(v, "subsection",
+ VISIT_SEQ_COMPAT_SUBSECTION,
+ &sh, &local_err);
+ vmstate_save_state(v, sub_vmsd, opaque);
+
+ visit_end_sequence_compat(v, "subsection",
+ VISIT_SEQ_COMPAT_SUBSECTION, &local_err);
}
sub++;
}
+ visit_end_sequence_compat(v, "subseclist", VISIT_SEQ_COMPAT_SUBSECLIST,
+ &local_err);
+ LOCAL_ERR_REPORT(return;);
}
/* bool */
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 09/16] Visitor: Load path
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (7 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 08/16] Visitor: Output path Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 10/16] Visitor: Common types to use visitors Dr. David Alan Gilbert (git)
` (7 subsequent siblings)
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Hacks in piix4.c, pci.c, spapr_vscsi.c for now
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
arch_init.c | 72 ++++++++++++++---------
hw/acpi/piix4.c | 5 +-
hw/pci/pci.c | 3 +-
hw/scsi/spapr_vscsi.c | 3 +-
include/migration/vmstate.h | 2 +-
savevm.c | 133 +++++++++++++++++++++++++++++++------------
vmstate.c | 135 +++++++++++++++++++++++++++++++++-----------
7 files changed, 251 insertions(+), 102 deletions(-)
diff --git a/arch_init.c b/arch_init.c
index 02bf78a..145666d 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -1004,15 +1004,12 @@ static int load_xbzrle(QEMUFile *f, ram_addr_t addr, void *host)
return rc;
}
-static inline void *host_from_stream_offset(QEMUFile *f,
- ram_addr_t offset,
- int flags)
+static inline void *host_from_stream_offset(ramsecentry_header *rse_hdr)
{
static RAMBlock *block = NULL;
- char id[256];
- uint8_t len;
+ ram_addr_t offset = rse_hdr->addr;
- if (flags & RAM_SAVE_FLAG_CONTINUE) {
+ if (rse_hdr->flags & RAM_SAVE_FLAG_CONTINUE) {
if (!block) {
fprintf(stderr, "Ack, bad migration stream!\n");
return NULL;
@@ -1021,16 +1018,13 @@ static inline void *host_from_stream_offset(QEMUFile *f,
return memory_region_get_ram_ptr(block->mr) + offset;
}
- len = qemu_get_byte(f);
- qemu_get_buffer(f, (uint8_t *)id, len);
- id[len] = 0;
-
QTAILQ_FOREACH(block, &ram_list.blocks, next) {
- if (!strncmp(id, block->idstr, sizeof(id)))
+ if (!strncmp(rse_hdr->idstr, block->idstr, sizeof(rse_hdr->idstr))) {
return memory_region_get_ram_ptr(block->mr) + offset;
+ }
}
- fprintf(stderr, "Can't find block %s!\n", id);
+ fprintf(stderr, "Can't find block %s!\n", rse_hdr->idstr);
return NULL;
}
@@ -1051,6 +1045,9 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
int flags, ret = 0;
int error;
static uint64_t seq_iter;
+ Error *local_err = NULL;
+ ramsecentry_header rse_hdr;
+ Visitor *v = qemu_file_get_tmp_visitor(f); // TODO
seq_iter++;
@@ -1058,11 +1055,18 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
return -EINVAL;
}
- do {
- addr = qemu_get_be64(f);
-
- flags = addr & ~TARGET_PAGE_MASK;
- addr &= TARGET_PAGE_MASK;
+ visit_start_sequence_compat(v, "ramseclist", VISIT_SEQ_COMPAT_RAMSECLIST,
+ NULL, &local_err);
+ LOCAL_ERR_REPORT(return -EINVAL;);
+ while (visit_get_next_type(v, &flags, NULL, "ramseclist", &local_err),
+ (!local_err && !(flags & RAM_SAVE_FLAG_EOS))) {
+ LOCAL_ERR_REPORT(return -EINVAL;);
+ visit_start_sequence_compat(v, "ramsecentry",
+ VISIT_SEQ_COMPAT_RAMSECENTRY, &rse_hdr,
+ &local_err);
+ addr = rse_hdr.addr;
+ flags = rse_hdr.flags;
+ LOCAL_ERR_REPORT(return -EINVAL;);
if (flags & RAM_SAVE_FLAG_MEM_SIZE) {
if (version_id == 4) {
@@ -1071,14 +1075,17 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
ram_addr_t length;
ram_addr_t total_ram_bytes = addr;
+ visit_start_list(v, "blocklist", &local_err);
while (total_ram_bytes) {
RAMBlock *block;
- uint8_t len;
+ visit_start_struct(v, NULL, "blockid", "blockid", 0,
+ &local_err);
+ visit_type_str256(v, id, "blockname", &local_err);
+ visit_type_uint64(v, &length, "blocklen", &local_err);
+ LOCAL_ERR_REPORT(ret = -EINVAL; goto done;);
+ visit_end_struct(v, &local_err);
+ LOCAL_ERR_REPORT(ret = -EINVAL; goto done;);
- len = qemu_get_byte(f);
- qemu_get_buffer(f, (uint8_t *)id, len);
- id[len] = 0;
- length = qemu_get_be64(f);
QTAILQ_FOREACH(block, &ram_list.blocks, next) {
if (!strncmp(id, block->idstr, sizeof(id))) {
@@ -1103,6 +1110,9 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
total_ram_bytes -= length;
}
+ LOCAL_ERR_REPORT(ret = -EINVAL; goto done;);
+ visit_end_list(v, &local_err);
+ LOCAL_ERR_REPORT(ret = -EINVAL; goto done;);
}
}
@@ -1110,24 +1120,25 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
void *host;
uint8_t ch;
- host = host_from_stream_offset(f, addr, flags);
+ host = host_from_stream_offset(&rse_hdr);
if (!host) {
return -EINVAL;
}
- ch = qemu_get_byte(f);
+ visit_type_uint8(v, &ch, "pagefill", &local_err);
ram_handle_compressed(host, ch, TARGET_PAGE_SIZE);
} else if (flags & RAM_SAVE_FLAG_PAGE) {
void *host;
- host = host_from_stream_offset(f, addr, flags);
+ host = host_from_stream_offset(&rse_hdr);
if (!host) {
return -EINVAL;
}
- qemu_get_buffer(f, host, TARGET_PAGE_SIZE);
+ visit_type_buffer(v, host, TARGET_PAGE_SIZE, false, "page",
+ &local_err);
} else if (flags & RAM_SAVE_FLAG_XBZRLE) {
- void *host = host_from_stream_offset(f, addr, flags);
+ void *host = host_from_stream_offset(&rse_hdr);
if (!host) {
return -EINVAL;
}
@@ -1144,7 +1155,12 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id)
ret = error;
goto done;
}
- } while (!(flags & RAM_SAVE_FLAG_EOS));
+ visit_end_sequence_compat(v, "ramsecentry",
+ VISIT_SEQ_COMPAT_RAMSECENTRY, &local_err);
+ };
+ visit_end_sequence_compat(v, "ramseclist", VISIT_SEQ_COMPAT_RAMSECLIST,
+ &local_err);
+ LOCAL_ERR_REPORT(return -EINVAL;);
done:
DPRINTF("Completed load of VM with exit code %d seq iteration "
diff --git a/hw/acpi/piix4.c b/hw/acpi/piix4.c
index 67dc075..a44f4b8 100644
--- a/hw/acpi/piix4.c
+++ b/hw/acpi/piix4.c
@@ -211,7 +211,8 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
qemu_get_be16s(f, &s->ar.pm1.evt.en);
qemu_get_be16s(f, &s->ar.pm1.cnt.cnt);
- ret = vmstate_load_state(f, &vmstate_apm, &s->apm, 1);
+ ret = vmstate_load_state(qemu_file_get_tmp_visitor(f), &vmstate_apm,
+ &s->apm, 1);
if (ret) {
return ret;
}
@@ -229,7 +230,7 @@ static int acpi_load_old(QEMUFile *f, void *opaque, int version_id)
qemu_get_be16s(f, &temp);
}
- ret = vmstate_load_state(f, &vmstate_pci_status,
+ ret = vmstate_load_state(qemu_file_get_tmp_visitor(f), &vmstate_pci_status,
&s->acpi_pci_hotplug.acpi_pcihp_pci_status[ACPI_PCIHP_BSEL_DEFAULT], 1);
return ret;
}
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index e91f729..2009c0f 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -523,7 +523,8 @@ void pci_device_save(PCIDevice *s, QEMUFile *f)
int pci_device_load(PCIDevice *s, QEMUFile *f)
{
int ret;
- ret = vmstate_load_state(f, pci_get_vmstate(s), s, s->version_id);
+ ret = vmstate_load_state(qemu_file_get_tmp_visitor(f),
+ pci_get_vmstate(s), s, s->version_id);
/* Restore the interrupt status bit. */
pci_update_irq_status(s);
return ret;
diff --git a/hw/scsi/spapr_vscsi.c b/hw/scsi/spapr_vscsi.c
index 8e799e2..c50a741 100644
--- a/hw/scsi/spapr_vscsi.c
+++ b/hw/scsi/spapr_vscsi.c
@@ -643,7 +643,8 @@ static void *vscsi_load_request(QEMUFile *f, SCSIRequest *sreq)
assert(!req->active);
memset(req, 0, sizeof(*req));
- rc = vmstate_load_state(f, &vmstate_spapr_vscsi_req, req, 1);
+ rc = vmstate_load_state(qemu_file_get_tmp_visitor(f),
+ &vmstate_spapr_vscsi_req, req, 1);
if (rc) {
fprintf(stderr, "VSCSI: failed loading request tag#%u\n", sreq->tag);
return NULL;
diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index b66342b..d22c4e3 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -759,7 +759,7 @@ extern const VMStateInfo vmstate_info_bitmap;
#define VMSTATE_END_OF_LIST() \
{}
-int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd,
+int vmstate_load_state(Visitor *v, const VMStateDescription *vmsd,
void *opaque, int version_id);
void vmstate_save_state(Visitor *v, const VMStateDescription *vmsd,
void *opaque);
diff --git a/savevm.c b/savevm.c
index 51efd4a..7431773 100644
--- a/savevm.c
+++ b/savevm.c
@@ -436,12 +436,41 @@ void vmstate_unregister(DeviceState *dev, const VMStateDescription *vmsd,
}
}
-static int vmstate_load(QEMUFile *f, SaveStateEntry *se, int version_id)
+static int vmstate_load(Visitor *v, SaveStateEntry *se, int version_id)
{
+ int ret;
+ Error *local_err = NULL;
+
if (!se->vmsd) { /* Old style */
- return se->ops->load_state(f, se->opaque, version_id);
+ QEMUFile *wrapperf;
+ bool is_iterative = se->ops->save_live_iterate != NULL;
+
+ /* Some visitors put old format things down separate QEMUFile's -
+ * but not the iterators which must use the visitors now -
+ * but TODO they need their interface changing
+ */
+
+ if (!is_iterative) {
+ visit_start_sequence_compat(v, "oldstate",
+ VISIT_SEQ_COMPAT_BLOB, &wrapperf, &local_err);
+ }
+ LOCAL_ERR_REPORT(return -EINVAL;);
+
+ ret = se->ops->load_state(is_iterative ? visitor_get_qemufile(v) :
+ wrapperf,
+ se->opaque, version_id);
+ if (!is_iterative) {
+ visit_end_sequence_compat(v, "oldstate", VISIT_SEQ_COMPAT_BLOB,
+ &local_err);
+ }
+
+ LOCAL_ERR_REPORT(return -EINVAL;);
+
+ return ret;
}
- return vmstate_load_state(f, se->vmsd, se->opaque, version_id);
+ ret = vmstate_load_state(v, se->vmsd, se->opaque, version_id);
+
+ return ret;
}
static int vmstate_save(Visitor *v, SaveStateEntry *se, int version_id)
@@ -827,57 +856,69 @@ int qemu_loadvm_state(QEMUFile *f)
QLIST_HEAD(, LoadStateEntry) loadvm_handlers =
QLIST_HEAD_INITIALIZER(loadvm_handlers);
LoadStateEntry *le, *new_le;
- uint8_t section_type;
- unsigned int v;
+ Error *local_err = NULL;
+ int32_t section_type;
+ unsigned int tmp;
int ret;
if (qemu_savevm_state_blocked(NULL)) {
return -EINVAL;
}
- v = qemu_get_be32(f);
- if (v != QEMU_VM_FILE_MAGIC) {
+ tmp = qemu_get_be32(f);
+ if (tmp != QEMU_VM_FILE_MAGIC) {
return -EINVAL;
}
- v = qemu_get_be32(f);
- if (v == QEMU_VM_FILE_VERSION_COMPAT) {
- fprintf(stderr, "SaveVM v2 format is obsolete and don't work anymore\n");
+ tmp = qemu_get_be32(f);
+ if (tmp == QEMU_VM_FILE_VERSION_COMPAT) {
+ error_report("SaveVM v2 format is obsolete and don't work anymore");
return -ENOTSUP;
}
- if (v != QEMU_VM_FILE_VERSION) {
+ if (tmp != QEMU_VM_FILE_VERSION) {
return -ENOTSUP;
}
- while ((section_type = qemu_get_byte(f)) != QEMU_VM_EOF) {
- uint32_t instance_id, version_id, section_id;
+ /* TODO: Here we should be able to figure out if it's a binary file
+ * or what and open the right type of visitor
+ */
+ QemuFileBinInputVisitor *qfbiv = qemu_file_bin_input_visitor_new(f);
+ Visitor *v = qemu_file_bin_input_get_visitor(qfbiv);
+ qemu_file_set_tmp_visitor(f, v);
+
+ visit_start_sequence_compat(v, "top", VISIT_SEQ_COMPAT_BYTE0TERM,
+ NULL, &local_err);
+ while (visit_get_next_type(v, §ion_type, NULL, "section", &local_err),
+ (!local_err && section_type != QEMU_VM_EOF)) {
SaveStateEntry *se;
- char idstr[257];
- int len;
+ SectionHeader sh;
switch (section_type) {
case QEMU_VM_SECTION_START:
case QEMU_VM_SECTION_FULL:
/* Read section start */
- section_id = qemu_get_be32(f);
- len = qemu_get_byte(f);
- qemu_get_buffer(f, (uint8_t *)idstr, len);
- idstr[len] = 0;
- instance_id = qemu_get_be32(f);
- version_id = qemu_get_be32(f);
+ visit_start_sequence_compat(v,
+ (section_type == QEMU_VM_SECTION_START) ?
+ "secstart" : "secfull",
+ VISIT_SEQ_COMPAT_SECTION_HEADER, &sh, &local_err);
+ if (local_err) {
+ ret = -EINVAL;
+ goto out;
+ }
/* Find savevm section */
- se = find_se(idstr, instance_id);
+ se = find_se(sh.idstr, sh.instance_id);
if (se == NULL) {
- fprintf(stderr, "Unknown savevm section or instance '%s' %d\n", idstr, instance_id);
+ error_report("Unknown savevm section or instance '%s' %d",
+ sh.idstr, sh.instance_id);
ret = -EINVAL;
goto out;
}
/* Validate version */
- if (version_id > se->version_id) {
+ if (sh.version_id > se->version_id) {
fprintf(stderr, "savevm: unsupported version %d for '%s' v%d\n",
- version_id, idstr, se->version_id);
+ sh.version_id, sh.idstr, se->version_id);
ret = -EINVAL;
goto out;
}
@@ -886,38 +927,51 @@ int qemu_loadvm_state(QEMUFile *f)
le = g_malloc0(sizeof(*le));
le->se = se;
- le->section_id = section_id;
- le->version_id = version_id;
+ le->section_id = sh.section_id;
+ le->version_id = sh.version_id;
QLIST_INSERT_HEAD(&loadvm_handlers, le, entry);
- ret = vmstate_load(f, le->se, le->version_id);
+ ret = vmstate_load(v, le->se, le->version_id);
if (ret < 0) {
- fprintf(stderr, "qemu: warning: error while loading state for instance 0x%x of device '%s'\n",
- instance_id, idstr);
+ error_report("warning: error while loading state for instance "
+ "0x%x of device '%s'", sh.instance_id, sh.idstr);
goto out;
}
+ visit_end_sequence_compat(v,
+ (section_type == QEMU_VM_SECTION_START) ?
+ "secstart" : "secfull",
+ VISIT_SEQ_COMPAT_SECTION_HEADER, &local_err);
break;
case QEMU_VM_SECTION_PART:
case QEMU_VM_SECTION_END:
- section_id = qemu_get_be32(f);
+ visit_start_sequence_compat(v,
+ (section_type == QEMU_VM_SECTION_PART) ? "secpart" : "secend",
+ VISIT_SEQ_COMPAT_SECTION_MIN, &sh, &local_err);
+ if (local_err) {
+ ret = -EINVAL;
+ goto out;
+ }
QLIST_FOREACH(le, &loadvm_handlers, entry) {
- if (le->section_id == section_id) {
+ if (le->section_id == sh.section_id) {
break;
}
}
if (le == NULL) {
- fprintf(stderr, "Unknown savevm section %d\n", section_id);
+ error_report("Unknown savevm section %d", sh.section_id);
ret = -EINVAL;
goto out;
}
- ret = vmstate_load(f, le->se, le->version_id);
+ ret = vmstate_load(v, le->se, le->version_id);
if (ret < 0) {
- fprintf(stderr, "qemu: warning: error while loading state section id %d\n",
- section_id);
+ error_report("warning: error while loading state section id %d",
+ sh.section_id);
goto out;
}
+ visit_end_sequence_compat(v,
+ (section_type == QEMU_VM_SECTION_PART) ? "secpart" : "secend",
+ VISIT_SEQ_COMPAT_SECTION_MIN, &local_err);
break;
default:
fprintf(stderr, "Unknown savevm section type %d\n", section_type);
@@ -936,10 +990,17 @@ out:
g_free(le);
}
+ visit_end_sequence_compat(v, "top", VISIT_SEQ_COMPAT_BYTE0TERM, &local_err);
+ if (local_err) {
+ error_report("%s", error_get_pretty(local_err));
+ ret = -EINVAL;
+ }
+
if (ret == 0) {
ret = qemu_file_get_error(f);
}
+ qemu_file_bin_input_visitor_cleanup(qfbiv);
return ret;
}
diff --git a/vmstate.c b/vmstate.c
index 46bf7b9..af332b7 100644
--- a/vmstate.c
+++ b/vmstate.c
@@ -9,7 +9,7 @@
static void vmstate_subsection_save(Visitor *v, const VMStateDescription *vmsd,
void *opaque);
-static int vmstate_subsection_load(QEMUFile *f, const VMStateDescription *vmsd,
+static int vmstate_subsection_load(Visitor *v, const VMStateDescription *vmsd,
void *opaque);
#define LOCAL_ERR_REPORT(fallout) \
@@ -19,11 +19,20 @@ static int vmstate_subsection_load(QEMUFile *f, const VMStateDescription *vmsd,
fallout \
}
-int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd,
+int vmstate_load_state(Visitor *v, const VMStateDescription *vmsd,
void *opaque, int version_id)
{
VMStateField *field = vmsd->fields;
int ret;
+ Error *local_err = NULL;
+ QEMUFile *wrapperf;
+ uint32_t tmp32;
+
+ /* BER type comes from the vmsd if it's set */
+ tmp32 = vmsd->ber_tag;
+ visit_start_sequence_compat(v, vmsd->name, VISIT_SEQ_COMPAT_VMSTATE,
+ &tmp32, &local_err);
+ LOCAL_ERR_REPORT(return -EINVAL;);
if (version_id > vmsd->version_id) {
return -EINVAL;
@@ -32,7 +41,18 @@ int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd,
return -EINVAL;
}
if (version_id < vmsd->minimum_version_id) {
- return vmsd->load_state_old(f, opaque, version_id);
+ /* Some visitors put old format things down separate QEMUFile's */
+ visit_start_sequence_compat(v, vmsd->name, VISIT_SEQ_COMPAT_BLOB,
+ &wrapperf, &local_err);
+ LOCAL_ERR_REPORT(return -EINVAL;);
+
+ ret = vmsd->load_state_old(visitor_get_qemufile(v), opaque, version_id);
+
+ visit_end_sequence_compat(v, vmsd->name, VISIT_SEQ_COMPAT_BLOB,
+ &local_err);
+ LOCAL_ERR_REPORT(return -EINVAL;);
+
+ return ret;
}
if (vmsd->pre_load) {
int ret = vmsd->pre_load(opaque);
@@ -69,6 +89,11 @@ int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd,
if (field->flags & VMS_POINTER) {
base_addr = *(void **)base_addr + field->start;
}
+
+ if (n_elems > 1) {
+ visit_start_array(v, NULL, field->name, n_elems, 0, &local_err);
+ }
+
for (i = 0; i < n_elems; i++) {
void *addr = base_addr + size * i;
@@ -76,30 +101,62 @@ int vmstate_load_state(QEMUFile *f, const VMStateDescription *vmsd,
addr = *(void **)addr;
}
if (field->flags & VMS_STRUCT) {
- ret = vmstate_load_state(f, field->vmsd, addr,
+ ret = vmstate_load_state(v, field->vmsd, addr,
field->vmsd->version_id);
} else {
- ret = field->info->get(f, addr, size);
+ if (field->info->visit) {
+ ret = field->info->visit(v, addr, field->name, size,
+ &local_err);
+ } else {
+ /*
+ * Some visitors put old format things down separate
+ * QEMUFile's
+ */
+ visit_start_sequence_compat(v, field->name,
+ VISIT_SEQ_COMPAT_BLOB,
+ &wrapperf, &local_err);
+ LOCAL_ERR_REPORT(return -EINVAL;);
+ ret = field->info->get(wrapperf, addr, size);
+
+ visit_end_sequence_compat(v, field->name,
+ VISIT_SEQ_COMPAT_BLOB,
+ &local_err);
+ LOCAL_ERR_REPORT(return -EINVAL;);
+ }
}
if (ret < 0) {
return ret;
}
+
+ if ((i+1) != n_elems) {
+ visit_next_array(v, &local_err);
+ }
+ }
+
+ if (n_elems > 1) {
+ visit_end_array(v, &local_err);
}
+
}
field++;
}
- ret = vmstate_subsection_load(f, vmsd, opaque);
+ ret = vmstate_subsection_load(v, vmsd, opaque);
if (ret != 0) {
return ret;
}
if (vmsd->post_load) {
- return vmsd->post_load(opaque, version_id);
+ ret = vmsd->post_load(opaque, version_id);
}
- return 0;
+
+ visit_end_sequence_compat(v, vmsd->name, VISIT_SEQ_COMPAT_VMSTATE,
+ &local_err);
+ LOCAL_ERR_REPORT(return -EINVAL;);
+
+ return ret;
}
-void vmstate_save_state(QEMUFile *f, const VMStateDescription *vmsd,
+void vmstate_save_state(Visitor *v, const VMStateDescription *vmsd,
void *opaque)
{
VMStateField *field = vmsd->fields;
@@ -208,44 +265,56 @@ static const VMStateDescription *
return NULL;
}
-static int vmstate_subsection_load(QEMUFile *f, const VMStateDescription *vmsd,
+static int vmstate_subsection_load(Visitor *v, const VMStateDescription *vmsd,
void *opaque)
{
- while (qemu_peek_byte(f, 0) == QEMU_VM_SUBSECTION) {
- char idstr[256];
+ Error *local_err = NULL;
+ int32_t section_type;
+
+ if (!vmsd->subsections) {
+ /*
+ * If the type has no subsection defined at all then skip completely
+ * Note that this means if we have conditional subsections we will
+ * expect a subseclist, even if it's empty because the conditions are
+ * all false.
+ */
+ return 0;
+ }
+
+ visit_start_sequence_compat(v, "subseclist", VISIT_SEQ_COMPAT_SUBSECLIST,
+ (char *)vmsd->name, &local_err);
+ while (visit_get_next_type(v, §ion_type, NULL, "subsec", &local_err),
+ ((local_err == NULL) && (section_type == QEMU_VM_SUBSECTION))) {
int ret;
- uint8_t version_id, len, size;
const VMStateDescription *sub_vmsd;
+ SectionHeader sh;
- len = qemu_peek_byte(f, 1);
- if (len < strlen(vmsd->name) + 1) {
- /* subsection name has be be "section_name/a" */
- return 0;
- }
- size = qemu_peek_buffer(f, (uint8_t *)idstr, len, 2);
- if (size != len) {
- return 0;
- }
- idstr[size] = 0;
+ visit_start_sequence_compat(v, "subsection",
+ VISIT_SEQ_COMPAT_SUBSECTION, &sh,
+ &local_err);
+ LOCAL_ERR_REPORT(return -EINVAL;);
- if (strncmp(vmsd->name, idstr, strlen(vmsd->name)) != 0) {
- /* it don't have a valid subsection name */
- return 0;
- }
- sub_vmsd = vmstate_get_subsection(vmsd->subsections, idstr);
+ sub_vmsd = vmstate_subsection_name_lookup(vmsd->subsections, sh.idstr);
if (sub_vmsd == NULL) {
+ error_report("vmstate_subsection_load: failed to find subsection "
+ "%s in %s", sh.idstr, vmsd->name);
return -ENOENT;
}
- qemu_file_skip(f, 1); /* subsection */
- qemu_file_skip(f, 1); /* len */
- qemu_file_skip(f, len); /* idstr */
- version_id = qemu_get_be32(f);
- ret = vmstate_load_state(f, sub_vmsd, opaque, version_id);
+ ret = vmstate_load_state(v, sub_vmsd, opaque, sh.version_id);
if (ret) {
+ error_report("vmstate_subsection_load: load_state for %s failed "
+ "in %s", sh.idstr, vmsd->name);
return ret;
}
+ visit_end_sequence_compat(v, "subsection", VISIT_SEQ_COMPAT_SUBSECTION,
+ &local_err);
}
+ LOCAL_ERR_REPORT(return -EINVAL;);
+ visit_end_sequence_compat(v, "subseclist", VISIT_SEQ_COMPAT_SUBSECLIST,
+ &local_err);
+ LOCAL_ERR_REPORT(return -EINVAL;);
+
return 0;
}
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 10/16] Visitor: Common types to use visitors
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (8 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 09/16] Visitor: Load path Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 11/16] Choose output visitor based on env variable Dr. David Alan Gilbert (git)
` (6 subsequent siblings)
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Rework the common types in vmstate.c to use .visit rather than .get/.put
and now call the correct visitor function.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
vmstate.c | 376 ++++++++++++--------------------------------------------------
1 file changed, 73 insertions(+), 303 deletions(-)
diff --git a/vmstate.c b/vmstate.c
index af332b7..5a90cbe 100644
--- a/vmstate.c
+++ b/vmstate.c
@@ -1,3 +1,5 @@
+#include <inttypes.h>
+
#include "qemu-common.h"
#include "migration/migration.h"
#include "migration/qemu-file.h"
@@ -357,320 +359,88 @@ static void vmstate_subsection_save(Visitor *v, const VMStateDescription *vmsd,
LOCAL_ERR_REPORT(return;);
}
-/* bool */
-
-static int get_bool(QEMUFile *f, void *pv, size_t size)
-{
- bool *v = pv;
- *v = qemu_get_byte(f);
- return 0;
-}
-
-static void put_bool(QEMUFile *f, void *pv, size_t size)
-{
- bool *v = pv;
- qemu_put_byte(f, *v);
-}
-
-const VMStateInfo vmstate_info_bool = {
- .name = "bool",
- .get = get_bool,
- .put = put_bool,
-};
-
-/* 8 bit int */
-
-static int get_int8(QEMUFile *f, void *pv, size_t size)
-{
- int8_t *v = pv;
- qemu_get_s8s(f, v);
- return 0;
-}
-
-static void put_int8(QEMUFile *f, void *pv, size_t size)
-{
- int8_t *v = pv;
- qemu_put_s8s(f, v);
-}
-
-const VMStateInfo vmstate_info_int8 = {
- .name = "int8",
- .get = get_int8,
- .put = put_int8,
-};
-
-/* 16 bit int */
-
-static int get_int16(QEMUFile *f, void *pv, size_t size)
-{
- int16_t *v = pv;
- qemu_get_sbe16s(f, v);
- return 0;
-}
-
-static void put_int16(QEMUFile *f, void *pv, size_t size)
-{
- int16_t *v = pv;
- qemu_put_sbe16s(f, v);
-}
-
-const VMStateInfo vmstate_info_int16 = {
- .name = "int16",
- .get = get_int16,
- .put = put_int16,
-};
-
-/* 32 bit int */
-
-static int get_int32(QEMUFile *f, void *pv, size_t size)
-{
- int32_t *v = pv;
- qemu_get_sbe32s(f, v);
- return 0;
-}
-
-static void put_int32(QEMUFile *f, void *pv, size_t size)
-{
- int32_t *v = pv;
- qemu_put_sbe32s(f, v);
-}
-
-const VMStateInfo vmstate_info_int32 = {
- .name = "int32",
- .get = get_int32,
- .put = put_int32,
-};
-
-/* 32 bit int. See that the received value is the same than the one
- in the field */
-
-static int get_int32_equal(QEMUFile *f, void *pv, size_t size)
-{
- int32_t *v = pv;
- int32_t v2;
- qemu_get_sbe32s(f, &v2);
-
- if (*v == v2) {
- return 0;
- }
- return -EINVAL;
-}
-
-const VMStateInfo vmstate_info_int32_equal = {
- .name = "int32 equal",
- .get = get_int32_equal,
- .put = put_int32,
-};
+#define VMSTATE_INFO_FORTYPE(vn, vt) \
+ static int visit_ ## vn(Visitor *v, void *pdata, const char *name, \
+ size_t size, Error **errp) \
+ { \
+ visit_type_ ## vn(v, (vt *)pdata, name, errp); \
+ return 0; \
+ } \
+ const VMStateInfo vmstate_info_ ## vn = { \
+ .name = #vn , \
+ .visit = visit_ ## vn, \
+}
+
+VMSTATE_INFO_FORTYPE(bool, bool);
+VMSTATE_INFO_FORTYPE(int8, int8_t);
+VMSTATE_INFO_FORTYPE(int16, int16_t);
+VMSTATE_INFO_FORTYPE(int32, int32_t);
+VMSTATE_INFO_FORTYPE(int64, int64_t);
+VMSTATE_INFO_FORTYPE(uint8, uint8_t);
+VMSTATE_INFO_FORTYPE(uint16, uint16_t);
+VMSTATE_INFO_FORTYPE(uint32, uint32_t);
+VMSTATE_INFO_FORTYPE(uint64, uint64_t);
+
+/* equality entries; Check that the value read from the stream is the same
+ * as the one in the field */
+#define VMSTATE_INFO_EQ_FORTYPE(vn, vt, vf) \
+ static int visit_ ## vn ## _equal(Visitor *v, void *pdata, \
+ const char *name, size_t size, Error **errp) \
+ { \
+ if (visitor_loading(v)) { \
+ vt received; \
+ \
+ visit_type_ ## vn(v, &received, name, errp); \
+ if (*(vt *)pdata == received) { \
+ return 0; \
+ } \
+ error_setg(errp, \
+ "Unexpected value for field '%s', expected 0x%" vf \
+ " but got 0x%" vf, name, *(vt *)pdata, received); \
+ return -EINVAL; \
+ } else { \
+ visit_type_ ## vn(v, (vt *)pdata, name, errp); \
+ return 0; \
+ } \
+ } \
+ const VMStateInfo vmstate_info_ ## vn ## _equal = { \
+ .name = #vn " equal", \
+ .visit = visit_ ## vn ## _equal, \
+}
+
+VMSTATE_INFO_EQ_FORTYPE(int32, int32_t, PRIx32);
+VMSTATE_INFO_EQ_FORTYPE(uint8, uint8_t, PRIx8);
+VMSTATE_INFO_EQ_FORTYPE(uint16, uint16_t, PRIx16);
+VMSTATE_INFO_EQ_FORTYPE(uint32, uint32_t, PRIx32);
+VMSTATE_INFO_EQ_FORTYPE(uint64, uint64_t, PRIx64);
/* 32 bit int. Check that the received value is less than or equal to
the one in the field */
-
-static int get_int32_le(QEMUFile *f, void *pv, size_t size)
+static int visit_int32_le(Visitor *v, void *pdata, const char *name,
+ size_t size, Error **errp)
{
- int32_t *cur = pv;
- int32_t loaded;
- qemu_get_sbe32s(f, &loaded);
+ if (visitor_loading(v)) {
+ int32_t loaded;
- if (loaded <= *cur) {
- *cur = loaded;
+ visit_type_int32(v, &loaded, name, errp);
+ if (loaded <= *(int32_t *)pdata) {
+ *(int32_t *)pdata = loaded;
+ return 0;
+ }
+ error_setg(errp, "Unexpected value for field '%s', expected a value"
+ " no larger than %d but got %d", name, *(int32_t *)pdata,
+ loaded);
+ return -EINVAL;
+ } else {
+ visit_type_int32(v, (int32_t *)pdata, name, errp);
return 0;
}
- return -EINVAL;
+
}
const VMStateInfo vmstate_info_int32_le = {
.name = "int32 le",
- .get = get_int32_le,
- .put = put_int32,
-};
-
-/* 64 bit int */
-
-static int get_int64(QEMUFile *f, void *pv, size_t size)
-{
- int64_t *v = pv;
- qemu_get_sbe64s(f, v);
- return 0;
-}
-
-static void put_int64(QEMUFile *f, void *pv, size_t size)
-{
- int64_t *v = pv;
- qemu_put_sbe64s(f, v);
-}
-
-const VMStateInfo vmstate_info_int64 = {
- .name = "int64",
- .get = get_int64,
- .put = put_int64,
-};
-
-/* 8 bit unsigned int */
-
-static int get_uint8(QEMUFile *f, void *pv, size_t size)
-{
- uint8_t *v = pv;
- qemu_get_8s(f, v);
- return 0;
-}
-
-static void put_uint8(QEMUFile *f, void *pv, size_t size)
-{
- uint8_t *v = pv;
- qemu_put_8s(f, v);
-}
-
-const VMStateInfo vmstate_info_uint8 = {
- .name = "uint8",
- .get = get_uint8,
- .put = put_uint8,
-};
-
-/* 16 bit unsigned int */
-
-static int get_uint16(QEMUFile *f, void *pv, size_t size)
-{
- uint16_t *v = pv;
- qemu_get_be16s(f, v);
- return 0;
-}
-
-static void put_uint16(QEMUFile *f, void *pv, size_t size)
-{
- uint16_t *v = pv;
- qemu_put_be16s(f, v);
-}
-
-const VMStateInfo vmstate_info_uint16 = {
- .name = "uint16",
- .get = get_uint16,
- .put = put_uint16,
-};
-
-/* 32 bit unsigned int */
-
-static int get_uint32(QEMUFile *f, void *pv, size_t size)
-{
- uint32_t *v = pv;
- qemu_get_be32s(f, v);
- return 0;
-}
-
-static void put_uint32(QEMUFile *f, void *pv, size_t size)
-{
- uint32_t *v = pv;
- qemu_put_be32s(f, v);
-}
-
-const VMStateInfo vmstate_info_uint32 = {
- .name = "uint32",
- .get = get_uint32,
- .put = put_uint32,
-};
-
-/* 32 bit uint. See that the received value is the same than the one
- in the field */
-
-static int get_uint32_equal(QEMUFile *f, void *pv, size_t size)
-{
- uint32_t *v = pv;
- uint32_t v2;
- qemu_get_be32s(f, &v2);
-
- if (*v == v2) {
- return 0;
- }
- return -EINVAL;
-}
-
-const VMStateInfo vmstate_info_uint32_equal = {
- .name = "uint32 equal",
- .get = get_uint32_equal,
- .put = put_uint32,
-};
-
-/* 64 bit unsigned int */
-
-static int get_uint64(QEMUFile *f, void *pv, size_t size)
-{
- uint64_t *v = pv;
- qemu_get_be64s(f, v);
- return 0;
-}
-
-static void put_uint64(QEMUFile *f, void *pv, size_t size)
-{
- uint64_t *v = pv;
- qemu_put_be64s(f, v);
-}
-
-const VMStateInfo vmstate_info_uint64 = {
- .name = "uint64",
- .get = get_uint64,
- .put = put_uint64,
-};
-
-/* 64 bit unsigned int. See that the received value is the same than the one
- in the field */
-
-static int get_uint64_equal(QEMUFile *f, void *pv, size_t size)
-{
- uint64_t *v = pv;
- uint64_t v2;
- qemu_get_be64s(f, &v2);
-
- if (*v == v2) {
- return 0;
- }
- return -EINVAL;
-}
-
-const VMStateInfo vmstate_info_uint64_equal = {
- .name = "int64 equal",
- .get = get_uint64_equal,
- .put = put_uint64,
-};
-
-/* 8 bit int. See that the received value is the same than the one
- in the field */
-
-static int get_uint8_equal(QEMUFile *f, void *pv, size_t size)
-{
- uint8_t *v = pv;
- uint8_t v2;
- qemu_get_8s(f, &v2);
-
- if (*v == v2) {
- return 0;
- }
- return -EINVAL;
-}
-
-const VMStateInfo vmstate_info_uint8_equal = {
- .name = "uint8 equal",
- .get = get_uint8_equal,
- .put = put_uint8,
-};
-
-/* 16 bit unsigned int int. See that the received value is the same than the one
- in the field */
-
-static int get_uint16_equal(QEMUFile *f, void *pv, size_t size)
-{
- uint16_t *v = pv;
- uint16_t v2;
- qemu_get_be16s(f, &v2);
-
- if (*v == v2) {
- return 0;
- }
- return -EINVAL;
-}
-
-const VMStateInfo vmstate_info_uint16_equal = {
- .name = "uint16 equal",
- .get = get_uint16_equal,
- .put = put_uint16,
+ .visit = visit_int32_le,
};
/* floating point */
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 11/16] Choose output visitor based on env variable
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (9 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 10/16] Visitor: Common types to use visitors Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 12/16] BER Visitor: Create output visitor Dr. David Alan Gilbert (git)
` (5 subsequent siblings)
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
savevm.c | 39 ++++++++++++++++++++++++++++++++-------
1 file changed, 32 insertions(+), 7 deletions(-)
diff --git a/savevm.c b/savevm.c
index 7431773..25e92f2 100644
--- a/savevm.c
+++ b/savevm.c
@@ -26,6 +26,7 @@
#include "qemu-common.h"
#include "qapi/qemu-file-binary-input-visitor.h"
#include "qapi/qemu-file-binary-output-visitor.h"
+#include "qapi/qemu-file-debug-output-visitor.h"
#include "hw/hw.h"
#include "hw/qdev.h"
#include "net/net.h"
@@ -524,6 +525,35 @@ bool qemu_savevm_state_blocked(Error **errp)
return false;
}
+/* Return a visitor for use on the QEMUFile; visit_destroy should be
+ * called on it to clean it up.
+ */
+static Visitor *qemu_savevm_get_visitor(QEMUFile *f)
+{
+ char *formatvar = getenv("QEMUMIGFORMAT");
+
+ if (formatvar && (!strcmp(formatvar, "debug"))) {
+ QemuFileDebugOutputVisitor *qfdov =
+ qemu_file_debug_output_visitor_new(f);
+ Visitor *v = qemu_file_debug_output_get_visitor(qfdov);
+
+ qemu_file_set_tmp_visitor(f, v);
+ return v;
+ }
+
+ if (formatvar) {
+ error_report("QEMUMIGFORMAT set to unknown value '%s'", formatvar);
+ assert(0);
+ }
+
+ QemuFileBinOutputVisitor *qfbov = qemu_file_bin_output_visitor_new(f);
+ Visitor *v = qemu_file_bin_output_get_visitor(qfbov);
+
+ qemu_file_set_tmp_visitor(f, v);
+
+ return v;
+}
+
void qemu_savevm_state_begin(QEMUFile *f,
const MigrationParams *params)
{
@@ -531,10 +561,7 @@ void qemu_savevm_state_begin(QEMUFile *f,
int ret;
Error *local_err = NULL;
SectionHeader sh;
-
- QemuFileBinOutputVisitor *qfbov = qemu_file_bin_output_visitor_new(f);
- Visitor *v = qemu_file_bin_output_get_visitor(qfbov);
- qemu_file_set_tmp_visitor(f, v);
+ Visitor *v = qemu_savevm_get_visitor(f);
QTAILQ_FOREACH(se, &savevm_handlers, entry) {
if (!se->ops || !se->ops->set_params) {
@@ -781,13 +808,11 @@ static int qemu_savevm_state(QEMUFile *f)
static int qemu_save_device_state(QEMUFile *f)
{
+ Visitor *v = qemu_savevm_get_visitor(f);
SaveStateEntry *se;
Error *local_err;
SectionHeader sh;
- QemuFileBinOutputVisitor *qfbov = qemu_file_bin_output_visitor_new(f);
- qemu_file_set_tmp_visitor(f, qemu_file_bin_output_get_visitor(qfbov));
- Visitor *v = qemu_file_bin_output_get_visitor(qfbov);
int32_t section_type;
cpu_synchronize_all_states();
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 12/16] BER Visitor: Create output visitor
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (10 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 11/16] Choose output visitor based on env variable Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 13/16] BER Visitor: Create input visitor Dr. David Alan Gilbert (git)
` (4 subsequent siblings)
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
ber.h: Add VMSTATE_CPU_COMMON
---
include/qapi/ber.h | 108 ++++
include/qapi/qemu-file-ber-output-visitor.h | 26 +
qapi/Makefile.objs | 1 +
qapi/qemu-file-ber-output-visitor.c | 916 ++++++++++++++++++++++++++++
4 files changed, 1051 insertions(+)
create mode 100644 include/qapi/ber.h
create mode 100644 include/qapi/qemu-file-ber-output-visitor.h
create mode 100644 qapi/qemu-file-ber-output-visitor.c
diff --git a/include/qapi/ber.h b/include/qapi/ber.h
new file mode 100644
index 0000000..960bb17
--- /dev/null
+++ b/include/qapi/ber.h
@@ -0,0 +1,108 @@
+/*
+ * ASN.1 Basic Encoding Rules Common functions
+ *
+ * Copyright IBM, Corp. 2011, 2013
+ * Copyright Red Hat, Inc. 2011
+ *
+ * Authors:
+ * Stefan Berger <stefanb@us.ibm.com>
+ * Michael Tsirkin <mst@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+#ifndef QAPI_BER_H
+#define QAPI_BER_H
+
+/*
+ * This is a subset of BER for QEMU use.
+ * QEMU will use the DER encoding always with one extension from
+ * CER: SET and SEQUENCE types can have indefinite-length encoding
+ * if the encoding is not all immediately available.
+ *
+ * We assume that SET encodings can be available or not available,
+ * and that SEQUENCE encodings are available unless a SEQUENCE includes
+ * a non-available SET.
+ *
+ * The last is an extension to allow an arbitrarily large SET
+ * to be produced online without knowing the length in advance.
+ *
+ * All types used shall be universal, with explicit tagging, to simplify
+ * use by external tools.
+ */
+
+
+#define BER_TYPE_CLASS_SHIFT 6
+#define BER_TYPE_PC_SHIFT 5
+
+typedef enum ber_type_class {
+ BER_TYPE_CLASS_UNIVERSAL = 0x0 << BER_TYPE_CLASS_SHIFT,
+ BER_TYPE_CLASS_APPLICATION = 0x1 << BER_TYPE_CLASS_SHIFT,
+ BER_TYPE_CLASS_CONTENT_SPECIFIC = 0x2 << BER_TYPE_CLASS_SHIFT,
+ BER_TYPE_CLASS_PRIVATE = 0x3 << BER_TYPE_CLASS_SHIFT,
+ BER_TYPE_CLASS_MASK = 0x3 << BER_TYPE_CLASS_SHIFT /* Mask to get class */
+} BERTypeClass;
+
+/* P/C bit */
+typedef enum ber_type_p_c {
+ BER_TYPE_PRIMITIVE = 0x0 << BER_TYPE_PC_SHIFT,
+ BER_TYPE_CONSTRUCTED = 0x1 << BER_TYPE_PC_SHIFT,
+ BER_TYPE_P_C_MASK = 0x1 << BER_TYPE_PC_SHIFT /* Mask to get P/C bit */
+} BERTypePC;
+
+typedef enum ber_type_tag {
+ BER_TYPE_EOC = 0 /* P 0 0*/,
+ BER_TYPE_BOOLEAN = 1 /* P 1 1*/,
+ BER_TYPE_INTEGER = 2 /* P 2 2*/,
+ BER_TYPE_OCTET_STRING = 4 /* P/C 4 4*/,
+ BER_TYPE_NULL = 5 /* P 5 5*/,
+ BER_TYPE_UTF8_STRING = 12 /* P/C 12 C*/,
+ BER_TYPE_SEQUENCE = 16 /* C 16 10*/,
+ BER_TYPE_SET = 17 /* C 17 11*/,
+ /*BER_TYPE_NUMERIC_STRING P/C 18 12*/
+ /*BER_TYPE_PRINTABLE_STRING P/C 19 13*/
+ /*BER_TYPE_T61STRING P/C 20 14*/
+ /*BER_TYPE_VIDEOTEX_STRING P/C 21 15*/
+ /*BER_TYPE_IA5_STRING P/C 22 16*/
+ /*BER_TYPE_UTCTIME P/C 23 17*/
+ /*BER_TYPE_GENERALIZED_TIME P/C 24 18*/
+ /*BER_TYPE_GRAPHIC_STRING P/C 25 19*/
+ /*BER_TYPE_VISIBLE_STRING P/C 26 1A*/
+ /*BER_TYPE_GENERAL_STRING P/C 27 1B*/
+ /*BER_TYPE_UNIVERSAL_STRING P/C 28 1C*/
+ /*BER_TYPE_CHARACTER_STRING P/C 29 1D*/
+ /*BER_TYPE_BMP_STRING P/C 30 1E*/
+ BER_TYPE_LONG_FORM = 31/* - 31 1F*/,
+ BER_TYPE_TAG_MASK = 0x1f /* Mask to get tag */,
+ BER_TYPE_CUSTOM_LIST = 0x20,
+
+ /* It would be nice to autoderive this from the spec file somehow */
+ BER_TYPE_QEMU_FILE = 1270481, /* come out as 7f cd c5 51 - the 51 is Q
+ * the c5 and cd being E,M but with the top
+ * bit set which BER requires.
+ */
+ BER_TYPE_QEMU_RAMSEC_ENTRY = 8914,
+ BER_TYPE_QEMU_RAMSEC_LIST = 9810,
+ BER_TYPE_QEMU_SEC_MIN = 211,
+ BER_TYPE_QEMU_SEC_FULL = 2003,
+ BER_TYPE_QEMU_SUBSEC_LIST = 10700,
+ BER_TYPE_QEMU_SUBSEC = 10707,
+
+ /* Specific VMState types */
+ BER_TYPE_QEMU_VMSTATE_CPU_COMMON = 20000,
+} BERTypeTag;
+
+typedef enum ber_length {
+ /* Special length values */
+ BER_LENGTH_INDEFINITE = 0x1 << 7,
+ BER_LENGTH_RESERVED = 0xFF,
+ /* Anything else is either short or long */
+ BER_LENGTH_SHORT = 0x0 << 7,
+ BER_LENGTH_LONG = 0x1 << 7,
+ BER_LENGTH_SHORT_LONG_MASK = 0x1 << 7,
+ BER_LENGTH_MASK = 0x7F,
+} BERLength;
+
+#endif /* QAPI_BER_H */
+
diff --git a/include/qapi/qemu-file-ber-output-visitor.h b/include/qapi/qemu-file-ber-output-visitor.h
new file mode 100644
index 0000000..341b28a
--- /dev/null
+++ b/include/qapi/qemu-file-ber-output-visitor.h
@@ -0,0 +1,26 @@
+/*
+ * QEMUFile BER format output visitor
+ *
+ * Copyright 2014 Red Hat, Inc. and/or its affiliates
+ *
+ * Authors:
+ * David Gilbert <dgilbert@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef QEMU_FILE_BER_OUTPUT_VISITOR_H
+#define QEMU_FILE_BER_OUTPUT_VISITOR_H
+
+#include "visitor.h"
+
+typedef struct QemuFileBEROutputVisitor QemuFileBEROutputVisitor;
+
+QemuFileBEROutputVisitor *qemu_file_ber_output_visitor_new(QEMUFile *f);
+void qemu_file_ber_output_visitor_cleanup(QemuFileBEROutputVisitor *d);
+
+Visitor *qemu_file_ber_output_get_visitor(QemuFileBEROutputVisitor *v);
+
+#endif
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index 3d9d47a..f8fb347 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -2,5 +2,6 @@ util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qmp-input-visitor.o
util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
util-obj-y += string-input-visitor.o string-output-visitor.o
util-obj-y += qemu-file-binary-output-visitor.o qemu-file-binary-input-visitor.o
+util-obj-y += qemu-file-ber-output-visitor.o
util-obj-y += qemu-file-debug-output-visitor.o
util-obj-y += opts-visitor.o
diff --git a/qapi/qemu-file-ber-output-visitor.c b/qapi/qemu-file-ber-output-visitor.c
new file mode 100644
index 0000000..b38a184
--- /dev/null
+++ b/qapi/qemu-file-ber-output-visitor.c
@@ -0,0 +1,916 @@
+/*
+ * QEMUFile Output Visitor
+ *
+ * Copyright 2014 Red Hat, Inc. and/or its affiliates
+ *
+ * Authors:
+ * David Gilbert <dgilbert@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include "qapi/ber.h"
+#include "qapi/qemu-file-ber-output-visitor.h"
+#include "qapi/qemu-file-binary-output-visitor.h"
+#include "qapi/visitor-impl.h"
+#include "qapi/qmp/qerror.h"
+#include "qemu/queue.h"
+#include "qemu-common.h"
+#include "hw/hw.h"
+#include "migration/migration.h"
+
+/* Note that this can generate so much debug virt-test times out */
+#if 0
+#define DPRINTF(v, fmt, ...) \
+ do { \
+ fprintf(stderr, "qfberov/%s/%d: " fmt "\n", __func__, \
+ __LINE__, ## __VA_ARGS__); \
+ } while (0)
+#else
+#define DPRINTF(v, fmt, ...) \
+ do { } while (0)
+#endif
+
+/* TODO: This needs sharing between input/output binary visitors */
+typedef struct {
+ size_t elem_count;
+ size_t elem_size;
+ size_t pos;
+} ArrayInfo;
+
+typedef struct {
+ Visit_seq_compat_mode mode;
+ const void *data;
+ QEMUFile *binfile;
+} SeqCompatInfo;
+
+typedef struct StackEntry {
+ enum {
+ QFOV_ARRAY,
+ QFOV_LIST,
+ QFOV_STRUCT,
+ QFOV_SEQCOMPAT,
+ } type;
+ ArrayInfo array_info;
+ SeqCompatInfo seqcompat_info;
+ bool is_list_head;
+ QTAILQ_ENTRY(StackEntry) node;
+} StackEntry;
+
+struct QemuFileBEROutputVisitor {
+ Visitor visitor;
+ QTAILQ_HEAD(, StackEntry) stack;
+ QEMUFile *file;
+};
+
+static QemuFileBEROutputVisitor *to_ov(Visitor *v)
+{
+ return container_of(v, QemuFileBEROutputVisitor, visitor);
+}
+
+/* Encode 'ber_type' into buffer, returning the number of bytes used */
+static unsigned int ber_encode_type(uint8_t *buffer, uint32_t buflen,
+ uint64_t ber_type,
+ uint8_t ber_type_flags,
+ Error **errp)
+{
+ unsigned int idx = 0;
+
+ if (buflen < 1) {
+ error_set(errp, QERR_BUFFER_OVERRUN);
+ return 0;
+ }
+
+ if (ber_type > BER_TYPE_LONG_FORM) {
+ int byte = 4;
+ uint32_t mask = 0x7f << (7 * byte);
+ bool do_write = false;
+
+ buffer[0] = ber_type_flags | BER_TYPE_LONG_FORM;
+
+ while (byte >= 0) {
+ if (!do_write) {
+ if ((mask & ber_type)) {
+ do_write = true;
+ if (1 + byte + 1 > buflen) {
+ error_set(errp, QERR_BUFFER_OVERRUN);
+ return 0;
+ }
+ }
+ }
+ if (do_write) {
+ buffer[1 + idx] = (ber_type >> (7 * byte)) & 0x7f;
+ if (byte > 0) {
+ buffer[1 + idx] |= 0x80;
+ }
+ idx++;
+ }
+ byte--;
+ mask = 0x7f << (7 * byte);
+ }
+ } else {
+ buffer[0] = ber_type | ber_type_flags;
+ }
+ return 1 + idx;
+}
+
+/* Write the header (to f) for a SEQUENCE with indefinite length
+ * of the given type/flags
+ */
+static void ber_start_indeflen_seq(QEMUFile *f, uint64_t ber_type,
+ uint8_t ber_type_flags,
+ Error **errp)
+{
+ const unsigned int BUFLEN = 16;
+ uint8_t tmpbuf[BUFLEN];
+ size_t count;
+
+ count = ber_encode_type(tmpbuf, BUFLEN, ber_type, ber_type_flags, errp);
+ if (*errp) {
+ return;
+ }
+ tmpbuf[count++] = BER_LENGTH_INDEFINITE;
+
+ qemu_put_buffer(f, tmpbuf, count);
+}
+
+/* Write the terminator (00 00) for an object of indefinite length */
+static void ber_write_eoc(QEMUFile *f)
+{
+ qemu_put_be16(f, 0);
+}
+
+static unsigned int ber_encode_len(uint8_t *buffer, uint32_t buflen,
+ uint64_t len, Error **errp)
+{
+ uint64_t mask = 0xFF00000000000000ULL;
+ int shift = 64 - 8;
+ int c = 0;
+
+ if (len <= 0x7f && buflen >= 1) {
+ buffer[0] = len;
+ return 1;
+ }
+
+ while (mask && (mask & len) == 0) {
+ mask >>= 8;
+ shift -= 8;
+ }
+
+ while (shift >= 0) {
+ if (1 + c + 1 > buflen) {
+ error_set(errp, QERR_BUFFER_OVERRUN);
+ return 0;
+ }
+ buffer[1+c] = len >> shift;
+ c++;
+ shift -= 8;
+ }
+
+ buffer[0] = BER_LENGTH_LONG | c;
+
+ return 1 + c;
+}
+
+/* BER integers are encoded in a packed format that gets longer depending
+ * on the value; you'd normally call this with
+ * ber_type=BER_TYPE_INTEGER ber_type_flags=BER_TYPE_CLASS_UNIVERSAL
+ */
+static unsigned int ber_write_uint(QEMUFile *f, uint64_t val, uint64_t ber_type,
+ uint8_t ber_type_flags, Error **errp)
+{
+ const unsigned int BUFLEN = 32; /* Can't ever run out of room for
+ upto 64bit type and value */
+ uint8_t buffer[BUFLEN];
+ unsigned int offset, len_offset;
+ int shift;
+
+ offset = ber_encode_type(buffer, BUFLEN, ber_type, ber_type_flags, errp);
+ if (*errp) {
+ return 0;
+ }
+
+ /* Leave a gap to put the length in */
+ len_offset = offset;
+ offset++;
+
+ /* Find the MSByte that's none-0 */
+ for (shift = (64-8); shift; shift -= 8) {
+ if (val & (0xfful << shift)) {
+ break;
+ }
+ }
+
+ /* BER integers are always (inconveniently) signed, so if the MSBit
+ * of the 1st byte to be sent is set, we must put a leading 0 in
+ */
+ if (val & (0x80ul << shift)) {
+ buffer[offset++] = 0;
+ }
+ do {
+ buffer[offset++] = (val >> shift) & 0xff;
+ shift -= 8;
+ } while (shift >= 0);
+
+ buffer[len_offset] = (offset - len_offset)-1;
+
+ qemu_put_buffer(f, buffer, offset);
+
+ return offset;
+}
+
+/* BER integers are encoded in a packed format that gets longer depending
+ * on the value; you'd normally call this with
+ * ber_type=BER_TYPE_INTEGER ber_type_flags=BER_TYPE_CLASS_UNIVERSAL
+ */
+static unsigned int ber_write_int(QEMUFile *f, int64_t val, uint64_t ber_type,
+ uint8_t ber_type_flags, Error **errp)
+{
+ const unsigned int BUFLEN = 32; /* Can't ever run out of room for
+ upto 64bit type and value */
+ uint8_t buffer[BUFLEN];
+ unsigned int offset, len_offset;
+ int shift;
+ bool isneg = val < 0;
+ /* What a byte of sign extension looks like */
+ uint8_t signbyte = isneg ? 0xff : 0;
+ uint64_t uval = (uint64_t) val;
+
+ offset = ber_encode_type(buffer, BUFLEN, ber_type, ber_type_flags, errp);
+ if (*errp) {
+ return 0;
+ }
+
+ /* Leave a gap to put the length in */
+ len_offset = offset;
+ offset++;
+
+ /* Find the MSByte that's none-sign-extension */
+ for (shift = (64-8); shift; shift -= 8) {
+ if (((uval >> shift) & 0xff) != signbyte) {
+ break;
+ }
+ }
+
+ do {
+ buffer[offset++] = (uval >> shift) & 0xff;
+ shift -= 8;
+ } while (shift >= 0);
+
+ buffer[len_offset] = (offset - len_offset)-1;
+
+ qemu_put_buffer(f, buffer, offset);
+
+ return offset;
+}
+
+/* BER string is type, length, and then data */
+static unsigned int ber_write_string(QEMUFile *f, const char* val,
+ uint64_t ber_type, uint8_t ber_type_flags,
+ Error **errp)
+{
+ const unsigned int BUFLEN = 32; /* Can't ever run out of room for
+ upto 64bit type and length */
+ uint8_t buffer[BUFLEN];
+ unsigned int offset;
+ uint64_t len;
+
+ offset = ber_encode_type(buffer, BUFLEN/2, ber_type, ber_type_flags, errp);
+ len = strlen(val);
+ offset += ber_encode_len(buffer+offset, BUFLEN/2, len, errp);
+ if (*errp) {
+ return 0;
+ }
+
+ qemu_put_buffer(f, buffer, offset);
+ qemu_put_buffer(f, (const uint8_t *)val, len);
+
+ return offset;
+}
+
+static void qfbero_push(QemuFileBEROutputVisitor *ov, StackEntry *e)
+{
+ QTAILQ_INSERT_HEAD(&ov->stack, e, node);
+}
+
+static void qfbero_push_array(QemuFileBEROutputVisitor *ov, ArrayInfo ai)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFOV_ARRAY;
+ e->array_info = ai;
+ qfbero_push(ov, e);
+}
+
+static void qfbero_push_list(QemuFileBEROutputVisitor *ov)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFOV_LIST;
+ e->is_list_head = true;
+ qfbero_push(ov, e);
+}
+
+static void qfbero_push_seqcompat(QemuFileBEROutputVisitor *ov,
+ SeqCompatInfo sci)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFOV_SEQCOMPAT;
+ e->seqcompat_info = sci;
+ qfbero_push(ov, e);
+}
+
+static void qfbero_push_struct(QemuFileBEROutputVisitor *ov)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFOV_STRUCT;
+ qfbero_push(ov, e);
+}
+
+static StackEntry *qfbero_pop(QemuFileBEROutputVisitor *ov)
+{
+ StackEntry *e = QTAILQ_FIRST(&ov->stack);
+ QTAILQ_REMOVE(&ov->stack, e, node);
+ return e;
+}
+
+static bool qfbero_is_array(QemuFileBEROutputVisitor *ov)
+{
+ StackEntry *e = QTAILQ_FIRST(&ov->stack);
+ return e && e->type == QFOV_ARRAY;
+}
+
+static bool qfbero_is_list(QemuFileBEROutputVisitor *ov)
+{
+ StackEntry *e = QTAILQ_FIRST(&ov->stack);
+ return e && e->type == QFOV_LIST;
+}
+
+/* If we are in a seqcompat list return true and fill in
+ * sci with the compat mode
+ */
+static bool qfbero_is_seqcompat(QemuFileBEROutputVisitor *ov,
+ SeqCompatInfo *sci)
+{
+ StackEntry *e = QTAILQ_FIRST(&ov->stack);
+ if (e && e->type == QFOV_SEQCOMPAT) {
+ *sci = e->seqcompat_info;
+ return true;
+ }
+ return false;
+}
+
+static void qfbero_start_struct(Visitor *v, void **obj,
+ const char *kind, const char *name,
+ size_t unused, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+
+ qfbero_push_struct(ov);
+
+ ber_start_indeflen_seq(ov->file, BER_TYPE_SEQUENCE,
+ BER_TYPE_CLASS_UNIVERSAL | BER_TYPE_CONSTRUCTED, errp);
+}
+
+static void qfbero_end_struct(Visitor *v, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ StackEntry *e = qfbero_pop(ov);
+
+ if (!e || e->type != QFOV_STRUCT) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+
+ ber_write_eoc(ov->file);
+
+ g_free(e);
+}
+
+static void qfbero_start_list(Visitor *v, const char *name,
+ Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ qfbero_push_list(ov);
+
+ ber_start_indeflen_seq(ov->file, BER_TYPE_SEQUENCE,
+ BER_TYPE_CLASS_UNIVERSAL | BER_TYPE_CONSTRUCTED, errp);
+}
+
+static GenericList *qfbero_next_list(Visitor *v, GenericList **list,
+ Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ GenericList *entry;
+ StackEntry *e = QTAILQ_FIRST(&ov->stack);
+
+ /* Some users don't actually have a list */
+ if (!list) {
+ return NULL;
+ }
+ entry = *list;
+
+ if (!entry || !qfbero_is_list(ov)) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ }
+
+ /* The way the list iterator is currently used unfortunately clobbers
+ * **list by subseqently assigning our return value to the same container.
+ * This can cause an infinite loop, but we can get around this by tracking
+ * a bit of state to note when we should pass back the next entry rather
+ * than the current one.
+ */
+ if (e->is_list_head) {
+ e->is_list_head = false;
+ return entry;
+ }
+
+ *list = entry->next;
+ return entry->next;
+}
+
+static void qfbero_end_list(Visitor *v, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ StackEntry *e = qfbero_pop(ov);
+ if (!e || e->type != QFOV_LIST) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ }
+
+ ber_write_eoc(ov->file);
+
+ g_free(e);
+}
+
+static void qfbero_start_array(Visitor *v, void **obj,
+ const char *name,
+ size_t elem_count,
+ size_t elem_size, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ ArrayInfo ai = {
+ .elem_count = elem_count,
+ .elem_size = elem_size,
+ .pos = 0
+ };
+ qfbero_push_array(ov, ai);
+
+ ber_start_indeflen_seq(ov->file, BER_TYPE_SEQUENCE,
+ BER_TYPE_CLASS_UNIVERSAL | BER_TYPE_CONSTRUCTED, errp);
+}
+
+static void qfbero_next_array(Visitor *v, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ StackEntry *e = QTAILQ_FIRST(&ov->stack);
+ if (!qfbero_is_array(ov) ||
+ e->array_info.pos >= e->array_info.elem_count) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ }
+
+ e->array_info.pos++;
+}
+
+static void qfbero_end_array(Visitor *v, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ StackEntry *e = qfbero_pop(ov);
+ if (!e || e->type != QFOV_ARRAY) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+ ber_write_eoc(ov->file);
+ g_free(e);
+}
+
+static void qfbero_type_str(Visitor *v, char **obj, const char *name,
+ Error **errp)
+{
+ if (obj) {
+ g_free(*obj);
+ }
+ assert(0);
+}
+
+/* A string upto 256 bytes in length (including terminator)
+ * output as length byte (not including term) followed by text
+ * (also not including term)
+ */
+static void qfbero_type_str256(Visitor *v, char *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ ber_write_string(ov->file, obj, BER_TYPE_UTF8_STRING,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+}
+
+static void qfbero_type_buffer(Visitor *v, void *data, size_t len, bool async,
+ const char *name, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ const unsigned int BUFLEN = 32; /* Can't ever run out of room for
+ upto 64bit type and length */
+ uint8_t buffer[BUFLEN];
+ unsigned int offset;
+
+ offset = ber_encode_type(buffer, BUFLEN/2, BER_TYPE_OCTET_STRING,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL,
+ errp);
+ offset += ber_encode_len(buffer+offset, BUFLEN/2, len, errp);
+ if (*errp) {
+ return;
+ }
+ qemu_put_buffer(ov->file, buffer, offset);
+
+ /* And now the data */
+ if (async) {
+ qemu_put_buffer_async(ov->file, data, len);
+ } else {
+ qemu_put_buffer(ov->file, data, len);
+ }
+}
+
+static void qfbero_type_uint8(Visitor *v, uint8_t *obj,
+ const char *name,
+ Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ ber_write_uint(ov->file, *obj, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+}
+
+static void qfbero_type_uint16(Visitor *v, uint16_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ ber_write_uint(ov->file, *obj, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+}
+
+static void qfbero_type_uint32(Visitor *v, uint32_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ ber_write_uint(ov->file, *obj, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+}
+
+static void qfbero_type_uint64(Visitor *v, uint64_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ ber_write_uint(ov->file, *obj, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+}
+
+static void qfbero_type_int8(Visitor *v, int8_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ ber_write_int(ov->file, *obj, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+}
+
+static void qfbero_type_int16(Visitor *v, int16_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ ber_write_int(ov->file, *obj, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+}
+
+static void qfbero_type_int32(Visitor *v, int32_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ ber_write_int(ov->file, *obj, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+}
+
+static void qfbero_type_int64(Visitor *v, int64_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ ber_write_int(ov->file, *obj, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+}
+
+static void qfbero_type_bool(Visitor *v, bool *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ uint8_t val = *obj;
+ uint8_t buffer[3];
+
+ buffer[0] = BER_TYPE_BOOLEAN | BER_TYPE_CLASS_UNIVERSAL |
+ BER_TYPE_PRIMITIVE;
+ buffer[1] = 1; /* Length byte! */
+ buffer[2] = val ? 0xff : 0;
+ qemu_put_buffer(ov->file, buffer, 3);
+}
+
+static QEMUFile *qfbero_get_qemufile(Visitor *v)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+
+ return ov->file;
+}
+
+static void qfbero_get_next_type(Visitor *v, int *kind, const int *qobjects,
+ const char *name, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ SeqCompatInfo sci;
+
+ if (qfbero_is_seqcompat(ov, &sci)) {
+ DPRINTF(ov, "qfbero_get_next_type for %s", name);
+
+ switch (sci.mode) {
+ case VISIT_SEQ_COMPAT_BYTE0TERM:
+ /* BER doesn't need to do anything */
+ break;
+
+ default:
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+ return;
+ }
+
+ /* Only dealing with SeqCompat's for the moment */
+ error_set(errp, QERR_UNDEFINED_ERROR);
+}
+
+static void qfbero_start_sequence_compat(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ void *opaque, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ SeqCompatInfo sci = {
+ .mode = compat_mode,
+ .data = opaque
+ };
+ SectionHeader *sh;
+ ramsecentry_header *rse_hdr;
+ uint32_t tmp32;
+
+ switch (compat_mode) {
+ case VISIT_SEQ_COMPAT_FILE:
+ ber_start_indeflen_seq(ov->file, BER_TYPE_QEMU_FILE,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED, errp);
+ ber_write_uint(ov->file, 3 /* our version */, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ break;
+
+ case VISIT_SEQ_COMPAT_BYTE0TERM: /* TODO: Rename, this is actually 'top' */
+ /* Forms a sequence of 'sections' - just a normal sequence */
+ ber_start_indeflen_seq(ov->file, BER_TYPE_SEQUENCE,
+ BER_TYPE_CONSTRUCTED | BER_TYPE_CLASS_UNIVERSAL, errp);
+ break;
+
+ case VISIT_SEQ_COMPAT_RAMSECLIST:
+ ber_start_indeflen_seq(ov->file, BER_TYPE_QEMU_RAMSEC_LIST,
+ BER_TYPE_CONSTRUCTED | BER_TYPE_CLASS_APPLICATION, errp);
+ break;
+
+ case VISIT_SEQ_COMPAT_SECTION_HEADER:
+ sh = opaque;
+ ber_start_indeflen_seq(ov->file, BER_TYPE_QEMU_SEC_FULL,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED, errp);
+ ber_write_string(ov->file, sh->idstr, BER_TYPE_UTF8_STRING,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ ber_write_uint(ov->file, sh->section_id, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ ber_write_uint(ov->file, sh->instance_id, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ ber_write_uint(ov->file, sh->version_id, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ break;
+
+ case VISIT_SEQ_COMPAT_SECTION_MIN:
+ /* for VM_SECTION_PART/END where the section name->ID is already known
+ TODO: Actually, lets route the name through anyway */
+ sh = opaque;
+ ber_start_indeflen_seq(ov->file, BER_TYPE_QEMU_SEC_MIN,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED, errp);
+ ber_write_uint(ov->file, sh->section_id, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ break;
+
+ case VISIT_SEQ_COMPAT_SUBSECTION: /* An element in a subsection list */
+ sh = opaque;
+ ber_start_indeflen_seq(ov->file, BER_TYPE_QEMU_SUBSEC,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED, errp);
+ ber_write_string(ov->file, sh->idstr, BER_TYPE_UTF8_STRING,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ ber_write_uint(ov->file, sh->version_id, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ break;
+
+ case VISIT_SEQ_COMPAT_SUBSECLIST:
+ ber_start_indeflen_seq(ov->file, BER_TYPE_QEMU_SUBSEC_LIST,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED, errp);
+ break;
+
+ case VISIT_SEQ_COMPAT_RAMSECENTRY:
+ rse_hdr = opaque;
+ /* TODO: This is a bit big for the 0 page cases */
+ ber_start_indeflen_seq(ov->file, BER_TYPE_QEMU_RAMSEC_ENTRY,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED, errp);
+ ber_write_uint(ov->file, rse_hdr->addr, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ ber_write_uint(ov->file, rse_hdr->flags, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+
+ if ((rse_hdr->flags &
+ (RAM_SAVE_FLAG_CONTINUE | RAM_SAVE_FLAG_MEM_SIZE |
+ RAM_SAVE_FLAG_HOOK)) == 0) {
+ ber_write_string(ov->file, rse_hdr->idstr, BER_TYPE_UTF8_STRING,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ }
+ break;
+
+ case VISIT_SEQ_COMPAT_VMSTATE:
+ /*
+ * VMState's are just sequences, but the type tag is determined by
+ * the opaque, as follows
+ * 0 - a flag meaning use a universal sequence tag
+ * else - the tag to use with an BER_TYPE_CLASS_APPLICATION class
+ */
+ tmp32 = *(uint32_t *)opaque;
+ if (tmp32 == 0) {
+ ber_start_indeflen_seq(ov->file, BER_TYPE_SEQUENCE,
+ BER_TYPE_CLASS_UNIVERSAL | BER_TYPE_CONSTRUCTED, errp);
+ } else {
+ ber_start_indeflen_seq(ov->file, (enum ber_type_tag)tmp32,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED, errp);
+ }
+ break;
+
+ case VISIT_SEQ_COMPAT_BLOB: {
+ /* Note: There should be no other calls to this visitor until the
+ * end_sequence_compat for this blob. */
+ QemuFileBinOutputVisitor *qfbov;
+ Visitor *bv;
+
+ /* Write the blob data into a qemu_buf file - i.e. memory blob
+ * and hand that back as a QEMUFile to the caller
+ */
+ sci.binfile = qemu_bufopen("w", NULL);
+ /* and give that wrapper a binary output visitor so that it keeps
+ * substructures in compatibility mode
+ */
+ qfbov = qemu_file_bin_output_visitor_new(sci.binfile);
+ bv = qemu_file_bin_output_get_visitor(qfbov);
+ qemu_file_set_tmp_visitor(sci.binfile, bv);
+
+ *(QEMUFile **)opaque = sci.binfile;
+ break;
+ }
+
+ }
+
+ DPRINTF(qfbov, "qfbero_start_sequence_compat for %s/%d", name, compat_mode);
+ qfbero_push_seqcompat(ov, sci);
+ /* We don't need to read anything at this point */
+}
+
+static void qfbero_end_sequence_compat(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+ StackEntry *e = qfbero_pop(ov);
+ Visitor *bv;
+ const QEMUSizedBuffer *qsb;
+ size_t len, cur_iov;
+ const unsigned int BUFLEN = 32; /* Can't ever run out of room for
+ upto 64bit type and length */
+ uint8_t hdrbuf[BUFLEN];
+ unsigned int offset;
+
+ if (!e || e->type != QFOV_SEQCOMPAT) {
+ error_setg(errp, "bad struct stack %d", e ? e->type : -1);
+ if (e) {
+ g_free(e);
+ }
+ return;
+ }
+ switch (e->seqcompat_info.mode) {
+ case VISIT_SEQ_COMPAT_FILE:
+ case VISIT_SEQ_COMPAT_BYTE0TERM:
+ case VISIT_SEQ_COMPAT_SECTION_MIN:
+ case VISIT_SEQ_COMPAT_SECTION_HEADER:
+ case VISIT_SEQ_COMPAT_RAMSECLIST:
+ case VISIT_SEQ_COMPAT_RAMSECENTRY:
+ case VISIT_SEQ_COMPAT_SUBSECLIST:
+ case VISIT_SEQ_COMPAT_SUBSECTION:
+ case VISIT_SEQ_COMPAT_VMSTATE:
+ ber_write_eoc(ov->file);
+ break;
+
+ case VISIT_SEQ_COMPAT_BLOB:
+ bv = qemu_file_get_tmp_visitor(e->seqcompat_info.binfile);
+ visit_destroy(bv, errp);
+ qsb = qemu_buf_get(e->seqcompat_info.binfile);
+ len = qsb_get_length(qsb);
+
+ /* Set up headers so the blob is an OCTET_STRING */
+ offset = ber_encode_type(hdrbuf, BUFLEN/2, BER_TYPE_OCTET_STRING,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL,
+ errp);
+ offset += ber_encode_len(hdrbuf+offset, BUFLEN/2, len, errp);
+ DPRINTF(ov, "end blob: len=%zd", len);
+ if (*errp) {
+ qemu_fclose(e->seqcompat_info.binfile);
+ return;
+ }
+ qemu_put_buffer(ov->file, hdrbuf, offset);
+
+ /* all the data follows (concatinating the iov's) */
+ for (cur_iov = 0; cur_iov < qsb->n_iov; cur_iov++) {
+ /* The iov entries are partially filled */
+ size_t towrite = (qsb->iov[cur_iov].iov_len > len) ?
+ len :
+ qsb->iov[cur_iov].iov_len;
+ len -= towrite;
+ DPRINTF(ov, "end blob writing %zd of %zd leaving %zd", towrite, \
+ qsb->iov[cur_iov].iov_len, len);
+
+ if (!towrite) {
+ break;
+ }
+ qemu_put_buffer(ov->file, qsb->iov[cur_iov].iov_base, towrite);
+ }
+ qemu_fclose(e->seqcompat_info.binfile);
+ break;
+ }
+
+ if (e->seqcompat_info.mode != compat_mode) {
+ error_setg(errp, "mismatched seqcompat mode %d/%d", compat_mode,
+ e->seqcompat_info.mode);
+ }
+
+ DPRINTF(ov, "qfbero_end_sequence_compat %s/%d", name, \
+ e->seqcompat_info.mode);
+ g_free(e);
+}
+
+static void qfbero_destroy(Visitor *v, Error **errp)
+{
+ QemuFileBEROutputVisitor *ov = to_ov(v);
+
+ qemu_file_ber_output_visitor_cleanup(ov);
+}
+
+Visitor *qemu_file_ber_output_get_visitor(QemuFileBEROutputVisitor *v)
+{
+ return &v->visitor;
+}
+
+void qemu_file_ber_output_visitor_cleanup(QemuFileBEROutputVisitor *ov)
+{
+ g_free(ov);
+}
+
+QemuFileBEROutputVisitor *qemu_file_ber_output_visitor_new(QEMUFile *f)
+{
+ QemuFileBEROutputVisitor *v;
+
+ v = g_malloc0(sizeof(*v));
+
+ v->file = f;
+
+ v->visitor.start_struct = qfbero_start_struct;
+ v->visitor.end_struct = qfbero_end_struct;
+ v->visitor.start_list = qfbero_start_list;
+ v->visitor.next_list = qfbero_next_list;
+ v->visitor.end_list = qfbero_end_list;
+ v->visitor.start_array = qfbero_start_array;
+ v->visitor.next_array = qfbero_next_array;
+ v->visitor.end_array = qfbero_end_array;
+ v->visitor.type_buffer = qfbero_type_buffer;
+ v->visitor.type_int = qfbero_type_int64;
+ v->visitor.type_uint8 = qfbero_type_uint8;
+ v->visitor.type_uint16 = qfbero_type_uint16;
+ v->visitor.type_uint32 = qfbero_type_uint32;
+ v->visitor.type_uint64 = qfbero_type_uint64;
+ v->visitor.type_int8 = qfbero_type_int8;
+ v->visitor.type_int16 = qfbero_type_int16;
+ v->visitor.type_int32 = qfbero_type_int32;
+ v->visitor.type_int64 = qfbero_type_int64;
+ v->visitor.type_bool = qfbero_type_bool;
+ v->visitor.type_str = qfbero_type_str;
+ v->visitor.type_str256 = qfbero_type_str256;
+ v->visitor.destroy = qfbero_destroy;
+ v->visitor.start_sequence_compat = qfbero_start_sequence_compat;
+ v->visitor.get_next_type = qfbero_get_next_type;
+ v->visitor.end_sequence_compat = qfbero_end_sequence_compat;
+ v->visitor.get_qemufile = qfbero_get_qemufile;
+
+ v->visitor.flags = VISITOR_SAVING;
+
+ QTAILQ_INIT(&v->stack);
+
+ return v;
+}
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 13/16] BER Visitor: Create input visitor
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (11 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 12/16] BER Visitor: Create output visitor Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 14/16] Start some BER format docs Dr. David Alan Gilbert (git)
` (3 subsequent siblings)
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
include/qapi/qemu-file-ber-input-visitor.h | 26 +
qapi/Makefile.objs | 2 +-
qapi/qemu-file-ber-input-visitor.c | 1163 ++++++++++++++++++++++++++++
3 files changed, 1190 insertions(+), 1 deletion(-)
create mode 100644 include/qapi/qemu-file-ber-input-visitor.h
create mode 100644 qapi/qemu-file-ber-input-visitor.c
diff --git a/include/qapi/qemu-file-ber-input-visitor.h b/include/qapi/qemu-file-ber-input-visitor.h
new file mode 100644
index 0000000..18045f7
--- /dev/null
+++ b/include/qapi/qemu-file-ber-input-visitor.h
@@ -0,0 +1,26 @@
+/*
+ * QEMUFile input visitor for BER format files
+ *
+ * Copyright 2014 Red Hat, Inc. and/or its affiliates
+ *
+ * Authors:
+ * David Gilbert <dgilbert@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef QEMU_FILE_BER_INPUT_VISITOR_H
+#define QEMU_FILE_BER_INPUT_VISITOR_H
+
+#include "visitor.h"
+
+typedef struct QemuFileBERInputVisitor QemuFileBERInputVisitor;
+
+QemuFileBERInputVisitor *qemu_file_ber_input_visitor_new(QEMUFile *f);
+void qemu_file_ber_input_visitor_cleanup(QemuFileBERInputVisitor *d);
+
+Visitor *qemu_file_ber_input_get_visitor(QemuFileBERInputVisitor *v);
+
+#endif
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index f8fb347..c91dc46 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -2,6 +2,6 @@ util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qmp-input-visitor.o
util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
util-obj-y += string-input-visitor.o string-output-visitor.o
util-obj-y += qemu-file-binary-output-visitor.o qemu-file-binary-input-visitor.o
-util-obj-y += qemu-file-ber-output-visitor.o
+util-obj-y += qemu-file-ber-input-visitor.o qemu-file-ber-output-visitor.o
util-obj-y += qemu-file-debug-output-visitor.o
util-obj-y += opts-visitor.o
diff --git a/qapi/qemu-file-ber-input-visitor.c b/qapi/qemu-file-ber-input-visitor.c
new file mode 100644
index 0000000..d730208
--- /dev/null
+++ b/qapi/qemu-file-ber-input-visitor.c
@@ -0,0 +1,1163 @@
+/*
+ * QEMUFile input visitor for BER format
+ *
+ * Copyright IBM, Corp. 2011, 2013
+ * Copyright Red Hat, Corp. 2011,2014
+ *
+ * Authors:
+ * Anthony Liguori <aliguori@us.ibm.com>
+ * David Gilbert <dgilbert@redhat.com>
+ * Michael Tsirkin <mst@redhat.com>
+ * Stefan Berger <stefanb@us.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include "qapi/ber.h"
+#include "qapi/qemu-file-ber-input-visitor.h"
+#include "qapi/qemu-file-binary-input-visitor.h"
+#include "qapi/visitor-impl.h"
+#include "qapi/qmp/qerror.h"
+#include "qemu/queue.h"
+#include "qemu-common.h"
+#include "hw/hw.h"
+#include "migration/migration.h"
+
+#if 0
+#define DPRINTF(v, fmt, ...) \
+ do { \
+ fprintf(stderr, "%*s qfberiv/%s/%d: " fmt "\n", v->depth, "", \
+ __func__, __LINE__, ## __VA_ARGS__); \
+ } while (0)
+#else
+#define DPRINTF(v, fmt, ...) do { } while (0)
+#endif
+
+typedef struct {
+ size_t elem_count;
+ size_t elem_size;
+ size_t pos;
+} ArrayInfo;
+
+typedef struct {
+ Visit_seq_compat_mode mode;
+ const void *data;
+ bool hit_end;
+ QEMUFile *binfile;
+ QEMUSizedBuffer *qsb;
+} SeqCompatInfo;
+
+typedef struct StackEntry {
+ enum {
+ QFIV_ARRAY,
+ QFIV_LIST,
+ QFIV_STRUCT,
+ QFIV_SEQCOMPAT,
+ } type;
+ ArrayInfo array_info;
+ SeqCompatInfo seqcompat_info;
+ QTAILQ_ENTRY(StackEntry) node;
+} StackEntry;
+
+struct QemuFileBERInputVisitor {
+ Visitor visitor;
+ QTAILQ_HEAD(, StackEntry) stack;
+ QEMUFile *file;
+ unsigned int depth;
+};
+
+static QemuFileBERInputVisitor *to_iv(Visitor *v)
+{
+ return container_of(v, QemuFileBERInputVisitor, visitor);
+}
+
+static void qfberi_push(QemuFileBERInputVisitor *iv, StackEntry *e)
+{
+ QTAILQ_INSERT_HEAD(&iv->stack, e, node);
+ iv->depth++;
+}
+
+static void qfberi_push_array(QemuFileBERInputVisitor *iv,
+ ArrayInfo ai)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFIV_ARRAY;
+ e->array_info = ai;
+ qfberi_push(iv, e);
+}
+
+static void qfberi_push_list(QemuFileBERInputVisitor *iv)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFIV_LIST;
+ qfberi_push(iv, e);
+}
+
+static void qfberi_push_seqcompat(QemuFileBERInputVisitor *iv,
+ SeqCompatInfo sci)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFIV_SEQCOMPAT;
+ e->seqcompat_info = sci;
+ qfberi_push(iv, e);
+}
+
+static void qfberi_push_struct(QemuFileBERInputVisitor *iv)
+{
+ StackEntry *e = g_malloc0(sizeof(*e));
+ e->type = QFIV_STRUCT;
+ qfberi_push(iv, e);
+}
+
+static void *qfberi_pop(QemuFileBERInputVisitor *iv)
+{
+ StackEntry *e = QTAILQ_FIRST(&iv->stack);
+ if (e) {
+ QTAILQ_REMOVE(&iv->stack, e, node);
+ iv->depth--;
+ }
+ return e;
+}
+
+static bool qfberi_is_array(QemuFileBERInputVisitor *iv)
+{
+ StackEntry *e = QTAILQ_FIRST(&iv->stack);
+ return e->type == QFIV_ARRAY;
+}
+
+static bool qfberi_is_list(QemuFileBERInputVisitor *iv)
+{
+ StackEntry *e = QTAILQ_FIRST(&iv->stack);
+ return e && e->type == QFIV_LIST;
+}
+
+/* If we are in a seqcompat list return true and fill in
+ * sci with the compat mode
+ */
+static bool qfberi_is_seqcompat(QemuFileBERInputVisitor *iv,
+ SeqCompatInfo **sci)
+{
+ StackEntry *e = QTAILQ_FIRST(&iv->stack);
+ if (e && e->type == QFIV_SEQCOMPAT) {
+ *sci = &e->seqcompat_info;
+ return true;
+ }
+ return false;
+}
+
+/*
+ * Read a type tag from the stream. Up-to 32 bit type tags are supported
+ * for reading and otherwise an error is returned. Anything larger than that
+ * would not be reasonable and could only be abused.
+ *
+ * peek: Don't consume the type from the stream
+ */
+static uint32_t ber_read_type(QemuFileBERInputVisitor *iv,
+ uint8_t *ber_type_flags, bool peek, Error **errp)
+{
+ uint32_t type;
+ uint8_t byte;
+ uint8_t ctr = 0;
+ size_t byte_counter = 0;
+
+ if (*errp) { /* Skip if already in error */
+ return 0;
+ }
+
+ if (qemu_peek_buffer(iv->file, &byte, 1, byte_counter++) != 1) {
+ error_setg(errp, "QEMUFile has an error, error was '%s'",
+ "Error while reading type");
+ return 0;
+ }
+ type = byte;
+
+ *ber_type_flags = type & (BER_TYPE_P_C_MASK | BER_TYPE_CLASS_MASK);
+
+ if ((type & BER_TYPE_TAG_MASK) == BER_TYPE_LONG_FORM) {
+ type = 0;
+ while (true) {
+ type <<= 7;
+ if (qemu_peek_buffer(iv->file, &byte, 1, byte_counter++) != 1) {
+ error_setg(errp, "QEMUFile has an error, error was '%s'",
+ "Error while reading long type");
+ return 0;
+ }
+
+ type |= (byte & 0x7f);
+ if ((byte & 0x80) == 0) {
+ break;
+ }
+ ctr += 7; /* read 7 bits */
+ if (ctr >= (sizeof(type) * 8)) {
+ /* only support 32 bit type tags */
+ error_setg(errp, "Data stream is invalid; type tag is larger "
+ "than 32 bit");
+ return 0;
+ }
+ }
+ } else {
+ type &= BER_TYPE_TAG_MASK;
+ }
+
+ if (!peek) {
+ qemu_file_skip(iv->file, byte_counter);
+ }
+ return type;
+}
+
+static uint64_t ber_read_length(QemuFileBERInputVisitor *aiv,
+ bool *is_indefinite, Error **errp)
+{
+ uint8_t byte, c, int_len;
+ uint64_t len = 0;
+ QEMUFile *qfile = aiv->file;
+ unsigned char int_array[sizeof(len)];
+ char buf[128];
+
+ *is_indefinite = false;
+
+ if (qemu_get_buffer(qfile, &byte, 1) != 1) {
+ error_setg(errp, "QEMUFile has an error, error was '%s'",
+ "Error while reading length indicator");
+ return ~0x0ULL;
+ }
+
+ if (byte == BER_LENGTH_INDEFINITE) {
+ *is_indefinite = true;
+ return ~0x0ULL;
+ }
+
+ if (!(byte & BER_LENGTH_LONG)) {
+ len = byte;
+ } else {
+ int_len = byte & BER_LENGTH_MASK;
+ if (int_len > sizeof(len)) {
+ snprintf(buf, sizeof(buf),
+ "ASN.1 integer length field %d > %" PRIu64,
+ int_len, sizeof(len));
+ /* Length can be up to 127 byte, but it seems
+ * safe to assume any input will be < 1TB in length. */
+ error_set(errp, QERR_INVALID_PARAMETER, buf);
+ return ~0x0ULL;
+ }
+ if (qemu_get_buffer(qfile, int_array, int_len) != int_len) {
+ error_setg(errp, "QEMUFile error: Error while reading length");
+ return ~0x0ULL;
+ }
+ for (c = 0; c < int_len; c++) {
+ len <<= 8;
+ len |= int_array[c];
+ }
+ }
+
+ return len;
+}
+
+/* Read a type tag from the stream, and compare it to the expected type and
+ * flags.
+ * get_length_indef - if true also fetch the length, which is expected to be
+ * indefinite
+ * Return:
+ * True if it matched
+ * False if it failed to read, or failed to match
+ * setting errp to explain
+ */
+static bool ber_expect_type(QemuFileBERInputVisitor *iv,
+ enum ber_type_tag expected_type,
+ uint8_t expected_flags, bool get_length_indef,
+ Error **errp)
+{
+ uint32_t read_type;
+ uint8_t read_flags = 0; /* Silence compiler warning of possible uninit */
+
+ read_type = ber_read_type(iv, &read_flags, false, errp);
+ if (*errp) {
+ return false;
+ }
+
+ if ((expected_flags != read_flags) || (expected_type != read_type)) {
+ error_setg(errp, "Read incorrect BER types/flags; expecting %x/%x "
+ "read %x/%x",
+ expected_type, expected_flags,
+ read_type, read_flags);
+ return false;
+ }
+
+ DPRINTF(iv, "got match %x/%x", expected_type, expected_flags);
+
+ if (get_length_indef) {
+ bool is_indef;
+ ber_read_length(iv, &is_indef, errp);
+ if (*errp) {
+ return false;
+ }
+ if (!is_indef) {
+ error_setg(errp, "Got defined length, expecting indef (for %d/%d)",
+ expected_type, expected_flags);
+ return false;
+ }
+ }
+ return true;
+}
+
+/* The next entry is expected to be an EOC (0 0)
+ * read and error if not found.
+ */
+static void ber_consume_eoc(QemuFileBERInputVisitor *iv, Error **errp)
+{
+ uint16_t tmp;
+
+ qemu_get_be16s(iv->file, &tmp);
+
+ if (tmp != 0) {
+ error_setg(errp, "Expecting EOC but found %x", tmp);
+ }
+}
+
+/* Read an integer from the stream, with the given type/flags
+ * (typically integer/(primitive|universal) - but can be app specific
+ * obj - pointer to uint64_t to hold result
+ * intsize - Integer byte size the result must fit into (i.e. 1 for
+ * uint8_t, 2 for uint16_t etc)
+ * is_signed - True if the integer can be signed (in which case
+ * obj is treated as an int64_t
+ */
+static void ber_input_integer(QemuFileBERInputVisitor *iv, uint64_t *obj,
+ uint8_t intsize, bool is_signed,
+ enum ber_type_tag expected_type,
+ uint8_t expected_flags,
+ Error **errp)
+{
+ bool is_indefinite;
+ uint64_t len;
+ uint64_t val = 0;
+ unsigned int maxbytes;
+ unsigned char int_array[sizeof(val)+1];
+ int c;
+
+ assert(intsize <= 8);
+
+ if (!ber_expect_type(iv, expected_type, expected_flags, false, errp)) {
+ return;
+ }
+
+ len = ber_read_length(iv, &is_indefinite, errp);
+ if (error_is_set(errp)) {
+ return;
+ }
+
+ if (is_indefinite) {
+ error_set(errp, QERR_INVALID_PARAMETER_VALUE,
+ "BER int indicator is indefinite",
+ "[1..8]");
+ return;
+ }
+
+ /*
+ * Since BER ints are always signed, an n-byte unsigned int might need
+ * a 00 prefix byte so that it isn't signed.
+ */
+ maxbytes = intsize + (is_signed ? 0 : 1);
+ DPRINTF(iv, "len=%" PRIu64 " intsize=%d maxbytes=%d", len, intsize,
+ maxbytes);
+
+ if (len > maxbytes) {
+ char buf[128];
+ snprintf(buf, sizeof(buf), "BER integer length indicator %" PRIi64
+ " is larger than expected (%u bytes)",
+ len, maxbytes);
+ error_set(errp, QERR_INVALID_PARAMETER_VALUE,
+ buf, "[1..8]");
+ return;
+ }
+
+ if (qemu_get_buffer(iv->file, int_array, len) != len) {
+ error_setg(errp, "QEMUFile error: Error while reading integer");
+ return;
+ }
+
+ if (!is_signed && len == (intsize+1)) {
+ /*
+ * In the unsigned case, the extra byte can only be a 00 to indicate
+ * the lack of sign, anything else and it's too big to fit in the int
+ */
+ if (int_array[0]) {
+ error_setg(errp, "BER Integer; invalid 1st byte (%x) on unsigned",
+ int_array[0]);
+ return;
+ }
+ }
+
+ for (c = 0; c < len ; c++) {
+ val <<= 8;
+ val |= int_array[c];
+ if (c == 0 && (val & 0x80) == 0x80) {
+ if (!is_signed) {
+ error_setg(errp, "Unsigned integer received with sign set");
+ return;
+ }
+ /* sign extend */
+ val |= 0xffffffffffffff00ULL;
+ }
+ }
+
+ *obj = val;
+ DPRINTF(iv, "Val=%" PRIx64 " len=%" PRIu64, val, len);
+}
+
+static void qfberi_start_struct(Visitor *v, void **obj,
+ const char *kind,
+ const char *name, size_t size,
+ Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ DPRINTF(iv, "for '%s' of '%s'", name, kind);
+
+ if (!ber_expect_type(iv, BER_TYPE_SEQUENCE,
+ BER_TYPE_CLASS_UNIVERSAL | BER_TYPE_CONSTRUCTED,
+ true, errp)) {
+ return;
+ }
+ qfberi_push_struct(iv);
+}
+
+static void qfberi_end_struct(Visitor *v, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ StackEntry *e = qfberi_pop(iv);
+
+ DPRINTF(iv, "<");
+ if (!e || e->type != QFIV_STRUCT) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+ ber_consume_eoc(iv, errp);
+ g_free(e);
+}
+
+static void qfberi_start_list(Visitor *v, const char *name,
+ Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ if (!ber_expect_type(iv, BER_TYPE_SEQUENCE,
+ BER_TYPE_CLASS_UNIVERSAL | BER_TYPE_CONSTRUCTED,
+ true, errp)) {
+ return;
+ }
+ qfberi_push_list(iv);
+}
+
+static GenericList *qfberi_next_list(Visitor *v, GenericList **list,
+ Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ GenericList *entry;
+
+ if (!qfberi_is_list(iv)) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ }
+
+ /* Some users maintain their own list structure */
+ if (!list) {
+ return NULL;
+ }
+
+ entry = g_malloc0(sizeof(*entry));
+ if (*list) {
+ (*list)->next = entry;
+ }
+
+ *list = entry;
+ return entry;
+}
+
+static void qfberi_end_list(Visitor *v, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ StackEntry *e = qfberi_pop(iv);
+ if (!e || e->type != QFIV_LIST) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+ ber_consume_eoc(iv, errp);
+ g_free(e);
+}
+
+static void qfberi_start_array(Visitor *v, void **obj,
+ const char *name,
+ size_t elem_count,
+ size_t elem_size,
+ Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ ArrayInfo ai = {
+ .elem_count = elem_count,
+ .elem_size = elem_size,
+ .pos = 0
+ };
+ if (obj && (*obj == NULL) && elem_size) {
+ *obj = g_malloc0(elem_count * elem_size);
+ }
+ if (!ber_expect_type(iv, BER_TYPE_SEQUENCE,
+ BER_TYPE_CLASS_UNIVERSAL | BER_TYPE_CONSTRUCTED,
+ true, errp)) {
+ return;
+ }
+ qfberi_push_array(iv, ai);
+}
+
+static void qfberi_next_array(Visitor *v, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ StackEntry *e = QTAILQ_FIRST(&iv->stack);
+
+ if (!qfberi_is_array(iv) ||
+ e->array_info.pos >= e->array_info.elem_count) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ }
+
+ e->array_info.pos++;
+}
+
+static void qfberi_end_array(Visitor *v, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ StackEntry *e = qfberi_pop(iv);
+ if (!e || e->type != QFIV_ARRAY) {
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+ ber_consume_eoc(iv, errp);
+ g_free(e);
+}
+
+static void qfberi_type_str(Visitor *v, char **obj, const char *name,
+ Error **errp)
+{
+ if (obj) {
+ g_free(*obj);
+ }
+ assert(0); /* Not implemented yet for BER */
+}
+
+/* Read in a byte+buffer -> giving a string. obj must be a buffer of
+ * at least 256 chars in length
+ */
+static void qfberi_type_str256(Visitor *v, char *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ uint64_t len;
+ bool read_indef;
+
+ obj[0] = '\0'; /* in case we errror out */
+ DPRINTF(iv, "for %s", name);
+
+ if (!ber_expect_type(iv, BER_TYPE_UTF8_STRING,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL,
+ false, errp)) {
+ return;
+ }
+ len = ber_read_length(iv, &read_indef, errp);
+ if (*errp) {
+ return;
+ }
+ DPRINTF(iv, "for %s len=%" PRIu64, name, len);
+ if (read_indef || (len > 255)) {
+ error_setg(errp, "Invalid length reading '%s' (len=%" PRIu64
+ ") indef=%d", name, len, read_indef);
+ return;
+ }
+
+ if (qemu_get_buffer(iv->file, (uint8_t *)obj, len) != len) {
+ error_setg(errp, "QEMUFile error: Error while reading string '%s'",
+ name);
+ return;
+ }
+
+ obj[len] = '\0';
+ DPRINTF(iv, "'%s' got %s", name, obj);
+}
+
+static void qfberi_type_buffer(Visitor *v, void *data, size_t len, bool async,
+ const char *name, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ uint64_t read_len;
+ bool read_indef;
+
+ DPRINTF(iv, "for %s", name);
+ if (!ber_expect_type(iv, BER_TYPE_OCTET_STRING,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL,
+ false, errp)) {
+ return;
+ }
+ read_len = ber_read_length(iv, &read_indef, errp);
+ if (*errp) {
+ return;
+ }
+ DPRINTF(iv, "for %s len=%zu read_len=%" PRIu64, name, len, read_len);
+ if (read_indef || (len != read_len)) {
+ error_setg(errp, "Invalid length reading '%s' (len=%zd read_len=%"
+ PRIu64 ") indef=%d", name, len, read_len, read_indef);
+ return;
+ }
+
+ if (qemu_get_buffer(iv->file, data, len) != len) {
+ error_setg(errp, "QEMUFile error: Error reading buffer for '%s'", name);
+ }
+ DPRINTF(iv, "'%s' read %zd bytes:", name, len);
+ /*qemu_hexdump(data, stderr, __FUNCTION__, len); */
+ /*DPRINTF(iv, "------------------");*/
+}
+
+static void qfberi_type_uint8(Visitor *v, uint8_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ uint64_t tmp64;
+ ber_input_integer(iv, &tmp64, 1, false, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ *obj = (uint8_t)tmp64;
+}
+
+static void qfberi_type_uint16(Visitor *v, uint16_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ uint64_t tmp64;
+ ber_input_integer(iv, &tmp64, 2, false, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ *obj = (uint16_t)tmp64;
+}
+
+static void qfberi_type_uint32(Visitor *v, uint32_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ uint64_t tmp64;
+ ber_input_integer(iv, &tmp64, 4, false, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ *obj = (uint32_t)tmp64;
+}
+
+static void qfberi_type_uint64(Visitor *v, uint64_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ uint64_t tmp64;
+ ber_input_integer(iv, &tmp64, 8, false, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ *obj = tmp64;
+}
+
+static void qfberi_type_int8(Visitor *v, int8_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ int64_t tmp64;
+ ber_input_integer(iv, (uint64_t *)&tmp64, 1, true, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ *obj = (int8_t)tmp64;
+}
+
+static void qfberi_type_int16(Visitor *v, int16_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ int64_t tmp64;
+ ber_input_integer(iv, (uint64_t *)&tmp64, 2, true, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ *obj = (int16_t)tmp64;
+}
+
+static void qfberi_type_int32(Visitor *v, int32_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ int64_t tmp64;
+ ber_input_integer(iv, (uint64_t *)&tmp64, 4, true, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ *obj = (int32_t)tmp64;
+ DPRINTF(iv, "for '%s' / %d", name, *obj);
+}
+
+static void qfberi_type_int64(Visitor *v, int64_t *obj,
+ const char *name, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ int64_t tmp64;
+ ber_input_integer(iv, (uint64_t *)&tmp64, 8, true, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ *obj = tmp64;
+ DPRINTF(iv, "for '%s' / %ld", name, *obj);
+}
+
+static void qfberi_type_bool(Visitor *v, bool *obj, const char *name,
+ Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ uint8_t tmp;
+ if (!ber_expect_type(iv, BER_TYPE_BOOLEAN,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL,
+ false, errp)) {
+ return;
+ }
+
+ tmp = qemu_get_byte(iv->file);
+ if (tmp != 1) {
+ error_setg(errp, "Invalid length (%d) for boolean '%s'", tmp, name);
+ return;
+ }
+
+ tmp = qemu_get_byte(iv->file);
+ *obj = (tmp != 0);
+}
+
+static QEMUFile *qfberi_get_qemufile(Visitor *v)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+
+ return iv->file;
+}
+
+static void qfberi_get_next_type(Visitor *v, int *kind, const int *qobjects,
+ const char *name, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ SeqCompatInfo *sci;
+ uint32_t read_type;
+ uint8_t read_flags;
+
+ read_type = ber_read_type(iv, &read_flags, true /* peek */, errp);
+
+ DPRINTF(iv, "for '%s' peeked %x/%x", name, read_type, read_flags);
+ if (qfberi_is_seqcompat(iv, &sci)) {
+ DPRINTF(iv, "/seqcompat for '%s'", name);
+ if (sci->hit_end) {
+ error_setg(errp, "Attempted to read beyond the end of list '%s'",
+ name);
+ *kind = -1;
+ return;
+ }
+
+ switch (sci->mode) {
+ case VISIT_SEQ_COMPAT_BYTE0TERM:
+ switch (read_type) {
+ case BER_TYPE_QEMU_SEC_MIN:
+ *kind = QEMU_VM_SECTION_PART; /* Caller expects part or end
+ we only represent one */
+ break;
+
+ case BER_TYPE_QEMU_SEC_FULL:
+ *kind = QEMU_VM_SECTION_FULL; /* Caller expects full or start
+ we only represent full */
+ break;
+
+ case BER_TYPE_EOC:
+ *kind = QEMU_VM_EOF;
+ sci->hit_end = true;
+ break;
+
+ default:
+ error_setg(errp, "Unexpected type 0x%x for %s", read_type,
+ name);
+ *kind = -1;
+ }
+ break;
+
+ case VISIT_SEQ_COMPAT_RAMSECLIST:
+ /* The only flag we're providing to the host here is the EOS, it
+ * will get the rest from a ramsecentry
+ * (Might change if we have a type for 0 page)
+ */
+ switch (read_type) {
+ case BER_TYPE_EOC:
+ *kind = RAM_SAVE_FLAG_EOS;
+ sci->hit_end = true;
+ break;
+
+ case BER_TYPE_QEMU_RAMSEC_ENTRY:
+ *kind = 0; /* Don't have visibility of type here */
+ break;
+
+ default:
+ error_setg(errp, "Unexpected type 0x%x for %s", read_type,
+ name);
+ break;
+ }
+ break;
+
+ case VISIT_SEQ_COMPAT_SUBSECLIST:
+ switch (read_type) {
+ case BER_TYPE_EOC:
+ *kind = QEMU_VM_EOF;
+ sci->hit_end = true;
+ break;
+
+ case BER_TYPE_QEMU_SUBSEC:
+ *kind = QEMU_VM_SUBSECTION;
+ break;
+
+ default:
+ error_setg(errp, "Unexpected type 0x%x for %s", read_type,
+ name);
+ break;
+ }
+ break;
+
+ default:
+ *kind = -1;
+ error_set(errp, QERR_UNDEFINED_ERROR);
+ return;
+ }
+ return;
+ }
+
+ /* Only dealing with SeqCompat's for the moment */
+ error_set(errp, QERR_UNDEFINED_ERROR);
+}
+
+static void qfberi_start_sequence_compat(Visitor *v, const char *name,
+ Visit_seq_compat_mode compat_mode,
+ void *opaque, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ SeqCompatInfo sci = {
+ .mode = compat_mode,
+ .data = opaque
+ };
+ SectionHeader *sh;
+ ramsecentry_header *rse_hdr;
+ uint64_t tmp64, len;
+ uint32_t tmp32;
+ uint8_t *buffer;
+ QemuFileBinInputVisitor *qfbiv;
+ Visitor *bv;
+ bool read_indef;
+
+ switch (compat_mode) {
+ case VISIT_SEQ_COMPAT_FILE:
+ DPRINTF(iv, "for '%s'", name);
+ if (!ber_expect_type(iv, BER_TYPE_QEMU_FILE,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED,
+ true, errp)) {
+ return;
+ }
+ DPRINTF(iv, "for '%s'", name);
+ ber_input_integer(iv, &tmp64, sizeof(tmp64), false, BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ if (*errp) {
+ return;
+ }
+ DPRINTF(iv, "for '%s'", name);
+
+ if (tmp64 != 3) {
+ error_setg(errp, "Unsupported file version %" PRIu64 " expecting 3",
+ tmp64);
+ return;
+ }
+ DPRINTF(iv, "for '%s'", name);
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+ case VISIT_SEQ_COMPAT_BYTE0TERM:
+ if (!ber_expect_type(iv, BER_TYPE_SEQUENCE,
+ BER_TYPE_CONSTRUCTED | BER_TYPE_CLASS_UNIVERSAL,
+ true, errp)) {
+ return;
+ }
+ break;
+
+ case VISIT_SEQ_COMPAT_RAMSECLIST:
+ if (!ber_expect_type(iv, BER_TYPE_QEMU_RAMSEC_LIST,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED,
+ true, errp)) {
+ return;
+ }
+ break;
+
+ case VISIT_SEQ_COMPAT_SECTION_HEADER:
+ DPRINTF(iv, "for '%s'", name);
+ if (!ber_expect_type(iv, BER_TYPE_QEMU_SEC_FULL,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED,
+ true, errp)) {
+ return;
+ }
+ /*
+ * for VM_SECTION_FULL and VM_SECTION_START
+ * 'opaque' points to a struct Sectionheader
+ */
+ sh = opaque;
+ qfberi_type_str256(v, sh->idstr, name, errp);
+ ber_input_integer(iv, &tmp64, sizeof(sh->section_id), false,
+ BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ sh->section_id = tmp64;
+ ber_input_integer(iv, &tmp64, sizeof(sh->instance_id), false,
+ BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ sh->instance_id = tmp64;
+ ber_input_integer(iv, &tmp64, sizeof(sh->version_id), false,
+ BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ sh->version_id = tmp64;
+ DPRINTF(iv, "'%s' got %s %d/%d/%d", name, sh->idstr, sh->section_id,
+ sh->instance_id, sh->version_id);
+
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+ case VISIT_SEQ_COMPAT_SECTION_MIN:
+ /*
+ * for VM_SECTION_PART/END where the section ID is already known
+ * 'opaque' points to a struct Sectionheader
+ */
+ sh = opaque;
+ if (!ber_expect_type(iv, BER_TYPE_QEMU_SEC_MIN,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED,
+ true, errp)) {
+ return;
+ }
+ ber_input_integer(iv, &tmp64, sizeof(sh->section_id), false,
+ BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ sh->section_id = tmp64;
+ DPRINTF(iv, "'%s' got %d", name, sh->section_id);
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+ case VISIT_SEQ_COMPAT_SUBSECLIST:
+ if (!ber_expect_type(iv, BER_TYPE_QEMU_SUBSEC_LIST,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED,
+ true, errp)) {
+ return;
+ }
+ break;
+
+ case VISIT_SEQ_COMPAT_SUBSECTION:
+ DPRINTF(iv, "for '%s'", name);
+ if (!ber_expect_type(iv, BER_TYPE_QEMU_SUBSEC,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED,
+ true, errp)) {
+ return;
+ }
+ /* 'opaque' points to a struct Sectionheader */
+ sh = opaque;
+ qfberi_type_str256(v, sh->idstr, name, errp);
+ ber_input_integer(iv, &tmp64, sizeof(sh->version_id), false,
+ BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ sh->version_id = tmp64;
+ DPRINTF(iv, "'%s' got %s %d", name, sh->idstr, sh->version_id);
+
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+ case VISIT_SEQ_COMPAT_RAMSECENTRY:
+ rse_hdr = opaque;
+ if (!ber_expect_type(iv, BER_TYPE_QEMU_RAMSEC_ENTRY,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED,
+ true, errp)) {
+ return;
+ }
+ ber_input_integer(iv, &rse_hdr->addr, sizeof(rse_hdr->addr), false,
+ BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ ber_input_integer(iv, &tmp64, sizeof(rse_hdr->flags), false,
+ BER_TYPE_INTEGER,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL, errp);
+ rse_hdr->flags = (uint16_t)tmp64;
+
+ if ((rse_hdr->flags &
+ (RAM_SAVE_FLAG_CONTINUE | RAM_SAVE_FLAG_MEM_SIZE |
+ RAM_SAVE_FLAG_HOOK)) == 0) {
+ qfberi_type_str256(v, rse_hdr->idstr, name, errp);
+ }
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+ case VISIT_SEQ_COMPAT_VMSTATE:
+ /*
+ * VMState's are just sequences, but the type tag is determined by
+ * the opaque, as follows
+ * 0 - a flag meaning use a universal sequence tag
+ * else - the tag to use with an BER_TYPE_CLASS_APPLICATION class
+ */
+ tmp32 = *(uint32_t *)opaque;
+ if (tmp32 == 0) {
+ if (!ber_expect_type(iv, BER_TYPE_SEQUENCE,
+ BER_TYPE_CLASS_UNIVERSAL | BER_TYPE_CONSTRUCTED,
+ true, errp)) {
+ return;
+ }
+ } else {
+ if (!ber_expect_type(iv, (enum ber_type_tag)tmp32,
+ BER_TYPE_CLASS_APPLICATION | BER_TYPE_CONSTRUCTED,
+ true, errp)) {
+ return;
+ }
+ }
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+ case VISIT_SEQ_COMPAT_BLOB:
+ /* TODO! We don't need to, and shouldn't load this into memory
+ * we can do this as a shim on QEMUFile that only allows so many
+ * bytes to be loaded.
+ */
+ if (!ber_expect_type(iv, BER_TYPE_OCTET_STRING,
+ BER_TYPE_PRIMITIVE | BER_TYPE_CLASS_UNIVERSAL,
+ false, errp)) {
+ return;
+ }
+ len = ber_read_length(iv, &read_indef, errp);
+ if (*errp) {
+ return;
+ }
+ if (read_indef) {
+ error_setg(errp, "Unexpected indefinite length on '%s'",
+ name);
+ return;
+ }
+ DPRINTF(iv, "for %s read_len=%" PRIu64, name, len);
+ if (len > (10*1024*1024)) {
+ error_setg(errp, "Suspicious size (%" PRIu64 ") for '%s' ",
+ len, name);
+ return;
+ }
+ /* Load this blob into memory and hand back a QEMUFile that the
+ * caller can read.
+ */
+ buffer = g_malloc(len);
+
+ if (qemu_get_buffer(iv->file, buffer, len) != len) {
+ error_setg(errp, "QEMUFile error: Error reading buffer for '%s'",
+ name);
+ g_free(buffer);
+ }
+ sci.qsb = qsb_create(buffer, len);
+ g_free(buffer); /* Because qsb_create copies */
+ sci.binfile = qemu_bufopen("r", sci.qsb);
+
+ qfbiv = qemu_file_bin_input_visitor_new(sci.binfile);
+ bv = qemu_file_bin_input_get_visitor(qfbiv);
+ qemu_file_set_tmp_visitor(sci.binfile, bv);
+ *(QEMUFile **)opaque = sci.binfile;
+
+ sci.hit_end = true; /* It doesn't iterate over this as a list */
+ break;
+
+ }
+
+ DPRINTF(iv, "for '%s'", name);
+ qfberi_push_seqcompat(iv, sci);
+
+
+ /* We don't need to read anything at this point */
+}
+
+static void qfberi_end_sequence_compat(Visitor *v, const char* name,
+ Visit_seq_compat_mode compat_mode,
+ Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+ StackEntry *e = qfberi_pop(iv);
+ DPRINTF(iv, "> for '%s'", name);
+ if (!e || e->type != QFIV_SEQCOMPAT) {
+ error_setg(errp, "bad struct stack %d", e ? e->type : -1);
+ if (e) {
+ g_free(e);
+ }
+ return;
+ }
+ if (e->seqcompat_info.mode != compat_mode) {
+ error_setg(errp, "mismatched seqcompat mode %d/%d", compat_mode,
+ e->seqcompat_info.mode);
+ }
+ if (!*errp && !e->seqcompat_info.hit_end) {
+ error_setg(errp, "Didn't read the whole of list");
+ }
+ if (!*errp) {
+ switch (e->seqcompat_info.mode) {
+ case VISIT_SEQ_COMPAT_BYTE0TERM:
+ case VISIT_SEQ_COMPAT_FILE:
+ case VISIT_SEQ_COMPAT_RAMSECENTRY:
+ case VISIT_SEQ_COMPAT_RAMSECLIST:
+ case VISIT_SEQ_COMPAT_SECTION_HEADER:
+ case VISIT_SEQ_COMPAT_SECTION_MIN:
+ case VISIT_SEQ_COMPAT_SUBSECLIST:
+ case VISIT_SEQ_COMPAT_SUBSECTION:
+ case VISIT_SEQ_COMPAT_VMSTATE:
+ ber_consume_eoc(iv, errp);
+ break;
+
+ case VISIT_SEQ_COMPAT_BLOB:
+ qemu_fclose(e->seqcompat_info.binfile); /* Also frees the qsb */
+ break;
+ }
+ }
+ g_free(e);
+}
+
+static void qfberi_destroy(Visitor *v, Error **errp)
+{
+ QemuFileBERInputVisitor *iv = to_iv(v);
+
+ qemu_file_ber_input_visitor_cleanup(iv);
+}
+
+Visitor *qemu_file_ber_input_get_visitor(QemuFileBERInputVisitor *iv)
+{
+ return &iv->visitor;
+}
+
+void qemu_file_ber_input_visitor_cleanup(QemuFileBERInputVisitor *iv)
+{
+ g_free(iv);
+}
+
+QemuFileBERInputVisitor *qemu_file_ber_input_visitor_new(QEMUFile *f)
+{
+ QemuFileBERInputVisitor *v;
+
+ v = g_malloc0(sizeof(*v));
+
+ v->file = f;
+
+ v->visitor.start_struct = qfberi_start_struct;
+ v->visitor.end_struct = qfberi_end_struct;
+ v->visitor.start_list = qfberi_start_list;
+ v->visitor.next_list = qfberi_next_list;
+ v->visitor.end_list = qfberi_end_list;
+ v->visitor.start_array = qfberi_start_array;
+ v->visitor.next_array = qfberi_next_array;
+ v->visitor.end_array = qfberi_end_array;
+ v->visitor.type_int = qfberi_type_int64;
+ v->visitor.type_buffer = qfberi_type_buffer;
+ v->visitor.type_uint8 = qfberi_type_uint8;
+ v->visitor.type_uint16 = qfberi_type_uint16;
+ v->visitor.type_uint32 = qfberi_type_uint32;
+ v->visitor.type_uint64 = qfberi_type_uint64;
+ v->visitor.type_int8 = qfberi_type_int8;
+ v->visitor.type_int16 = qfberi_type_int16;
+ v->visitor.type_int32 = qfberi_type_int32;
+ v->visitor.type_int64 = qfberi_type_int64;
+ v->visitor.type_bool = qfberi_type_bool;
+ v->visitor.type_str = qfberi_type_str;
+ v->visitor.type_str256 = qfberi_type_str256;
+ v->visitor.destroy = qfberi_destroy;
+ v->visitor.start_sequence_compat = qfberi_start_sequence_compat;
+ v->visitor.get_next_type = qfberi_get_next_type;
+ v->visitor.end_sequence_compat = qfberi_end_sequence_compat;
+ v->visitor.get_qemufile = qfberi_get_qemufile;
+
+ v->visitor.flags = VISITOR_LOADING;
+
+ QTAILQ_INIT(&v->stack);
+ v->depth = 0;
+
+ return v;
+}
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 14/16] Start some BER format docs
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (12 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 13/16] BER Visitor: Create input visitor Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 15/16] ASN.1 schema for new migration format Dr. David Alan Gilbert (git)
` (2 subsequent siblings)
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Add information on the QEMUFile/Visitor/VMState split
Describe how to verify a BER stream against the schema
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
docs/migration.txt | 34 ++++++++++++++++++++++++++++++++--
1 file changed, 32 insertions(+), 2 deletions(-)
diff --git a/docs/migration.txt b/docs/migration.txt
index 0e0a1d4..894abf3 100644
--- a/docs/migration.txt
+++ b/docs/migration.txt
@@ -48,8 +48,17 @@ This is used for RAM and block devices. It is not yet ported to vmstate.
=== What is the common infrastructure ===
-QEMU uses a QEMUFile abstraction to be able to do migration. Any type
-of migration that wants to use QEMU infrastructure has to create a
+Migration consists of a number of layers that handle different parts of the
+problem:
+ QEMUFile - Buffering/reading/writing data without worrying about it's
+ contents. Versions exist for file I/O, sockets, RDMA and
+ to put data into a QEMU Block device.
+ Visitor - Visitors format data onto the QEMUFile. A binary format and
+ a BER format currently exist.
+ VMState - Represent the structure of the data for a given device with
+ support for versioning.
+
+Any type of migration that wants to use QEMU infrastructure has to create a
QEMUFile with:
QEMUFile *qemu_fopen_ops(void *opaque,
@@ -296,3 +305,24 @@ save/send this state when we are in the middle of a pio operation
(that is what ide_drive_pio_state_needed() checks). If DRQ_STAT is
not enabled, the values on that fields are garbage and don't need to
be sent.
+
+ASN.1/BER format migration
+==========================
+
+In BER format migration the migration stream is described by an ASN.1 schema
+(that's defined in docs/specs/migration.schema).
+
+The 'asn1c' program can be used to validate a migration stream against the schema
+ - although it'll probably use a lot of memory doing it - so it's best to do it
+with a small RAM image.
+
+To validate an image against the schema:
+ a) Make a temporary directory and cd into it
+ b) asn1c -fcompound-names /path/to/migration.schema
+ c) make -f Makefile.am.sample CFLAGS+="-DPDU=QemuFile -DEMIT_ASN_DEBUG=1 -I."
+ d) ./progname -iber log.ber -d 2> debug > decoding
+
+ If it's happy the 'decoding' output should be an XML like dump of the input file;
+if it's not happy, the debug file might help you find out why (Search backwards
+from the end for the first 'Freeing' to find the point at which it detected something
+bad).
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 15/16] ASN.1 schema for new migration format
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (13 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 14/16] Start some BER format docs Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 16/16] Wire in BER visitors Dr. David Alan Gilbert (git)
2014-03-25 20:43 ` [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Michael S. Tsirkin
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
docs/specs/migration.schema | 113 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 113 insertions(+)
create mode 100644 docs/specs/migration.schema
diff --git a/docs/specs/migration.schema b/docs/specs/migration.schema
new file mode 100644
index 0000000..d433127
--- /dev/null
+++ b/docs/specs/migration.schema
@@ -0,0 +1,113 @@
+-- Copyright 2014 Red Hat, Inc. and/or its affiliates
+--
+-- Authors:
+-- Dave Gilbert <dgilbert@redhat.com>
+--
+-- This work is licensed under the terms of the GNU LGPL, version 2 or later.
+-- See the COPYING.LIB file in the top-level directory.
+
+Qemu {}
+DEFINITIONS IMPLICIT TAGS ::=
+BEGIN
+
+-- Some basic types used in multiple places --
+QemuString ::= UTF8String (SIZE (1..255))
+
+-- TODO: 4096 is actually page size whatever that is
+FullPage ::= OCTET STRING (SIZE (4096))
+
+-- Types for specific devices --
+
+VMState-CPU-Common ::= [ APPLICATION 20000 ] SEQUENCE {
+ halted INTEGER,
+ interrupt-request INTEGER
+}
+
+--------------------------------
+
+RAMBlockID ::= SEQUENCE {
+ name QemuString,
+ len INTEGER
+}
+
+RAMSecEntry ::= [ APPLICATION 8914 ] SEQUENCE {
+ addr INTEGER, -- Address or offset or size
+ flags INTEGER, -- maybe more explicit type?
+ name QemuString OPTIONAL,
+
+ body CHOICE {
+ bl SEQUENCE OF RAMBlockID,
+ compr INTEGER (0..255), -- Page filled with this value
+ page FullPage
+ -- TODO xbzrle --
+ }
+}
+
+RAMSecList ::= [ APPLICATION 9810 ] SEQUENCE OF RAMSecEntry
+
+SubSection ::= [ APPLICATION 10707 ] SEQUENCE {
+ name QemuString,
+ versionid INTEGER,
+
+ contents SEQUENCE OF VMStateEntries
+}
+
+SubSecList ::= [ APPLICATION 10700 ] SEQUENCE OF SubSection
+
+VMStateEntries ::= CHOICE {
+ -- Hmm need to think more --
+ array SEQUENCE OF VMStateEntries,
+ bool BOOLEAN,
+ int INTEGER,
+ oldblob OCTET STRING,
+ subsecl SubSecList
+}
+
+VMState ::= CHOICE {
+ SEQUENCE OF VMStateEntries,
+
+ VMState-CPU-Common
+}
+
+-- Restrict to unsigned?
+SectionID ::= INTEGER
+
+SecFull ::= [ APPLICATION 2003 ] SEQUENCE {
+ name QemuString,
+ sectionid SectionID,
+ instanceid INTEGER,
+ versionid INTEGER,
+
+ contents CHOICE {
+ ramsec RAMSecList,
+ -- TODO other iterator initial stuff --
+ vmstate VMState,
+ oldblob OCTET STRING
+ }
+}
+
+SecMin ::= [ APPLICATION 211 ] SEQUENCE {
+ sectionid SectionID,
+
+ contents CHOICE {
+ ramsec RAMSecList
+ -- TODO other iterator general/end stuff --
+ }
+}
+
+Sections ::= CHOICE {
+ full SecFull,
+ min SecMin
+}
+
+-- The whole file --
+-- Application tag used to get first 32bits of file
+-- to come out as 7f cd c5 51 - the 51 is Q
+-- the c5 and cd being E,M but with the top bit set
+-- which BER requires
+QemuFile ::= [ APPLICATION 1270481 ] SEQUENCE {
+ version INTEGER (3),
+ top SEQUENCE OF Sections
+}
+
+END
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [Qemu-devel] [RFC PATCH 16/16] Wire in BER visitors
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (14 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 15/16] ASN.1 schema for new migration format Dr. David Alan Gilbert (git)
@ 2014-03-25 20:17 ` Dr. David Alan Gilbert (git)
2014-03-25 20:43 ` [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Michael S. Tsirkin
16 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert (git) @ 2014-03-25 20:17 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanb, quintela, mdroth, agraf, mst, aliguori, afaerber
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
BER output visitor to be selected with env (as other output visitors)
BER input visitor recognized by file header
exec.c: Set cpu-common to use it's own type tag (as example)
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
exec.c | 2 ++
include/migration/migration.h | 2 ++
savevm.c | 76 +++++++++++++++++++++++++++++++------------
3 files changed, 60 insertions(+), 20 deletions(-)
diff --git a/exec.c b/exec.c
index 91513c6..e2f3907 100644
--- a/exec.c
+++ b/exec.c
@@ -23,6 +23,7 @@
#endif
#include "qemu-common.h"
+#include "qapi/ber.h"
#include "cpu.h"
#include "tcg.h"
#include "hw/hw.h"
@@ -427,6 +428,7 @@ static int cpu_common_post_load(void *opaque, int version_id)
const VMStateDescription vmstate_cpu_common = {
.name = "cpu_common",
+ .ber_tag = BER_TYPE_QEMU_VMSTATE_CPU_COMMON,
.version_id = 1,
.minimum_version_id = 1,
.minimum_version_id_old = 1,
diff --git a/include/migration/migration.h b/include/migration/migration.h
index 1987e79..7ddb934 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -27,6 +27,8 @@
#define QEMU_VM_FILE_VERSION_COMPAT 0x00000002
#define QEMU_VM_FILE_VERSION 0x00000003
+#define QEMU_VM_BER_FILE_MAGIC 0x7fcdc551
+
#define QEMU_VM_EOF 0x00
#define QEMU_VM_SECTION_START 0x01
#define QEMU_VM_SECTION_PART 0x02
diff --git a/savevm.c b/savevm.c
index 25e92f2..3dbfd79 100644
--- a/savevm.c
+++ b/savevm.c
@@ -24,6 +24,8 @@
#include "config-host.h"
#include "qemu-common.h"
+#include "qapi/qemu-file-ber-output-visitor.h"
+#include "qapi/qemu-file-ber-input-visitor.h"
#include "qapi/qemu-file-binary-input-visitor.h"
#include "qapi/qemu-file-binary-output-visitor.h"
#include "qapi/qemu-file-debug-output-visitor.h"
@@ -532,6 +534,14 @@ static Visitor *qemu_savevm_get_visitor(QEMUFile *f)
{
char *formatvar = getenv("QEMUMIGFORMAT");
+ if (formatvar && (!strcmp(formatvar, "BER"))) {
+ QemuFileBEROutputVisitor *qfberov = qemu_file_ber_output_visitor_new(f);
+ Visitor *v = qemu_file_ber_output_get_visitor(qfberov);
+
+ qemu_file_set_tmp_visitor(f, v);
+ return v;
+ }
+
if (formatvar && (!strcmp(formatvar, "debug"))) {
QemuFileDebugOutputVisitor *qfdov =
qemu_file_debug_output_visitor_new(f);
@@ -876,41 +886,65 @@ typedef struct LoadStateEntry {
int version_id;
} LoadStateEntry;
+/* Given a just opened input QEMUFile, peak at the header and figure
+ * out which visitor should be used for it.
+ * Return the visitor ready for use.
+ */
+static Visitor *pick_input_visitor(QEMUFile *f)
+{
+ uint32_t tmp32;
+
+ if (qemu_peek_buffer(f, (uint8_t *)&tmp32, 4, 0) != 4) {
+ fprintf(stderr, "Failed to read SaveVM header word\n");
+ return NULL;
+ }
+
+ tmp32 = be32_to_cpu(tmp32);
+
+ switch (tmp32) {
+ case QEMU_VM_FILE_MAGIC: {
+ QemuFileBinInputVisitor *qfbiv = qemu_file_bin_input_visitor_new(f);
+ Visitor *v = qemu_file_bin_input_get_visitor(qfbiv);
+ qemu_file_set_tmp_visitor(f, v);
+
+ return v;
+ }
+
+ case QEMU_VM_BER_FILE_MAGIC: {
+ QemuFileBERInputVisitor *qfberiv = qemu_file_ber_input_visitor_new(f);
+ Visitor *v = qemu_file_ber_input_get_visitor(qfberiv);
+ qemu_file_set_tmp_visitor(f, v);
+
+ return v;
+ }
+
+ default:
+ fprintf(stderr, "Invalid SaveVM header (0x%x)\n", tmp32);
+ return NULL;
+ }
+}
+
int qemu_loadvm_state(QEMUFile *f)
{
QLIST_HEAD(, LoadStateEntry) loadvm_handlers =
QLIST_HEAD_INITIALIZER(loadvm_handlers);
LoadStateEntry *le, *new_le;
+ Visitor *v;
Error *local_err = NULL;
int32_t section_type;
- unsigned int tmp;
int ret;
if (qemu_savevm_state_blocked(NULL)) {
return -EINVAL;
}
- tmp = qemu_get_be32(f);
- if (tmp != QEMU_VM_FILE_MAGIC) {
- return -EINVAL;
- }
-
- tmp = qemu_get_be32(f);
- if (tmp == QEMU_VM_FILE_VERSION_COMPAT) {
- error_report("SaveVM v2 format is obsolete and don't work anymore");
+ v = pick_input_visitor(f);
+ if (!v) {
return -ENOTSUP;
}
- if (tmp != QEMU_VM_FILE_VERSION) {
- return -ENOTSUP;
- }
-
- /* TODO: Here we should be able to figure out if it's a binary file
- * or what and open the right type of visitor
- */
- QemuFileBinInputVisitor *qfbiv = qemu_file_bin_input_visitor_new(f);
- Visitor *v = qemu_file_bin_input_get_visitor(qfbiv);
- qemu_file_set_tmp_visitor(f, v);
+ visit_start_sequence_compat(v, "file", VISIT_SEQ_COMPAT_FILE, NULL,
+ &local_err);
visit_start_sequence_compat(v, "top", VISIT_SEQ_COMPAT_BYTE0TERM,
NULL, &local_err);
while (visit_get_next_type(v, §ion_type, NULL, "section", &local_err),
@@ -927,6 +961,7 @@ int qemu_loadvm_state(QEMUFile *f)
"secstart" : "secfull",
VISIT_SEQ_COMPAT_SECTION_HEADER, &sh, &local_err);
if (local_err) {
+ error_report("%s", error_get_pretty(local_err));
ret = -EINVAL;
goto out;
}
@@ -1016,6 +1051,7 @@ out:
}
visit_end_sequence_compat(v, "top", VISIT_SEQ_COMPAT_BYTE0TERM, &local_err);
+ visit_end_sequence_compat(v, "file", VISIT_SEQ_COMPAT_FILE, &local_err);
if (local_err) {
error_report("%s", error_get_pretty(local_err));
ret = -EINVAL;
@@ -1025,7 +1061,7 @@ out:
ret = qemu_file_get_error(f);
}
- qemu_file_bin_input_visitor_cleanup(qfbiv);
+ visit_destroy(v, &local_err);
return ret;
}
--
1.8.5.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
` (15 preceding siblings ...)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 16/16] Wire in BER visitors Dr. David Alan Gilbert (git)
@ 2014-03-25 20:43 ` Michael S. Tsirkin
2014-03-26 9:16 ` Dr. David Alan Gilbert
16 siblings, 1 reply; 24+ messages in thread
From: Michael S. Tsirkin @ 2014-03-25 20:43 UTC (permalink / raw)
To: Dr. David Alan Gilbert (git)
Cc: agraf, stefanb, quintela, mdroth, qemu-devel, aliguori, afaerber
On Tue, Mar 25, 2014 at 08:17:11PM +0000, Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> This is a work in progress cut of my visitor+BER format migration
> world; there's lots to do but it's starting to get there.
> I'd appreciate if anyone who has more experience with either the
> visitor code, or ASN.1/BER in general has a look to see if I'm
> doing anything particularly odd.
>
> I've cherry picked bits of Michael Roth's visitor set from 2011 and
> Stefan Berger's BER code from this time last year, but not necessarily
> used things in the same way.
Nice! Will take a look at patches later, just responding to cover
letter now:
> The good:
> It can perform a migration using a BER format file that's described
> and verifiable against a schema.
>
> It can also perform a migration using the old binary format.
>
> A lot of the nasty detail of the old format is hidden in the compatibility
> visitor (e.g. the subsection end detection).
>
> The ugly:
> 1) The encoding for zero pages is bloated, it can be simplified a
> lot (see previous discussion with Michael from a week or two back
> that I've not yet implemented)
> 2) There are a few shims that get passed visitors but need files
> or the other way that should be removable. The shim to get from
> a qemu file back to a visitor is patch 3 and marked as 'tmp' and
> the main uses are in piix4, pci.c, spapr_vscsi.c.
> 3) There are places where the visitor interface is too tied to
> the old file format, but most of those can be removed with effort.
> 4) I've made extensions to the Visitor type, many of these are very
> migration specific; and use things like get_next_type and the list
> visitors in quite a different way from the current uses.
> Should I really have a separate type?
> (see patch 1)
> 5) At the moment you select BER output format by setting an environment
> variable ( export QEMUMIGFORMAT=BER ) , I need to put more thought
> in to the right way to do this, there are some harder questions like
> what happens to devices that are still using pre-vmstate encodings
> (that are currently sent as blobs) when they eventually convert over
> and thus how to keep compatibility with earlier BER output versions
> where they were blobs.
A simple way is the way we always changed migration
format - trying this to the machine type.
Old machine type gets you binary, newer one BER with blobs,
then we can add more flags gradually.
With BER we'll actually be able to do smarter things in the future,
such as send out multiple formats and have destination select
the one it understands, but this isn't mandatory as a first step.
> 6) At the moment for blobs (i.e. pre-vmstate) the whole blob gets loaded
> into memory and then processed; that's not necessary and can be done
> with a QEMUFile shim.
> 7) I need to sort out my Error** handling (probably in a lot of places)
That's mostly broken in migration as it is.
> 8) The visitors should be able to share much more code.
>
> The not-yet-done:
> a) XBZRLE
> b) block-migration
> c) SPAPR (it's an iterative migration like RAM and block)
> d) RDMA (I'm guessing I've broken it, but haven't tried)
> e) Floats are currently sent as blobs.
So at least a,b and c need to be done.
Not sure about RDMA - it was never supported and no one
seems to work on it.
Maybe, somehow, it can use the old format when enabled?
> What I've done:
> There are separate output visitors (binary compatible, debug and BER) and
> they're selected by setting QEMUMIGFORMAT to one of 'debug', 'BER' or just
> leaving it unset.
>
> The Input visitor is selected automatically from the first 4 bytes of the
> file.
Not really necessary if you use machine type.
>
> In general most types are BER sequences of indefinite length, with some
> types that I've allocated an Application specific type tag to. There is a hook
> to give any VMState it's own type tag (which I've shown using one example
> for cpu-common). Integers and strings are standard 'universal' types.
>
> Objects with .get/.put methods or register_savevm are saved as an 'octet string'.
> There are a few places where a device registered with .get/.put calls back into
> vmstate_save_state for part of their state (typically for devices built on PCI)
> and when they do that, even when in BER mode, those components get stored inside
> the octet string in the old format.
>
> I've used the 'asn1c' tool to validate the schema (which is
> in docs/specs/migration.schema) and also to verify streams that I've produced
> match the schema.
>
> I've tested it with virt-test (hacked to have different source/dest qemu's)
> and tried bin-bin, ber-ber, and pre-visitor-qemu -> this and
> this -> pre-visitor-qemu just on the standard migration.default.tcp migration
> test, but I've not tried a large combination of configurations.
>
> My fix for qemu_peek_buffer that's on the list is needed to stabilise
> the binary format input visitor.
>
> I'll keep chipping away at that list, and would expect to pop another
> version out in a month or so.
>
> Dave
>
> Dr. David Alan Gilbert (16):
> Visitor: Add methods for migration format use
> QEMUSizedBuffer/QEMUFile
> qemu-file: Add set/get tmp_visitor
> Header/constant/types fixes for visitors
> Visitor: Binary compatible output visitor
> Visitor: Debug output visitor
> Visitor: Binary compatible input visitor
> Visitor: Output path
> Visitor: Load path
> Visitor: Common types to use visitors
> Choose output visitor based on env variable
> BER Visitor: Create output visitor
> BER Visitor: Create input visitor
> Start some BER format docs
> ASN.1 schema for new migration format
> Wire in BER visitors
One nifty feature of Stefan's patches was that a
set of unit tests for parser was included.
Worth reusing?
> arch_init.c | 231 +++--
> block-migration.c | 13 +-
> docs/migration.txt | 34 +-
> docs/specs/migration.schema | 113 +++
> exec.c | 2 +
> hw/acpi/piix4.c | 5 +-
> hw/pci/pci.c | 5 +-
> hw/ppc/spapr.c | 9 +-
> hw/scsi/spapr_vscsi.c | 6 +-
> include/migration/migration.h | 17 +
> include/migration/qemu-file.h | 31 +
> include/migration/vmstate.h | 28 +-
> include/qapi/ber.h | 108 +++
> include/qapi/qemu-file-ber-input-visitor.h | 26 +
> include/qapi/qemu-file-ber-output-visitor.h | 26 +
> include/qapi/qemu-file-binary-input-visitor.h | 27 +
> include/qapi/qemu-file-binary-output-visitor.h | 26 +
> include/qapi/qemu-file-debug-output-visitor.h | 26 +
> include/qapi/visitor-impl.h | 23 +
> include/qapi/visitor.h | 51 ++
> include/qemu/typedefs.h | 4 +-
> qapi/Makefile.objs | 4 +-
> qapi/qapi-visit-core.c | 80 ++
> qapi/qemu-file-ber-input-visitor.c | 1163 ++++++++++++++++++++++++
> qapi/qemu-file-ber-output-visitor.c | 916 +++++++++++++++++++
> qapi/qemu-file-binary-input-visitor.c | 688 ++++++++++++++
> qapi/qemu-file-binary-output-visitor.c | 564 ++++++++++++
> qapi/qemu-file-debug-output-visitor.c | 471 ++++++++++
> qemu-file.c | 423 +++++++++
> savevm.c | 412 +++++++--
> vmstate.c | 619 ++++++-------
> 31 files changed, 5593 insertions(+), 558 deletions(-)
> create mode 100644 docs/specs/migration.schema
> create mode 100644 include/qapi/ber.h
> create mode 100644 include/qapi/qemu-file-ber-input-visitor.h
> create mode 100644 include/qapi/qemu-file-ber-output-visitor.h
> create mode 100644 include/qapi/qemu-file-binary-input-visitor.h
> create mode 100644 include/qapi/qemu-file-binary-output-visitor.h
> create mode 100644 include/qapi/qemu-file-debug-output-visitor.h
> create mode 100644 qapi/qemu-file-ber-input-visitor.c
> create mode 100644 qapi/qemu-file-ber-output-visitor.c
> create mode 100644 qapi/qemu-file-binary-input-visitor.c
> create mode 100644 qapi/qemu-file-binary-output-visitor.c
> create mode 100644 qapi/qemu-file-debug-output-visitor.c
>
> --
> 1.8.5.3
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format
2014-03-25 20:43 ` [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Michael S. Tsirkin
@ 2014-03-26 9:16 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert @ 2014-03-26 9:16 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: agraf, quintela, stefanb, mdroth, qemu-devel, aliguori, afaerber
* Michael S. Tsirkin (mst@redhat.com) wrote:
> On Tue, Mar 25, 2014 at 08:17:11PM +0000, Dr. David Alan Gilbert (git) wrote:
<snip>
> > 5) At the moment you select BER output format by setting an environment
> > variable ( export QEMUMIGFORMAT=BER ) , I need to put more thought
> > in to the right way to do this, there are some harder questions like
> > what happens to devices that are still using pre-vmstate encodings
> > (that are currently sent as blobs) when they eventually convert over
> > and thus how to keep compatibility with earlier BER output versions
> > where they were blobs.
>
> A simple way is the way we always changed migration
> format - trying this to the machine type.
> Old machine type gets you binary, newer one BER with blobs,
> then we can add more flags gradually.
Right, that should work; it gets a little messy to tie machine type
to formats for individual devices, but no big issue.
> With BER we'll actually be able to do smarter things in the future,
> such as send out multiple formats and have destination select
> the one it understands, but this isn't mandatory as a first step.
>
> > 6) At the moment for blobs (i.e. pre-vmstate) the whole blob gets loaded
> > into memory and then processed; that's not necessary and can be done
> > with a QEMUFile shim.
> > 7) I need to sort out my Error** handling (probably in a lot of places)
>
> That's mostly broken in migration as it is.
>
> > 8) The visitors should be able to share much more code.
> >
> > The not-yet-done:
> > a) XBZRLE
> > b) block-migration
> > c) SPAPR (it's an iterative migration like RAM and block)
> > d) RDMA (I'm guessing I've broken it, but haven't tried)
> > e) Floats are currently sent as blobs.
>
> So at least a,b and c need to be done.
Yep, agreed; iterative migration stuff has direct byte/word putting all over
so is rather manual, so I just did enough to get it going first.
> Not sure about RDMA - it was never supported and no one
> seems to work on it.
> Maybe, somehow, it can use the old format when enabled?
I think Michael still throws bug fixes in from time to time, however my worry
is actually more about the Visitor conversion than the actual format change
that might break it; anyway needs testing and then we'll see.
> > What I've done:
> > There are separate output visitors (binary compatible, debug and BER) and
> > they're selected by setting QEMUMIGFORMAT to one of 'debug', 'BER' or just
> > leaving it unset.
> >
> > The Input visitor is selected automatically from the first 4 bytes of the
> > file.
>
> Not really necessary if you use machine type.
It's easy anyway.
> > In general most types are BER sequences of indefinite length, with some
> > types that I've allocated an Application specific type tag to. There is a hook
> > to give any VMState it's own type tag (which I've shown using one example
> > for cpu-common). Integers and strings are standard 'universal' types.
> >
> > Objects with .get/.put methods or register_savevm are saved as an 'octet string'.
> > There are a few places where a device registered with .get/.put calls back into
> > vmstate_save_state for part of their state (typically for devices built on PCI)
> > and when they do that, even when in BER mode, those components get stored inside
> > the octet string in the old format.
> >
> > I've used the 'asn1c' tool to validate the schema (which is
> > in docs/specs/migration.schema) and also to verify streams that I've produced
> > match the schema.
> >
> > I've tested it with virt-test (hacked to have different source/dest qemu's)
> > and tried bin-bin, ber-ber, and pre-visitor-qemu -> this and
> > this -> pre-visitor-qemu just on the standard migration.default.tcp migration
> > test, but I've not tried a large combination of configurations.
> >
> > My fix for qemu_peek_buffer that's on the list is needed to stabilise
> > the binary format input visitor.
> >
> > I'll keep chipping away at that list, and would expect to pop another
> > version out in a month or so.
> >
> > Dave
> >
> > Dr. David Alan Gilbert (16):
> > Visitor: Add methods for migration format use
> > QEMUSizedBuffer/QEMUFile
> > qemu-file: Add set/get tmp_visitor
> > Header/constant/types fixes for visitors
> > Visitor: Binary compatible output visitor
> > Visitor: Debug output visitor
> > Visitor: Binary compatible input visitor
> > Visitor: Output path
> > Visitor: Load path
> > Visitor: Common types to use visitors
> > Choose output visitor based on env variable
> > BER Visitor: Create output visitor
> > BER Visitor: Create input visitor
> > Start some BER format docs
> > ASN.1 schema for new migration format
> > Wire in BER visitors
>
> One nifty feature of Stefan's patches was that a
> set of unit tests for parser was included.
> Worth reusing?
Yes, unit tests are one thing I missed off my list but know
need doing.
Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 01/16] Visitor: Add methods for migration format use
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 01/16] Visitor: Add methods for migration format use Dr. David Alan Gilbert (git)
@ 2014-06-05 17:00 ` Markus Armbruster
2014-06-05 17:43 ` Dr. David Alan Gilbert
0 siblings, 1 reply; 24+ messages in thread
From: Markus Armbruster @ 2014-06-05 17:00 UTC (permalink / raw)
To: Dr. David Alan Gilbert (git)
Cc: quintela, qemu-devel, mst, mdroth, agraf, aliguori, stefanb,
afaerber
"Dr. David Alan Gilbert (git)" <dgilbert@redhat.com> writes:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> array types
> From https://lists.gnu.org/archive/html/qemu-devel/2011-09/msg02465.html
>
> str256 type
> For the upto 256byte strings QEMU commonly uses for IDs
Naive question: why do you need this when you have arrays?
> buffer type
> For a blob of data that the caller wants to deliver whole (e.g.
> a page of RAM or block of disk)
This smells like an array, too.
> Load/save flags to let a user perform pre-save/post-load checking
Odd. I'd expect separate visitors, one for save, one for load.
> An accessor to get the underlying QEMUFile* (for compatibility)
>
> compat-sequences
> Provide enough information for a visitor providing compatibility
> with the old format to generate it's byte stream, while allowing a new
> visitor to do something sensible.
>
> destroy
> Allows the caller to destroy a visitor without knowing what type of
> visitor it is.
When the commit message is basically a list, splitting it into one
commit per list item often makes sense.
Some of them (the one introducing destroy, perhaps?) can then be put to
use in the same or the next patch. Makes review much easier.
But on to the actual code.
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> ---
> include/qapi/visitor-impl.h | 23 +++++++++++++
> include/qapi/visitor.h | 51 +++++++++++++++++++++++++++++
> qapi/qapi-visit-core.c | 80 +++++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 154 insertions(+)
>
> diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
> index f3fa420..10cdbf7 100644
> --- a/include/qapi/visitor-impl.h
> +++ b/include/qapi/visitor-impl.h
> @@ -15,6 +15,9 @@
> #include "qapi/error.h"
> #include "qapi/visitor.h"
>
> +#define VISITOR_SAVING (1<<0)
> +#define VISITOR_LOADING (1<<1)
> +
Comment explaining their meaning and the connection to Visitor member
flags needed.
Are the flags independent, i.e. all four combinations meaningful?
> struct Visitor
> {
> /* Must be set */
> @@ -29,6 +32,10 @@ struct Visitor
> void (*start_list)(Visitor *v, const char *name, Error **errp);
> GenericList *(*next_list)(Visitor *v, GenericList **list, Error **errp);
> void (*end_list)(Visitor *v, Error **errp);
> + void (*start_array)(Visitor *v, void **obj, const char *name,
> + size_t elem_count, size_t elem_size, Error **errp);
> + void (*next_array)(Visitor *v, Error **errp);
> + void (*end_array)(Visitor *v, Error **errp);
>
> void (*type_enum)(Visitor *v, int *obj, const char *strings[],
> const char *kind, const char *name, Error **errp);
Since these "must be set", you need to update all existing visitors to
actually set them, in the same patch, don't you?
I'm not sure the existing visitors completely obey "must be set"
vs. "may be null". But let's not make it worse.
> @@ -38,6 +45,7 @@ struct Visitor
> void (*type_int)(Visitor *v, int64_t *obj, const char *name, Error **errp);
> void (*type_bool)(Visitor *v, bool *obj, const char *name, Error **errp);
> void (*type_str)(Visitor *v, char **obj, const char *name, Error **errp);
> + void (*type_str256)(Visitor *v, char *obj, const char *name, Error **errp);
> void (*type_number)(Visitor *v, double *obj, const char *name,
> Error **errp);
>
/* May be NULL */
More context ^^^
Members above must be set, members below may be null.
> @@ -49,6 +57,8 @@ struct Visitor
> void (*start_handle)(Visitor *v, void **obj, const char *kind,
> const char *name, Error **errp);
> void (*end_handle)(Visitor *v, Error **errp);
No longer applies (I killed end_handle() in commit cbc955).
> + void (*type_buffer)(Visitor *v, void *data, size_t len, bool async,
> + const char *name, Error **errp);
> void (*type_uint8)(Visitor *v, uint8_t *obj, const char *name, Error **errp);
> void (*type_uint16)(Visitor *v, uint16_t *obj, const char *name, Error **errp);
> void (*type_uint32)(Visitor *v, uint32_t *obj, const char *name, Error **errp);
> @@ -59,6 +69,19 @@ struct Visitor
> void (*type_int64)(Visitor *v, int64_t *obj, const char *name, Error **errp);
> /* visit_type_size() falls back to (*type_uint64)() if type_size is unset */
> void (*type_size)(Visitor *v, uint64_t *obj, const char *name, Error **errp);
> +
> + void (*destroy)(Visitor *v, Error **errp);
> +
> + void (*start_sequence_compat)(Visitor *v, const char *name,
> + Visit_seq_compat_mode compat_mode,
> + void *opaque, Error **errp);
> + void (*end_sequence_compat)(Visitor *v, const char *name,
> + Visit_seq_compat_mode compat_mode,
> + Error **errp);
> +
> + QEMUFile* (*get_qemufile)(Visitor *v);
Style nit: QEMUFile *(*get_qemufile)(Visitor *v);
The empty line before destroy makes me wonder whether these still belong
to the "May be NULL" heading.
> +
> + uint64_t flags;
This one can't, because it can't be null :)
> };
>
> void input_type_enum(Visitor *v, int *obj, const char *strings[],
Your patch drags migration-specific stuff into the until now perfectly
generic struct Visitor:
* get_qemufile()
Looks temporary, thus tolerable.
* Compat sequences
* Load/save flags
These look permanent :(
I'd have to review how they're used to convince myself we actually
need them.
> diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
> index 29da211..70c20df 100644
> --- a/include/qapi/visitor.h
> +++ b/include/qapi/visitor.h
> @@ -39,11 +39,22 @@ void visit_end_implicit_struct(Visitor *v, Error **errp);
> void visit_start_list(Visitor *v, const char *name, Error **errp);
> GenericList *visit_next_list(Visitor *v, GenericList **list, Error **errp);
> void visit_end_list(Visitor *v, Error **errp);
> +void visit_start_array(Visitor *v, void **obj, const char *name,
> + size_t elem_count, size_t elem_size, Error **errp);
> +void visit_next_array(Visitor *v, Error **errp);
> +void visit_end_array(Visitor *v, Error **errp);
> +
> void visit_start_optional(Visitor *v, bool *present, const char *name,
> Error **errp);
> void visit_end_optional(Visitor *v, Error **errp);
> void visit_get_next_type(Visitor *v, int *obj, const int *qtypes,
> const char *name, Error **errp);
> +/* Blocks of guest memory,disk or otherwise opaque data that there is a lot
> + * of and must be handled efficiently. 'async' true if the write can happen
> + * 'later'
> + */
> +void visit_type_buffer(Visitor *v, void *data, size_t len, bool async,
> + const char *name, Error **errp);
I'm afraid your specification of async is tied to a specific kind of
visitor: one that "writes". Many don't.
> void visit_type_enum(Visitor *v, int *obj, const char *strings[],
> const char *kind, const char *name, Error **errp);
> void visit_type_int(Visitor *v, int64_t *obj, const char *name, Error **errp);
> @@ -58,6 +69,46 @@ void visit_type_int64(Visitor *v, int64_t *obj, const char *name, Error **errp);
> void visit_type_size(Visitor *v, uint64_t *obj, const char *name, Error **errp);
> void visit_type_bool(Visitor *v, bool *obj, const char *name, Error **errp);
> void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp);
> +/* A string no more than 256 (including term) characters in length */
> +void visit_type_str256(Visitor *v, char *obj, const char *name, Error **errp);
> void visit_type_number(Visitor *v, double *obj, const char *name, Error **errp);
> +void visit_destroy(Visitor *v, Error **errp);
> +
> +/* Type of visitor, used where actions have to be taken when (de)serializing */
> +bool visitor_loading(Visitor *v);
> +bool visitor_saving(Visitor *v);
> +
> +typedef enum {
> + VISIT_SEQ_COMPAT_BYTE0TERM, /* list terminated with a 0 byte */
> + VISIT_SEQ_COMPAT_FILE, /* The top level file object */
> + VISIT_SEQ_COMPAT_SUBSECLIST, /* list terminated by
> + historical complexity */
> + VISIT_SEQ_COMPAT_SUBSECTION, /* One subsection */
> + VISIT_SEQ_COMPAT_SECTION_HEADER, /* SECTION_FULL/START, header + name */
> + VISIT_SEQ_COMPAT_SECTION_MIN, /* SECTION_PART/END - minimal header */
> + VISIT_SEQ_COMPAT_RAMSECLIST, /* list terminated by int64 bit
> + RAM_SAVE_FLAG_EOS */
> + VISIT_SEQ_COMPAT_RAMSECENTRY, /* Entry in RAM Sec list */
> + VISIT_SEQ_COMPAT_VMSTATE, /* One VMState */
> + VISIT_SEQ_COMPAT_BLOB /* A binary old-format blob */
> +} Visit_seq_compat_mode;
> +
> +/* Start a sequence of items (which may be of unknown length and unknown
> + * mix of some subset of types), specify a compatibility mode that's only
> + * used by an implementation trying to match the existing binary migration
> + * format.
> + * opaque is compat_mode specific
> + */
> +void visit_start_sequence_compat(Visitor *v, const char *name,
> + Visit_seq_compat_mode compat_mode,
> + void *opaque,
> + Error **errp);
> +/* Use visit_get_next_type for each entry including the first */
I'm afraid I don't get this comment.
> +void visit_end_sequence_compat(Visitor *v, const char *name,
> + Visit_seq_compat_mode compat_mode,
> + Error **errp);
> +
> +/* Don't Use! - lets us move forward until we can get rid of all file uses */
> +QEMUFile *visitor_get_qemufile(Visitor *v);
Likewise.
>
> #endif
> diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
> index 6451a21..2d20fde 100644
> --- a/qapi/qapi-visit-core.c
> +++ b/qapi/qapi-visit-core.c
> @@ -84,6 +84,28 @@ void visit_end_list(Visitor *v, Error **errp)
> v->end_list(v, errp);
> }
>
> +void visit_start_array(Visitor *v, void **obj, const char *name,
> + size_t elem_count, size_t elem_size, Error **errp)
> +{
> + if (!error_is_set(errp)) {
> + v->start_array(v, obj, name, elem_count, elem_size, errp);
> + }
> +}
> +
> +void visit_next_array(Visitor *v, Error **errp)
> +{
> + if (!error_is_set(errp)) {
> + v->next_array(v, errp);
> + }
> +}
> +
> +void visit_end_array(Visitor *v, Error **errp)
> +{
> + if (!error_is_set(errp)) {
> + v->end_array(v, errp);
> + }
> +}
> +
> void visit_start_optional(Visitor *v, bool *present, const char *name,
> Error **errp)
> {
> @@ -107,6 +129,14 @@ void visit_get_next_type(Visitor *v, int *obj, const int *qtypes,
> }
> }
>
> +void visit_type_buffer(Visitor *v, void *data, size_t len, bool async,
> + const char *name, Error **errp)
> +{
> + if (!error_is_set(errp)) {
> + v->type_buffer(v, data, len, async, name, errp);
> + }
> +}
> +
Looks like Visitor member type_buffer must be set. However, you put it
under the "may be null" heading in visitor-impl.h.
> void visit_type_enum(Visitor *v, int *obj, const char *strings[],
> const char *kind, const char *name, Error **errp)
> {
> @@ -291,6 +321,13 @@ void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp)
> }
> }
>
> +void visit_type_str256(Visitor *v, char *obj, const char *name, Error **errp)
> +{
> + if (!error_is_set(errp)) {
> + v->type_str256(v, obj, name, errp);
> + }
> +}
> +
> void visit_type_number(Visitor *v, double *obj, const char *name, Error **errp)
> {
> if (!error_is_set(errp)) {
> @@ -347,3 +384,46 @@ void input_type_enum(Visitor *v, int *obj, const char *strings[],
> g_free(enum_str);
> *obj = value;
> }
> +
> +void visit_destroy(Visitor *v, Error **errp)
> +{
> + v->destroy(v, errp);
> +}
Likewise.
> +
> +void visit_start_sequence_compat(Visitor *v, const char *name,
> + Visit_seq_compat_mode compat_mode,
> + void *opaque, Error **errp)
> +{
> + if (error_is_set(errp)) {
> + return;
> + }
> +
> + v->start_sequence_compat(v, name, compat_mode, opaque, errp);
> +}
Likewise.
> +
> +void visit_end_sequence_compat(Visitor *v, const char *name,
> + Visit_seq_compat_mode compat_mode,
> + Error **errp)
> +{
> + if (error_is_set(errp)) {
> + return;
> + }
> +
> + v->end_sequence_compat(v, name, compat_mode, errp);
> +}
Likewise.
> +
> +QEMUFile *visitor_get_qemufile(Visitor *v)
> +{
> + return v->get_qemufile(v);
> +}
Likewise.
> +
> +bool visitor_loading(Visitor *v)
> +{
> + return v->flags & VISITOR_LOADING;
> +}
> +
> +bool visitor_saving(Visitor *v)
> +{
> + return v->flags & VISITOR_SAVING;
> +}
> +
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 01/16] Visitor: Add methods for migration format use
2014-06-05 17:00 ` Markus Armbruster
@ 2014-06-05 17:43 ` Dr. David Alan Gilbert
2014-06-05 18:59 ` Dr. David Alan Gilbert
2014-06-06 8:19 ` Markus Armbruster
0 siblings, 2 replies; 24+ messages in thread
From: Dr. David Alan Gilbert @ 2014-06-05 17:43 UTC (permalink / raw)
To: Markus Armbruster
Cc: quintela, qemu-devel, mst, mdroth, agraf, aliguori, stefanb,
afaerber
* Markus Armbruster (armbru@redhat.com) wrote:
> "Dr. David Alan Gilbert (git)" <dgilbert@redhat.com> writes:
>
Hi Markus,
Thanks for the review. I've reordered your comments to group some of the
replies together.
> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> Your patch drags migration-specific stuff into the until now perfectly
> generic struct Visitor:
>
> * get_qemufile()
>
> Looks temporary, thus tolerable.
>
> * Compat sequences
> * Load/save flags
>
> These look permanent :(
>
> I'd have to review how they're used to convince myself we actually
> need them.
Right - and this is the one I actually wanted you to look at this series
for; I know it feels weird, and I'm up for better suggestions
I've toyed with the idea of having a MigrationVisitor subclass - ie just like
the visitor except the few additions; but how would I cleanly make it a subclass
in this setup?
> > array types
> > From https://lists.gnu.org/archive/html/qemu-devel/2011-09/msg02465.html
> >
> > str256 type
> > For the upto 256byte strings QEMU commonly uses for IDs
>
> Naive question: why do you need this when you have arrays?
> > buffer type
> > For a blob of data that the caller wants to deliver whole (e.g.
> > a page of RAM or block of disk)
>
> This smells like an array, too.
I'd assumed that the point of the 'array' was something where you started
an array and then visited each member (with the appropriate visitor type
for the type of the member).
I use arrays for things that are declared as such in vmstate and which
I know the internal types of;
BER has various string types and it makes sense to me to use that
for readable strings. The other reason in this case is that the
binary-visitor knows to format this in the way compatible with the
existing length+text string formatting.
Where there is a blob of data which the visitor code isn't going to
get to know the inside of, that's a buffer.
> > Load/save flags to let a user perform pre-save/post-load checking
>
> Odd. I'd expect separate visitors, one for save, one for load.
Indeed they are - these flags are to allow a caller to know whether
it's being visited for the purpose of saving or loading;
In a non-visitor world you typically have a:
load_state
save_state
function; in the visit world you more commonly have a:
visit_state
function; for a lot of the simpler types this works fine - but when
you want to do some reformatting or checking of your data then
you really want to know if you're saving or loading.
The other way of doing it would still be to have a load_state/save_state
that just happen to call the same function in most cases.
> > +#define VISITOR_SAVING (1<<0)
> > +#define VISITOR_LOADING (1<<1)
> > +
>
> Comment explaining their meaning and the connection to Visitor member
> flags needed.
>
> Are the flags independent, i.e. all four combinations meaningful?
They're only two states, and I could have encoded it as one bit in the flag.
> > An accessor to get the underlying QEMUFile* (for compatibility)
> >
> > compat-sequences
> > Provide enough information for a visitor providing compatibility
> > with the old format to generate it's byte stream, while allowing a new
> > visitor to do something sensible.
> >
> > destroy
> > Allows the caller to destroy a visitor without knowing what type of
> > visitor it is.
>
> When the commit message is basically a list, splitting it into one
> commit per list item often makes sense.
>
> Some of them (the one introducing destroy, perhaps?) can then be put to
> use in the same or the next patch. Makes review much easier.
Yep, happy to split.
> But on to the actual code.
>
> > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> > ---
> > include/qapi/visitor-impl.h | 23 +++++++++++++
> > include/qapi/visitor.h | 51 +++++++++++++++++++++++++++++
> > qapi/qapi-visit-core.c | 80 +++++++++++++++++++++++++++++++++++++++++++++
> > 3 files changed, 154 insertions(+)
> >
> > diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
> > index f3fa420..10cdbf7 100644
> > --- a/include/qapi/visitor-impl.h
> > +++ b/include/qapi/visitor-impl.h
> > @@ -15,6 +15,9 @@
> > #include "qapi/error.h"
> > #include "qapi/visitor.h"
> >
>
> > struct Visitor
> > {
> > /* Must be set */
> > @@ -29,6 +32,10 @@ struct Visitor
> > void (*start_list)(Visitor *v, const char *name, Error **errp);
> > GenericList *(*next_list)(Visitor *v, GenericList **list, Error **errp);
> > void (*end_list)(Visitor *v, Error **errp);
> > + void (*start_array)(Visitor *v, void **obj, const char *name,
> > + size_t elem_count, size_t elem_size, Error **errp);
> > + void (*next_array)(Visitor *v, Error **errp);
> > + void (*end_array)(Visitor *v, Error **errp);
> >
> > void (*type_enum)(Visitor *v, int *obj, const char *strings[],
> > const char *kind, const char *name, Error **errp);
>
> Since these "must be set", you need to update all existing visitors to
> actually set them, in the same patch, don't you?
>
> I'm not sure the existing visitors completely obey "must be set"
> vs. "may be null". But let's not make it worse.
Yep; you've picked me up on a whole bunch of the 'must be set'/'may be null'
- I'll treat those all as one and go through and tidy them up.
> Members above must be set, members below may be null.
>
> > @@ -49,6 +57,8 @@ struct Visitor
> > void (*start_handle)(Visitor *v, void **obj, const char *kind,
> > const char *name, Error **errp);
> > void (*end_handle)(Visitor *v, Error **errp);
>
> No longer applies (I killed end_handle() in commit cbc955).
Yep; it's dead in my working tree.
> > + void (*type_buffer)(Visitor *v, void *data, size_t len, bool async,
> > + const char *name, Error **errp);
> > void (*type_uint8)(Visitor *v, uint8_t *obj, const char *name, Error **errp);
> > void (*type_uint16)(Visitor *v, uint16_t *obj, const char *name, Error **errp);
> > void (*type_uint32)(Visitor *v, uint32_t *obj, const char *name, Error **errp);
> > @@ -59,6 +69,19 @@ struct Visitor
> > void (*type_int64)(Visitor *v, int64_t *obj, const char *name, Error **errp);
> > /* visit_type_size() falls back to (*type_uint64)() if type_size is unset */
> > void (*type_size)(Visitor *v, uint64_t *obj, const char *name, Error **errp);
> > +
> > + void (*destroy)(Visitor *v, Error **errp);
> > +
> > + void (*start_sequence_compat)(Visitor *v, const char *name,
> > + Visit_seq_compat_mode compat_mode,
> > + void *opaque, Error **errp);
> > + void (*end_sequence_compat)(Visitor *v, const char *name,
> > + Visit_seq_compat_mode compat_mode,
> > + Error **errp);
> > +
> > + QEMUFile* (*get_qemufile)(Visitor *v);
>
> Style nit: QEMUFile *(*get_qemufile)(Visitor *v);
Fixed.
> The empty line before destroy makes me wonder whether these still belong
> to the "May be NULL" heading.
>
> > +
> > + uint64_t flags;
>
> This one can't, because it can't be null :)
>
> > +/* Blocks of guest memory,disk or otherwise opaque data that there is a lot
> > + * of and must be handled efficiently. 'async' true if the write can happen
> > + * 'later'
> > + */
> > +void visit_type_buffer(Visitor *v, void *data, size_t len, bool async,
> > + const char *name, Error **errp);
>
> I'm afraid your specification of async is tied to a specific kind of
> visitor: one that "writes". Many don't.
Do you have a suggestion on how to do this better? Async accesses were
put in some time ago apparently for a useful speed increase, so I can't
remove them, and anyway they make sense for the writes; I guess you could
argue that they could be used for read as well, if the time at which
the 'async' had to have completed was defined, but that's needed for the
write anyway.
> > void visit_type_enum(Visitor *v, int *obj, const char *strings[],
> > const char *kind, const char *name, Error **errp);
> > void visit_type_int(Visitor *v, int64_t *obj, const char *name, Error **errp);
> > @@ -58,6 +69,46 @@ void visit_type_int64(Visitor *v, int64_t *obj, const char *name, Error **errp);
> > void visit_type_size(Visitor *v, uint64_t *obj, const char *name, Error **errp);
> > void visit_type_bool(Visitor *v, bool *obj, const char *name, Error **errp);
> > void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp);
> > +/* A string no more than 256 (including term) characters in length */
> > +void visit_type_str256(Visitor *v, char *obj, const char *name, Error **errp);
> > void visit_type_number(Visitor *v, double *obj, const char *name, Error **errp);
> > +void visit_destroy(Visitor *v, Error **errp);
> > +
> > +/* Type of visitor, used where actions have to be taken when (de)serializing */
> > +bool visitor_loading(Visitor *v);
> > +bool visitor_saving(Visitor *v);
> > +
> > +typedef enum {
> > + VISIT_SEQ_COMPAT_BYTE0TERM, /* list terminated with a 0 byte */
> > + VISIT_SEQ_COMPAT_FILE, /* The top level file object */
> > + VISIT_SEQ_COMPAT_SUBSECLIST, /* list terminated by
> > + historical complexity */
> > + VISIT_SEQ_COMPAT_SUBSECTION, /* One subsection */
> > + VISIT_SEQ_COMPAT_SECTION_HEADER, /* SECTION_FULL/START, header + name */
> > + VISIT_SEQ_COMPAT_SECTION_MIN, /* SECTION_PART/END - minimal header */
> > + VISIT_SEQ_COMPAT_RAMSECLIST, /* list terminated by int64 bit
> > + RAM_SAVE_FLAG_EOS */
> > + VISIT_SEQ_COMPAT_RAMSECENTRY, /* Entry in RAM Sec list */
> > + VISIT_SEQ_COMPAT_VMSTATE, /* One VMState */
> > + VISIT_SEQ_COMPAT_BLOB /* A binary old-format blob */
> > +} Visit_seq_compat_mode;
> > +
> > +/* Start a sequence of items (which may be of unknown length and unknown
> > + * mix of some subset of types), specify a compatibility mode that's only
> > + * used by an implementation trying to match the existing binary migration
> > + * format.
> > + * opaque is compat_mode specific
> > + */
> > +void visit_start_sequence_compat(Visitor *v, const char *name,
> > + Visit_seq_compat_mode compat_mode,
> > + void *opaque,
> > + Error **errp);
> > +/* Use visit_get_next_type for each entry including the first */
> > +void visit_end_sequence_compat(Visitor *v, const char *name,
> > + Visit_seq_compat_mode compat_mode,
> > + Error **errp);
> > +
>
> I'm afraid I don't get this comment.
Quite a lot of the structures I need to keep compatibility with are
lists of some mixture of different things with a terminator on the end
(there's not much consistency in the binary format about how you tell
what's next or what the terminator is).
So the typical use is:
visit_start_sequence_compat(...)
while (visit_get_next_type( &flag ), flag != end_flag) {
switch (flag) {
case A:
do whatever you need to do now that we've found an 'A' in the list
(typically another visit_start_sequence_compat for that type)
case B:
do whatever you need to do now that we've found an 'B' in the list
....
}
}
visit_end_sequence_compat(...)
An example is a RAM migration section; it's a list of initialisation data,
normal pages, all-zero pages, XBZRLE pages
> > +/* Don't Use! - lets us move forward until we can get rid of all file uses */
> > +QEMUFile *visitor_get_qemufile(Visitor *v);
>
> Likewise.
I'm hoping this is a temporary shim - my hope eventually is that we won't need to
use this and everywhere that uses Visitors won't also need to get a QEMUFile
except internally to the visitor.
Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 01/16] Visitor: Add methods for migration format use
2014-06-05 17:43 ` Dr. David Alan Gilbert
@ 2014-06-05 18:59 ` Dr. David Alan Gilbert
2014-06-06 8:19 ` Markus Armbruster
1 sibling, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert @ 2014-06-05 18:59 UTC (permalink / raw)
To: Markus Armbruster
Cc: mst, qemu-devel, quintela, mdroth, agraf, aliguori, stefanb,
afaerber
* Dr. David Alan Gilbert (dgilbert@redhat.com) wrote:
> * Markus Armbruster (armbru@redhat.com) wrote:
> > * Dr. David Alan Gilbert (dgilbert@redhat.com) wrote:
<snip>
> > > Load/save flags to let a user perform pre-save/post-load checking
> >
> > Odd. I'd expect separate visitors, one for save, one for load.
>
> Indeed they are - these flags are to allow a caller to know whether
Oops that's a typo - 'Indeed there are separate load and save visitors'
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 01/16] Visitor: Add methods for migration format use
2014-06-05 17:43 ` Dr. David Alan Gilbert
2014-06-05 18:59 ` Dr. David Alan Gilbert
@ 2014-06-06 8:19 ` Markus Armbruster
2014-06-06 11:44 ` Dr. David Alan Gilbert
1 sibling, 1 reply; 24+ messages in thread
From: Markus Armbruster @ 2014-06-06 8:19 UTC (permalink / raw)
To: Dr. David Alan Gilbert
Cc: mst, qemu-devel, quintela, mdroth, agraf, aliguori, stefanb,
afaerber
"Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
> * Markus Armbruster (armbru@redhat.com) wrote:
>> "Dr. David Alan Gilbert (git)" <dgilbert@redhat.com> writes:
>>
>
> Hi Markus,
> Thanks for the review. I've reordered your comments to group some of the
> replies together.
>
>> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
>> Your patch drags migration-specific stuff into the until now perfectly
>> generic struct Visitor:
>>
>> * get_qemufile()
>>
>> Looks temporary, thus tolerable.
>>
>> * Compat sequences
>> * Load/save flags
>>
>> These look permanent :(
>>
>> I'd have to review how they're used to convince myself we actually
>> need them.
>
> Right - and this is the one I actually wanted you to look at this series
> for; I know it feels weird, and I'm up for better suggestions
> I've toyed with the idea of having a MigrationVisitor subclass - ie just like
> the visitor except the few additions; but how would I cleanly make it a subclass
> in this setup?
To help with that, I need to understand the problems your additional
functionality solves. I could try to find out by studying the rest of
this series, but it'll take time. Perhaps I can find some next week.
>> > array types
>> > From
>> > https://lists.gnu.org/archive/html/qemu-devel/2011-09/msg02465.html
>> >
>> > str256 type
>> > For the upto 256byte strings QEMU commonly uses for IDs
>>
>> Naive question: why do you need this when you have arrays?
>
>> > buffer type
>> > For a blob of data that the caller wants to deliver whole (e.g.
>> > a page of RAM or block of disk)
>>
>> This smells like an array, too.
>
> I'd assumed that the point of the 'array' was something where you started
> an array and then visited each member (with the appropriate visitor type
> for the type of the member).
>
> I use arrays for things that are declared as such in vmstate and which
> I know the internal types of;
>
> BER has various string types and it makes sense to me to use that
> for readable strings. The other reason in this case is that the
> binary-visitor knows to format this in the way compatible with the
> existing length+text string formatting.
Point taken: while (unboxed) strings really are just special arrays, it
can make sense to treat them specially.
But limiting it to arrays of exactly 256 characters is lame :) Would
generalizing to arbitrary size be hard?
This one's not specific to migration.
> Where there is a blob of data which the visitor code isn't going to
> get to know the inside of, that's a buffer.
Similar argument as for strings: blobs are just special arrays, but it
can make sense to treat them specially. Not sure I find it convincing,
but examining actual users tends to fix that.
Again, not necessarily specific to migration.
>> > Load/save flags to let a user perform pre-save/post-load checking
>>
>> Odd. I'd expect separate visitors, one for save, one for load.
>
> Indeed they are - these flags are to allow a caller to know whether
> it's being visited for the purpose of saving or loading;
> In a non-visitor world you typically have a:
> load_state
> save_state
> function; in the visit world you more commonly have a:
> visit_state
> function; for a lot of the simpler types this works fine - but when
> you want to do some reformatting or checking of your data then
> you really want to know if you're saving or loading.
> The other way of doing it would still be to have a load_state/save_state
> that just happen to call the same function in most cases.
The purpose of visitors is to separate data structure traversal and what
to do with the data.
An implementation of the Visitor type encapsulates "what to do". The
code passing a Visitor instance v to the visit_FOO(v, ...) does the
traversal.
In other words, the abstract Visitor type hides "what to do".
Your flags leak "what to do" bits into the abstract Visitor type
interface. Unclean.
The orthodox way to provide additional data to a concrete visitor is
extending the abstract Visitor type:
struct FooVisitor {
Visitor visitor;
... Foo-specific stuff ...
}
This is how the existing visitors work. Could this work for you, too?
>> > +#define VISITOR_SAVING (1<<0)
>> > +#define VISITOR_LOADING (1<<1)
>> > +
>>
>> Comment explaining their meaning and the connection to Visitor member
>> flags needed.
>>
>> Are the flags independent, i.e. all four combinations meaningful?
>
> They're only two states, and I could have encoded it as one bit in the flag.
I prefer to avoid invalid states.
>> > An accessor to get the underlying QEMUFile* (for compatibility)
>> >
>> > compat-sequences
>> > Provide enough information for a visitor providing compatibility
>> > with the old format to generate it's byte stream, while allowing a new
>> > visitor to do something sensible.
>> >
>> > destroy
>> > Allows the caller to destroy a visitor without knowing what type of
>> > visitor it is.
>>
>> When the commit message is basically a list, splitting it into one
>> commit per list item often makes sense.
>>
>> Some of them (the one introducing destroy, perhaps?) can then be put to
>> use in the same or the next patch. Makes review much easier.
>
> Yep, happy to split.
>
>> But on to the actual code.
>>
>> > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
>> > ---
>> > include/qapi/visitor-impl.h | 23 +++++++++++++
>> > include/qapi/visitor.h | 51 +++++++++++++++++++++++++++++
>> > qapi/qapi-visit-core.c | 80
>> > +++++++++++++++++++++++++++++++++++++++++++++
>> > 3 files changed, 154 insertions(+)
>> >
>> > diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
>> > index f3fa420..10cdbf7 100644
>> > --- a/include/qapi/visitor-impl.h
>> > +++ b/include/qapi/visitor-impl.h
>> > @@ -15,6 +15,9 @@
>> > #include "qapi/error.h"
>> > #include "qapi/visitor.h"
>> >
>>
>> > struct Visitor
>> > {
>> > /* Must be set */
>> > @@ -29,6 +32,10 @@ struct Visitor
>> > void (*start_list)(Visitor *v, const char *name, Error **errp);
>> > GenericList *(*next_list)(Visitor *v, GenericList **list, Error **errp);
>> > void (*end_list)(Visitor *v, Error **errp);
>> > + void (*start_array)(Visitor *v, void **obj, const char *name,
>> > + size_t elem_count, size_t elem_size, Error **errp);
>> > + void (*next_array)(Visitor *v, Error **errp);
>> > + void (*end_array)(Visitor *v, Error **errp);
>> >
>> > void (*type_enum)(Visitor *v, int *obj, const char *strings[],
>> > const char *kind, const char *name, Error **errp);
>>
>> Since these "must be set", you need to update all existing visitors to
>> actually set them, in the same patch, don't you?
>>
>> I'm not sure the existing visitors completely obey "must be set"
>> vs. "may be null". But let's not make it worse.
>
> Yep; you've picked me up on a whole bunch of the 'must be set'/'may be null'
> - I'll treat those all as one and go through and tidy them up.
>
>> Members above must be set, members below may be null.
>>
>> > @@ -49,6 +57,8 @@ struct Visitor
>> > void (*start_handle)(Visitor *v, void **obj, const char *kind,
>> > const char *name, Error **errp);
>> > void (*end_handle)(Visitor *v, Error **errp);
>>
>> No longer applies (I killed end_handle() in commit cbc955).
>
> Yep; it's dead in my working tree.
When I get to studying how you put this stuff to use, I'll want a
current version. Happy to pull, just tell me where from.
>> > + void (*type_buffer)(Visitor *v, void *data, size_t len, bool async,
>> > + const char *name, Error **errp);
>> > void (*type_uint8)(Visitor *v, uint8_t *obj, const char
>> > *name, Error **errp);
>> > void (*type_uint16)(Visitor *v, uint16_t *obj, const char
>> > *name, Error **errp);
>> > void (*type_uint32)(Visitor *v, uint32_t *obj, const char
>> > *name, Error **errp);
>> > @@ -59,6 +69,19 @@ struct Visitor
>> > void (*type_int64)(Visitor *v, int64_t *obj, const char
>> > *name, Error **errp);
>> > /* visit_type_size() falls back to (*type_uint64)() if
>> > type_size is unset */
>> > void (*type_size)(Visitor *v, uint64_t *obj, const char
>> > *name, Error **errp);
>> > +
>> > + void (*destroy)(Visitor *v, Error **errp);
>> > +
>> > + void (*start_sequence_compat)(Visitor *v, const char *name,
>> > + Visit_seq_compat_mode compat_mode,
>> > + void *opaque, Error **errp);
>> > + void (*end_sequence_compat)(Visitor *v, const char *name,
>> > + Visit_seq_compat_mode compat_mode,
>> > + Error **errp);
>> > +
>> > + QEMUFile* (*get_qemufile)(Visitor *v);
>>
>> Style nit: QEMUFile *(*get_qemufile)(Visitor *v);
>
> Fixed.
>
>> The empty line before destroy makes me wonder whether these still belong
>> to the "May be NULL" heading.
>>
>> > +
>> > + uint64_t flags;
>>
>> This one can't, because it can't be null :)
>>
>
>> > +/* Blocks of guest memory,disk or otherwise opaque data that there is a lot
>> > + * of and must be handled efficiently. 'async' true if the write can happen
>> > + * 'later'
>> > + */
>> > +void visit_type_buffer(Visitor *v, void *data, size_t len, bool async,
>> > + const char *name, Error **errp);
>>
>> I'm afraid your specification of async is tied to a specific kind of
>> visitor: one that "writes". Many don't.
>
> Do you have a suggestion on how to do this better? Async accesses were
> put in some time ago apparently for a useful speed increase, so I can't
> remove them, and anyway they make sense for the writes; I guess you could
> argue that they could be used for read as well, if the time at which
> the 'async' had to have completed was defined, but that's needed for the
> write anyway.
How do we decide async / sync?
>> > void visit_type_enum(Visitor *v, int *obj, const char *strings[],
>> > const char *kind, const char *name, Error **errp);
>> > void visit_type_int(Visitor *v, int64_t *obj, const char *name, Error **errp);
>> > @@ -58,6 +69,46 @@ void visit_type_int64(Visitor *v, int64_t *obj, const char *name, Error **errp);
>> > void visit_type_size(Visitor *v, uint64_t *obj, const char *name, Error **errp);
>> > void visit_type_bool(Visitor *v, bool *obj, const char *name, Error **errp);
>> > void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp);
>> > +/* A string no more than 256 (including term) characters in length */
>> > +void visit_type_str256(Visitor *v, char *obj, const char *name, Error **errp);
>> > void visit_type_number(Visitor *v, double *obj, const char *name, Error **errp);
>> > +void visit_destroy(Visitor *v, Error **errp);
>> > +
>> > +/* Type of visitor, used where actions have to be taken when (de)serializing */
>> > +bool visitor_loading(Visitor *v);
>> > +bool visitor_saving(Visitor *v);
>> > +
>> > +typedef enum {
>> > + VISIT_SEQ_COMPAT_BYTE0TERM, /* list terminated with a 0 byte */
>> > + VISIT_SEQ_COMPAT_FILE, /* The top level file object */
>> > + VISIT_SEQ_COMPAT_SUBSECLIST, /* list terminated by
>> > + historical complexity */
>> > + VISIT_SEQ_COMPAT_SUBSECTION, /* One subsection */
>> > + VISIT_SEQ_COMPAT_SECTION_HEADER, /* SECTION_FULL/START, header + name */
>> > + VISIT_SEQ_COMPAT_SECTION_MIN, /* SECTION_PART/END - minimal header */
>> > + VISIT_SEQ_COMPAT_RAMSECLIST, /* list terminated by int64 bit
>> > + RAM_SAVE_FLAG_EOS */
>> > + VISIT_SEQ_COMPAT_RAMSECENTRY, /* Entry in RAM Sec list */
>> > + VISIT_SEQ_COMPAT_VMSTATE, /* One VMState */
>> > + VISIT_SEQ_COMPAT_BLOB /* A binary old-format blob */
>> > +} Visit_seq_compat_mode;
>> > +
>> > +/* Start a sequence of items (which may be of unknown length and unknown
>> > + * mix of some subset of types), specify a compatibility mode that's only
>> > + * used by an implementation trying to match the existing binary migration
>> > + * format.
>> > + * opaque is compat_mode specific
>> > + */
>> > +void visit_start_sequence_compat(Visitor *v, const char *name,
>> > + Visit_seq_compat_mode compat_mode,
>> > + void *opaque,
>> > + Error **errp);
>> > +/* Use visit_get_next_type for each entry including the first */
>> > +void visit_end_sequence_compat(Visitor *v, const char *name,
>> > + Visit_seq_compat_mode compat_mode,
>> > + Error **errp);
>> > +
>>
>> I'm afraid I don't get this comment.
>
> Quite a lot of the structures I need to keep compatibility with are
> lists of some mixture of different things with a terminator on the end
> (there's not much consistency in the binary format about how you tell
> what's next or what the terminator is).
> So the typical use is:
>
> visit_start_sequence_compat(...)
> while (visit_get_next_type( &flag ), flag != end_flag) {
> switch (flag) {
> case A:
> do whatever you need to do now that we've found an 'A' in the list
> (typically another visit_start_sequence_compat for that type)
> case B:
> do whatever you need to do now that we've found an 'B' in the list
> ....
> }
> }
> visit_end_sequence_compat(...)
>
> An example is a RAM migration section; it's a list of initialisation data,
> normal pages, all-zero pages, XBZRLE pages
Doesn't match the existing visit_get_next_type(). I guess I need more
context. One way to get it is study your patches.
>> > +/* Don't Use! - lets us move forward until we can get rid of all
>> > file uses */
>> > +QEMUFile *visitor_get_qemufile(Visitor *v);
>>
>> Likewise.
>
> I'm hoping this is a temporary shim - my hope eventually is that we
> won't need to
> use this and everywhere that uses Visitors won't also need to get a QEMUFile
> except internally to the visitor.
Suggest to put that in the comment.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [Qemu-devel] [RFC PATCH 01/16] Visitor: Add methods for migration format use
2014-06-06 8:19 ` Markus Armbruster
@ 2014-06-06 11:44 ` Dr. David Alan Gilbert
0 siblings, 0 replies; 24+ messages in thread
From: Dr. David Alan Gilbert @ 2014-06-06 11:44 UTC (permalink / raw)
To: Markus Armbruster
Cc: quintela, qemu-devel, mst, mdroth, agraf, aliguori, stefanb,
afaerber
* Markus Armbruster (armbru@redhat.com) wrote:
> "Dr. David Alan Gilbert" <dgilbert@redhat.com> writes:
>
> > * Markus Armbruster (armbru@redhat.com) wrote:
> >> "Dr. David Alan Gilbert (git)" <dgilbert@redhat.com> writes:
> >>
> >
> > Hi Markus,
> > Thanks for the review. I've reordered your comments to group some of the
> > replies together.
> >
> >> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> >
> >> Your patch drags migration-specific stuff into the until now perfectly
> >> generic struct Visitor:
> >>
> >> * get_qemufile()
> >>
> >> Looks temporary, thus tolerable.
> >>
> >> * Compat sequences
> >> * Load/save flags
> >>
> >> These look permanent :(
> >>
> >> I'd have to review how they're used to convince myself we actually
> >> need them.
> >
> > Right - and this is the one I actually wanted you to look at this series
> > for; I know it feels weird, and I'm up for better suggestions
> > I've toyed with the idea of having a MigrationVisitor subclass - ie just like
> > the visitor except the few additions; but how would I cleanly make it a subclass
> > in this setup?
>
> To help with that, I need to understand the problems your additional
> functionality solves. I could try to find out by studying the rest of
> this series, but it'll take time. Perhaps I can find some next week.
Hmm, that means I need to beef up my comments/commit messages to make that
more obvious.
OK, the quick overview is that I want to abstract out of the devices/ram/etc code
any notion of the format of the migration file, and then support two different formats,
the current binary format and the new BER format. I end up with 4 visitors
for each of (Binary, BER) x (Input, output).
The tricky bit for the visitor inteface is matching the existing binary format;
ignoring the VMState (which is pretty easy) all the rest of the migration format
is frankly pretty random in the way it deals with lists of items or special formatting,
and each one needs treating slightly specially; although many are similar.
I didn't want to create a separate visitor method for each weird case in the binary format,
and so I created 'visit_start_sequence_compat' which takes an enum telling it which of
the special cases it's dealing with (and a pointer to a data structure for that case).
> >> > array types
> >> > From
> >> > https://lists.gnu.org/archive/html/qemu-devel/2011-09/msg02465.html
> >> >
> >> > str256 type
> >> > For the upto 256byte strings QEMU commonly uses for IDs
> >>
> >> Naive question: why do you need this when you have arrays?
> >
> >> > buffer type
> >> > For a blob of data that the caller wants to deliver whole (e.g.
> >> > a page of RAM or block of disk)
> >>
> >> This smells like an array, too.
> >
> > I'd assumed that the point of the 'array' was something where you started
> > an array and then visited each member (with the appropriate visitor type
> > for the type of the member).
> >
> > I use arrays for things that are declared as such in vmstate and which
> > I know the internal types of;
> >
> > BER has various string types and it makes sense to me to use that
> > for readable strings. The other reason in this case is that the
> > binary-visitor knows to format this in the way compatible with the
> > existing length+text string formatting.
>
> Point taken: while (unboxed) strings really are just special arrays, it
> can make sense to treat them specially.
>
> But limiting it to arrays of exactly 256 characters is lame :) Would
> generalizing to arbitrary size be hard?
>
> This one's not specific to migration.
Probably not too hard; the problem I have here is that I know that
the binary visitor must encode that as a single byte length followed
by the string; now I could just take the '256' off the name and
error in the case of someone passing a string longer than 255 bytes
to it for the binary visitor. The other thing I would have to
do is add a string copy (at the moment the input visitor here
assumes it's passed a pointer to a nice roomy 256 byte buffer).
> > Where there is a blob of data which the visitor code isn't going to
> > get to know the inside of, that's a buffer.
>
> Similar argument as for strings: blobs are just special arrays, but it
> can make sense to treat them specially. Not sure I find it convincing,
> but examining actual users tends to fix that.
>
> Again, not necessarily specific to migration.
Let me give some examples from a decoded BER stream (from the unber tool):
A string:
<P O="105" T="[UNIVERSAL 12]" TL="2" V="22" A="UTF8String">0000:00:03.0/e1000.rom</P>
A buffer:
<P O="289" T="[UNIVERSAL 4]" TL="4" V="4096" A="OCTET STRING"> (Now follows 4k of hex data)</P>
An array:
<I O="1589207" T="[UNIVERSAL 16]" TL="2" V="Indefinite" A="SEQUENCE">
<P O="1589209" T="[UNIVERSAL 2]" TL="2" V="2" A="INTEGER" F>4416</P>
<P O="1589213" T="[UNIVERSAL 2]" TL="2" V="2" A="INTEGER" F>31053</P>
<P O="1589217" T="[UNIVERSAL 2]" TL="2" V="2" A="INTEGER" F>321</P>
<P O="1589221" T="[UNIVERSAL 2]" TL="2" V="2" A="INTEGER" F>3104</P>
<P O="1589311" T="[UNIVERSAL 2]" TL="2" V="1" A="INTEGER" F>0</P>
<P O="1589314" T="[UNIVERSAL 2]" TL="2" V="1" A="INTEGER" F>0</P>
</I O="1589317" T="[UNIVERSAL 0]" TL="2" L="112">
> >> > Load/save flags to let a user perform pre-save/post-load checking
> >>
> >> Odd. I'd expect separate visitors, one for save, one for load.
> >
> > Indeed they are - these flags are to allow a caller to know whether
> > it's being visited for the purpose of saving or loading;
> > In a non-visitor world you typically have a:
> > load_state
> > save_state
> > function; in the visit world you more commonly have a:
> > visit_state
> > function; for a lot of the simpler types this works fine - but when
> > you want to do some reformatting or checking of your data then
> > you really want to know if you're saving or loading.
> > The other way of doing it would still be to have a load_state/save_state
> > that just happen to call the same function in most cases.
>
> The purpose of visitors is to separate data structure traversal and what
> to do with the data.
>
> An implementation of the Visitor type encapsulates "what to do". The
> code passing a Visitor instance v to the visit_FOO(v, ...) does the
> traversal.
>
> In other words, the abstract Visitor type hides "what to do".
>
> Your flags leak "what to do" bits into the abstract Visitor type
> interface. Unclean.
>
> The orthodox way to provide additional data to a concrete visitor is
> extending the abstract Visitor type:
>
> struct FooVisitor {
> Visitor visitor;
> ... Foo-specific stuff ...
> }
>
> This is how the existing visitors work. Could this work for you, too?
Note that these flags are only accessed by the:
bool visitor_loading(Visitor *v);
bool visitor_saving(Visitor *v);
methods; those magic constants are never used by the callers.
an example user is traversing the int32_le types in vmstate:
static int visit_int32_le(Visitor *v, void *pdata, const char *name,
size_t size, Error **errp)
{
if (visitor_loading(v)) {
int32_t loaded;
visit_type_int32(v, &loaded, name, errp);
if (loaded >= 0 && loaded <= *(int32_t *)pdata) {
*(int32_t *)pdata = loaded;
return 0;
}
error_setg(errp, "Unexpected value for field '%s', expected a value"
" no larger than %d but got %d", name, *(int32_t *)pdata,
loaded);
return -EINVAL;
} else {
visit_type_int32(v, (int32_t *)pdata, name, errp);
return 0;
}
}
all the more basic vmstate wrappers just directly call visit_type_whatever without
careing if they're load/store.
> >> > +#define VISITOR_SAVING (1<<0)
> >> > +#define VISITOR_LOADING (1<<1)
> >> > +
> >>
> >> Comment explaining their meaning and the connection to Visitor member
> >> flags needed.
> >>
> >> Are the flags independent, i.e. all four combinations meaningful?
> >
> > They're only two states, and I could have encoded it as one bit in the flag.
>
> I prefer to avoid invalid states.
Yep, fair enough.
> >> > An accessor to get the underlying QEMUFile* (for compatibility)
> >> >
> >> > compat-sequences
> >> > Provide enough information for a visitor providing compatibility
> >> > with the old format to generate it's byte stream, while allowing a new
> >> > visitor to do something sensible.
> >> >
> >> > destroy
> >> > Allows the caller to destroy a visitor without knowing what type of
> >> > visitor it is.
> >>
> >> When the commit message is basically a list, splitting it into one
> >> commit per list item often makes sense.
> >>
> >> Some of them (the one introducing destroy, perhaps?) can then be put to
> >> use in the same or the next patch. Makes review much easier.
> >
> > Yep, happy to split.
> >
> >> But on to the actual code.
> >>
> >> > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> >> > ---
> >> > include/qapi/visitor-impl.h | 23 +++++++++++++
> >> > include/qapi/visitor.h | 51 +++++++++++++++++++++++++++++
> >> > qapi/qapi-visit-core.c | 80
> >> > +++++++++++++++++++++++++++++++++++++++++++++
> >> > 3 files changed, 154 insertions(+)
> >> >
> >> > diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
> >> > index f3fa420..10cdbf7 100644
> >> > --- a/include/qapi/visitor-impl.h
> >> > +++ b/include/qapi/visitor-impl.h
> >> > @@ -15,6 +15,9 @@
> >> > #include "qapi/error.h"
> >> > #include "qapi/visitor.h"
> >> >
> >>
> >> > struct Visitor
> >> > {
> >> > /* Must be set */
> >> > @@ -29,6 +32,10 @@ struct Visitor
> >> > void (*start_list)(Visitor *v, const char *name, Error **errp);
> >> > GenericList *(*next_list)(Visitor *v, GenericList **list, Error **errp);
> >> > void (*end_list)(Visitor *v, Error **errp);
> >> > + void (*start_array)(Visitor *v, void **obj, const char *name,
> >> > + size_t elem_count, size_t elem_size, Error **errp);
> >> > + void (*next_array)(Visitor *v, Error **errp);
> >> > + void (*end_array)(Visitor *v, Error **errp);
> >> >
> >> > void (*type_enum)(Visitor *v, int *obj, const char *strings[],
> >> > const char *kind, const char *name, Error **errp);
> >>
> >> Since these "must be set", you need to update all existing visitors to
> >> actually set them, in the same patch, don't you?
> >>
> >> I'm not sure the existing visitors completely obey "must be set"
> >> vs. "may be null". But let's not make it worse.
> >
> > Yep; you've picked me up on a whole bunch of the 'must be set'/'may be null'
> > - I'll treat those all as one and go through and tidy them up.
> >
> >> Members above must be set, members below may be null.
> >>
> >> > @@ -49,6 +57,8 @@ struct Visitor
> >> > void (*start_handle)(Visitor *v, void **obj, const char *kind,
> >> > const char *name, Error **errp);
> >> > void (*end_handle)(Visitor *v, Error **errp);
> >>
> >> No longer applies (I killed end_handle() in commit cbc955).
> >
> > Yep; it's dead in my working tree.
>
> When I get to studying how you put this stuff to use, I'll want a
> current version. Happy to pull, just tell me where from.
OK, will do.
> >> > + void (*type_buffer)(Visitor *v, void *data, size_t len, bool async,
> >> > + const char *name, Error **errp);
> >> > void (*type_uint8)(Visitor *v, uint8_t *obj, const char
> >> > *name, Error **errp);
> >> > void (*type_uint16)(Visitor *v, uint16_t *obj, const char
> >> > *name, Error **errp);
> >> > void (*type_uint32)(Visitor *v, uint32_t *obj, const char
> >> > *name, Error **errp);
> >> > @@ -59,6 +69,19 @@ struct Visitor
> >> > void (*type_int64)(Visitor *v, int64_t *obj, const char
> >> > *name, Error **errp);
> >> > /* visit_type_size() falls back to (*type_uint64)() if
> >> > type_size is unset */
> >> > void (*type_size)(Visitor *v, uint64_t *obj, const char
> >> > *name, Error **errp);
> >> > +
> >> > + void (*destroy)(Visitor *v, Error **errp);
> >> > +
> >> > + void (*start_sequence_compat)(Visitor *v, const char *name,
> >> > + Visit_seq_compat_mode compat_mode,
> >> > + void *opaque, Error **errp);
> >> > + void (*end_sequence_compat)(Visitor *v, const char *name,
> >> > + Visit_seq_compat_mode compat_mode,
> >> > + Error **errp);
> >> > +
> >> > + QEMUFile* (*get_qemufile)(Visitor *v);
> >>
> >> Style nit: QEMUFile *(*get_qemufile)(Visitor *v);
> >
> > Fixed.
> >
> >> The empty line before destroy makes me wonder whether these still belong
> >> to the "May be NULL" heading.
> >>
> >> > +
> >> > + uint64_t flags;
> >>
> >> This one can't, because it can't be null :)
> >>
> >
> >> > +/* Blocks of guest memory,disk or otherwise opaque data that there is a lot
> >> > + * of and must be handled efficiently. 'async' true if the write can happen
> >> > + * 'later'
> >> > + */
> >> > +void visit_type_buffer(Visitor *v, void *data, size_t len, bool async,
> >> > + const char *name, Error **errp);
> >>
> >> I'm afraid your specification of async is tied to a specific kind of
> >> visitor: one that "writes". Many don't.
> >
> > Do you have a suggestion on how to do this better? Async accesses were
> > put in some time ago apparently for a useful speed increase, so I can't
> > remove them, and anyway they make sense for the writes; I guess you could
> > argue that they could be used for read as well, if the time at which
> > the 'async' had to have completed was defined, but that's needed for the
> > write anyway.
>
> How do we decide async / sync?
The one current use is for RAM migration; in most cases we just want to throw
the 'current' ram data out on the wire and don't particularly care if we
send a newer version out by the time the write() gets to it and hence set
async; the gotcha is in the xbzrle case where the xbzrle cache has to match
exactly what was sent to the destination and hence we send it sync.
Note that the 'async' in this case is being used to deal with the version/validity
of the data and not really caring about knowing whether it's actually hit the
wire or not.
> >> > void visit_type_enum(Visitor *v, int *obj, const char *strings[],
> >> > const char *kind, const char *name, Error **errp);
> >> > void visit_type_int(Visitor *v, int64_t *obj, const char *name, Error **errp);
> >> > @@ -58,6 +69,46 @@ void visit_type_int64(Visitor *v, int64_t *obj, const char *name, Error **errp);
> >> > void visit_type_size(Visitor *v, uint64_t *obj, const char *name, Error **errp);
> >> > void visit_type_bool(Visitor *v, bool *obj, const char *name, Error **errp);
> >> > void visit_type_str(Visitor *v, char **obj, const char *name, Error **errp);
> >> > +/* A string no more than 256 (including term) characters in length */
> >> > +void visit_type_str256(Visitor *v, char *obj, const char *name, Error **errp);
> >> > void visit_type_number(Visitor *v, double *obj, const char *name, Error **errp);
> >> > +void visit_destroy(Visitor *v, Error **errp);
> >> > +
> >> > +/* Type of visitor, used where actions have to be taken when (de)serializing */
> >> > +bool visitor_loading(Visitor *v);
> >> > +bool visitor_saving(Visitor *v);
> >> > +
> >> > +typedef enum {
> >> > + VISIT_SEQ_COMPAT_BYTE0TERM, /* list terminated with a 0 byte */
> >> > + VISIT_SEQ_COMPAT_FILE, /* The top level file object */
> >> > + VISIT_SEQ_COMPAT_SUBSECLIST, /* list terminated by
> >> > + historical complexity */
> >> > + VISIT_SEQ_COMPAT_SUBSECTION, /* One subsection */
> >> > + VISIT_SEQ_COMPAT_SECTION_HEADER, /* SECTION_FULL/START, header + name */
> >> > + VISIT_SEQ_COMPAT_SECTION_MIN, /* SECTION_PART/END - minimal header */
> >> > + VISIT_SEQ_COMPAT_RAMSECLIST, /* list terminated by int64 bit
> >> > + RAM_SAVE_FLAG_EOS */
> >> > + VISIT_SEQ_COMPAT_RAMSECENTRY, /* Entry in RAM Sec list */
> >> > + VISIT_SEQ_COMPAT_VMSTATE, /* One VMState */
> >> > + VISIT_SEQ_COMPAT_BLOB /* A binary old-format blob */
> >> > +} Visit_seq_compat_mode;
> >> > +
> >> > +/* Start a sequence of items (which may be of unknown length and unknown
> >> > + * mix of some subset of types), specify a compatibility mode that's only
> >> > + * used by an implementation trying to match the existing binary migration
> >> > + * format.
> >> > + * opaque is compat_mode specific
> >> > + */
> >> > +void visit_start_sequence_compat(Visitor *v, const char *name,
> >> > + Visit_seq_compat_mode compat_mode,
> >> > + void *opaque,
> >> > + Error **errp);
> >> > +/* Use visit_get_next_type for each entry including the first */
> >> > +void visit_end_sequence_compat(Visitor *v, const char *name,
> >> > + Visit_seq_compat_mode compat_mode,
> >> > + Error **errp);
> >> > +
> >>
> >> I'm afraid I don't get this comment.
> >
> > Quite a lot of the structures I need to keep compatibility with are
> > lists of some mixture of different things with a terminator on the end
> > (there's not much consistency in the binary format about how you tell
> > what's next or what the terminator is).
> > So the typical use is:
> >
> > visit_start_sequence_compat(...)
> > while (visit_get_next_type( &flag ), flag != end_flag) {
> > switch (flag) {
> > case A:
> > do whatever you need to do now that we've found an 'A' in the list
> > (typically another visit_start_sequence_compat for that type)
> > case B:
> > do whatever you need to do now that we've found an 'B' in the list
> > ....
> > }
> > }
> > visit_end_sequence_compat(...)
> >
> > An example is a RAM migration section; it's a list of initialisation data,
> > normal pages, all-zero pages, XBZRLE pages
>
> Doesn't match the existing visit_get_next_type(). I guess I need more
> context. One way to get it is study your patches.
Probably a good one to look at is the load path change;
http://lists.gnu.org/archive/html/qemu-devel/2014-04/msg03583.html
in particular the changes to ram_load and qemu_loadvm_state that both have
examples of that use.
> >> > +/* Don't Use! - lets us move forward until we can get rid of all
> >> > file uses */
> >> > +QEMUFile *visitor_get_qemufile(Visitor *v);
> >>
> >> Likewise.
> >
> > I'm hoping this is a temporary shim - my hope eventually is that we
> > won't need to
> > use this and everywhere that uses Visitors won't also need to get a QEMUFile
> > except internally to the visitor.
>
> Suggest to put that in the comment.
Will do.
Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2014-06-06 11:44 UTC | newest]
Thread overview: 24+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-03-25 20:17 [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 01/16] Visitor: Add methods for migration format use Dr. David Alan Gilbert (git)
2014-06-05 17:00 ` Markus Armbruster
2014-06-05 17:43 ` Dr. David Alan Gilbert
2014-06-05 18:59 ` Dr. David Alan Gilbert
2014-06-06 8:19 ` Markus Armbruster
2014-06-06 11:44 ` Dr. David Alan Gilbert
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 02/16] QEMUSizedBuffer/QEMUFile Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 03/16] qemu-file: Add set/get tmp_visitor Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 04/16] Header/constant/types fixes for visitors Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 05/16] Visitor: Binary compatible output visitor Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 06/16] Visitor: Debug " Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 07/16] Visitor: Binary compatible input visitor Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 08/16] Visitor: Output path Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 09/16] Visitor: Load path Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 10/16] Visitor: Common types to use visitors Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 11/16] Choose output visitor based on env variable Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 12/16] BER Visitor: Create output visitor Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 13/16] BER Visitor: Create input visitor Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 14/16] Start some BER format docs Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 15/16] ASN.1 schema for new migration format Dr. David Alan Gilbert (git)
2014-03-25 20:17 ` [Qemu-devel] [RFC PATCH 16/16] Wire in BER visitors Dr. David Alan Gilbert (git)
2014-03-25 20:43 ` [Qemu-devel] [RFC PATCH 00/16] visitor+BER migration format Michael S. Tsirkin
2014-03-26 9:16 ` Dr. David Alan Gilbert
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).