From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48117) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1b5SWE-0001li-IC for qemu-devel@nongnu.org; Wed, 25 May 2016 02:42:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1b5SWA-0000lX-8J for qemu-devel@nongnu.org; Wed, 25 May 2016 02:42:14 -0400 Date: Wed, 25 May 2016 15:38:24 +1000 From: David Gibson Message-ID: <20160525053824.GM17226@voom.fritz.box> References: <1464112509-21806-1-git-send-email-duanj@linux.vnet.ibm.com> <1464112509-21806-5-git-send-email-duanj@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="MsEL38XAg4rx1uDx" Content-Disposition: inline In-Reply-To: <1464112509-21806-5-git-send-email-duanj@linux.vnet.ibm.com> Subject: Re: [Qemu-devel] [QEMU RFC PATCH v2 4/6] Migration: migrate QTAILQ List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Jianjun Duan Cc: qemu-devel@nongnu.org, qemu-ppc@nongnu.org, mdroth@linux.vnet.ibm.com, amit.shah@redhat.com, quintela@redhat.com --MsEL38XAg4rx1uDx Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, May 24, 2016 at 10:55:07AM -0700, Jianjun Duan wrote: > A recursive structure has elements of the same type in itself. Currently > we cannot directly transfer a QTAILQ instance, or any recursive > structure such as lists in migration because of the limitation in the > migration code. Here we introduce a general approach to transfer such > structures. In our approach a recursive structure is tagged with VMS_CSTM. > We extend VMStateField with 3 fields: meta_data to store the meta data > about the recursive structure in question, extend_get to load the structu= re > from migration stream to memory, and extend_put to dump the structure into > the migration stream. This extension mirrors VMStateInfo. We then modify > vmstate_save_state and vmstate_load_state so that when VMS_CSTM is > encountered, extend_put and extend_get are called respectively with the > knowledge of the meta data. All the talk about recursive structures just obscures the issue. The point is you're building a way of migrating QTAILQs - just stick to that. > To make it work for QTAILQ in qemu/queue.h, we created the meta data > format, extend_get and extend_put for it. We will use this approach to > transfer pending_events and ccs_list in spapr state. >=20 > We also create some macros in qemu/queue.h to get the layout information > about QTAILQ. This ensures that we do not depend on the implementation > details about QTAILQ in the migration code. I'm interested to see what Juan and Dave Gilbert think about this. >=20 > Signed-off-by: Jianjun Duan > --- > include/migration/vmstate.h | 59 +++++++++++++++++++++++++++++++ > include/qemu/queue.h | 38 ++++++++++++++++++++ > migration/vmstate.c | 84 +++++++++++++++++++++++++++++++++++++++= ++++++ > 3 files changed, 181 insertions(+) >=20 > diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h > index 1622638..bf57b25 100644 > --- a/include/migration/vmstate.h > +++ b/include/migration/vmstate.h > @@ -183,6 +183,8 @@ enum VMStateFlags { > * to determine the number of entries in the array. Only valid in > * combination with one of VMS_VARRAY*. */ > VMS_MULTIPLY_ELEMENTS =3D 0x4000, > + /* For fields which need customized handling, such as QTAILQ in queu= e.h*/ > + VMS_CSTM =3D 0x8000, > }; > =20 > typedef struct { > @@ -198,6 +200,14 @@ typedef struct { > const VMStateDescription *vmsd; > int version_id; > bool (*field_exists)(void *opaque, int version_id); > + /* > + * Following 3 fields are for VMStateField which needs customized ha= ndling, > + * such as QTAILQ in qemu/queue.h, lists, and tree. > + */ > + const void *meta_data; > + int (*extend_get)(QEMUFile *f, const void *metadata, void *opaque); > + void (*extend_put)(QEMUFile *f, const void *metadata, void *opaque, > + QJSON *vmdesc); > } VMStateField; > =20 > struct VMStateDescription { > @@ -654,6 +664,18 @@ extern const VMStateInfo vmstate_info_bitmap; > .offset =3D offsetof(_state, _field), \ > } > =20 > +/* For fields that need customized handling, such as queue, list */ > +#define VMSTATE_CSTM_V(_field, _state, _version, _meta_data, _get, _put)= \ > +{ = \ > + .name =3D (stringify(_field)), = \ > + .version_id =3D (_version), = \ > + .flags =3D VMS_CSTM, = \ > + .offset =3D offsetof(_state, _field), = \ > + .meta_data =3D &(_meta_data), = \ > + .extend_get =3D (_get), = \ > + .extend_put =3D (_put), = \ > +} > + > /* _f : field name > _f_n : num of elements field_name > _n : num of elements > @@ -970,4 +992,41 @@ int64_t self_announce_delay(int round) > =20 > void dump_vmstate_json_to_file(FILE *out_fp); > =20 > + > +/* Meta data for QTAILQ */ > +typedef struct QTAILQMetaData { > + /* the offset of tqh_first in QTAILQ_HEAD */ > + size_t first; > + /* the offset of tqh_last in QTAILQ_HEAD */ > + size_t last; > + /* the offset of tqe_next in QTAILQ_ENTRY */ > + size_t next; > + /* the offset of tqe_prev in QTAILQ_ENTRY */ > + size_t prev; > + /* the offset of QTAILQ_ENTRY in a QTAILQ element*/ > + size_t entry; > + /* size of a QTAILQ element */ > + size_t size; > + /* vmsd of a QTAILQ element */ > + const VMStateDescription *vmsd; > +} QTAILQMetaData; > + > +#define VMSTATE_QTAILQ_METADATA(_field, _state, _type, _next, _vmsd) { \ > + .first =3D QTAILQ_FIRST_OFFSET(typeof_field(_state, _field)), = \ > + .last =3D QTAILQ_LAST_OFFSET(typeof_field(_state, _field)), = \ > + .next =3D QTAILQ_NEXT_OFFSET(_type, _next), = \ > + .prev =3D QTAILQ_PREV_OFFSET(_type, _next), = \ > + .entry =3D offsetof(_type, _next), = \ > + .size =3D sizeof(_type), = \ > + .vmsd =3D &(_vmsd), = \ > +} > + > +int vmstate_get_qtailq(QEMUFile *f, const void *metadata, void *opaque); > +void vmstate_put_qtailq(QEMUFile *f, const void *metadata, > + void *opaque, QJSON *vmdesc); > + > +/* VMStateField for QTAILQ field */ > +#define VMSTATE_QTAILQ_V(_field, _state, _version, _meta_data) = \ > + VMSTATE_CSTM_V(_field, _state, _version, _meta_data, vmstate_get_qta= ilq, \ > + vmstate_put_qtailq) > #endif > diff --git a/include/qemu/queue.h b/include/qemu/queue.h > index f781aa2..46962d7 100644 > --- a/include/qemu/queue.h > +++ b/include/qemu/queue.h > @@ -437,3 +437,41 @@ struct { = \ > (*(((struct headname *)((elm)->field.tqe_prev))->tqh_last)) > =20 > #endif /* !QEMU_SYS_QUEUE_H_ */ > + > +/* > + * Offsets of layout of a tail queue head. > + */ > +#define QTAILQ_FIRST_OFFSET(head_type) \ > + ((size_t) ((char *) &((head_type *)0)->tqh_first - (char *)0)) > + > +#define QTAILQ_LAST_OFFSET(head_type) \ > + ((size_t) ((char *) &((head_type *)0)->tqh_last - (char *)0)) > + > +/* > + * Offsets of layout of a tail queue element. > + */ > +#define QTAILQ_NEXT_OFFSET(ele_type, field) \ > + ((size_t) ((char *) &((ele_type *)0)->field.tqe_next - \ > + (char *) &((ele_type *)0)->field)) > + > +#define QTAILQ_PREV_OFFSET(ele_type, field) \ > + ((size_t) ((char *) &((ele_type *)0)->field.tqe_prev - \ > + (char *) &((ele_type *)0)->field)) > +/* > + * Tail queue tranversal using pointer arithmetic. > + */ > +#define QTAILQ_RAW_FOREACH(elm, head, entry, first, next) \ > + for ((elm) =3D *((void **) ((char *) (head) + (first))); = \ > + (elm); \ > + (elm) =3D *((void **) ((char *) (elm) + (entry) + (next)))) > +/* > + * Tail queue insertion using pointer arithmetic. > + */ > +#define QTAILQ_RAW_INSERT_TAIL(head, elm, entry, first, last, next, prev= ) do { \ > + *((void **) ((char *) (elm) + (entry) + (next))) =3D NULL; = \ > + *((void **) ((char *) (elm) + (entry) + (prev))) =3D = \ > + *((void **) ((char *) (head) + (last))); = \ > + **((void ***)((char *) (head) + (last))) =3D (elm); = \ > + *((void **) ((char *) (head) + (last))) =3D = \ > + (void *) ((char *) (elm) + (entry) + (next)); = \ > +} while (/*CONSTCOND*/0) > diff --git a/migration/vmstate.c b/migration/vmstate.c > index bf3d5db..47cd052 100644 > --- a/migration/vmstate.c > +++ b/migration/vmstate.c > @@ -5,6 +5,7 @@ > #include "migration/vmstate.h" > #include "qemu/bitops.h" > #include "qemu/error-report.h" > +#include "qemu/queue.h" > #include "trace.h" > #include "qjson.h" > =20 > @@ -121,6 +122,8 @@ int vmstate_load_state(QEMUFile *f, const VMStateDesc= ription *vmsd, > if (field->flags & VMS_STRUCT) { > ret =3D vmstate_load_state(f, field->vmsd, addr, > field->vmsd->version_id); > + } else if (field->flags & VMS_CSTM) { > + ret =3D field->extend_get(f, field->meta_data, addr); > } else { > ret =3D field->info->get(f, addr, size); > =20 > @@ -193,6 +196,8 @@ static const char *vmfield_get_type_name(VMStateField= *field) > =20 > if (field->flags & VMS_STRUCT) { > type =3D "struct"; > + } else if (field->flags & VMS_CSTM) { > + type =3D "customized"; > } else if (field->info->name) { > type =3D field->info->name; > } > @@ -327,6 +332,8 @@ void vmstate_save_state(QEMUFile *f, const VMStateDes= cription *vmsd, > } > if (field->flags & VMS_STRUCT) { > vmstate_save_state(f, field->vmsd, addr, vmdesc_loop= ); > + } else if (field->flags & VMS_CSTM) { > + field->extend_put(f, field->meta_data, addr, vmdesc_= loop); > } else { > field->info->put(f, addr, size); > } > @@ -916,3 +923,80 @@ const VMStateInfo vmstate_info_bitmap =3D { > .get =3D get_bitmap, > .put =3D put_bitmap, > }; > + > +/* extend_get for QTAILQ */ > +int vmstate_get_qtailq(QEMUFile *f, const void *metadata, void *opaque) > +{ > + bool link; > + int ret =3D 0; > + size_t first, last, next, prev, entry, size; > + QTAILQMetaData *data =3D (QTAILQMetaData *)metadata; > + const VMStateDescription *vmsd =3D data->vmsd; > + int version_id =3D vmsd->version_id; > + void *elm; > + > + first =3D data->first; > + last =3D data->last; > + next =3D data->next; > + prev =3D data->prev; > + entry =3D data->entry; > + size =3D data->size; > + > + trace_vmstate_load_state(vmsd->name, version_id); > + if (version_id > vmsd->version_id) { > + trace_vmstate_load_state_end(vmsd->name, "too new", -EINVAL); > + return -EINVAL; > + } > + if (version_id < vmsd->minimum_version_id) { > + trace_vmstate_load_state_end(vmsd->name, "too old", -EINVAL); > + return -EINVAL; > + } > + > + while (true) { > + vmstate_info_bool.get(f, &link, sizeof(bool)); > + if (!link) { > + break; > + } > + > + elm =3D g_malloc(size); > + ret =3D vmstate_load_state(f, vmsd, elm, version_id); > + if (ret) { > + return ret; > + } > + QTAILQ_RAW_INSERT_TAIL(opaque, elm, entry, first, last, next, pr= ev); > + } > + > + trace_vmstate_load_state_end(vmsd->name, "end", ret); > + return ret; > +} > + > +/* extend_get for QTAILQ */ > +void vmstate_put_qtailq(QEMUFile *f, const void *metadata, void *opaque, > + QJSON *vmdesc) > +{ > + bool link =3D true; > + size_t first, next, entry; > + QTAILQMetaData *data =3D (QTAILQMetaData *)metadata; > + const VMStateDescription *vmsd =3D data->vmsd; > + void *elm; > + > + first =3D data->first; > + next =3D data->next; > + entry =3D data->entry; > + > + if (vmdesc) { > + json_prop_str(vmdesc, "vmsd_name", vmsd->name); > + json_prop_int(vmdesc, "version", vmsd->version_id); > + json_start_array(vmdesc, "fields"); > + } > + > + QTAILQ_RAW_FOREACH(elm, opaque, entry, first, next) { > + vmstate_info_bool.put(f, &link, sizeof(bool)); > + vmstate_save_state(f, vmsd, elm, vmdesc); > + } > + link =3D false; > + vmstate_info_bool.put(f, &link, sizeof(bool)); > + if (vmdesc) { > + json_end_array(vmdesc); > + } > +} --=20 David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson --MsEL38XAg4rx1uDx Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJXRTpQAAoJEGw4ysog2bOSCdkP/3aMVXRYk0QEnrk5XNTyPZRH 6pHS+Vnj57hHfnAv3IJ3bCYd6AtFFJWg7RVEnRU23EjAdxQgkzXhZ5ciQgq6aucr muUXl4RsnDyoKOcVmKUY1vHGRqiBGRl/vUWCnPgSRGK9myCvvvvY147aHoQPaPXt aFMGI9tqNrv49YXGRSf3/EItXqPBg98cIBPMY3z7Wfp6OkBv4TlNc86CwGZjF4kp jHD557uW0u2Wrbjsp6Hfmtb/5xW7DAmsXI4/EKdSGJq/K/VH8WhlKcu9HMRxCy2s THwWnFCFsoruVqTDOpPM5Lu1rbJMuNRviG6+AQhnQs+nwyTwFTDoVhAlnSmhsaYN BX5WxPxbSoDTel2rRkB/ur6WzUQ0y+Q7Ghb3FxvxQs8BTGsJlH2iUIBopgYVBeN0 lZ7b0eKgTquaXagIZfYllFUcvJcmh6DgWe1fhbQmxZEY3Dm+wqIi5poLtBZ13NXa Jh8F17aRUvApMj8a9YVs68t3fgNSO2u6dhR/w2epyasU/dBz9ZSNedEEO8RtlaYe /LRGUxW1K6QOpOX53E18bqdf4QJFA7NxvDp9T5huFB8c2nIFXjl3FSpfhFNYGi1U /atW5Rbl32m0SdPOvdZ6kO6Nxz/oHKFmEAfTFiJTN2VoCCjTMSXkyD8fgFy6yU2r pwIqabWDnkq1r5iMxzPq =h40n -----END PGP SIGNATURE----- --MsEL38XAg4rx1uDx--