qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
To: Amit Shah <amit.shah@redhat.com>
Cc: aarcange@redhat.com, yamahata@private.email.ne.jp,
	quintela@redhat.com, liang.z.li@intel.com, qemu-devel@nongnu.org,
	luis@cs.umu.se, pbonzini@redhat.com, david@gibson.dropbear.id.au
Subject: Re: [Qemu-devel] [PATCH v7 19/42] MIG_CMD_PACKAGED: Send a packaged chunk of migration stream
Date: Mon, 27 Jul 2015 18:28:39 +0100	[thread overview]
Message-ID: <20150727172838.GB27613@work-vm> (raw)
In-Reply-To: <20150721061129.GC10280@grmbl.mre>

* Amit Shah (amit.shah@redhat.com) wrote:
> On (Tue) 16 Jun 2015 [11:26:32], Dr. David Alan Gilbert (git) wrote:
> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > 
> > MIG_CMD_PACKAGED is a migration command that wraps a chunk of migration
> > stream inside a package whose length can be determined purely by reading
> > its header.  The destination guarantees that the whole MIG_CMD_PACKAGED
> > is read off the stream prior to parsing the contents.
> > 
> > This is used by postcopy to load device state (from the package)
> > while leaving the main stream free to receive memory pages.
> 
> Not sure why this is necessary.  I suppose I'll have to go read the
> documentation in patch 1..

Yep - or one of the previous replies where I explained it.

> However:
> 
> > --- a/migration/savevm.c
> > +++ b/migration/savevm.c
> > @@ -718,6 +718,50 @@ void qemu_savevm_send_open_return_path(QEMUFile *f)
> >      qemu_savevm_command_send(f, MIG_CMD_OPEN_RETURN_PATH, 0, NULL);
> >  }
> >  
> > +/* We have a buffer of data to send; we don't want that all to be loaded
> > + * by the command itself, so the command contains just the length of the
> > + * extra buffer that we then send straight after it.
> > + * TODO: Must be a better way to organise that
> > + *
> > + * Returns:
> > + *    0 on success
> > + *    -ve on error
> > + */
> > +int qemu_savevm_send_packaged(QEMUFile *f, const QEMUSizedBuffer *qsb)
> > +{
> > +    size_t cur_iov;
> > +    size_t len = qsb_get_length(qsb);
> > +    uint32_t tmp;
> > +
> > +    if (len > MAX_VM_CMD_PACKAGED_SIZE) {
> > +        error_report("%s: Unreasonably large packaged state: %zu",
> > +                     __func__, len);
> > +        return -1;
> > +    }
> > +
> > +    tmp = cpu_to_be32(len);
> > +
> > +    trace_qemu_savevm_send_packaged();
> > +    qemu_savevm_command_send(f, MIG_CMD_PACKAGED, 4, (uint8_t *)&tmp);
> > +
> > +    /* all the data follows (concatinating the iov's) */
> > +    for (cur_iov = 0; cur_iov < qsb->n_iov; cur_iov++) {
> > +        /* The iov entries are partially filled */
> > +        size_t towrite = (qsb->iov[cur_iov].iov_len > len) ?
> > +                              len :
> > +                              qsb->iov[cur_iov].iov_len;
> 
> If iov_len was > len, we only wrote part of the current buffer, and we
> skip to the next?

Yes; this is just the end case; the qsb allocates iov entries in 'chunks'
but then the data that gets added often doesn't use the whole chunk.
'len'  - set above from qsb_get_length - gives the used contents of
the qsb, and that's all we want to write.  This is normally the case
on the last entry in the qsb anyway; however since 'len' gets
decremented by the amount written we might go once more around the loop
and the 'if (!towrite) { break; }' might break us out of the loop instead.

Dave

> 
> 
> 		Amit
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

  reply	other threads:[~2015-07-27 18:23 UTC|newest]

Thread overview: 209+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-06-16 10:26 [Qemu-devel] [PATCH v7 00/42] Postcopy implementation Dr. David Alan Gilbert (git)
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 01/42] Start documenting how postcopy works Dr. David Alan Gilbert (git)
2015-06-17 11:42   ` Juan Quintela
2015-06-17 12:30     ` Dr. David Alan Gilbert
2015-06-18  7:50   ` Li, Liang Z
2015-06-18  8:10     ` Dr. David Alan Gilbert
2015-06-18  8:28     ` Paolo Bonzini
2015-06-19 17:52       ` Dr. David Alan Gilbert
2015-06-26  6:46   ` Yang Hongyang
2015-06-26  7:53     ` zhanghailiang
2015-06-26  8:00       ` Yang Hongyang
2015-06-26  8:10         ` Dr. David Alan Gilbert
2015-06-26  8:19           ` Yang Hongyang
2015-08-04  5:20   ` Amit Shah
2015-08-05 12:21     ` Dr. David Alan Gilbert
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 02/42] Provide runtime Target page information Dr. David Alan Gilbert (git)
2015-06-17 11:43   ` Juan Quintela
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 03/42] Init page sizes in qtest Dr. David Alan Gilbert (git)
2015-06-17 11:49   ` Juan Quintela
2015-07-06  6:14   ` Amit Shah
2015-08-04  5:23   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 04/42] qemu_ram_block_from_host Dr. David Alan Gilbert (git)
2015-06-17 11:54   ` Juan Quintela
2015-07-10  8:36   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 05/42] Add qemu_get_buffer_less_copy to avoid copies some of the time Dr. David Alan Gilbert (git)
2015-06-17 11:57   ` Juan Quintela
2015-06-17 12:33     ` Dr. David Alan Gilbert
2015-07-13  9:08   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 06/42] Add wrapper for setting blocking status on a QEMUFile Dr. David Alan Gilbert (git)
2015-06-17 11:59   ` Juan Quintela
2015-06-17 12:34     ` Dr. David Alan Gilbert
2015-06-17 12:57       ` Juan Quintela
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 07/42] ram_debug_dump_bitmap: Dump a migration bitmap as text Dr. David Alan Gilbert (git)
2015-06-17 12:17   ` Juan Quintela
2015-06-19 17:04     ` Dr. David Alan Gilbert
2015-07-13 10:15       ` Juan Quintela
2015-07-13  9:12   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 08/42] migrate_init: Call from savevm Dr. David Alan Gilbert (git)
2015-06-17 12:18   ` Juan Quintela
2015-07-13  9:13   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 09/42] Rename save_live_complete to save_live_complete_precopy Dr. David Alan Gilbert (git)
2015-06-17 12:20   ` Juan Quintela
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 10/42] Return path: Open a return path on QEMUFile for sockets Dr. David Alan Gilbert (git)
2015-06-17 12:23   ` Juan Quintela
2015-06-17 17:07     ` Dr. David Alan Gilbert
2015-07-13 10:12   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 11/42] Return path: socket_writev_buffer: Block even on non-blocking fd's Dr. David Alan Gilbert (git)
2015-06-17 12:28   ` Juan Quintela
2015-06-19 17:18     ` Dr. David Alan Gilbert
2015-07-13 12:37   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 12/42] Migration commands Dr. David Alan Gilbert (git)
2015-06-17 12:31   ` Juan Quintela
2015-06-19 17:38     ` Dr. David Alan Gilbert
2015-07-13 12:45   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 13/42] Return path: Control commands Dr. David Alan Gilbert (git)
2015-06-17 12:49   ` Juan Quintela
2015-06-23 18:57     ` Dr. David Alan Gilbert
2015-07-13 12:55   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 14/42] Return path: Send responses from destination to source Dr. David Alan Gilbert (git)
2015-06-17 16:30   ` Juan Quintela
2015-06-19 18:42     ` Dr. David Alan Gilbert
2015-07-01  9:29       ` Juan Quintela
2015-08-06 12:18         ` Dr. David Alan Gilbert
2015-07-15  7:31   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 15/42] Return path: Source handling of return path Dr. David Alan Gilbert (git)
2015-07-13 10:29   ` Juan Quintela
2015-08-18 10:23     ` Dr. David Alan Gilbert
2015-07-15  7:50   ` Amit Shah
2015-07-16 11:32     ` Dr. David Alan Gilbert
2015-08-05  8:06   ` zhanghailiang
2015-08-18 10:45     ` Dr. David Alan Gilbert
2015-08-18 11:29       ` zhanghailiang
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 16/42] Rework loadvm path for subloops Dr. David Alan Gilbert (git)
2015-07-13 10:33   ` Juan Quintela
2015-07-15  9:34   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 17/42] Add migration-capability boolean for postcopy-ram Dr. David Alan Gilbert (git)
2015-06-16 15:43   ` Eric Blake
2015-06-16 15:58     ` Dr. David Alan Gilbert
2015-07-15  9:39       ` Amit Shah
2015-07-13 10:35   ` Juan Quintela
2015-07-15  9:40   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 18/42] Add wrappers and handlers for sending/receiving the postcopy-ram migration messages Dr. David Alan Gilbert (git)
2015-07-13 11:02   ` Juan Quintela
2015-07-20 10:13     ` Amit Shah
2015-08-26 14:48     ` Dr. David Alan Gilbert
2015-07-20 10:06   ` Amit Shah
2015-07-27  9:55     ` Dr. David Alan Gilbert
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 19/42] MIG_CMD_PACKAGED: Send a packaged chunk of migration stream Dr. David Alan Gilbert (git)
2015-07-13 11:07   ` Juan Quintela
2015-07-21  6:11   ` Amit Shah
2015-07-27 17:28     ` Dr. David Alan Gilbert [this message]
2015-08-04  5:27   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 20/42] Modify save_live_pending for postcopy Dr. David Alan Gilbert (git)
2015-07-13 11:12   ` Juan Quintela
2015-07-31 16:13     ` Dr. David Alan Gilbert
2015-07-21  6:17   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 21/42] postcopy: OS support test Dr. David Alan Gilbert (git)
2015-07-13 11:20   ` Juan Quintela
2015-07-13 16:31     ` Dr. David Alan Gilbert
2015-07-21  7:29   ` Amit Shah
2015-07-27 17:38     ` Dr. David Alan Gilbert
2015-08-04  5:28   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 22/42] migrate_start_postcopy: Command to trigger transition to postcopy Dr. David Alan Gilbert (git)
2015-07-13 11:23   ` Juan Quintela
2015-07-13 17:13     ` Dr. David Alan Gilbert
2015-07-13 18:07       ` Juan Quintela
2015-07-21  7:40         ` Amit Shah
2015-09-24  9:59           ` Dr. David Alan Gilbert
2015-09-24 14:20         ` Dr. David Alan Gilbert
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 23/42] MIGRATION_STATUS_POSTCOPY_ACTIVE: Add new migration state Dr. David Alan Gilbert (git)
2015-07-13 11:27   ` Juan Quintela
2015-07-13 15:53     ` Dr. David Alan Gilbert
2015-07-13 16:26       ` Juan Quintela
2015-07-13 16:48         ` Dr. David Alan Gilbert
2015-07-13 18:05           ` Juan Quintela
2015-07-21 10:33   ` Amit Shah
2015-09-23 17:04     ` Dr. David Alan Gilbert
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 24/42] Add qemu_savevm_state_complete_postcopy Dr. David Alan Gilbert (git)
2015-07-13 11:35   ` Juan Quintela
2015-07-13 15:33     ` Dr. David Alan Gilbert
2015-07-21 10:42   ` Amit Shah
2015-07-27 17:58     ` Dr. David Alan Gilbert
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 25/42] Postcopy: Maintain sentmap and calculate discard Dr. David Alan Gilbert (git)
2015-07-13 11:47   ` Juan Quintela
2015-09-15 17:01     ` Dr. David Alan Gilbert
2015-07-21 11:36   ` Amit Shah
2015-07-31 16:51     ` Dr. David Alan Gilbert
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 26/42] postcopy: Incoming initialisation Dr. David Alan Gilbert (git)
2015-07-13 12:04   ` Juan Quintela
2015-09-23 19:06     ` Dr. David Alan Gilbert
2015-07-22  6:19   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 27/42] postcopy: ram_enable_notify to switch on userfault Dr. David Alan Gilbert (git)
2015-07-13 12:10   ` Juan Quintela
2015-07-13 17:36     ` Dr. David Alan Gilbert
2015-07-23  5:22   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 28/42] Postcopy: Postcopy startup in migration thread Dr. David Alan Gilbert (git)
2015-07-13 12:56   ` Juan Quintela
2015-07-13 17:56     ` Dr. David Alan Gilbert
2015-07-13 18:09       ` Juan Quintela
2015-09-23 17:56         ` Dr. David Alan Gilbert
2015-07-23  5:53       ` Amit Shah
2015-07-23  5:55   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 29/42] Postcopy end in migration_thread Dr. David Alan Gilbert (git)
2015-07-13 13:15   ` Juan Quintela
2015-07-23  6:41     ` Amit Shah
2015-08-04 11:31     ` Dr. David Alan Gilbert
2015-07-23  6:41   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 30/42] Page request: Add MIG_RP_MSG_REQ_PAGES reverse command Dr. David Alan Gilbert (git)
2015-07-13 13:24   ` Juan Quintela
2015-08-06 14:15     ` Dr. David Alan Gilbert
2015-07-23  6:50   ` Amit Shah
2015-08-06 14:21     ` Dr. David Alan Gilbert
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 31/42] Page request: Process incoming page request Dr. David Alan Gilbert (git)
2015-07-14  9:18   ` Juan Quintela
2015-08-06 10:45     ` Dr. David Alan Gilbert
2015-10-20 10:29       ` Juan Quintela
2015-07-23 12:23   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 32/42] Page request: Consume pages off the post-copy queue Dr. David Alan Gilbert (git)
2015-07-14  9:40   ` Juan Quintela
2015-09-16 18:36     ` Dr. David Alan Gilbert
2015-07-27  6:05   ` Amit Shah
2015-09-16 18:48     ` Dr. David Alan Gilbert
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 33/42] postcopy_ram.c: place_page and helpers Dr. David Alan Gilbert (git)
2015-07-14 10:05   ` Juan Quintela
2015-07-27  6:11     ` Amit Shah
2015-09-23 16:45     ` Dr. David Alan Gilbert
2015-07-27  6:11   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 34/42] Postcopy: Use helpers to map pages during migration Dr. David Alan Gilbert (git)
2015-07-14 12:34   ` Juan Quintela
2015-07-17 17:31     ` Dr. David Alan Gilbert
2015-07-27  7:39   ` Amit Shah
2015-08-06 11:22     ` Dr. David Alan Gilbert
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 35/42] Don't sync dirty bitmaps in postcopy Dr. David Alan Gilbert (git)
2015-07-14 12:36   ` Juan Quintela
2015-07-14 13:13     ` Dr. David Alan Gilbert
2015-07-27  7:43   ` Amit Shah
2015-07-31  9:50     ` Dr. David Alan Gilbert
2015-08-04  5:46       ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 36/42] Host page!=target page: Cleanup bitmaps Dr. David Alan Gilbert (git)
2015-07-14 15:01   ` Juan Quintela
2015-07-31 15:53     ` Dr. David Alan Gilbert
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 37/42] Postcopy; Handle userfault requests Dr. David Alan Gilbert (git)
2015-07-14 15:10   ` Juan Quintela
2015-07-14 15:15     ` Dr. David Alan Gilbert
2015-07-14 15:25       ` Juan Quintela
2015-07-27 14:29   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 38/42] Start up a postcopy/listener thread ready for incoming page data Dr. David Alan Gilbert (git)
2015-07-14 15:12   ` Juan Quintela
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 39/42] postcopy: Wire up loadvm_postcopy_handle_ commands Dr. David Alan Gilbert (git)
2015-07-14 15:14   ` Juan Quintela
2015-07-28  5:53   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 40/42] End of migration for postcopy Dr. David Alan Gilbert (git)
2015-07-14 15:15   ` Juan Quintela
2015-07-28  5:55   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 41/42] Disable mlock around incoming postcopy Dr. David Alan Gilbert (git)
2015-07-14 15:22   ` Juan Quintela
2015-07-28  6:02     ` Amit Shah
2015-07-28 11:32       ` Juan Quintela
2015-08-06 14:55         ` Dr. David Alan Gilbert
2015-08-07  3:05           ` zhanghailiang
2015-09-24 10:36     ` Dr. David Alan Gilbert
2015-07-28  6:02   ` Amit Shah
2015-06-16 10:26 ` [Qemu-devel] [PATCH v7 42/42] Inhibit ballooning during postcopy Dr. David Alan Gilbert (git)
2015-07-14 15:24   ` Juan Quintela
2015-07-28  6:15   ` Amit Shah
2015-07-28  9:08     ` Dr. David Alan Gilbert
2015-07-28 10:01       ` Amit Shah
2015-07-28 11:16         ` Dr. David Alan Gilbert
2015-07-28  6:21 ` [Qemu-devel] [PATCH v7 00/42] Postcopy implementation Amit Shah

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20150727172838.GB27613@work-vm \
    --to=dgilbert@redhat.com \
    --cc=aarcange@redhat.com \
    --cc=amit.shah@redhat.com \
    --cc=david@gibson.dropbear.id.au \
    --cc=liang.z.li@intel.com \
    --cc=luis@cs.umu.se \
    --cc=pbonzini@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=quintela@redhat.com \
    --cc=yamahata@private.email.ne.jp \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).