qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Daniel P. Berrange" <berrange@redhat.com>
To: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Cc: aarcange@redhat.com, quintela@redhat.com, liang.z.li@intel.com,
	qemu-devel@nongnu.org, luis@cs.umu.se,
	bharata@linux.vnet.ibm.com, amit.shah@redhat.com,
	pbonzini@redhat.com
Subject: Re: [Qemu-devel] [PATCH v8 16/54] Return path: Open a return path on QEMUFile for sockets
Date: Fri, 2 Oct 2015 18:03:47 +0100	[thread overview]
Message-ID: <20151002170347.GJ28469@redhat.com> (raw)
In-Reply-To: <20151002163218.GB8601@work-vm>

On Fri, Oct 02, 2015 at 05:32:18PM +0100, Dr. David Alan Gilbert wrote:
> * Daniel P. Berrange (berrange@redhat.com) wrote:
> > On Tue, Sep 29, 2015 at 09:37:40AM +0100, Dr. David Alan Gilbert (git) wrote:
> > > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> > > 
> > > Postcopy needs a method to send messages from the destination back to
> > > the source, this is the 'return path'.
> > > 
> > > Wire it up for 'socket' QEMUFile's.
> > 
> > I find this to be a pretty wierd approach to the problem. THe underlying
> > transport is bi-directional, so I would expect to have a single QEMUFile
> > object that allowed bi-directional I/O on it, rather than creating a
> > second QEMUFile for the back channel, which was forbidden from closing
> > the shared FD.
> > 
> > I can understand why you've done this though - since we only have a
> > single buffer embedded in QEMUFile.  I wonder though if we'd be better
> > off changing QEMUFile to have a 'inbuf' and 'outbuf' intead of just
> > 'buf' and likewise iniov & outiov. Then we can allow bi-directional
> > I/O on the single QEMUFile object which is a more natural fit.
> 
> The 'c' FILE* is one directional, and I just took it that the QEMUFile* is
> like that; i.e. a buffered layer on top of an underlying one directional
> transport. stdin,stdout are two separate FILE*'s.

Yep, QEMUFile was really designed as a FILE* alternative, so makes sense
from that POV.

> Your iniov, outiov would be basically the same, so you'd end up duplicating
> code for the in and out parts; where as what you really have is two of the same
> thing wired up in opposite directions.

I don't think it'd actually end up duplicating any code - mostly just
updating which variable was accessed in each existing method, depending
on whether it was a read or write related method.

> Having said that, for things like RDMA, they have to do special stuff for
> each direction and the QEMUFile is really a shim on top of that.

Similarly when we add TLS into the mix, there is a single shared TLS
session context that is used by both I/O directionals. Now this would
not be visible to the QEMUFile regardless, since its hidden in the
QIOChannel object I'm defining, so its not a show stopper either but
I guess my general thought is that there is a mixture of state that
we maintain some different for read vs write and some shared. You
workaround the fact that the FD is shared by having a comment saying
we should not call close() on the FD kept by the QEMUFile for the
return path.

All that said, I don't think it is too critical to change this right
now. It would be fine to leave it to a later date, unless there's a
more pressing reason.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

  reply	other threads:[~2015-10-02 17:03 UTC|newest]

Thread overview: 118+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-09-29  8:37 [Qemu-devel] [PATCH v8 00/54] Postcopy implementation Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 01/54] Add postcopy documentation Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 02/54] Provide runtime Target page information Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 03/54] Init page sizes in qtest Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 04/54] Move configuration section writing Dr. David Alan Gilbert (git)
2015-10-05  6:44   ` Amit Shah
2015-10-30 12:47     ` Dr. David Alan Gilbert
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 05/54] qemu_ram_block_from_host Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 06/54] Rename mis->file to from_src_file Dr. David Alan Gilbert (git)
2015-09-29 10:41   ` Amit Shah
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 07/54] Add qemu_get_buffer_in_place to avoid copies some of the time Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 08/54] Add wrapper for setting blocking status on a QEMUFile Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 09/54] Add QEMU_MADV_NOHUGEPAGE Dr. David Alan Gilbert (git)
2015-10-28 10:35   ` Amit Shah
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 10/54] migration/ram.c: Use RAMBlock rather than MemoryRegion Dr. David Alan Gilbert (git)
2015-10-28 10:36   ` Amit Shah
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 11/54] ram_debug_dump_bitmap: Dump a migration bitmap as text Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 12/54] migrate_init: Call from savevm Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 13/54] Move dirty page search state into separate structure Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 14/54] ram_find_and_save_block: Split out the finding Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 15/54] Rename save_live_complete to save_live_complete_precopy Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 16/54] Return path: Open a return path on QEMUFile for sockets Dr. David Alan Gilbert (git)
2015-10-02 15:29   ` Daniel P. Berrange
2015-10-02 16:32     ` Dr. David Alan Gilbert
2015-10-02 17:03       ` Daniel P. Berrange [this message]
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 17/54] Return path: socket_writev_buffer: Block even on non-blocking fd's Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 18/54] Migration commands Dr. David Alan Gilbert (git)
2015-10-20 11:22   ` Juan Quintela
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 19/54] Return path: Control commands Dr. David Alan Gilbert (git)
2015-10-20 11:27   ` Juan Quintela
2015-10-26 11:42     ` Dr. David Alan Gilbert
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 20/54] Return path: Send responses from destination to source Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 21/54] Return path: Source handling of return path Dr. David Alan Gilbert (git)
2015-10-20 11:33   ` Juan Quintela
2015-10-26 12:06     ` Dr. David Alan Gilbert
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 22/54] Rework loadvm path for subloops Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 23/54] Add migration-capability boolean for postcopy-ram Dr. David Alan Gilbert (git)
2015-09-29 20:22   ` Eric Blake
2015-09-30  7:00     ` Amit Shah
2015-09-30 12:44       ` Eric Blake
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 24/54] Add wrappers and handlers for sending/receiving the postcopy-ram migration messages Dr. David Alan Gilbert (git)
2015-10-20 11:50   ` Juan Quintela
2015-10-26 12:22     ` Dr. David Alan Gilbert
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 25/54] MIG_CMD_PACKAGED: Send a packaged chunk of migration stream Dr. David Alan Gilbert (git)
2015-10-20 13:25   ` Juan Quintela
2015-10-26 16:21     ` Dr. David Alan Gilbert
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 26/54] Modify save_live_pending for postcopy Dr. David Alan Gilbert (git)
2015-10-28 11:03   ` Amit Shah
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 27/54] postcopy: OS support test Dr. David Alan Gilbert (git)
2015-10-20 13:31   ` Juan Quintela
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 28/54] migrate_start_postcopy: Command to trigger transition to postcopy Dr. David Alan Gilbert (git)
2015-09-30 16:25   ` Eric Blake
2015-09-30 16:30     ` Dr. David Alan Gilbert
2015-10-20 13:33   ` Juan Quintela
2015-10-28 11:17   ` Amit Shah
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 29/54] MIGRATION_STATUS_POSTCOPY_ACTIVE: Add new migration state Dr. David Alan Gilbert (git)
2015-10-20 13:35   ` Juan Quintela
2015-10-30 18:19     ` Dr. David Alan Gilbert
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 30/54] Avoid sending vmdescription during postcopy Dr. David Alan Gilbert (git)
2015-10-20 13:35   ` Juan Quintela
2015-10-28 11:19   ` Amit Shah
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 31/54] Add qemu_savevm_state_complete_postcopy Dr. David Alan Gilbert (git)
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 32/54] Postcopy: Maintain sentmap and calculate discard Dr. David Alan Gilbert (git)
2015-10-21 11:17   ` Juan Quintela
2015-10-30 18:43     ` Dr. David Alan Gilbert
2015-11-02 17:31     ` Dr. David Alan Gilbert
2015-11-02 18:19     ` Dr. David Alan Gilbert
2015-11-02 20:14     ` Dr. David Alan Gilbert
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 33/54] postcopy: Incoming initialisation Dr. David Alan Gilbert (git)
2015-10-21  8:35   ` Juan Quintela
2015-11-03 17:59     ` Dr. David Alan Gilbert
2015-11-03 18:32       ` Juan Quintela
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 34/54] postcopy: ram_enable_notify to switch on userfault Dr. David Alan Gilbert (git)
2015-10-28 11:40   ` Amit Shah
2015-09-29  8:37 ` [Qemu-devel] [PATCH v8 35/54] Postcopy: Postcopy startup in migration thread Dr. David Alan Gilbert (git)
2015-10-21  8:57   ` Juan Quintela
2015-10-26 17:12     ` Dr. David Alan Gilbert
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 36/54] Split out end of migration code from migration_thread Dr. David Alan Gilbert (git)
2015-10-21  9:11   ` Juan Quintela
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 37/54] Postcopy: End of iteration Dr. David Alan Gilbert (git)
2015-10-21  9:16   ` Juan Quintela
2015-10-29  5:10   ` Amit Shah
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 38/54] Page request: Add MIG_RP_MSG_REQ_PAGES reverse command Dr. David Alan Gilbert (git)
2015-10-21 11:12   ` Juan Quintela
2015-10-26 16:58     ` Dr. David Alan Gilbert
2015-10-29  5:17   ` Amit Shah
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 39/54] Page request: Process incoming page request Dr. David Alan Gilbert (git)
2015-10-21 11:17   ` Juan Quintela
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 40/54] Page request: Consume pages off the post-copy queue Dr. David Alan Gilbert (git)
2015-10-26 16:32   ` Juan Quintela
2015-11-03 11:52     ` Dr. David Alan Gilbert
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 41/54] postcopy_ram.c: place_page and helpers Dr. David Alan Gilbert (git)
2015-10-28 10:28   ` Juan Quintela
2015-10-28 13:11     ` Dr. David Alan Gilbert
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 42/54] Postcopy: Use helpers to map pages during migration Dr. David Alan Gilbert (git)
2015-10-28 10:58   ` Juan Quintela
2015-10-30 12:59     ` Dr. David Alan Gilbert
2015-10-30 16:35     ` Dr. David Alan Gilbert
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 43/54] Don't sync dirty bitmaps in postcopy Dr. David Alan Gilbert (git)
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 44/54] Don't iterate on precopy-only devices during postcopy Dr. David Alan Gilbert (git)
2015-10-28 11:01   ` Juan Quintela
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 45/54] Host page!=target page: Cleanup bitmaps Dr. David Alan Gilbert (git)
2015-10-28 11:24   ` Juan Quintela
2015-11-03 17:32     ` Dr. David Alan Gilbert
2015-11-03 18:30       ` Juan Quintela
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 46/54] postcopy: Check order of received target pages Dr. David Alan Gilbert (git)
2015-10-28 11:26   ` Juan Quintela
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 47/54] Round up RAMBlock sizes to host page sizes Dr. David Alan Gilbert (git)
2015-10-28 11:28   ` Juan Quintela
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 48/54] Postcopy; Handle userfault requests Dr. David Alan Gilbert (git)
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 49/54] Start up a postcopy/listener thread ready for incoming page data Dr. David Alan Gilbert (git)
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 50/54] postcopy: Wire up loadvm_postcopy_handle_ commands Dr. David Alan Gilbert (git)
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 51/54] Postcopy: Mark nohugepage before discard Dr. David Alan Gilbert (git)
2015-10-28 14:02   ` Juan Quintela
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 52/54] End of migration for postcopy Dr. David Alan Gilbert (git)
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 53/54] Disable mlock around incoming postcopy Dr. David Alan Gilbert (git)
2015-10-21  9:17   ` Juan Quintela
2015-09-29  8:38 ` [Qemu-devel] [PATCH v8 54/54] Inhibit ballooning during postcopy Dr. David Alan Gilbert (git)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20151002170347.GJ28469@redhat.com \
    --to=berrange@redhat.com \
    --cc=aarcange@redhat.com \
    --cc=amit.shah@redhat.com \
    --cc=bharata@linux.vnet.ibm.com \
    --cc=dgilbert@redhat.com \
    --cc=liang.z.li@intel.com \
    --cc=luis@cs.umu.se \
    --cc=pbonzini@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=quintela@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).