qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: "Dr. David Alan Gilbert (git)" <dgilbert@redhat.com>
To: qemu-devel@nongnu.org
Cc: aarcange@redhat.com, yamahata@private.email.ne.jp,
	quintela@redhat.com, amit.shah@redhat.com, pbonzini@redhat.com,
	david@gibson.dropbear.id.au, yayanghy@cn.fujitsu.com
Subject: [Qemu-devel] [PATCH v6 08/47] Add qemu_get_buffer_less_copy to avoid copies some of the time
Date: Tue, 14 Apr 2015 18:03:34 +0100	[thread overview]
Message-ID: <1429031053-4454-9-git-send-email-dgilbert@redhat.com> (raw)
In-Reply-To: <1429031053-4454-1-git-send-email-dgilbert@redhat.com>

From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

qemu_get_buffer always copies the data it reads to a users buffer,
however in many cases the file buffer inside qemu_file could be given
back to the caller, avoiding the copy.  This isn't always possible
depending on the size and alignment of the data.

Thus 'qemu_get_buffer_less_copy' either copies the data to a supplied
buffer or updates a pointer to the internal buffer if convenient.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 include/migration/qemu-file.h |  2 ++
 migration/qemu-file.c         | 45 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 47 insertions(+)

diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index 3fe545e..4cac58f 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -159,6 +159,8 @@ void qemu_put_be32(QEMUFile *f, unsigned int v);
 void qemu_put_be64(QEMUFile *f, uint64_t v);
 int qemu_peek_buffer(QEMUFile *f, uint8_t **buf, int size, size_t offset);
 int qemu_get_buffer(QEMUFile *f, uint8_t *buf, int size);
+int qemu_get_buffer_less_copy(QEMUFile *f, uint8_t **buf, int size);
+
 /*
  * Note that you can only peek continuous bytes from where the current pointer
  * is; you aren't guaranteed to be able to peak to +n bytes unless you've
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 8dc5767..ec3a598 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -426,6 +426,51 @@ int qemu_get_buffer(QEMUFile *f, uint8_t *buf, int size)
 }
 
 /*
+ * Read 'size' bytes of data from the file.
+ * 'size' can be larger than the internal buffer.
+ *
+ * The data:
+ *   may be held on an internal buffer (in which case *buf is updated
+ *     to point to it) that is valid until the next qemu_file operation.
+ * OR
+ *   will be copied to the *buf that was passed in.
+ *
+ * The code tries to avoid the copy if possible.
+ *
+ * It will return size bytes unless there was an error, in which case it will
+ * return as many as it managed to read (assuming blocking fd's which
+ * all current QEMUFile are)
+ */
+int qemu_get_buffer_less_copy(QEMUFile *f, uint8_t **buf, int size)
+{
+    int pending = size;
+    int done = 0;
+    bool first = true;
+
+    while (pending > 0) {
+        int res;
+        uint8_t *src;
+
+        res = qemu_peek_buffer(f, &src, MIN(pending, IO_BUF_SIZE), 0);
+        if (res == 0) {
+            return done;
+        }
+        qemu_file_skip(f, res);
+        done += res;
+        pending -= res;
+        if (first && res == size) {
+            *buf = src;
+            return done;
+        } else {
+            first = false;
+            memcpy(buf, src, res);
+            buf += res;
+        }
+    }
+    return done;
+}
+
+/*
  * Peeks a single byte from the buffer; this isn't guaranteed to work if
  * offset leaves a gap after the previous read/peeked data.
  */
-- 
2.1.0

  parent reply	other threads:[~2015-04-14 17:04 UTC|newest]

Thread overview: 74+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-04-14 17:03 [Qemu-devel] [PATCH v6 00/47] Postcopy implementation Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 01/47] Start documenting how postcopy works Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 02/47] Split header writing out of qemu_savevm_state_begin Dr. David Alan Gilbert (git)
2015-05-11 11:16   ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 03/47] qemu_ram_foreach_block: pass up error value, and down the ramblock name Dr. David Alan Gilbert (git)
2015-05-15 10:38   ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 04/47] Add qemu_get_counted_string to read a string prefixed by a count byte Dr. David Alan Gilbert (git)
2015-05-15 13:50   ` Amit Shah
2015-05-15 14:06     ` Dr. David Alan Gilbert
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 05/47] Create MigrationIncomingState Dr. David Alan Gilbert (git)
2015-05-18  6:58   ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 06/47] Provide runtime Target page information Dr. David Alan Gilbert (git)
2015-05-18  7:06   ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 07/47] Move copy out of qemu_peek_buffer Dr. David Alan Gilbert (git)
2015-05-21  6:47   ` Amit Shah
2015-04-14 17:03 ` Dr. David Alan Gilbert (git) [this message]
2015-05-21  7:09   ` [Qemu-devel] [PATCH v6 08/47] Add qemu_get_buffer_less_copy to avoid copies some of the time Amit Shah
2015-05-21  8:45     ` Dr. David Alan Gilbert
2015-05-21  8:58       ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 09/47] Add wrapper for setting blocking status on a QEMUFile Dr. David Alan Gilbert (git)
2015-05-18  7:35   ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 10/47] Rename save_live_complete to save_live_complete_precopy Dr. David Alan Gilbert (git)
2015-05-18  7:35   ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 11/47] Return path: Open a return path on QEMUFile for sockets Dr. David Alan Gilbert (git)
2015-06-10  9:00   ` Amit Shah
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 12/47] Return path: socket_writev_buffer: Block even on non-blocking fd's Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 13/47] Migration commands Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 14/47] Return path: Control commands Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 15/47] Return path: Send responses from destination to source Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 16/47] Return path: Source handling of return path Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 17/47] ram_debug_dump_bitmap: Dump a migration bitmap as text Dr. David Alan Gilbert (git)
2015-05-21  9:21   ` Amit Shah
2015-05-21 10:10     ` Dr. David Alan Gilbert
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 18/47] Move loadvm_handlers into MigrationIncomingState Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 19/47] Rework loadvm path for subloops Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 20/47] Add migration-capability boolean for postcopy-ram Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 21/47] Add wrappers and handlers for sending/receiving the postcopy-ram migration messages Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 22/47] MIG_CMD_PACKAGED: Send a packaged chunk of migration stream Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 23/47] migrate_init: Call from savevm Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 24/47] Modify save_live_pending for postcopy Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 25/47] postcopy: OS support test Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 26/47] migrate_start_postcopy: Command to trigger transition to postcopy Dr. David Alan Gilbert (git)
2015-04-14 17:38   ` Eric Blake
2015-04-14 17:40     ` Dr. David Alan Gilbert
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 27/47] MIGRATION_STATUS_POSTCOPY_ACTIVE: Add new migration state Dr. David Alan Gilbert (git)
2015-04-14 17:40   ` Eric Blake
2015-04-14 18:00     ` Dr. David Alan Gilbert
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 28/47] Add qemu_savevm_state_complete_postcopy Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 29/47] Postcopy: Maintain sentmap and calculate discard Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 30/47] postcopy: Incoming initialisation Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 31/47] postcopy: ram_enable_notify to switch on userfault Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 32/47] Postcopy: Postcopy startup in migration thread Dr. David Alan Gilbert (git)
2015-04-14 17:03 ` [Qemu-devel] [PATCH v6 33/47] Postcopy end in migration_thread Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 34/47] Page request: Add MIG_RP_MSG_REQ_PAGES reverse command Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 35/47] Page request: Process incoming page request Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 36/47] Page request: Consume pages off the post-copy queue Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 37/47] postcopy_ram.c: place_page and helpers Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 38/47] Postcopy: Use helpers to map pages during migration Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 39/47] qemu_ram_block_from_host Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 40/47] Don't sync dirty bitmaps in postcopy Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 41/47] Host page!=target page: Cleanup bitmaps Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 42/47] Postcopy; Handle userfault requests Dr. David Alan Gilbert (git)
2015-05-25  9:18   ` zhanghailiang
2015-05-26  9:50     ` Dr. David Alan Gilbert
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 43/47] Start up a postcopy/listener thread ready for incoming page data Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 44/47] postcopy: Wire up loadvm_postcopy_handle_ commands Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 45/47] End of migration for postcopy Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 46/47] Disable mlock around incoming postcopy Dr. David Alan Gilbert (git)
2015-04-14 17:04 ` [Qemu-devel] [PATCH v6 47/47] Inhibit ballooning during postcopy Dr. David Alan Gilbert (git)
2015-04-27  8:04 ` [Qemu-devel] [PATCH v6 00/47] Postcopy implementation Li, Liang Z
2015-04-29 17:23   ` Dr. David Alan Gilbert
2015-04-30  1:09     ` Li, Liang Z
     [not found]       ` <20150505150112.GM2126@work-vm>
     [not found]         ` <F2CBF3009FA73547804AE4C663CAB28E50F0E1@shsmsx102.ccr.corp.intel.com>
     [not found]           ` <20150506083056.GB2204@work-vm>
2015-05-07  1:21             ` Li, Liang Z
2015-05-07  8:01               ` Dr. David Alan Gilbert

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1429031053-4454-9-git-send-email-dgilbert@redhat.com \
    --to=dgilbert@redhat.com \
    --cc=aarcange@redhat.com \
    --cc=amit.shah@redhat.com \
    --cc=david@gibson.dropbear.id.au \
    --cc=pbonzini@redhat.com \
    --cc=qemu-devel@nongnu.org \
    --cc=quintela@redhat.com \
    --cc=yamahata@private.email.ne.jp \
    --cc=yayanghy@cn.fujitsu.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).