[Qemu-devel] [PATCH 0/3] More migration file cleanup

[Qemu-devel] [PATCH 0/3] More migration file cleanup

[Dr. David Alan Gilbert (git)]

From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>


This is a small set of patches that shuffle migration code
around, but doesn't change the behaviour:

  1) Move a lot of the migration source into a separate 'migration' directory
     Note this moves a lot of files around, git format-patch -M -B spots the
     renames
  2) Split the 'struct QEMUFile' out into a private header
     Because the QEMU Buffered file code wants to access fields, and it's about
     to be in a separate file from QEMUFile.
  3) Split the QEMU buffered file code out in the same way as the stdio and
     unix/socket code has been split out.

Dave

Dr. David Alan Gilbert (3):
  Start moving migration code into a migration directory
  Split struct QEMUFile out
  Split the QEMU buffered file code out

 Makefile.objs                                    |   9 +-
 migration/Makefile.objs                          |   7 +
 migration-exec.c => migration/migration-exec.c   |   0
 migration-fd.c => migration/migration-fd.c       |   0
 migration-rdma.c => migration/migration-rdma.c   |   0
 migration-tcp.c => migration/migration-tcp.c     |   0
 migration-unix.c => migration/migration-unix.c   |   0
 migration.c => migration/migration.c             |   0
 qemu-file.c => migration/qemu-file-buf.c         | 511 +---------------------
 migration/qemu-file-internal.h                   |  53 +++
 qemu-file-stdio.c => migration/qemu-file-stdio.c |   0
 qemu-file-unix.c => migration/qemu-file-unix.c   |   0
 migration/qemu-file.c                            | 519 +++++++++++++++++++++++
 vmstate.c => migration/vmstate.c                 |   0
 xbzrle.c => migration/xbzrle.c                   |   0
 tests/Makefile                                   |   7 +-
 16 files changed, 586 insertions(+), 520 deletions(-)
 create mode 100644 migration/Makefile.objs
 rename migration-exec.c => migration/migration-exec.c (100%)
 rename migration-fd.c => migration/migration-fd.c (100%)
 rename migration-rdma.c => migration/migration-rdma.c (100%)
 rename migration-tcp.c => migration/migration-tcp.c (100%)
 rename migration-unix.c => migration/migration-unix.c (100%)
 rename migration.c => migration/migration.c (100%)
 rename qemu-file.c => migration/qemu-file-buf.c (51%)
 create mode 100644 migration/qemu-file-internal.h
 rename qemu-file-stdio.c => migration/qemu-file-stdio.c (100%)
 rename qemu-file-unix.c => migration/qemu-file-unix.c (100%)
 create mode 100644 migration/qemu-file.c
 rename vmstate.c => migration/vmstate.c (100%)
 rename xbzrle.c => migration/xbzrle.c (100%)

-- 
1.9.3

[Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Dr. David Alan Gilbert (git)]

From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

The migration code now occupies a fair chunk of the top level .c
files, it seems time to give it it's own directory.

I've not touched:
   arch_init.c - that's mostly RAM migration but has a few random other
                 bits
   savevm.c    - because it's built target specific
   block-migration.c - should that go in block/ or migration/ ?

This is purely a code move; no code has changed.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 Makefile.objs                                    | 9 ++-------
 migration/Makefile.objs                          | 7 +++++++
 migration-exec.c => migration/migration-exec.c   | 0
 migration-fd.c => migration/migration-fd.c       | 0
 migration-rdma.c => migration/migration-rdma.c   | 0
 migration-tcp.c => migration/migration-tcp.c     | 0
 migration-unix.c => migration/migration-unix.c   | 0
 migration.c => migration/migration.c             | 0
 qemu-file-stdio.c => migration/qemu-file-stdio.c | 0
 qemu-file-unix.c => migration/qemu-file-unix.c   | 0
 qemu-file.c => migration/qemu-file.c             | 0
 vmstate.c => migration/vmstate.c                 | 0
 xbzrle.c => migration/xbzrle.c                   | 0
 tests/Makefile                                   | 6 +++---
 14 files changed, 12 insertions(+), 10 deletions(-)
 create mode 100644 migration/Makefile.objs
 rename migration-exec.c => migration/migration-exec.c (100%)
 rename migration-fd.c => migration/migration-fd.c (100%)
 rename migration-rdma.c => migration/migration-rdma.c (100%)
 rename migration-tcp.c => migration/migration-tcp.c (100%)
 rename migration-unix.c => migration/migration-unix.c (100%)
 rename migration.c => migration/migration.c (100%)
 rename qemu-file-stdio.c => migration/qemu-file-stdio.c (100%)
 rename qemu-file-unix.c => migration/qemu-file-unix.c (100%)
 rename qemu-file.c => migration/qemu-file.c (100%)
 rename vmstate.c => migration/vmstate.c (100%)
 rename xbzrle.c => migration/xbzrle.c (100%)

diff --git a/Makefile.objs b/Makefile.objs
index 18fd35c..71b4b79 100644
--- a/Makefile.objs
+++ b/Makefile.objs
@@ -48,15 +48,10 @@ common-obj-$(CONFIG_POSIX) += os-posix.o
 
 common-obj-$(CONFIG_LINUX) += fsdev/
 
-common-obj-y += migration.o migration-tcp.o
-common-obj-y += vmstate.o
-common-obj-y += qemu-file.o qemu-file-unix.o qemu-file-stdio.o
-common-obj-$(CONFIG_RDMA) += migration-rdma.o
+common-obj-y += migration/
 common-obj-y += qemu-char.o #aio.o
 common-obj-y += block-migration.o
-common-obj-y += page_cache.o xbzrle.o
-
-common-obj-$(CONFIG_POSIX) += migration-exec.o migration-unix.o migration-fd.o
+common-obj-y += page_cache.o
 
 common-obj-$(CONFIG_SPICE) += spice-qemu-char.o
 
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
new file mode 100644
index 0000000..681bae9
--- /dev/null
+++ b/migration/Makefile.objs
@@ -0,0 +1,7 @@
+common-obj-y += migration.o migration-tcp.o
+common-obj-y += vmstate.o
+common-obj-y += qemu-file.o qemu-file-unix.o qemu-file-stdio.o
+common-obj-$(CONFIG_RDMA) += migration-rdma.o
+common-obj-y += xbzrle.o
+
+common-obj-$(CONFIG_POSIX) += migration-exec.o migration-unix.o migration-fd.o
diff --git a/migration-exec.c b/migration/migration-exec.c
similarity index 100%
rename from migration-exec.c
rename to migration/migration-exec.c
diff --git a/migration-fd.c b/migration/migration-fd.c
similarity index 100%
rename from migration-fd.c
rename to migration/migration-fd.c
diff --git a/migration-rdma.c b/migration/migration-rdma.c
similarity index 100%
rename from migration-rdma.c
rename to migration/migration-rdma.c
diff --git a/migration-tcp.c b/migration/migration-tcp.c
similarity index 100%
rename from migration-tcp.c
rename to migration/migration-tcp.c
diff --git a/migration-unix.c b/migration/migration-unix.c
similarity index 100%
rename from migration-unix.c
rename to migration/migration-unix.c
diff --git a/migration.c b/migration/migration.c
similarity index 100%
rename from migration.c
rename to migration/migration.c
diff --git a/qemu-file-stdio.c b/migration/qemu-file-stdio.c
similarity index 100%
rename from qemu-file-stdio.c
rename to migration/qemu-file-stdio.c
diff --git a/qemu-file-unix.c b/migration/qemu-file-unix.c
similarity index 100%
rename from qemu-file-unix.c
rename to migration/qemu-file-unix.c
diff --git a/qemu-file.c b/migration/qemu-file.c
similarity index 100%
rename from qemu-file.c
rename to migration/qemu-file.c
diff --git a/vmstate.c b/migration/vmstate.c
similarity index 100%
rename from vmstate.c
rename to migration/vmstate.c
diff --git a/xbzrle.c b/migration/xbzrle.c
similarity index 100%
rename from xbzrle.c
rename to migration/xbzrle.c
diff --git a/tests/Makefile b/tests/Makefile
index 16f0e4c..3a03979 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -49,7 +49,7 @@ check-unit-y += tests/test-x86-cpuid$(EXESUF)
 # all code tested by test-x86-cpuid is inside topology.h
 gcov-files-test-x86-cpuid-y =
 check-unit-y += tests/test-xbzrle$(EXESUF)
-gcov-files-test-xbzrle-y = xbzrle.c
+gcov-files-test-xbzrle-y = migration/xbzrle.c
 check-unit-y += tests/test-cutils$(EXESUF)
 gcov-files-test-cutils-y += util/cutils.c
 check-unit-y += tests/test-mul64$(EXESUF)
@@ -247,7 +247,7 @@ tests/test-thread-pool$(EXESUF): tests/test-thread-pool.o $(block-obj-y) libqemu
 tests/test-iov$(EXESUF): tests/test-iov.o libqemuutil.a
 tests/test-hbitmap$(EXESUF): tests/test-hbitmap.o libqemuutil.a libqemustub.a
 tests/test-x86-cpuid$(EXESUF): tests/test-x86-cpuid.o
-tests/test-xbzrle$(EXESUF): tests/test-xbzrle.o xbzrle.o page_cache.o libqemuutil.a
+tests/test-xbzrle$(EXESUF): tests/test-xbzrle.o migration/xbzrle.o page_cache.o libqemuutil.a
 tests/test-cutils$(EXESUF): tests/test-cutils.o util/cutils.o
 tests/test-int128$(EXESUF): tests/test-int128.o
 tests/test-qdev-global-props$(EXESUF): tests/test-qdev-global-props.o \
@@ -258,7 +258,7 @@ tests/test-qdev-global-props$(EXESUF): tests/test-qdev-global-props.o \
 	$(test-qapi-obj-y) \
 	libqemuutil.a libqemustub.a
 tests/test-vmstate$(EXESUF): tests/test-vmstate.o \
-	vmstate.o qemu-file.o qemu-file-unix.o \
+	migration/vmstate.o migration/qemu-file.o migration/qemu-file-unix.o \
 	libqemuutil.a libqemustub.a
 
 tests/test-qapi-types.c tests/test-qapi-types.h :\
-- 
1.9.3

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Juan Quintela]

"Dr. David Alan Gilbert (git)" <dgilbert@redhat.com> wrote:
G> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> The migration code now occupies a fair chunk of the top level .c
> files, it seems time to give it it's own directory.
>
> I've not touched:
>    arch_init.c - that's mostly RAM migration but has a few random other
>                  bits

Will split the memory bits, and can go to migration.

>    savevm.c    - because it's built target specific

Damn, would have to look at it.

>    block-migration.c - should that go in block/ or migration/ ?

It is really on migration.  we have basically:

- ram-migration
- block-migration

We can call other names if people preffer.

>
> This is purely a code move; no code has changed.
>
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Thanks, Juan.

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Dr. David Alan Gilbert]

* Juan Quintela (quintela@redhat.com) wrote:
> "Dr. David Alan Gilbert (git)" <dgilbert@redhat.com> wrote:
> G> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> >
> > The migration code now occupies a fair chunk of the top level .c
> > files, it seems time to give it it's own directory.
> >
> > I've not touched:
> >    arch_init.c - that's mostly RAM migration but has a few random other
> >                  bits
> 
> Will split the memory bits, and can go to migration.
> 
> >    savevm.c    - because it's built target specific
> 
> Damn, would have to look at it.

Yes; I suspect it's the vmstate_register_ram that uses TARGET_PAGE_MASK,
one of the patches in my postcopy world adds a function in exec.c that
returns TARGET_PAGE_BITS so that stuff that needs to know target page size
can make a call to find it out; that might solve that.

Dave

> 
> >    block-migration.c - should that go in block/ or migration/ ?
> 
> It is really on migration.  we have basically:
> 
> - ram-migration
> - block-migration
> 
> We can call other names if people preffer.
> 
> >
> > This is purely a code move; no code has changed.
> >
> > Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> 
> Thanks, Juan.
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Paolo Bonzini]

On 10/16/2014 10:12 AM, Dr. David Alan Gilbert wrote:
> Yes; I suspect it's the vmstate_register_ram that uses TARGET_PAGE_MASK,
> one of the patches in my postcopy world adds a function in exec.c that
> returns TARGET_PAGE_BITS so that stuff that needs to know target page size
> can make a call to find it out; that might solve that.

Yeah, or you can just move vmstate_register_ram to migration/ram.c 
together with the arch_init.c bits.

Paolo

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Juan Quintela]

"Dr. David Alan Gilbert (git)" <dgilbert@redhat.com> wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> The migration code now occupies a fair chunk of the top level .c
> files, it seems time to give it it's own directory.
>
> I've not touched:
>    arch_init.c - that's mostly RAM migration but has a few random other
>                  bits
>    savevm.c    - because it's built target specific
>    block-migration.c - should that go in block/ or migration/ ?
>
> This is purely a code move; no code has changed.
>
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Amit Shah]

On (Thu) 16 Oct 2014 [08:53:52], Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> 
> The migration code now occupies a fair chunk of the top level .c
> files, it seems time to give it it's own directory.

s/it's/its

> I've not touched:
>    arch_init.c - that's mostly RAM migration but has a few random other
>                  bits
>    savevm.c    - because it's built target specific
>    block-migration.c - should that go in block/ or migration/ ?
> 
> This is purely a code move; no code has changed.
> 
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Reviewed-by: Amit Shah <amit.shah@redhat.com>


		Amit

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Gary Hook]

On 10/30/14, 7:26 AM, "Amit Shah" <amit.shah@redhat.com> wrote:

>On (Thu) 16 Oct 2014 [08:53:52], Dr. David Alan Gilbert (git) wrote:
>> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>> 
>> The migration code now occupies a fair chunk of the top level .c
>> files, it seems time to give it it's own directory.
>
>s/it's/its

6 out of 87 .c files, and approximately 370 blocks out of 2840 (based on
du output). 13% is a "fair chunk"?

But tidy organization is a good thing while needless renaming is not. The
only goal that the suggested renames would appear to accomplish is
additional obfuscation. How about just moving them into a subdirectory and
leave their names alone?

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Dr. David Alan Gilbert]

* Gary Hook (gary.hook@nimboxx.com) wrote:
> 
> 
> On 10/30/14, 7:26 AM, "Amit Shah" <amit.shah@redhat.com> wrote:
> 
> >On (Thu) 16 Oct 2014 [08:53:52], Dr. David Alan Gilbert (git) wrote:
> >> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> >> 
> >> The migration code now occupies a fair chunk of the top level .c
> >> files, it seems time to give it it's own directory.
> >
> >s/it's/its
> 
> 6 out of 87 .c files, and approximately 370 blocks out of 2840 (based on
> du output). 13% is a "fair chunk"?

I'm not sure how you got 6;
  migration.c
  migration-exec.c
  migration-fd.c
  migration-rdma.c
  migration-tcp.c
  migration-unix.c
  qemu-file-buf.c
  qemu-file.c
  qemu-file-stdio.c
  qemu-file-unix.c
  vmstate.c
  xbzrle.c

so that's 12, and there are another 3 in the commit message saying they could
do with being moved.  That would be 15 files, or 17% - and so yes, I do call that
a fair chunk.

> But tidy organization is a good thing while needless renaming is not. The
> only goal that the suggested renames would appear to accomplish is
> additional obfuscation. How about just moving them into a subdirectory and
> leave their names alone?

Which is what I did; however I have sympathy with those that think that in a
directory called 'migration' starting a bunch of the files with 'migration-' is
excessive.

Dave
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Eduardo Habkost]

On Mon, Nov 03, 2014 at 08:53:54AM +0000, Dr. David Alan Gilbert wrote:
[...]
> > But tidy organization is a good thing while needless renaming is not. The
> > only goal that the suggested renames would appear to accomplish is
> > additional obfuscation. How about just moving them into a subdirectory and
> > leave their names alone?
> 
> Which is what I did; however I have sympathy with those that think that in a
> directory called 'migration' starting a bunch of the files with 'migration-' is
> excessive.

I, for one, don't even see the difference between moving and renaming.  :)

-- 
Eduardo

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Amit Shah]

On (Thu) 16 Oct 2014 [08:53:52], Dr. David Alan Gilbert (git) wrote:

>  rename migration-exec.c => migration/migration-exec.c (100%)
>  rename migration-fd.c => migration/migration-fd.c (100%)
>  rename migration-rdma.c => migration/migration-rdma.c (100%)
>  rename migration-tcp.c => migration/migration-tcp.c (100%)
>  rename migration-unix.c => migration/migration-unix.c (100%)
>  rename migration.c => migration/migration.c (100%)

I'm wondering if we should also use the opportunity to cleanup the
file names:

migration.c => main.c
migration-X.c => X.c

?

		Amit

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Dr. David Alan Gilbert]

* Amit Shah (amit.shah@redhat.com) wrote:
> On (Thu) 16 Oct 2014 [08:53:52], Dr. David Alan Gilbert (git) wrote:
> 
> >  rename migration-exec.c => migration/migration-exec.c (100%)
> >  rename migration-fd.c => migration/migration-fd.c (100%)
> >  rename migration-rdma.c => migration/migration-rdma.c (100%)
> >  rename migration-tcp.c => migration/migration-tcp.c (100%)
> >  rename migration-unix.c => migration/migration-unix.c (100%)
> >  rename migration.c => migration/migration.c (100%)
> 
> I'm wondering if we should also use the opportunity to cleanup the
> file names:
> 
> migration.c => main.c
> migration-X.c => X.c
> 
> ?

I'd be OK with changing filenames, but they would have to
be fairly clear; I don't like having a 'main.c' because that's
where I'd expect to find a main() - but we do have a few core.c's
as the core of each of a few hw subdirs.

Dave

> 
> 		Amit
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Amit Shah]

On (Thu) 30 Oct 2014 [12:37:27], Dr. David Alan Gilbert wrote:
> * Amit Shah (amit.shah@redhat.com) wrote:
> > On (Thu) 16 Oct 2014 [08:53:52], Dr. David Alan Gilbert (git) wrote:
> > 
> > >  rename migration-exec.c => migration/migration-exec.c (100%)
> > >  rename migration-fd.c => migration/migration-fd.c (100%)
> > >  rename migration-rdma.c => migration/migration-rdma.c (100%)
> > >  rename migration-tcp.c => migration/migration-tcp.c (100%)
> > >  rename migration-unix.c => migration/migration-unix.c (100%)
> > >  rename migration.c => migration/migration.c (100%)
> > 
> > I'm wondering if we should also use the opportunity to cleanup the
> > file names:
> > 
> > migration.c => main.c
> > migration-X.c => X.c
> > 
> > ?
> 
> I'd be OK with changing filenames, but they would have to
> be fairly clear; I don't like having a 'main.c' because that's
> where I'd expect to find a main() - but we do have a few core.c's
> as the core of each of a few hw subdirs.

I think main should be OK in a subdir, because it's the main file for
that subsystem (not necessarily the whole program).  Checking the src
for main.c, though, it looks like no one uses it in such a way (Linux
has lots of these, btw, hence my initial suggestion).

core.c is fine, too.

		Amit

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Peter Maydell]

On 30 October 2014 12:37, Dr. David Alan Gilbert <dgilbert@redhat.com> wrote:
> * Amit Shah (amit.shah@redhat.com) wrote:
>> On (Thu) 16 Oct 2014 [08:53:52], Dr. David Alan Gilbert (git) wrote:
>>
>> >  rename migration-exec.c => migration/migration-exec.c (100%)
>> >  rename migration-fd.c => migration/migration-fd.c (100%)
>> >  rename migration-rdma.c => migration/migration-rdma.c (100%)
>> >  rename migration-tcp.c => migration/migration-tcp.c (100%)
>> >  rename migration-unix.c => migration/migration-unix.c (100%)
>> >  rename migration.c => migration/migration.c (100%)
>>
>> I'm wondering if we should also use the opportunity to cleanup the
>> file names:
>>
>> migration.c => main.c
>> migration-X.c => X.c
>>
>> ?
>
> I'd be OK with changing filenames, but they would have to
> be fairly clear; I don't like having a 'main.c' because that's
> where I'd expect to find a main() - but we do have a few core.c's
> as the core of each of a few hw subdirs.

Yeah, I think I agree that 'main.c' is not a great choice
(but core.c or migration.c would be fine); for the others I
agree with Amit that we should drop the migration- prefix
as we move them. (It's generally how we've handled other
moves-into-subdirectories in the past; various arm_* and
alpha_* files lost their prefixes when they moved into hw/arm
and hw/alpha, for instance.)

thanks
-- PMM

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Peter Maydell]

On 16 October 2014 08:53, Dr. David Alan Gilbert (git)
<dgilbert@redhat.com> wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> The migration code now occupies a fair chunk of the top level .c
> files, it seems time to give it it's own directory.

Missed opportunity to make the patch summary line
"Start migrating migration code into a migration directory" :-)

-- PMM

Re: [Qemu-devel] [PATCH 1/3] Start moving migration code into a migration directory

[Dr. David Alan Gilbert]

* Peter Maydell (peter.maydell@linaro.org) wrote:
> On 16 October 2014 08:53, Dr. David Alan Gilbert (git)
> <dgilbert@redhat.com> wrote:
> > From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> >
> > The migration code now occupies a fair chunk of the top level .c
> > files, it seems time to give it it's own directory.
> 
> Missed opportunity to make the patch summary line
> "Start migrating migration code into a migration directory" :-)

I would certainly allow a committer to fix such a missed opportunity.

Dave

> 
> -- PMM
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

[Qemu-devel] [PATCH 2/3] Split struct QEMUFile out

[Dr. David Alan Gilbert (git)]

From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

Now we've got multiple QEMUFile source files, some of them need
access to things that were defined in qemu-file.c, so create
a -internal header for them.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/qemu-file-internal.h | 53 ++++++++++++++++++++++++++++++++++++++++++
 migration/qemu-file.c          | 23 +-----------------
 2 files changed, 54 insertions(+), 22 deletions(-)
 create mode 100644 migration/qemu-file-internal.h

diff --git a/migration/qemu-file-internal.h b/migration/qemu-file-internal.h
new file mode 100644
index 0000000..d95e853
--- /dev/null
+++ b/migration/qemu-file-internal.h
@@ -0,0 +1,53 @@
+/*
+ * QEMU System Emulator
+ *
+ * Copyright (c) 2003-2008 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#ifndef QEMU_FILE_INTERNAL_H
+#define QEMU_FILE_INTERNAL_H 1
+
+#include "qemu-common.h"
+#include "qemu/iov.h"
+
+#define IO_BUF_SIZE 32768
+#define MAX_IOV_SIZE MIN(IOV_MAX, 64)
+
+struct QEMUFile {
+    const QEMUFileOps *ops;
+    void *opaque;
+
+    int64_t bytes_xfer;
+    int64_t xfer_limit;
+
+    int64_t pos; /* start of buffer when writing, end of buffer
+                    when reading */
+    int buf_index;
+    int buf_size; /* 0 when writing */
+    uint8_t buf[IO_BUF_SIZE];
+
+    struct iovec iov[MAX_IOV_SIZE];
+    unsigned int iovcnt;
+
+    int last_error;
+};
+
+#endif
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index f938e36..671fba9 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -27,30 +27,9 @@
 #include "block/coroutine.h"
 #include "migration/migration.h"
 #include "migration/qemu-file.h"
+#include "migration/qemu-file-internal.h"
 #include "trace.h"
 
-#define IO_BUF_SIZE 32768
-#define MAX_IOV_SIZE MIN(IOV_MAX, 64)
-
-struct QEMUFile {
-    const QEMUFileOps *ops;
-    void *opaque;
-
-    int64_t bytes_xfer;
-    int64_t xfer_limit;
-
-    int64_t pos; /* start of buffer when writing, end of buffer
-                    when reading */
-    int buf_index;
-    int buf_size; /* 0 when writing */
-    uint8_t buf[IO_BUF_SIZE];
-
-    struct iovec iov[MAX_IOV_SIZE];
-    unsigned int iovcnt;
-
-    int last_error;
-};
-
 bool qemu_file_mode_is_not_valid(const char *mode)
 {
     if (mode == NULL ||
-- 
1.9.3

Re: [Qemu-devel] [PATCH 2/3] Split struct QEMUFile out

[Juan Quintela]

"Dr. David Alan Gilbert (git)" <dgilbert@redhat.com> wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> Now we've got multiple QEMUFile source files, some of them need
> access to things that were defined in qemu-file.c, so create
> a -internal header for them.
>
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

Re: [Qemu-devel] [PATCH 2/3] Split struct QEMUFile out

[Amit Shah]

On (Thu) 16 Oct 2014 [08:53:53], Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> 
> Now we've got multiple QEMUFile source files, some of them need
> access to things that were defined in qemu-file.c, so create
> a -internal header for them.
> 
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Reviewed-by: Amit Shah <amit.shah@redhat.com>

		Amit

[Qemu-devel] [PATCH 3/3] Split the QEMU buffered file code out

[Dr. David Alan Gilbert (git)]

From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>

The splitting of qemu-file and addition of the buffered file landed
at the same time; so now split the buffered file code out.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
---
 migration/Makefile.objs   |   2 +-
 migration/qemu-file-buf.c | 486 ++++++++++++++++++++++++++++++++++++++++++++++
 migration/qemu-file.c     | 455 -------------------------------------------
 tests/Makefile            |   3 +-
 4 files changed, 489 insertions(+), 457 deletions(-)
 create mode 100644 migration/qemu-file-buf.c

diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index 681bae9..14e130a 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -1,6 +1,6 @@
 common-obj-y += migration.o migration-tcp.o
 common-obj-y += vmstate.o
-common-obj-y += qemu-file.o qemu-file-unix.o qemu-file-stdio.o
+common-obj-y += qemu-file.o qemu-file-buf.o qemu-file-unix.o qemu-file-stdio.o
 common-obj-$(CONFIG_RDMA) += migration-rdma.o
 common-obj-y += xbzrle.o
 
diff --git a/migration/qemu-file-buf.c b/migration/qemu-file-buf.c
new file mode 100644
index 0000000..d33dd44
--- /dev/null
+++ b/migration/qemu-file-buf.c
@@ -0,0 +1,486 @@
+/*
+ * QEMU System Emulator
+ *
+ * Copyright (c) 2003-2008 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+#include "qemu-common.h"
+#include "qemu/iov.h"
+#include "qemu/sockets.h"
+#include "block/coroutine.h"
+#include "migration/migration.h"
+#include "migration/qemu-file.h"
+#include "migration/qemu-file-internal.h"
+#include "trace.h"
+
+#define QSB_CHUNK_SIZE      (1 << 10)
+#define QSB_MAX_CHUNK_SIZE  (16 * QSB_CHUNK_SIZE)
+
+/**
+ * Create a QEMUSizedBuffer
+ * This type of buffer uses scatter-gather lists internally and
+ * can grow to any size. Any data array in the scatter-gather list
+ * can hold different amount of bytes.
+ *
+ * @buffer: Optional buffer to copy into the QSB
+ * @len: size of initial buffer; if @buffer is given, buffer must
+ *       hold at least len bytes
+ *
+ * Returns a pointer to a QEMUSizedBuffer or NULL on allocation failure
+ */
+QEMUSizedBuffer *qsb_create(const uint8_t *buffer, size_t len)
+{
+    QEMUSizedBuffer *qsb;
+    size_t alloc_len, num_chunks, i, to_copy;
+    size_t chunk_size = (len > QSB_MAX_CHUNK_SIZE)
+                        ? QSB_MAX_CHUNK_SIZE
+                        : QSB_CHUNK_SIZE;
+
+    num_chunks = DIV_ROUND_UP(len ? len : QSB_CHUNK_SIZE, chunk_size);
+    alloc_len = num_chunks * chunk_size;
+
+    qsb = g_try_new0(QEMUSizedBuffer, 1);
+    if (!qsb) {
+        return NULL;
+    }
+
+    qsb->iov = g_try_new0(struct iovec, num_chunks);
+    if (!qsb->iov) {
+        g_free(qsb);
+        return NULL;
+    }
+
+    qsb->n_iov = num_chunks;
+
+    for (i = 0; i < num_chunks; i++) {
+        qsb->iov[i].iov_base = g_try_malloc0(chunk_size);
+        if (!qsb->iov[i].iov_base) {
+            /* qsb_free is safe since g_free can cope with NULL */
+            qsb_free(qsb);
+            return NULL;
+        }
+
+        qsb->iov[i].iov_len = chunk_size;
+        if (buffer) {
+            to_copy = (len - qsb->used) > chunk_size
+                      ? chunk_size : (len - qsb->used);
+            memcpy(qsb->iov[i].iov_base, &buffer[qsb->used], to_copy);
+            qsb->used += to_copy;
+        }
+    }
+
+    qsb->size = alloc_len;
+
+    return qsb;
+}
+
+/**
+ * Free the QEMUSizedBuffer
+ *
+ * @qsb: The QEMUSizedBuffer to free
+ */
+void qsb_free(QEMUSizedBuffer *qsb)
+{
+    size_t i;
+
+    if (!qsb) {
+        return;
+    }
+
+    for (i = 0; i < qsb->n_iov; i++) {
+        g_free(qsb->iov[i].iov_base);
+    }
+    g_free(qsb->iov);
+    g_free(qsb);
+}
+
+/**
+ * Get the number of used bytes in the QEMUSizedBuffer
+ *
+ * @qsb: A QEMUSizedBuffer
+ *
+ * Returns the number of bytes currently used in this buffer
+ */
+size_t qsb_get_length(const QEMUSizedBuffer *qsb)
+{
+    return qsb->used;
+}
+
+/**
+ * Set the length of the buffer; the primary usage of this
+ * function is to truncate the number of used bytes in the buffer.
+ * The size will not be extended beyond the current number of
+ * allocated bytes in the QEMUSizedBuffer.
+ *
+ * @qsb: A QEMUSizedBuffer
+ * @new_len: The new length of bytes in the buffer
+ *
+ * Returns the number of bytes the buffer was truncated or extended
+ * to.
+ */
+size_t qsb_set_length(QEMUSizedBuffer *qsb, size_t new_len)
+{
+    if (new_len <= qsb->size) {
+        qsb->used = new_len;
+    } else {
+        qsb->used = qsb->size;
+    }
+    return qsb->used;
+}
+
+/**
+ * Get the iovec that holds the data for a given position @pos.
+ *
+ * @qsb: A QEMUSizedBuffer
+ * @pos: The index of a byte in the buffer
+ * @d_off: Pointer to an offset that this function will indicate
+ *         at what position within the returned iovec the byte
+ *         is to be found
+ *
+ * Returns the index of the iovec that holds the byte at the given
+ * index @pos in the byte stream; a negative number if the iovec
+ * for the given position @pos does not exist.
+ */
+static ssize_t qsb_get_iovec(const QEMUSizedBuffer *qsb,
+                             off_t pos, off_t *d_off)
+{
+    ssize_t i;
+    off_t curr = 0;
+
+    if (pos > qsb->used) {
+        return -1;
+    }
+
+    for (i = 0; i < qsb->n_iov; i++) {
+        if (curr + qsb->iov[i].iov_len > pos) {
+            *d_off = pos - curr;
+            return i;
+        }
+        curr += qsb->iov[i].iov_len;
+    }
+    return -1;
+}
+
+/*
+ * Convert the QEMUSizedBuffer into a flat buffer.
+ *
+ * Note: If at all possible, try to avoid this function since it
+ *       may unnecessarily copy memory around.
+ *
+ * @qsb: pointer to QEMUSizedBuffer
+ * @start: offset to start at
+ * @count: number of bytes to copy
+ * @buf: a pointer to a buffer to write into (at least @count bytes)
+ *
+ * Returns the number of bytes copied into the output buffer
+ */
+ssize_t qsb_get_buffer(const QEMUSizedBuffer *qsb, off_t start,
+                       size_t count, uint8_t *buffer)
+{
+    const struct iovec *iov;
+    size_t to_copy, all_copy;
+    ssize_t index;
+    off_t s_off;
+    off_t d_off = 0;
+    char *s;
+
+    if (start > qsb->used) {
+        return 0;
+    }
+
+    all_copy = qsb->used - start;
+    if (all_copy > count) {
+        all_copy = count;
+    } else {
+        count = all_copy;
+    }
+
+    index = qsb_get_iovec(qsb, start, &s_off);
+    if (index < 0) {
+        return 0;
+    }
+
+    while (all_copy > 0) {
+        iov = &qsb->iov[index];
+
+        s = iov->iov_base;
+
+        to_copy = iov->iov_len - s_off;
+        if (to_copy > all_copy) {
+            to_copy = all_copy;
+        }
+        memcpy(&buffer[d_off], &s[s_off], to_copy);
+
+        d_off += to_copy;
+        all_copy -= to_copy;
+
+        s_off = 0;
+        index++;
+    }
+
+    return count;
+}
+
+/**
+ * Grow the QEMUSizedBuffer to the given size and allocate
+ * memory for it.
+ *
+ * @qsb: A QEMUSizedBuffer
+ * @new_size: The new size of the buffer
+ *
+ * Return:
+ *    a negative error code in case of memory allocation failure
+ * or
+ *    the new size of the buffer. The returned size may be greater or equal
+ *    to @new_size.
+ */
+static ssize_t qsb_grow(QEMUSizedBuffer *qsb, size_t new_size)
+{
+    size_t needed_chunks, i;
+
+    if (qsb->size < new_size) {
+        struct iovec *new_iov;
+        size_t size_diff = new_size - qsb->size;
+        size_t chunk_size = (size_diff > QSB_MAX_CHUNK_SIZE)
+                             ? QSB_MAX_CHUNK_SIZE : QSB_CHUNK_SIZE;
+
+        needed_chunks = DIV_ROUND_UP(size_diff, chunk_size);
+
+        new_iov = g_try_new(struct iovec, qsb->n_iov + needed_chunks);
+        if (new_iov == NULL) {
+            return -ENOMEM;
+        }
+
+        /* Allocate new chunks as needed into new_iov */
+        for (i = qsb->n_iov; i < qsb->n_iov + needed_chunks; i++) {
+            new_iov[i].iov_base = g_try_malloc0(chunk_size);
+            new_iov[i].iov_len = chunk_size;
+            if (!new_iov[i].iov_base) {
+                size_t j;
+
+                /* Free previously allocated new chunks */
+                for (j = qsb->n_iov; j < i; j++) {
+                    g_free(new_iov[j].iov_base);
+                }
+                g_free(new_iov);
+
+                return -ENOMEM;
+            }
+        }
+
+        /*
+         * Now we can't get any allocation errors, copy over to new iov
+         * and switch.
+         */
+        for (i = 0; i < qsb->n_iov; i++) {
+            new_iov[i] = qsb->iov[i];
+        }
+
+        qsb->n_iov += needed_chunks;
+        g_free(qsb->iov);
+        qsb->iov = new_iov;
+        qsb->size += (needed_chunks * chunk_size);
+    }
+
+    return qsb->size;
+}
+
+/**
+ * Write into the QEMUSizedBuffer at a given position and a given
+ * number of bytes. This function will automatically grow the
+ * QEMUSizedBuffer.
+ *
+ * @qsb: A QEMUSizedBuffer
+ * @source: A byte array to copy data from
+ * @pos: The position within the @qsb to write data to
+ * @size: The number of bytes to copy into the @qsb
+ *
+ * Returns @size or a negative error code in case of memory allocation failure,
+ *           or with an invalid 'pos'
+ */
+ssize_t qsb_write_at(QEMUSizedBuffer *qsb, const uint8_t *source,
+                     off_t pos, size_t count)
+{
+    ssize_t rc = qsb_grow(qsb, pos + count);
+    size_t to_copy;
+    size_t all_copy = count;
+    const struct iovec *iov;
+    ssize_t index;
+    char *dest;
+    off_t d_off, s_off = 0;
+
+    if (rc < 0) {
+        return rc;
+    }
+
+    if (pos + count > qsb->used) {
+        qsb->used = pos + count;
+    }
+
+    index = qsb_get_iovec(qsb, pos, &d_off);
+    if (index < 0) {
+        return -EINVAL;
+    }
+
+    while (all_copy > 0) {
+        iov = &qsb->iov[index];
+
+        dest = iov->iov_base;
+
+        to_copy = iov->iov_len - d_off;
+        if (to_copy > all_copy) {
+            to_copy = all_copy;
+        }
+
+        memcpy(&dest[d_off], &source[s_off], to_copy);
+
+        s_off += to_copy;
+        all_copy -= to_copy;
+
+        d_off = 0;
+        index++;
+    }
+
+    return count;
+}
+
+/**
+ * Create a deep copy of the given QEMUSizedBuffer.
+ *
+ * @qsb: A QEMUSizedBuffer
+ *
+ * Returns a clone of @qsb or NULL on allocation failure
+ */
+QEMUSizedBuffer *qsb_clone(const QEMUSizedBuffer *qsb)
+{
+    QEMUSizedBuffer *out = qsb_create(NULL, qsb_get_length(qsb));
+    size_t i;
+    ssize_t res;
+    off_t pos = 0;
+
+    if (!out) {
+        return NULL;
+    }
+
+    for (i = 0; i < qsb->n_iov; i++) {
+        res =  qsb_write_at(out, qsb->iov[i].iov_base,
+                            pos, qsb->iov[i].iov_len);
+        if (res < 0) {
+            qsb_free(out);
+            return NULL;
+        }
+        pos += res;
+    }
+
+    return out;
+}
+
+typedef struct QEMUBuffer {
+    QEMUSizedBuffer *qsb;
+    QEMUFile *file;
+} QEMUBuffer;
+
+static int buf_get_buffer(void *opaque, uint8_t *buf, int64_t pos, int size)
+{
+    QEMUBuffer *s = opaque;
+    ssize_t len = qsb_get_length(s->qsb) - pos;
+
+    if (len <= 0) {
+        return 0;
+    }
+
+    if (len > size) {
+        len = size;
+    }
+    return qsb_get_buffer(s->qsb, pos, len, buf);
+}
+
+static int buf_put_buffer(void *opaque, const uint8_t *buf,
+                          int64_t pos, int size)
+{
+    QEMUBuffer *s = opaque;
+
+    return qsb_write_at(s->qsb, buf, pos, size);
+}
+
+static int buf_close(void *opaque)
+{
+    QEMUBuffer *s = opaque;
+
+    qsb_free(s->qsb);
+
+    g_free(s);
+
+    return 0;
+}
+
+const QEMUSizedBuffer *qemu_buf_get(QEMUFile *f)
+{
+    QEMUBuffer *p;
+
+    qemu_fflush(f);
+
+    p = f->opaque;
+
+    return p->qsb;
+}
+
+static const QEMUFileOps buf_read_ops = {
+    .get_buffer = buf_get_buffer,
+    .close =      buf_close,
+};
+
+static const QEMUFileOps buf_write_ops = {
+    .put_buffer = buf_put_buffer,
+    .close =      buf_close,
+};
+
+QEMUFile *qemu_bufopen(const char *mode, QEMUSizedBuffer *input)
+{
+    QEMUBuffer *s;
+
+    if (mode == NULL || (mode[0] != 'r' && mode[0] != 'w') ||
+        mode[1] != '\0') {
+        error_report("qemu_bufopen: Argument validity check failed");
+        return NULL;
+    }
+
+    s = g_malloc0(sizeof(QEMUBuffer));
+    if (mode[0] == 'r') {
+        s->qsb = input;
+    }
+
+    if (s->qsb == NULL) {
+        s->qsb = qsb_create(NULL, 0);
+    }
+    if (!s->qsb) {
+        g_free(s);
+        error_report("qemu_bufopen: qsb_create failed");
+        return NULL;
+    }
+
+
+    if (mode[0] == 'r') {
+        s->file = qemu_fopen_ops(s, &buf_read_ops);
+    } else {
+        s->file = qemu_fopen_ops(s, &buf_write_ops);
+    }
+    return s->file;
+}
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 671fba9..d2d4007 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -517,458 +517,3 @@ uint64_t qemu_get_be64(QEMUFile *f)
     v |= qemu_get_be32(f);
     return v;
 }
-
-#define QSB_CHUNK_SIZE      (1 << 10)
-#define QSB_MAX_CHUNK_SIZE  (16 * QSB_CHUNK_SIZE)
-
-/**
- * Create a QEMUSizedBuffer
- * This type of buffer uses scatter-gather lists internally and
- * can grow to any size. Any data array in the scatter-gather list
- * can hold different amount of bytes.
- *
- * @buffer: Optional buffer to copy into the QSB
- * @len: size of initial buffer; if @buffer is given, buffer must
- *       hold at least len bytes
- *
- * Returns a pointer to a QEMUSizedBuffer or NULL on allocation failure
- */
-QEMUSizedBuffer *qsb_create(const uint8_t *buffer, size_t len)
-{
-    QEMUSizedBuffer *qsb;
-    size_t alloc_len, num_chunks, i, to_copy;
-    size_t chunk_size = (len > QSB_MAX_CHUNK_SIZE)
-                        ? QSB_MAX_CHUNK_SIZE
-                        : QSB_CHUNK_SIZE;
-
-    num_chunks = DIV_ROUND_UP(len ? len : QSB_CHUNK_SIZE, chunk_size);
-    alloc_len = num_chunks * chunk_size;
-
-    qsb = g_try_new0(QEMUSizedBuffer, 1);
-    if (!qsb) {
-        return NULL;
-    }
-
-    qsb->iov = g_try_new0(struct iovec, num_chunks);
-    if (!qsb->iov) {
-        g_free(qsb);
-        return NULL;
-    }
-
-    qsb->n_iov = num_chunks;
-
-    for (i = 0; i < num_chunks; i++) {
-        qsb->iov[i].iov_base = g_try_malloc0(chunk_size);
-        if (!qsb->iov[i].iov_base) {
-            /* qsb_free is safe since g_free can cope with NULL */
-            qsb_free(qsb);
-            return NULL;
-        }
-
-        qsb->iov[i].iov_len = chunk_size;
-        if (buffer) {
-            to_copy = (len - qsb->used) > chunk_size
-                      ? chunk_size : (len - qsb->used);
-            memcpy(qsb->iov[i].iov_base, &buffer[qsb->used], to_copy);
-            qsb->used += to_copy;
-        }
-    }
-
-    qsb->size = alloc_len;
-
-    return qsb;
-}
-
-/**
- * Free the QEMUSizedBuffer
- *
- * @qsb: The QEMUSizedBuffer to free
- */
-void qsb_free(QEMUSizedBuffer *qsb)
-{
-    size_t i;
-
-    if (!qsb) {
-        return;
-    }
-
-    for (i = 0; i < qsb->n_iov; i++) {
-        g_free(qsb->iov[i].iov_base);
-    }
-    g_free(qsb->iov);
-    g_free(qsb);
-}
-
-/**
- * Get the number of used bytes in the QEMUSizedBuffer
- *
- * @qsb: A QEMUSizedBuffer
- *
- * Returns the number of bytes currently used in this buffer
- */
-size_t qsb_get_length(const QEMUSizedBuffer *qsb)
-{
-    return qsb->used;
-}
-
-/**
- * Set the length of the buffer; the primary usage of this
- * function is to truncate the number of used bytes in the buffer.
- * The size will not be extended beyond the current number of
- * allocated bytes in the QEMUSizedBuffer.
- *
- * @qsb: A QEMUSizedBuffer
- * @new_len: The new length of bytes in the buffer
- *
- * Returns the number of bytes the buffer was truncated or extended
- * to.
- */
-size_t qsb_set_length(QEMUSizedBuffer *qsb, size_t new_len)
-{
-    if (new_len <= qsb->size) {
-        qsb->used = new_len;
-    } else {
-        qsb->used = qsb->size;
-    }
-    return qsb->used;
-}
-
-/**
- * Get the iovec that holds the data for a given position @pos.
- *
- * @qsb: A QEMUSizedBuffer
- * @pos: The index of a byte in the buffer
- * @d_off: Pointer to an offset that this function will indicate
- *         at what position within the returned iovec the byte
- *         is to be found
- *
- * Returns the index of the iovec that holds the byte at the given
- * index @pos in the byte stream; a negative number if the iovec
- * for the given position @pos does not exist.
- */
-static ssize_t qsb_get_iovec(const QEMUSizedBuffer *qsb,
-                             off_t pos, off_t *d_off)
-{
-    ssize_t i;
-    off_t curr = 0;
-
-    if (pos > qsb->used) {
-        return -1;
-    }
-
-    for (i = 0; i < qsb->n_iov; i++) {
-        if (curr + qsb->iov[i].iov_len > pos) {
-            *d_off = pos - curr;
-            return i;
-        }
-        curr += qsb->iov[i].iov_len;
-    }
-    return -1;
-}
-
-/*
- * Convert the QEMUSizedBuffer into a flat buffer.
- *
- * Note: If at all possible, try to avoid this function since it
- *       may unnecessarily copy memory around.
- *
- * @qsb: pointer to QEMUSizedBuffer
- * @start: offset to start at
- * @count: number of bytes to copy
- * @buf: a pointer to a buffer to write into (at least @count bytes)
- *
- * Returns the number of bytes copied into the output buffer
- */
-ssize_t qsb_get_buffer(const QEMUSizedBuffer *qsb, off_t start,
-                       size_t count, uint8_t *buffer)
-{
-    const struct iovec *iov;
-    size_t to_copy, all_copy;
-    ssize_t index;
-    off_t s_off;
-    off_t d_off = 0;
-    char *s;
-
-    if (start > qsb->used) {
-        return 0;
-    }
-
-    all_copy = qsb->used - start;
-    if (all_copy > count) {
-        all_copy = count;
-    } else {
-        count = all_copy;
-    }
-
-    index = qsb_get_iovec(qsb, start, &s_off);
-    if (index < 0) {
-        return 0;
-    }
-
-    while (all_copy > 0) {
-        iov = &qsb->iov[index];
-
-        s = iov->iov_base;
-
-        to_copy = iov->iov_len - s_off;
-        if (to_copy > all_copy) {
-            to_copy = all_copy;
-        }
-        memcpy(&buffer[d_off], &s[s_off], to_copy);
-
-        d_off += to_copy;
-        all_copy -= to_copy;
-
-        s_off = 0;
-        index++;
-    }
-
-    return count;
-}
-
-/**
- * Grow the QEMUSizedBuffer to the given size and allocate
- * memory for it.
- *
- * @qsb: A QEMUSizedBuffer
- * @new_size: The new size of the buffer
- *
- * Return:
- *    a negative error code in case of memory allocation failure
- * or
- *    the new size of the buffer. The returned size may be greater or equal
- *    to @new_size.
- */
-static ssize_t qsb_grow(QEMUSizedBuffer *qsb, size_t new_size)
-{
-    size_t needed_chunks, i;
-
-    if (qsb->size < new_size) {
-        struct iovec *new_iov;
-        size_t size_diff = new_size - qsb->size;
-        size_t chunk_size = (size_diff > QSB_MAX_CHUNK_SIZE)
-                             ? QSB_MAX_CHUNK_SIZE : QSB_CHUNK_SIZE;
-
-        needed_chunks = DIV_ROUND_UP(size_diff, chunk_size);
-
-        new_iov = g_try_new(struct iovec, qsb->n_iov + needed_chunks);
-        if (new_iov == NULL) {
-            return -ENOMEM;
-        }
-
-        /* Allocate new chunks as needed into new_iov */
-        for (i = qsb->n_iov; i < qsb->n_iov + needed_chunks; i++) {
-            new_iov[i].iov_base = g_try_malloc0(chunk_size);
-            new_iov[i].iov_len = chunk_size;
-            if (!new_iov[i].iov_base) {
-                size_t j;
-
-                /* Free previously allocated new chunks */
-                for (j = qsb->n_iov; j < i; j++) {
-                    g_free(new_iov[j].iov_base);
-                }
-                g_free(new_iov);
-
-                return -ENOMEM;
-            }
-        }
-
-        /*
-         * Now we can't get any allocation errors, copy over to new iov
-         * and switch.
-         */
-        for (i = 0; i < qsb->n_iov; i++) {
-            new_iov[i] = qsb->iov[i];
-        }
-
-        qsb->n_iov += needed_chunks;
-        g_free(qsb->iov);
-        qsb->iov = new_iov;
-        qsb->size += (needed_chunks * chunk_size);
-    }
-
-    return qsb->size;
-}
-
-/**
- * Write into the QEMUSizedBuffer at a given position and a given
- * number of bytes. This function will automatically grow the
- * QEMUSizedBuffer.
- *
- * @qsb: A QEMUSizedBuffer
- * @source: A byte array to copy data from
- * @pos: The position within the @qsb to write data to
- * @size: The number of bytes to copy into the @qsb
- *
- * Returns @size or a negative error code in case of memory allocation failure,
- *           or with an invalid 'pos'
- */
-ssize_t qsb_write_at(QEMUSizedBuffer *qsb, const uint8_t *source,
-                     off_t pos, size_t count)
-{
-    ssize_t rc = qsb_grow(qsb, pos + count);
-    size_t to_copy;
-    size_t all_copy = count;
-    const struct iovec *iov;
-    ssize_t index;
-    char *dest;
-    off_t d_off, s_off = 0;
-
-    if (rc < 0) {
-        return rc;
-    }
-
-    if (pos + count > qsb->used) {
-        qsb->used = pos + count;
-    }
-
-    index = qsb_get_iovec(qsb, pos, &d_off);
-    if (index < 0) {
-        return -EINVAL;
-    }
-
-    while (all_copy > 0) {
-        iov = &qsb->iov[index];
-
-        dest = iov->iov_base;
-
-        to_copy = iov->iov_len - d_off;
-        if (to_copy > all_copy) {
-            to_copy = all_copy;
-        }
-
-        memcpy(&dest[d_off], &source[s_off], to_copy);
-
-        s_off += to_copy;
-        all_copy -= to_copy;
-
-        d_off = 0;
-        index++;
-    }
-
-    return count;
-}
-
-/**
- * Create a deep copy of the given QEMUSizedBuffer.
- *
- * @qsb: A QEMUSizedBuffer
- *
- * Returns a clone of @qsb or NULL on allocation failure
- */
-QEMUSizedBuffer *qsb_clone(const QEMUSizedBuffer *qsb)
-{
-    QEMUSizedBuffer *out = qsb_create(NULL, qsb_get_length(qsb));
-    size_t i;
-    ssize_t res;
-    off_t pos = 0;
-
-    if (!out) {
-        return NULL;
-    }
-
-    for (i = 0; i < qsb->n_iov; i++) {
-        res =  qsb_write_at(out, qsb->iov[i].iov_base,
-                            pos, qsb->iov[i].iov_len);
-        if (res < 0) {
-            qsb_free(out);
-            return NULL;
-        }
-        pos += res;
-    }
-
-    return out;
-}
-
-typedef struct QEMUBuffer {
-    QEMUSizedBuffer *qsb;
-    QEMUFile *file;
-} QEMUBuffer;
-
-static int buf_get_buffer(void *opaque, uint8_t *buf, int64_t pos, int size)
-{
-    QEMUBuffer *s = opaque;
-    ssize_t len = qsb_get_length(s->qsb) - pos;
-
-    if (len <= 0) {
-        return 0;
-    }
-
-    if (len > size) {
-        len = size;
-    }
-    return qsb_get_buffer(s->qsb, pos, len, buf);
-}
-
-static int buf_put_buffer(void *opaque, const uint8_t *buf,
-                          int64_t pos, int size)
-{
-    QEMUBuffer *s = opaque;
-
-    return qsb_write_at(s->qsb, buf, pos, size);
-}
-
-static int buf_close(void *opaque)
-{
-    QEMUBuffer *s = opaque;
-
-    qsb_free(s->qsb);
-
-    g_free(s);
-
-    return 0;
-}
-
-const QEMUSizedBuffer *qemu_buf_get(QEMUFile *f)
-{
-    QEMUBuffer *p;
-
-    qemu_fflush(f);
-
-    p = f->opaque;
-
-    return p->qsb;
-}
-
-static const QEMUFileOps buf_read_ops = {
-    .get_buffer = buf_get_buffer,
-    .close =      buf_close,
-};
-
-static const QEMUFileOps buf_write_ops = {
-    .put_buffer = buf_put_buffer,
-    .close =      buf_close,
-};
-
-QEMUFile *qemu_bufopen(const char *mode, QEMUSizedBuffer *input)
-{
-    QEMUBuffer *s;
-
-    if (mode == NULL || (mode[0] != 'r' && mode[0] != 'w') ||
-        mode[1] != '\0') {
-        error_report("qemu_bufopen: Argument validity check failed");
-        return NULL;
-    }
-
-    s = g_malloc0(sizeof(QEMUBuffer));
-    if (mode[0] == 'r') {
-        s->qsb = input;
-    }
-
-    if (s->qsb == NULL) {
-        s->qsb = qsb_create(NULL, 0);
-    }
-    if (!s->qsb) {
-        g_free(s);
-        error_report("qemu_bufopen: qsb_create failed");
-        return NULL;
-    }
-
-
-    if (mode[0] == 'r') {
-        s->file = qemu_fopen_ops(s, &buf_read_ops);
-    } else {
-        s->file = qemu_fopen_ops(s, &buf_write_ops);
-    }
-    return s->file;
-}
diff --git a/tests/Makefile b/tests/Makefile
index 3a03979..3f4c0d7 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -258,7 +258,8 @@ tests/test-qdev-global-props$(EXESUF): tests/test-qdev-global-props.o \
 	$(test-qapi-obj-y) \
 	libqemuutil.a libqemustub.a
 tests/test-vmstate$(EXESUF): tests/test-vmstate.o \
-	migration/vmstate.o migration/qemu-file.o migration/qemu-file-unix.o \
+	migration/vmstate.o migration/qemu-file.o migration/qemu-file-buf.o \
+        migration/qemu-file-unix.o \
 	libqemuutil.a libqemustub.a
 
 tests/test-qapi-types.c tests/test-qapi-types.h :\
-- 
1.9.3

Re: [Qemu-devel] [PATCH 3/3] Split the QEMU buffered file code out

[Juan Quintela]

"Dr. David Alan Gilbert (git)" <dgilbert@redhat.com> wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
>
> The splitting of qemu-file and addition of the buffered file landed
> at the same time; so now split the buffered file code out.
>
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Reviewed-by: Juan Quintela <quintela@redhat.com>

Re: [Qemu-devel] [PATCH 3/3] Split the QEMU buffered file code out

[Amit Shah]

On (Thu) 16 Oct 2014 [08:53:54], Dr. David Alan Gilbert (git) wrote:
> From: "Dr. David Alan Gilbert" <dgilbert@redhat.com>
> 
> The splitting of qemu-file and addition of the buffered file landed
> at the same time; so now split the buffered file code out.
> 
> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

Reviewed-by: Amit Shah <amit.shah@redhat.com>


		Amit