Re: [Qemu-devel] [patch] non-blocking disk IO

Re: [Qemu-devel] [patch] non-blocking disk IO

[John Coiner]

Magnus,

I don't think the Windows 2000 install hack will ever be obsolete.

The installer assumes that a hard disk will take nonzero time to read 
some data. QEMU always services a read in zero-guest-time. (With the 
nonblocking IO patch, zero-guest-time reads still occur, when the 
requested data is in the host's file cache.)

I doubt the IDE spec allows Windows to make this assumption... but the 
assumption is there, and we work around it by adding a delay that's 
visible to the guest.

-- John

Re: [Qemu-devel] [patch] non-blocking disk IO

[Troy Benjegerdes]

On Mon, Oct 03, 2005 at 08:41:46AM -0400, John Coiner wrote:
> 
> Magnus,
> 
> I don't think the Windows 2000 install hack will ever be obsolete.
> 
> The installer assumes that a hard disk will take nonzero time to read 
> some data. QEMU always services a read in zero-guest-time. (With the 
> nonblocking IO patch, zero-guest-time reads still occur, when the 
> requested data is in the host's file cache.)
> 
> I doubt the IDE spec allows Windows to make this assumption... but the 
> assumption is there, and we work around it by adding a delay that's 
> visible to the guest.

There are other bugs in the IDE emulation..

I need to try this latest patch, but with the previous DMA patch
(without non-blocking support) having a disk image on an AFS filesystem
just did not work at all.

I am also haveing trouble getting a fresh win2k install under qemu to actually
be able to run windows update.

Re: [Qemu-devel] [patch] non-blocking disk IO

[John Coiner]

Troy Benjegerdes wrote:
> I am also haveing trouble getting a fresh win2k install under qemu to actually
> be able to run windows update.

I had to download and install Win2k SP4, then Win2k SP4 "Hotfixes," and 
also an IE6 upgrade, before windows update ran.

-john

Re: [Qemu-devel] [patch] non-blocking disk IO

[Troy Benjegerdes]

On Mon, Oct 03, 2005 at 11:16:18PM -0400, John Coiner wrote:
> 
> 
> Troy Benjegerdes wrote:
> >I am also haveing trouble getting a fresh win2k install under qemu to 
> >actually
> >be able to run windows update.
> 
> I had to download and install Win2k SP4, then Win2k SP4 "Hotfixes," and 
> also an IE6 upgrade, before windows update ran.

Are you running with a tun/tap device, or -user-net ? I'm still unable
to get windowsupdate to work. Getting it to work in vmware seems to work
on the first try.

Re: [Qemu-devel] [patch] non-blocking disk IO

[John Coiner]

>>I had to download and install Win2k SP4, then Win2k SP4 "Hotfixes," and 
>>also an IE6 upgrade, before windows update ran.
> 
> Are you running with a tun/tap device, or -user-net ? I'm still unable
> to get windowsupdate to work. Getting it to work in vmware seems to work
> on the first try.

I've been running -user-net.

Re: [Qemu-devel] [patch] non-blocking disk IO

[Troy Benjegerdes]

On Tue, Oct 04, 2005 at 06:11:41PM -0400, John Coiner wrote:
> 
> >>I had to download and install Win2k SP4, then Win2k SP4 "Hotfixes," and 
> >>also an IE6 upgrade, before windows update ran.
> >
> >Are you running with a tun/tap device, or -user-net ? I'm still unable
> >to get windowsupdate to work. Getting it to work in vmware seems to work
> >on the first try.
> 
> I've been running -user-net.

Okay, after some magic combination of SP4, hotfixes, and IE6 upgrade,
plus some fiddling, and a full rebuild of qemu with your latest patch, I
have a win2k image that seems to be able to run windows update
successfully.

Now, if I could just get vnc to be more reponsive when running qemu on a
fast server on the other side of a DSL line ;)

[Qemu-devel] [patch] non-blocking disk IO

[John Coiner]

Non-blocking disk IO now works for any type of disk image, not just 
"raw" format. There is no longer any format-specific code in the patch:

http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html

You might want this patch if:
  * you run a multitasking guest OS,
  * you access a disk sometimes, and
  * you wouldn't mind if QEMU ran a little faster.

Why I have not got feedback in droves I do not understand ;)

-John

Re: [Qemu-devel] [patch] non-blocking disk IO

[Magnus Damm]

John,

On 10/3/05, John Coiner <jcoiner@stanfordalumni.org> wrote:
>
> Non-blocking disk IO now works for any type of disk image, not just
> "raw" format. There is no longer any format-specific code in the patch:
>
> http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html
>
> You might want this patch if:
>   * you run a multitasking guest OS,
>   * you access a disk sometimes, and
>   * you wouldn't mind if QEMU ran a little faster.
>
> Why I have not got feedback in droves I do not understand ;)

I'd love to try out your patch, but I have too little free time as
always. I have very high hopes for this patch and I do think that this
need to be done somehow. One thing that I'd really like to see is if
this patch makes the w2k install hack obsolete...

Keeep up the good work,

/ magnus

Re: [Qemu-devel] [patch] non-blocking disk IO

[Elefterios Stamatogiannakis]

With which pthreads library have you compiled it under windows?

I've tried it with the one here:

ftp://sources.redhat.com/pub/pthreads-win32/

but it didn't compile.

teris.

John Coiner wrote:
> 
> Non-blocking disk IO now works for any type of disk image, not just 
> "raw" format. There is no longer any format-specific code in the patch:
> 
> http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html
> 
> You might want this patch if:
>  * you run a multitasking guest OS,
>  * you access a disk sometimes, and
>  * you wouldn't mind if QEMU ran a little faster.
> 
> Why I have not got feedback in droves I do not understand ;)
> 
> -John
> 
> 
> 
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel

Re: [Qemu-devel] [patch] non-blocking disk IO

[Christian MICHON]

I managed to make it work (qemu+non blocking IO on windows host),
with a rough estimation of 10% speed increase at the early stage of
windows setup. I expect more once windows installs itself in true
multitasking mode. :)

What you need to do is:
- download pthreads-w32-2-6-0-release.tar.gz and compile it from scratch
 (make clean GC)
- you also need to add the following lines inside block.c (usleep is not
present in win32/mingw32)

 #ifdef WIN32
void usleep(unsigned long usec)
{
    Sleep(usec/1000);
}
#endif

- you need to update the link to include "-lpthreadGC2 -lz", and
keep a copy of the pthreadGC2.dll where qemu.exe will be
(to pu it just in from of "-lz" is just my personal suggestion)

HIH,
Christian

On 10/3/05, Elefterios Stamatogiannakis <estama@dblab.ece.ntua.gr> wrote:
> With which pthreads library have you compiled it under windows?
>
> I've tried it with the one here:
>
> ftp://sources.redhat.com/pub/pthreads-win32/
>
> but it didn't compile.
>
> teris.

Re: [Qemu-devel] [patch] non-blocking disk IO

[John Coiner]

Whoops. I had the mistaken belief that Qemu already used pthreads.

So, the makefiles and configure script need to handle pthreads somehow 
(especially on windows.) Pthreads-w32 is refusing to 'make install' 
itself into standard lib and include directories. Unlike SDL, there is 
no 'pthread-config' to tell us where it lives.

Suppose that on Windows, we tell people to unpack the pthread 
distribution within the qemu tree. Qemu's configure script could find 
it, build a static pthread library, and link that into Qemu.

This removes the need for the user to:
  * read pthreads-w32 documents,
  * issue a separate build command for pthreads-w32,
  * get confused before realizing that pthreads-w32 doesn't really 
support 'make install',
  * hack at qemu until it points to the uninstalled pthreads dir,
  * copy the pthreads DLL into c:\windows

I'll look at adding this to the configure script.

-john



Christian MICHON wrote:
> I managed to make it work (qemu+non blocking IO on windows host),
> with a rough estimation of 10% speed increase at the early stage of
> windows setup. I expect more once windows installs itself in true
> multitasking mode. :)
> 
> What you need to do is:
> - download pthreads-w32-2-6-0-release.tar.gz and compile it from scratch
>  (make clean GC)
> - you also need to add the following lines inside block.c (usleep is not
> present in win32/mingw32)
> 
>  #ifdef WIN32
> void usleep(unsigned long usec)
> {
>     Sleep(usec/1000);
> }
> #endif
> 
> - you need to update the link to include "-lpthreadGC2 -lz", and
> keep a copy of the pthreadGC2.dll where qemu.exe will be
> (to pu it just in from of "-lz" is just my personal suggestion)
> 
> HIH,
> Christian
> 
> On 10/3/05, Elefterios Stamatogiannakis <estama@dblab.ece.ntua.gr> wrote:
> 
>>With which pthreads library have you compiled it under windows?
>>
>>I've tried it with the one here:
>>
>>ftp://sources.redhat.com/pub/pthreads-win32/
>>
>>but it didn't compile.
>>
>>teris.
> 
> 
> 
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel
> 
> 

Re: [Qemu-devel] [patch] non-blocking disk IO

[Christian MICHON]

I did not manage a static link on win32 yesterday. I needed the
pthread dll in the same directory as qemu.exe.

I'll look more into it.

On 10/4/05, John Coiner <jcoiner@stanfordalumni.org> wrote:
> Suppose that on Windows, we tell people to unpack the pthread
> distribution within the qemu tree. Qemu's configure script could find
> it, build a static pthread library, and link that into Qemu.

Re: [Qemu-devel] [patch] non-blocking disk IO

[Elefterios Stamatogiannakis]

Thanks for the info.

>>  #ifdef WIN32
>> void usleep(unsigned long usec)
>> {
>>     Sleep(usec/1000);
>> }
>> #endif
>>

The division "usec/1000" is an expensive operation on any processor.

Maybe it would be better just to use "_sleep(1)" which is analogous to 
usleep(1000) ?

This is what i did to have it compiled and it worked like a charm.

teris.

Re: [Qemu-devel] [patch] non-blocking disk IO

[Christian MICHON]

indeed, this was a quick and dirty hack :)
I just managed to compile pthreadGC2 statically...
pthread.h has to be modified to include extra
PTW32_STATIC_LIB info, like this:

#define PTW32_VERSION 2,6,0,0
#define PTW32_VERSION_STRING "2, 6, 0, 0\0"
#define PTW32_STATIC_LIB 1

I'll test it later today and will report realistic benchmarks
on windows hosts.

On 10/4/05, Elefterios Stamatogiannakis <estama@dblab.ece.ntua.gr> wrote:
> Thanks for the info.
>
> >>  #ifdef WIN32
> >> void usleep(unsigned long usec)
> >> {
> >>     Sleep(usec/1000);
> >> }
> >> #endif
> >>
>
> The division "usec/1000" is an expensive operation on any processor.
>
> Maybe it would be better just to use "_sleep(1)" which is analogous to
> usleep(1000) ?
>
> This is what i did to have it compiled and it worked like a charm.
>
> teris.
>
>
>


--
Christian

Re: [Qemu-devel] [patch] non-blocking disk IO

[Christian MICHON]

hmmm....

the static binary crashes, the one with ptreadGC2.dll is ok.

What I tried to measure, on windows host, kqemu-0.7.2
loaded, was:
(1) tar -jxf linux-2.6.10.tar.bz2
(2) make allnoconfig; make

I confirm ~18% speed improvement in case (1), but no change
more or less for case (2). I believe maybe someone with a
windows host SMP could get different figures. If so, please
update these accordingly and let the list know.

Indeed, threading on a busy single CPU doesn't make it faster.
A kernel compilation with kqemu makes full use of the CPU. :(

I think we'll benefit mostly when copying files or setup OSes when
the CPU is quite in IDLE state. I've no figures without kqemu yet.

At least, it's a good start for speed improvement.
Thanks for the patch! :)

On 10/4/05, Christian MICHON <christian.michon@gmail.com> wrote:
> indeed, this was a quick and dirty hack :)
> I just managed to compile pthreadGC2 statically...
> pthread.h has to be modified to include extra
> PTW32_STATIC_LIB info, like this:
>
> #define PTW32_VERSION 2,6,0,0
> #define PTW32_VERSION_STRING "2, 6, 0, 0\0"
> #define PTW32_STATIC_LIB 1
>
> I'll test it later today and will report realistic benchmarks
> on windows hosts.
>

--
Christian

Re: [Qemu-devel] [patch] non-blocking disk IO

[Jens Axboe]

On Tue, Oct 04 2005, Christian MICHON wrote:
> hmmm....
> 
> the static binary crashes, the one with ptreadGC2.dll is ok.
> 
> What I tried to measure, on windows host, kqemu-0.7.2
> loaded, was:
> (1) tar -jxf linux-2.6.10.tar.bz2
> (2) make allnoconfig; make
> 
> I confirm ~18% speed improvement in case (1), but no change
> more or less for case (2). I believe maybe someone with a
> windows host SMP could get different figures. If so, please
> update these accordingly and let the list know.
> 
> Indeed, threading on a busy single CPU doesn't make it faster.

That's not quite true. The io thread basically takes no real CPU, but it
often has to wait for io to become available. So even (especially!) for
a single cpu system, you definitely want to make sure that blocking io
doesn't eat away cpu cycles from a cpu bound case like the compile.

If you did the kernel compile with a hot disk cache, I'm not surprised
you're not seeing a performance benefit of the non-blocking io patch.
Even for a cold cache compile it will generally be cpu bound.

-- 
Jens Axboe

Re: [Qemu-devel] [patch] non-blocking disk IO

[John Coiner]

Christian MICHON wrote:
> hmmm....
> 
> the static binary crashes, the one with ptreadGC2.dll is ok.
> 
> What I tried to measure, on windows host, kqemu-0.7.2
> loaded, was:
> (1) tar -jxf linux-2.6.10.tar.bz2
> (2) make allnoconfig; make
> 
> I confirm ~18% speed improvement in case (1), but no change
> more or less for case (2).

(1) is a parallel task -- 'tar' and 'bunzip2' run concurrently. You see 
a benefit here when one process can use the CPU while the other is 
blocked waiting for IO.

(2) is a serial task. It can't benefit from non-blocking IO. 'make -j 2' 
should benefit, in cases where the host file cache is cold.

-- john

Re: [Qemu-devel] [patch] non-blocking disk IO

[Christian MICHON]

Then all is good and I should not use (2) as a benchmark.
Thanks for clarifying :)

On 10/4/05, John Coiner <jcoiner@stanfordalumni.org> wrote:
> (1) is a parallel task -- 'tar' and 'bunzip2' run concurrently. You see
> a benefit here when one process can use the CPU while the other is
> blocked waiting for IO.
>
> (2) is a serial task. It can't benefit from non-blocking IO. 'make -j 2'
> should benefit, in cases where the host file cache is cold.
--
Christian

Re: [Qemu-devel] [patch] non-blocking disk IO

[Jens Axboe]

On Mon, Oct 03 2005, John Coiner wrote:
> 
> Non-blocking disk IO now works for any type of disk image, not just 
> "raw" format. There is no longer any format-specific code in the patch:
> 
> http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html
> 
> You might want this patch if:
>  * you run a multitasking guest OS,
>  * you access a disk sometimes, and
>  * you wouldn't mind if QEMU ran a little faster.
> 
> Why I have not got feedback in droves I do not understand ;)

Why not use aio for this instead, seems like a better fit than spawning
a thread per block device? That would still require a thread for
handling completions, but you could easily just use a single completion
thread for all devices for this as it would not need to do any real
work.

-- 
Jens Axboe

Re: [Qemu-devel] [patch] non-blocking disk IO

[John Coiner]

Jens Axboe wrote:
  > Why not use aio for this instead, seems like a better fit than spawning
> a thread per block device? That would still require a thread for
> handling completions, but you could easily just use a single completion
> thread for all devices for this as it would not need to do any real
> work.

Pthreads vs. AIO is an open question.

The pthreads implementation works with all disk image formats, without 
modifying any format-specific code. To use AIO, each image format driver 
would have to be modified.

QEMU already uses pthreads, therefore using pthreads does not add a new 
dependency on the host platform. (Or so I thought. The build on Windows 
broke anyway. Haha... my bad...)

My questions are:
  * Which QEMU host platforms support Posix AIO?
  * Which host platform has the slowest pthreads library, and how many 
cycles will it waste on pthreads overhead?
  * What is the likely performance benefit of using AIO?

When I have time, I'll hack AIO into the "raw" image driver, and take 
some benchmarks on a linux host.

Thanks.

-John

Re: [Qemu-devel] [patch] non-blocking disk IO

[Jens Axboe]

On Mon, Oct 03 2005, John Coiner wrote:
> 
> 
> Jens Axboe wrote:
>  > Why not use aio for this instead, seems like a better fit than spawning
> >a thread per block device? That would still require a thread for
> >handling completions, but you could easily just use a single completion
> >thread for all devices for this as it would not need to do any real
> >work.
> 
> Pthreads vs. AIO is an open question.
> 
> The pthreads implementation works with all disk image formats, without 
> modifying any format-specific code. To use AIO, each image format driver 
> would have to be modified.

That is true, I appreciate why you did it the way you did. Using libaio
would be more invasive, indeed.

> QEMU already uses pthreads, therefore using pthreads does not add a new 
> dependency on the host platform. (Or so I thought. The build on Windows 
> broke anyway. Haha... my bad...)
> 
> My questions are:
>  * Which QEMU host platforms support Posix AIO?
>  * Which host platform has the slowest pthreads library, and how many 
> cycles will it waste on pthreads overhead?
>  * What is the likely performance benefit of using AIO?
> 
> When I have time, I'll hack AIO into the "raw" image driver, and take 
> some benchmarks on a linux host.

Using aio would result in cleaner design in the end, but it would
require extra host specific code for the abstraction. It's also
non-buffered, which may or may not be a win for qemu (it should be,
would be trivial to test this with the existing code already).

For the normal case of a single or two block devices, performance of the
threaded vs libaio model will likely be the same. For many block
devices, libaio would definitely be a big improvement.

At least on Linux, the posix aio model is not worth spending time on, as
it just relies on threading as well on the glibc side. Then you might as
well just keep the design you have now.

BTW, thanks for starting this work, it's sorely needed! A future
experiment would be to write a sata/scsi qemu driver so you could do
sane command queueing. Since file layout isn't always perfect, that
should further increase the io performance.

-- 
Jens Axboe

Re: [Qemu-devel] [patch] non-blocking disk IO

[Troy Benjegerdes]

On Mon, Oct 03, 2005 at 03:58:40PM +0200, Jens Axboe wrote:
> On Mon, Oct 03 2005, John Coiner wrote:
> > 
> > Non-blocking disk IO now works for any type of disk image, not just 
> > "raw" format. There is no longer any format-specific code in the patch:
> > 
> > http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html
> > 
> > You might want this patch if:
> >  * you run a multitasking guest OS,
> >  * you access a disk sometimes, and
> >  * you wouldn't mind if QEMU ran a little faster.
> > 
> > Why I have not got feedback in droves I do not understand ;)
> 
> Why not use aio for this instead, seems like a better fit than spawning
> a thread per block device? That would still require a thread for
> handling completions, but you could easily just use a single completion
> thread for all devices for this as it would not need to do any real
> work.

Why is there a posix async IO 'standard', but a different linux AIO standard?
Having something that wraps both Linux-native AIO and posix async I/O
might make supporting it a little easier to swallow on cross-platform
applications.

Re: [Qemu-devel] [patch] non-blocking disk IO

[John Coiner]

> Why is there a posix async IO 'standard', but a different linux AIO standard?

Linux has a Posix AIO library that sits on top of linux's native AIO:

   http://www.bullopensource.org/posix/

It's pretty new, version 0.6. We might not want qemu to depend on it, 
because most distros probably do not bundle it yet, so it would be an 
extra dependency for all new users to chase down.

-john

Re: [Qemu-devel] [patch] non-blocking disk IO

[Henrik Nordstrom]

On Mon, 3 Oct 2005, Troy Benjegerdes wrote:

> Why is there a posix async IO 'standard', but a different linux AIO standard?

Because the Posix AIO sucks quite badly both to implement and use.

But all Linuxes I know of have a Posix AIO library. Some even have a Posix 
AIO library using Linux AIO.

> Having something that wraps both Linux-native AIO and posix async I/O
> might make supporting it a little easier to swallow on cross-platform
> applications.

If you anyway plan on having Posix AIO support then go for the Posix AIO 
interface. The performance reasons why Linux AIO exists is very unlikely 
to be an issue to qemu as you need to be quite I/O intensive to see any 
performance difference.

Regards
Henrik

Re: [Qemu-devel] [patch] non-blocking disk IO

[Troy Benjegerdes]

> 
> If you anyway plan on having Posix AIO support then go for the Posix AIO 
> interface. The performance reasons why Linux AIO exists is very unlikely 
> to be an issue to qemu as you need to be quite I/O intensive to see any 
> performance difference.

Ideally, we should be able to use a Posix AIO support libary that uses
native Linux AIO underneath.

What we want is to be able to have the guest OS request some DMA
I/O operation, and have qemu be able to use AIO so that the actual disk
hardware can dump the data directly in the pages the userspace process
on the guest OS ends up wanting it in, avoiding several expensive memcopy
and context switch operations.

Re: [Qemu-devel] [patch] non-blocking disk IO

[Jens Axboe]

On Tue, Oct 04 2005, Troy Benjegerdes wrote:
> What we want is to be able to have the guest OS request some DMA
> I/O operation, and have qemu be able to use AIO so that the actual disk
> hardware can dump the data directly in the pages the userspace process
> on the guest OS ends up wanting it in, avoiding several expensive memcopy
> and context switch operations.

That should be easy enough to do already, with or without the
nonblocking patch. Just make sure to open the files O_DIRECT and align
the io buffers and lengths. With a 2.6 host, you can usually get away
with 512-b aligment, on 2.4 you may have to ensure 1k/4k alignment.

-- 
Jens Axboe

Re: [Qemu-devel] [patch] non-blocking disk IO

[Fabrice Bellard]

Thank you for the patch. I'll look at it ASAP. If there is no regression 
(in terms of bugs and performance) I will include it soon: I consider 
that non blocking I/O is now a critical feature in QEMU in order to get 
better performances.

Fabrice.

John Coiner wrote:
> 
> Non-blocking disk IO now works for any type of disk image, not just 
> "raw" format. There is no longer any format-specific code in the patch:
> 
> http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html
> 
> You might want this patch if:
>  * you run a multitasking guest OS,
>  * you access a disk sometimes, and
>  * you wouldn't mind if QEMU ran a little faster.
> 
> Why I have not got feedback in droves I do not understand ;)
> 
> -John
> 
> 
> 
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel
> 
> 

Re: [Qemu-devel] [patch] non-blocking disk IO

[Brad Campbell]

John Coiner wrote:
> 
> Non-blocking disk IO now works for any type of disk image, not just 
> "raw" format. There is no longer any format-specific code in the patch:
> 
> http://people.brandeis.edu/~jcoiner/qemu_idedma/qemu_dma_patch.html
> 
> You might want this patch if:
>  * you run a multitasking guest OS,
>  * you access a disk sometimes, and
>  * you wouldn't mind if QEMU ran a little faster.
> 
> Why I have not got feedback in droves I do not understand ;)

I have 3 or 4 images here (of different flavours of windows from 95 -> XP) that I have base images 
about 4G and then I use qcow files to do my testing and other damaging stuff to.
With this patch qemu will not boot the qcow files but will boot the base images (I copy them and 
then try to boot to test). As soon as I patch -R && make, qemu will boot the qcow files no problems..

Let me clarify that..
qemu-img create -f qcow xp.img 4G
qemu -cdrom xp.iso -hda xp.img -boot d (Install XP from scratch)
qemu-img create -b xp.img -f qcow xp2.img
qemu -hda xp2.img

This gives me an invalid hard disk error. If I revert the patch I can boot from xp2.img with no 
issues. With or without the patch I can boot from xp.img.

I have tried installing XP with and without the patch and that works fine..

Regards,
Brad
-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams

Re: [Qemu-devel] [patch] non-blocking disk IO

[John Coiner]

Brad Campbell wrote:
> With this patch qemu will not boot the qcow files but will boot the base 
> images (I copy them and then try to boot to test). As soon as I patch -R 
> && make, qemu will boot the qcow files no problems..
> 
> Let me clarify that..
> qemu-img create -f qcow xp.img 4G
> qemu -cdrom xp.iso -hda xp.img -boot d (Install XP from scratch)
> qemu-img create -b xp.img -f qcow xp2.img
> qemu -hda xp2.img
> 
> This gives me an invalid hard disk error. If I revert the patch I can 
> boot from xp2.img with no issues. With or without the patch I can boot 
> from xp.img.
> 
> I have tried installing XP with and without the patch and that works fine..
> 
> Regards,
> Brad

Thanks for the report. I was able to reproduce this.

The problem was due to missing Makefile dependencies. The patch modifies 
'block_int.h', which is #included in several '.c' files. The file 
'block.c' recompiled at 'make' time, but 'block-qcow.c' didn't. The 
resulting '.o' files had different opinions about what a 
BlockDriverState struct looked like, which broke the qcow code.

Unaware of this, I added a printf to 'block-qcow.c', and the printf 
fixed the problem! The man page for printf doesn't mention this... and 
then I was enlightened. :P

To be safe, try a 'make clean ; make install' and you should be all 
good. Let me know if not.

-- John

Re: [Qemu-devel] [patch] non-blocking disk IO

[Brad Campbell]

John Coiner wrote:

> 
> Thanks for the report. I was able to reproduce this.
> 
> The problem was due to missing Makefile dependencies. The patch modifies 
> 'block_int.h', which is #included in several '.c' files. The file 
> 'block.c' recompiled at 'make' time, but 'block-qcow.c' didn't. The 
> resulting '.o' files had different opinions about what a 
> BlockDriverState struct looked like, which broke the qcow code.

Having had it running for a while I did some performance tests and found no performance gains _at 
all_, but then I'm running my entire system over the network on an NFS filesystem (this machine has 
no disks in it).

*NOW* having said that, one of the tricks I do to keep qemu from "going to sleep" when running a 
long process like a big software install (sometimes it appears to stall until I move the mouse 
around a bit) is to ping the machine constantly from the host.
Prior to this patch I would get pings that varied from 1ms to 250ms. With this patch I get a spread 
from about 1ms to 15ms tops. So it has smoothed out the operation of the host noticably (also 
noticable in screen redraws and UI responsiveness while under a heavy disk load).

No performance for me, but it just "feels" better to use all round.

So, no functional regressions, better usability but no performance here (probably due to running the 
whole thing over nfs)
One day I'll get a local disk and test it out.

As a lark, I also did a fresh compile of the BOCHS latest CVS bios with the apm and dma patches 
included. It appears to give me a slight speed improvement on booting my XP guest, but given the 
nature of my system that could be statistical noise.. If anyone wants to give it a whirl I have put 
my newly compiled bios here http://fnarfbargle.dyndns.org:81/bios

Regards,
Brad
-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams

Re: [Qemu-devel] [patch] non-blocking disk IO

[Troy Benjegerdes]

On Wed, Oct 05, 2005 at 03:27:18PM +0400, Brad Campbell wrote:
> John Coiner wrote:
> 
> >
> >Thanks for the report. I was able to reproduce this.
> >
> >The problem was due to missing Makefile dependencies. The patch modifies 
> >'block_int.h', which is #included in several '.c' files. The file 
> >'block.c' recompiled at 'make' time, but 'block-qcow.c' didn't. The 
> >resulting '.o' files had different opinions about what a 
> >BlockDriverState struct looked like, which broke the qcow code.
> 
> Having had it running for a while I did some performance tests and found no 
> performance gains _at all_, but then I'm running my entire system over the 
> network on an NFS filesystem (this machine has no disks in it).

What network do you have between the client and nfs server? Also, what
throughput do you see to the NFS server? (and what are the nfs 'wsize'
and 'rsize' settings)

> *NOW* having said that, one of the tricks I do to keep qemu from "going to 
> sleep" when running a long process like a big software install (sometimes 
> it appears to stall until I move the mouse around a bit) is to ping the 
> machine constantly from the host.
> Prior to this patch I would get pings that varied from 1ms to 250ms. With 
> this patch I get a spread from about 1ms to 15ms tops. So it has smoothed 
> out the operation of the host noticably (also noticable in screen redraws 
> and UI responsiveness while under a heavy disk load).

Does the stall still happen if you don't ping the guest?

Re: [Qemu-devel] [patch] non-blocking disk IO

[Ryan Rempel]

I've been using the non-blocking IO / DMA patch with a FreeBSD host
and Windows 98 guest. It seems to work well -- I haven't done any
benchmarking, but subjectively it seems to make a difference.

However, I tried enabling DMA in the Windows 98 system settings for
the C: drive, and that doesn't work -- when I reboot the emulator, it
stalls fairly early in the boot process.

Have others been able to get DMA working (with the patch) with Windows
98? Is there something I could to try to track down the problem
(assuming it is supposed to work).