Re: [ANNOUNCE] kvm-14 release

Laurent Vivier
2007-02-22T16:35:13Z

[-- Attachment #1.1.1: Type: text/plain, Size: 3199 bytes --]

Anthony Liguori wrote:
> Laurent Vivier wrote:
>> Avi Kivity wrote:
>>
>>> Aurelien Jarno wrote:
>>>
>>>>
>>>>
>>>>> What is your disk image file format, or are you using a partition?
>>>>>
>>>>>
>>>> I am using a raw image file on an ext3 partition.
>>>>
>>>>
>>>>
>>>>> Do the results change (on kvm-14) if you pin the guest to a core with
>>>>> 'taskset 1 qemu ...'
>>>>>
>>>>>
>>>> Bingo. It now works even faster than kvm-13!
>>>>
>>>> real 0m22.307s
>>>> user 0m13.935s
>>>> sys 0m4.720
>>>>
>>>>
>>>>
>>> I'm guessing this is due to the glibc aio implementation, which uses
>>> threads instead of true aio. The threads may cause the vcpu to migrate
>>> frequently from one code to another.
>>>
>>> There are two possible solutions:
>>>
>>> - use native aio from http://www.bullopensource.org/posix/. I think
>>> the aio signal patches are not yet in, so this may not work.
>>> - teach the scheduler about the cost of migrating vcpus
>>> The first approach will solve itself eventually, though slowly if the
>>> current slow rate of aio merging continues. We'll have to do the second.
>>>
>>>
>> if you prefer the first one, Sébastien will release very soon aio patches for
>> 2.6.20 with an up-to-date libposix-aio.
>>
>
> Hi Laurent,
>
> I gave that a shot a little bit ago. Ran into two problems.
>
> 1) Couldn't avoid linking to -lrt as QEMU uses time functions from it.
> 2) While I could get things compiling (with patches), QEMU would SEGV
> almost immediately.
>
> Could you guys maybe give compiling QEMU w/libposix-aio a shot? I'm
> really interested to see if it makes a difference.
>
> Regards,
>
> Anthony Liguori
>
>> [advertising] Keep an eye on the bullopensource website. [/advertising]
>> :-P
>>
>> Regards,
>> Laurent

OK, I didn' have time to test the performance of the result, but you can find
attached some patches to enable libposix-aio with kvm-14.

first take last patches for linux-2.6.20 and libposix-aio-.0.8.2 from website :

http://sourceforge.net/projects/paiol

if you are using AMD64, you must patch libposix-aio because there is a little
problem remaining to detect lio_submit syscall (first attachment)

then apply following patch to kvm-14 (second attachment).

It works fine on my system except when I use "-hda /dev/sdb" : qemu crashes just
after mounting filesystems when "init" tries to set kernel parameters with
"sysctl" (I use a debian 4.0). If I boot in emergency mode, mounting manually
filesystems and running manually sysctl, all works fine. It looks like a
synchronization problem.

There are remaining issues : libposix-aio uses kernel AIO, so files must be
opened using O_DIRECT and buffers must be aligned. libposix-aio is able to
manage other cases but this has a performance cost.

Aurélien, do you have any time to test this on your system ?

Regards,
Laurent
--
------------- Laurent.Vivier-6ktuUTfB/bM@public.gmane.org --------------
"Any sufficiently advanced technology is
indistinguishable from magic." - Arthur C. Clarke

[-- Attachment #1.1.2: configure_ac-x86_64.patch --]
[-- Type: text/plain, Size: 1191 bytes --]

--- libposix-aio-0.8.2-org/configure.ac 2007-02-21 15:58:43.000000000 +0100
+++ libposix-aio-0.8.2/configure.ac 2007-02-22 09:43:39.000000000 +0100
@@ -217,6 +217,24 @@

# test if kernel supports signal on LIO completion

+case $host_cpu in
+
+ x86_64*)
+ lio_submit_syscall=280
+ ;;
+
+ i386*)
+ lio_submit_syscall=320
+ ;;
+ *)
+ AC_MSG_WARN(unknown syscall number for lio_submit)
+ lio_submit_syscall=
+ ;;
+esac
+
+if test "${lio_submit_syscall}" != ""
+then
+
AC_ARG_ENABLE(
[lio-signal],
[AS_HELP_STRING(--enable-lio-signal,
@@ -236,7 +254,7 @@

#define IO_CMD_PWRITE 1

-#define SYS_lio_submit 320
+#define SYS_lio_submit ${lio_submit_syscall}

#define LIO_WAIT 0
#define LIO_NOWAIT 1
@@ -397,7 +415,7 @@

#define IO_CMD_PWRITE 1

-#define SYS_lio_submit 320
+#define SYS_lio_submit ${lio_submit_syscall}

#define LIO_WAIT 0
#define LIO_NOWAIT 1
@@ -514,6 +532,7 @@
Define to 1 if kernel supports LIO wait)],[ac_lio_wait=no
AC_MSG_RESULT(no)
],[]))
+fi # end of test lio_submit_syscall

# check for support of fd for io_cancel

[-- Attachment #1.1.3: posixaio.diff --]
[-- Type: text/plain, Size: 3432 bytes --]

Index: kvm-14/configure
===================================================================
--- kvm-14.orig/configure 2007-02-22 11:46:09.000000000 +0100
+++ kvm-14/configure 2007-02-22 11:46:55.000000000 +0100
@@ -87,7 +87,7 @@
--enable-kvm --kernel-path="$libkvm_kerneldir" \
--enable-alsa \
${disable_gcc_check:+"--disable-gcc-check"} \
- --prefix="$prefix"
+ --prefix="$prefix" --enable-libposix-aio
)

Index: kvm-14/qemu/Makefile
===================================================================
--- kvm-14.orig/qemu/Makefile 2007-02-22 11:46:09.000000000 +0100
+++ kvm-14/qemu/Makefile 2007-02-22 11:46:12.000000000 +0100
@@ -27,10 +27,14 @@
ifndef CONFIG_DARWIN
ifndef CONFIG_WIN32
ifndef CONFIG_SOLARIS
+ifdef USE_LIBPOSIX_AIO
+LIBS+=-lposix-aio
+else
LIBS+=-lrt
endif
endif
endif
+endif

all: $(TOOLS) $(DOCS) recurse-all

Index: kvm-14/qemu/Makefile.target
===================================================================
--- kvm-14.orig/qemu/Makefile.target 2007-02-22 11:46:09.000000000 +0100
+++ kvm-14/qemu/Makefile.target 2007-02-22 11:46:12.000000000 +0100
@@ -452,7 +452,11 @@
ifndef CONFIG_DARWIN
ifndef CONFIG_WIN32
ifndef CONFIG_SOLARIS
-VL_LIBS=-lutil -lrt -luuid
+VL_LIBS=-lutil
+ifdef USE_LIBPOSIX_AIO
+VL_LIBS+=-lposix-aio
+endif
+VL_LIBS+=-lrt -luuid
endif
endif
endif
Index: kvm-14/qemu/block-raw.c
===================================================================
--- kvm-14.orig/qemu/block-raw.c 2007-02-22 11:46:09.000000000 +0100
+++ kvm-14/qemu/block-raw.c 2007-02-22 11:46:12.000000000 +0100
@@ -197,7 +197,7 @@
act.sa_handler = aio_signal_handler;
sigaction(aio_sig_num, &act, NULL);

-#if defined(__GLIBC__) && defined(__linux__)
+#if defined(__GLIBC__) && defined(__linux__) && !defined(USE_LIBPOSIX_AIO)
{
/* XXX: aio thread exit seems to hang on RedHat 9 and this init
seems to fix the problem. */
Index: kvm-14/qemu/configure
===================================================================
--- kvm-14.orig/qemu/configure 2007-02-22 11:46:09.000000000 +0100
+++ kvm-14/qemu/configure 2007-02-22 11:46:12.000000000 +0100
@@ -99,6 +99,7 @@
linux_user="no"
darwin_user="no"
build_docs="no"
+libposix_aio="no"
uname_release=""

# OS specific
@@ -262,6 +263,8 @@
;;
--enable-uname-release=*) uname_release="$optarg"
;;
+ --enable-libposix-aio) libposix_aio="yes"
+ ;;
esac
done

@@ -312,6 +315,7 @@
echo " --fmod-lib path to FMOD library"
echo " --fmod-inc path to FMOD includes"
echo " --enable-uname-release=R Return R for uname -r in usermode emulation"
+echo " --enable-libposix-aio use libposix-aio instead of glibc aio"
echo ""
echo "NOTE: The object files are built at the place where configure is launched"
exit 1
@@ -635,6 +639,7 @@
echo "kqemu support $kqemu"
echo "kvm support $kvm"
echo "Documentation $build_docs"
+echo "libposix-aio $libposix_aio"
[ ! -z "$uname_release" ] && \
echo "uname -r $uname_release"

@@ -795,6 +800,8 @@
echo "#define MAP_ANONYMOUS MAP_ANON" >> $config_h
echo "#define _BSD 1" >> $config_h
fi
+echo "USE_LIBPOSIX_AIO=yes" >> $config_mak
+echo "#define USE_LIBPOSIX_AIO 1" >> $config_h

echo "#define CONFIG_UNAME_RELEASE \"$uname_release\"" >> $config_h

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 345 bytes --]

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[-- Attachment #3: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org