Re: [ANNOUNCE] kvm-14 release

[Laurent Vivier <Laurent.Vivier-6ktuUTfB/bM@public.gmane.org>]

[-- Attachment #1.1.1: Type: text/plain, Size: 3199 bytes --]

Anthony Liguori wrote:
> Laurent Vivier wrote:
>> Avi Kivity wrote:
>>   
>>> Aurelien Jarno wrote:
>>>     
>>>>   
>>>>       
>>>>> What is your disk image file format, or are you using a partition?
>>>>>     
>>>>>         
>>>> I am using a raw image file on an ext3 partition.
>>>>
>>>>   
>>>>       
>>>>> Do the results change (on kvm-14) if you pin the guest to a core with 
>>>>> 'taskset 1 qemu ...'
>>>>>     
>>>>>         
>>>> Bingo. It now works even faster than kvm-13!
>>>>
>>>> real    0m22.307s
>>>> user    0m13.935s
>>>> sys     0m4.720
>>>>
>>>>   
>>>>       
>>> I'm guessing this is due to the glibc aio implementation, which uses 
>>> threads instead of true aio.  The threads may cause the vcpu to migrate 
>>> frequently from one code to another.
>>>
>>> There are two possible solutions:
>>>
>>>  - use native aio from http://www.bullopensource.org/posix/.  I think 
>>> the aio signal patches are not yet in, so this may not work.
>>>  - teach the scheduler about the cost of migrating vcpus
>>> The first approach will solve itself eventually, though slowly if the 
>>> current slow rate of aio merging continues.  We'll have to do the second.
>>>
>>>     
>> if you prefer the first one, Sébastien will release very soon aio patches for
>> 2.6.20 with an up-to-date libposix-aio.
>>   
> 
> Hi Laurent,
> 
> I gave that a shot a little bit ago.  Ran into two problems.
> 
> 1) Couldn't avoid linking to -lrt as QEMU uses time functions from it.
> 2) While I could get things compiling (with patches), QEMU would SEGV 
> almost immediately.
> 
> Could you guys maybe give compiling QEMU w/libposix-aio a shot?  I'm 
> really interested to see if it makes a difference.
> 
> Regards,
> 
> Anthony Liguori
> 
>> [advertising] Keep an eye on the bullopensource website. [/advertising]
>> :-P
>>
>> Regards,
>> Laurent

OK, I didn' have time to test the performance of the result, but you can find
attached some patches to enable libposix-aio with kvm-14.

first take last patches for linux-2.6.20 and libposix-aio-.0.8.2 from website :

http://sourceforge.net/projects/paiol

if you are using AMD64, you must patch libposix-aio because there is a little
problem remaining to detect lio_submit syscall (first attachment)

then apply following patch to kvm-14 (second attachment).

It works fine on my system except when I use "-hda /dev/sdb" : qemu crashes just
after mounting filesystems when "init" tries to set kernel parameters with
"sysctl" (I use a debian 4.0). If I boot in emergency mode, mounting manually
filesystems and running manually sysctl, all works fine. It looks like a
synchronization problem.

There are remaining issues : libposix-aio uses kernel AIO, so files must be
opened using O_DIRECT and buffers must be aligned. libposix-aio is able to
manage other cases but this has a performance cost.

Aurélien, do you have any time to test this on your system ?

Regards,
Laurent
-- 
------------- Laurent.Vivier-6ktuUTfB/bM@public.gmane.org  --------------
       "Any sufficiently advanced technology is
  indistinguishable from magic." - Arthur C. Clarke

[-- Attachment #1.1.2: configure_ac-x86_64.patch --]
[-- Type: text/plain, Size: 1191 bytes --]

--- libposix-aio-0.8.2-org/configure.ac	2007-02-21 15:58:43.000000000 +0100
+++ libposix-aio-0.8.2/configure.ac	2007-02-22 09:43:39.000000000 +0100
@@ -217,6 +217,24 @@
 
 # test if kernel supports signal on LIO completion
 
+case $host_cpu in
+
+        x86_64*)
+                lio_submit_syscall=280
+        ;;
+  
+        i386*)
+                lio_submit_syscall=320
+        ;;
+	*)
+		AC_MSG_WARN(unknown syscall number for lio_submit)
+		lio_submit_syscall=
+        ;;
+esac
+
+if test "${lio_submit_syscall}" != ""
+then
+
 AC_ARG_ENABLE(
 	[lio-signal],
 	[AS_HELP_STRING(--enable-lio-signal,
@@ -236,7 +254,7 @@
 
 #define IO_CMD_PWRITE 1
 
-#define SYS_lio_submit 320
+#define SYS_lio_submit ${lio_submit_syscall}
 
 #define LIO_WAIT   0
 #define LIO_NOWAIT 1
@@ -397,7 +415,7 @@
 
 #define IO_CMD_PWRITE 1
 
-#define SYS_lio_submit 320
+#define SYS_lio_submit ${lio_submit_syscall}
 
 #define LIO_WAIT   0
 #define LIO_NOWAIT 1
@@ -514,6 +532,7 @@
 			Define to 1 if kernel supports LIO wait)],[ac_lio_wait=no
 	AC_MSG_RESULT(no)
 ],[]))
+fi # end of test lio_submit_syscall
 
 # check for support of fd for io_cancel
 

[-- Attachment #1.1.3: posixaio.diff --]
[-- Type: text/plain, Size: 3432 bytes --]

Index: kvm-14/configure
===================================================================
--- kvm-14.orig/configure	2007-02-22 11:46:09.000000000 +0100
+++ kvm-14/configure	2007-02-22 11:46:55.000000000 +0100
@@ -87,7 +87,7 @@
     --enable-kvm --kernel-path="$libkvm_kerneldir" \
     --enable-alsa \
     ${disable_gcc_check:+"--disable-gcc-check"} \
-    --prefix="$prefix"
+    --prefix="$prefix" --enable-libposix-aio
 )
 
 
Index: kvm-14/qemu/Makefile
===================================================================
--- kvm-14.orig/qemu/Makefile	2007-02-22 11:46:09.000000000 +0100
+++ kvm-14/qemu/Makefile	2007-02-22 11:46:12.000000000 +0100
@@ -27,10 +27,14 @@
 ifndef CONFIG_DARWIN
 ifndef CONFIG_WIN32
 ifndef CONFIG_SOLARIS
+ifdef USE_LIBPOSIX_AIO
+LIBS+=-lposix-aio
+else
 LIBS+=-lrt
 endif
 endif
 endif
+endif
 
 all: $(TOOLS) $(DOCS) recurse-all
 
Index: kvm-14/qemu/Makefile.target
===================================================================
--- kvm-14.orig/qemu/Makefile.target	2007-02-22 11:46:09.000000000 +0100
+++ kvm-14/qemu/Makefile.target	2007-02-22 11:46:12.000000000 +0100
@@ -452,7 +452,11 @@
 ifndef CONFIG_DARWIN
 ifndef CONFIG_WIN32
 ifndef CONFIG_SOLARIS
-VL_LIBS=-lutil -lrt -luuid
+VL_LIBS=-lutil
+ifdef USE_LIBPOSIX_AIO
+VL_LIBS+=-lposix-aio
+endif
+VL_LIBS+=-lrt -luuid
 endif
 endif
 endif
Index: kvm-14/qemu/block-raw.c
===================================================================
--- kvm-14.orig/qemu/block-raw.c	2007-02-22 11:46:09.000000000 +0100
+++ kvm-14/qemu/block-raw.c	2007-02-22 11:46:12.000000000 +0100
@@ -197,7 +197,7 @@
     act.sa_handler = aio_signal_handler;
     sigaction(aio_sig_num, &act, NULL);
 
-#if defined(__GLIBC__) && defined(__linux__)
+#if defined(__GLIBC__) && defined(__linux__) && !defined(USE_LIBPOSIX_AIO)
     {
         /* XXX: aio thread exit seems to hang on RedHat 9 and this init
            seems to fix the problem. */
Index: kvm-14/qemu/configure
===================================================================
--- kvm-14.orig/qemu/configure	2007-02-22 11:46:09.000000000 +0100
+++ kvm-14/qemu/configure	2007-02-22 11:46:12.000000000 +0100
@@ -99,6 +99,7 @@
 linux_user="no"
 darwin_user="no"
 build_docs="no"
+libposix_aio="no"
 uname_release=""
 
 # OS specific
@@ -262,6 +263,8 @@
   ;;
   --enable-uname-release=*) uname_release="$optarg"
   ;;
+  --enable-libposix-aio) libposix_aio="yes"
+  ;;
   esac
 done
 
@@ -312,6 +315,7 @@
 echo "  --fmod-lib               path to FMOD library"
 echo "  --fmod-inc               path to FMOD includes"
 echo "  --enable-uname-release=R Return R for uname -r in usermode emulation"
+echo "  --enable-libposix-aio    use libposix-aio instead of glibc aio"
 echo ""
 echo "NOTE: The object files are built at the place where configure is launched"
 exit 1
@@ -635,6 +639,7 @@
 echo "kqemu support     $kqemu"
 echo "kvm support       $kvm"
 echo "Documentation     $build_docs"
+echo "libposix-aio      $libposix_aio"
 [ ! -z "$uname_release" ] && \
 echo "uname -r          $uname_release"
 
@@ -795,6 +800,8 @@
   echo "#define MAP_ANONYMOUS MAP_ANON" >> $config_h
   echo "#define _BSD 1" >> $config_h
 fi
+echo "USE_LIBPOSIX_AIO=yes" >> $config_mak
+echo "#define USE_LIBPOSIX_AIO 1" >> $config_h
 
 echo "#define CONFIG_UNAME_RELEASE \"$uname_release\"" >> $config_h
 

[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 345 bytes --]

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

[-- Attachment #3: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel