[PATCH] Make KVM compile on split source/object kernel configurations

[PATCH] Make KVM compile on split source/object kernel configurations

[Alexander Graf]

[-- Attachment #1: Type: text/plain, Size: 653 bytes --]

KVM as is assumes that the kernel obj dir and the kernel source dir are 
at the same location. This is true for most self-built vanilla kernels, 
but some distributions split these up (e.g. SUSE).
To keep compatible and have users have a good experience on building KVM 
on any distribution, this patch attempts to rebuild the logic from the 
kernel Makefile as closely as possible. With it I successfully built KVM 
on a current SUSE system.

Signed-off-by: Alexander Graf <agraf@suse.de>

Please check and see if it breaks the build process for anyone else. 
Building with IA64 on SUSE-kernels is still broken due to similar but 
separate problems.



[-- Attachment #2: kvm-split-source-obj.diff --]
[-- Type: text/x-patch, Size: 1500 bytes --]

diff --git a/configure b/configure
index 3bb10ce..5e9cbab 100755
--- a/configure
+++ b/configure
@@ -1,6 +1,7 @@
 #!/bin/bash
 
 prefix=/usr/local
+kernelsourcedir=
 kerneldir=/lib/modules/$(uname -r)/build
 cc=gcc
 ld=ld
@@ -102,6 +103,11 @@ if [ "$arch" = "powerpc" ]; then
     qemu_ldflags="$qemu_ldflags -L $PWD/libfdt"
 fi
 
+# see if we have split build and source directories
+if [ -d "$kerneldir/include2" ]; then
+    kernelsourcedir=${kerneldir%/*}/source
+fi
+
 #configure user dir
 (cd user; ./configure --prefix="$prefix" --kerneldir="$libkvm_kerneldir" \
           --arch="$arch" \
@@ -124,6 +130,7 @@ cat <<EOF > config.mak
 ARCH=$arch
 PREFIX=$prefix
 KERNELDIR=$kerneldir
+KERNELSOURCEDIR=$kernelsourcedir
 LIBKVM_KERNELDIR=$libkvm_kerneldir
 WANT_MODULE=$want_module
 CROSS_COMPILE=$cross_prefix
diff --git a/kernel/Makefile b/kernel/Makefile
index 3f5f6da..d2cf89c 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -36,7 +36,9 @@ hack-files = $(hack-files-$(ARCH_DIR))
 all:: header-link prerequisite
 #	include header priority 1) $LINUX 2) $KERNELDIR 3) include-compat
 	$(MAKE) -C $(KERNELDIR) M=`pwd` \
-		LINUXINCLUDE="-I`pwd`/include -Iinclude -Iarch/${ARCH_DIR}/include -I`pwd`/include-compat \
+		LINUXINCLUDE="-I`pwd`/include -Iinclude \
+		$(if $(KERNELSOURCEDIR),-Iinclude2 -I$(KERNELSOURCEDIR)/include) \
+		-Iarch/${ARCH_DIR}/include -I`pwd`/include-compat \
 		-include include/linux/autoconf.h \
 		-include `pwd`/$(ARCH_DIR)/external-module-compat.h"
 		"$$@"

RE: [PATCH] Make KVM compile on split source/object kernel configurations

[Zhang, Xiantao]

Alexander Graf wrote:
> KVM as is assumes that the kernel obj dir and the kernel source dir
> are at the same location. This is true for most self-built vanilla
> kernels, but some distributions split these up (e.g. SUSE).
> To keep compatible and have users have a good experience on building
> KVM on any distribution, this patch attempts to rebuild the logic
> from the kernel Makefile as closely as possible. With it I
> successfully built KVM on a current SUSE system.
> 
> Signed-off-by: Alexander Graf <agraf@suse.de>
> 
> Please check and see if it breaks the build process for anyone else.
> Building with IA64 on SUSE-kernels is still broken due to similar but
> separate problems.

Hi, Graf
	what problems did you meet when building kvm/ia64 ?
Xiantao

Re: [PATCH] Make KVM compile on split source/object kernel configurations

[Alexander Graf]

[-- Attachment #1: Type: text/plain, Size: 1865 bytes --]


On Sep 16, 2008, at 9:22 AM, Zhang, Xiantao wrote:

> Alexander Graf wrote:
>> KVM as is assumes that the kernel obj dir and the kernel source dir
>> are at the same location. This is true for most self-built vanilla
>> kernels, but some distributions split these up (e.g. SUSE).
>> To keep compatible and have users have a good experience on building
>> KVM on any distribution, this patch attempts to rebuild the logic
>> from the kernel Makefile as closely as possible. With it I
>> successfully built KVM on a current SUSE system.
>>
>> Signed-off-by: Alexander Graf <agraf@suse.de>
>>
>> Please check and see if it breaks the build process for anyone else.
>> Building with IA64 on SUSE-kernels is still broken due to similar but
>> separate problems.
>
> Hi, Graf
> 	what problems did you meet when building kvm/ia64 ?
> Xiantao


This is the list of stuff I had to modify to get kvm-74 to run. I  
haven't tried anything more recent yet.

1) -mno-sdata does not work. Qemu CFLAGS need to be -msdata
2) In qemu/ia64.ld SEARCH_DIR is invalid for SUSE. It shouldn't search  
in /usr/ia64-linux but in /usr/ia64-suse-linux.
3) TARGET_PAGE_BITS is wrong
4) kernel/ia64/Makefile.pre assumes to find .S files in $KERNELDIR.  
They are in $SOURCEDIR.
5) kernel/ia64/Makefile.pre suffers from the same problem my original  
fix for x86 addressed.
6) The IA64 kernel build system is broken. The file asm-ia64/nr-irqs.h  
gets generated during the kernel build, but is not exported as obj, so  
it won't be in the split obj directory. Since the source-dir is clean  
though, the file simply does not exist on SUSE rpms.

I will attach the patch I use in our local buildsystem to make KVM  
compile on IA64. Please do not take this as fixes, most of them are  
simple workarounds and should rather be fixed properly. We were just  
hitting deadlines internally.

Alex


[-- Attachment #2: IA64-kvm-suse.patch --]
[-- Type: application/octet-stream, Size: 3148 bytes --]

From 8ec9fd6df253413cd0d2a07202e31c86d22e8f6d Mon Sep 17 00:00:00 2001
From: Raymund Will <rw@iridium.suse.de>
Date: Thu, 4 Sep 2008 11:14:59 +0200
Subject: [PATCH] KVM-ia64 build fix for SuSE Linux

---
 qemu/Makefile.target   |    6 ++++--
 qemu/ia64.ld           |    2 +-
 qemu/target-ia64/cpu.h |    2 +-
 3 files changed, 6 insertions(+), 4 deletions(-)

Index: kvm-74/qemu/Makefile.target
===================================================================
--- kvm-74.orig/qemu/Makefile.target
+++ kvm-74/qemu/Makefile.target
@@ -150,8 +150,10 @@ BASE_LDFLAGS+=-Wl,-T,$(SRC_PATH)/$(ARCH)
 endif
 
 ifeq ($(ARCH),ia64)
-CFLAGS+=-mno-sdata
-OP_CFLAGS+=-mno-sdata
+#CFLAGS+=-mno-sdata
+#OP_CFLAGS+=-mno-sdata
+CFLAGS+=-msdata
+OP_CFLAGS+=-msdata
 endif
 
 ifeq ($(ARCH),arm)
Index: kvm-74/qemu/ia64.ld
===================================================================
--- kvm-74.orig/qemu/ia64.ld
+++ kvm-74/qemu/ia64.ld
@@ -3,7 +3,7 @@ OUTPUT_FORMAT("elf64-ia64-little", "elf6
 	      "elf64-ia64-little")
 OUTPUT_ARCH(ia64)
 ENTRY(_start)
-SEARCH_DIR("/usr/ia64-linux/lib"); SEARCH_DIR("/usr/local/lib"); SEARCH_DIR("/lib"); SEARCH_DIR("/usr/lib");
+SEARCH_DIR("/usr/ia64-suse-linux/lib"); SEARCH_DIR("/usr/local/lib"); SEARCH_DIR("/lib"); SEARCH_DIR("/usr/lib");
 /* Do we need any of these for elf?
    __DYNAMIC = 0;    */
 SECTIONS
Index: kvm-74/qemu/target-ia64/cpu.h
===================================================================
--- kvm-74.orig/qemu/target-ia64/cpu.h
+++ kvm-74/qemu/target-ia64/cpu.h
@@ -30,7 +30,7 @@
 
 #define TARGET_LONG_BITS 64
 
-#define TARGET_PAGE_BITS 16
+#define TARGET_PAGE_BITS 14
 
 #define ELF_MACHINE	EM_IA_64
 
Index: kvm-74/kernel/ia64/Makefile.pre
===================================================================
--- kvm-74.orig/kernel/ia64/Makefile.pre
+++ kvm-74/kernel/ia64/Makefile.pre
@@ -1,6 +1,6 @@
 prerequisite: asm-offsets.h ia64/memset.S ia64/memcpy.S
-	cp -f $(KERNELDIR)/arch/ia64/lib/memcpy.S ia64/memcpy.S
-	cp -f $(KERNELDIR)/arch/ia64/lib/memset.S ia64/memset.S
+	cp -f /usr/src/linux/arch/ia64/lib/memcpy.S ia64/memcpy.S
+	cp -f /usr/src/linux/arch/ia64/lib/memset.S ia64/memset.S
 	cmp -s asm-offset.h ia64/asm-offset.h || mv -f asm-offsets.* ia64/
 
 asm-offsets.h: asm-offsets.s
@@ -20,4 +20,4 @@ asm-offsets.h: asm-offsets.s
 	  echo "#endif") <$< >$@
 
 asm-offsets.s: ia64/asm-offsets.c
-	gcc -S -D__KERNEL__ -I$(KERNELDIR)/include -I$(KERNELDIR)/arch/ia64/include -I./include ia64/asm-offsets.c
+	gcc -S -D__KERNEL__ -I$(KERNELDIR)/include -I$(KERNELDIR)/arch/ia64/include -Iinclude2 -I/usr/src/linux/include -I/usr/src/linux/arch/ia64/include -I./include ia64/asm-offsets.c
Index: kvm-74/kernel/include/asm-ia64/nr-irqs.h
===================================================================
--- /dev/null
+++ kvm-74/kernel/include/asm-ia64/nr-irqs.h
@@ -0,0 +1,12 @@
+#ifndef __ASM_NR_IRQS_H__
+#define __ASM_NR_IRQS_H__
+/*
+ * DO NOT MODIFY.
+ *
+ * This file was generated by Kbuild
+ *
+ */
+
+#define NR_IRQS 1024 /* sizeof (union paravirt_nr_irqs_max)	// */
+
+#endif

[-- Attachment #3: Type: text/plain, Size: 1 bytes --]

RE: [PATCH] Make KVM compile on split source/object kernel configurations

[Zhang, Xiantao]

Alexander Graf wrote:
> KVM as is assumes that the kernel obj dir and the kernel source dir
> are at the same location. This is true for most self-built vanilla
> kernels, but some distributions split these up (e.g. SUSE).
> To keep compatible and have users have a good experience on building
> KVM on any distribution, this patch attempts to rebuild the logic
> from the kernel Makefile as closely as possible. With it I
> successfully built KVM on a current SUSE system.
> 
> Signed-off-by: Alexander Graf <agraf@suse.de>
> 
> Please check and see if it breaks the build process for anyone else.
> Building with IA64 on SUSE-kernels is still broken due to similar but
> separate problems.

Hi, Graf
I am confused by this patch. Why not use ./configure --kerneldir=
your_kernel_source_directory ?  It should be also workable for you. 
In the Makefile, KERNELDIR means  the kernel source directory of target
machine, and is set to /lib/modules/$(uname -r)/build of build machine
by default. But you can change it at your will through configure or
modifying config.mak. 
Xiantao

RE: [PATCH] Make KVM compile on split source/object kernel configurations

[Zhang, Xiantao]

Alexander Graf wrote:
> On Sep 16, 2008, at 9:22 AM, Zhang, Xiantao wrote:
> 
>> Alexander Graf wrote:
>>> KVM as is assumes that the kernel obj dir and the kernel source dir
>>> are at the same location. This is true for most self-built vanilla
>>> kernels, but some distributions split these up (e.g. SUSE).
>>> To keep compatible and have users have a good experience on building
>>> KVM on any distribution, this patch attempts to rebuild the logic
>>> from the kernel Makefile as closely as possible. With it I
>>> successfully built KVM on a current SUSE system.
>>> 
>>> Signed-off-by: Alexander Graf <agraf@suse.de>
>>> 
>>> Please check and see if it breaks the build process for anyone else.
>>> Building with IA64 on SUSE-kernels is still broken due to similar
>>> but separate problems.
>> 
>> Hi, Graf
>> 	what problems did you meet when building kvm/ia64 ?
>> Xiantao
> 
> 
> This is the list of stuff I had to modify to get kvm-74 to run. I
> haven't tried anything more recent yet.
> 
> 1) -mno-sdata does not work. Qemu CFLAGS need to be -msdata 

Which version of GCC are you using ? 

> 2) In qemu/ia64.ld SEARCH_DIR is invalid for SUSE. It shouldn't search
> in /usr/ia64-linux but in /usr/ia64-suse-linux.
You can make a patch to fix it. 

> 3) TARGET_PAGE_BITS is wrong
Currently, we just set TARGET_PAGE_BITS to 16 by hardcoding in Qemu. In
the future, we should add the logic to make it equal to kernel's
PAGE_SHIFT.  

> 4) kernel/ia64/Makefile.pre assumes to find .S files in $KERNELDIR.
> They are in $SOURCEDIR.

There is no issue here, I think.  You have to set corrent $KERNELDIR
through ./configure --kerneldir=xxxx once target machine and build
machine is not same.

> 5) kernel/ia64/Makefile.pre suffers from the same problem my original
> fix for x86 addressed.

Ditto. 

> 6) The IA64 kernel build system is broken. The file asm-ia64/nr-irqs.h
> gets generated during the kernel build, but is not exported as obj, so
> it won't be in the split obj directory. Since the source-dir is clean
> though, the file simply does not exist on SUSE rpms.

You mean the source-dir is clean ?  So the generated header files
including (linux/autoconf.h, asm-offsets.h...)should be removed wholely?
I don't think they can be built with a clean source for most modules. 

> I will attach the patch I use in our local buildsystem to make KVM
> compile on IA64. Please do not take this as fixes, most of them are
> simple workarounds and should rather be fixed properly. We were just
> hitting deadlines internally.
> 
> Alex

Re: [PATCH] Make KVM compile on split source/object kernel configurations

[Alexander Graf]

On Sep 16, 2008, at 12:01 PM, Zhang, Xiantao wrote:

> Alexander Graf wrote:
>> KVM as is assumes that the kernel obj dir and the kernel source dir
>> are at the same location. This is true for most self-built vanilla
>> kernels, but some distributions split these up (e.g. SUSE).
>> To keep compatible and have users have a good experience on building
>> KVM on any distribution, this patch attempts to rebuild the logic
>> from the kernel Makefile as closely as possible. With it I
>> successfully built KVM on a current SUSE system.
>>
>> Signed-off-by: Alexander Graf <agraf@suse.de>
>>
>> Please check and see if it breaks the build process for anyone else.
>> Building with IA64 on SUSE-kernels is still broken due to similar but
>> separate problems.
>
> Hi, Graf
> I am confused by this patch. Why not use ./configure --kerneldir=
> your_kernel_source_directory ?  It should be also workable for you.
> In the Makefile, KERNELDIR means  the kernel source directory of  
> target
> machine, and is set to /lib/modules/$(uname -r)/build of build machine
> by default. But you can change it at your will through configure or
> modifying config.mak.

On SUSE kernels, there are two directories that are completely  
separate. One is linked to from /lib/modules/$(uname -r)/build (that's  
the object dir), while the other one is linked to from /lib/modules/$ 
(uname -r)/source (that's the source dir).

The build dir contains files that were built during a kernel build.  
The source dir is the one the kernel was built from. Almost all of the  
code in KVM assumes, that these are at one location. This is true for  
self-built kernels, as they generate the obj files in your source dir.  
It is not true for SUSE kernels you install from RPMs though.

Alex

Re: [PATCH] Make KVM compile on split source/object kernel configurations

[Alexander Graf]

On Sep 16, 2008, at 12:22 PM, Zhang, Xiantao wrote:

> Alexander Graf wrote:
>> On Sep 16, 2008, at 9:22 AM, Zhang, Xiantao wrote:
>>
>>> Alexander Graf wrote:
>>>> KVM as is assumes that the kernel obj dir and the kernel source dir
>>>> are at the same location. This is true for most self-built vanilla
>>>> kernels, but some distributions split these up (e.g. SUSE).
>>>> To keep compatible and have users have a good experience on  
>>>> building
>>>> KVM on any distribution, this patch attempts to rebuild the logic
>>>> from the kernel Makefile as closely as possible. With it I
>>>> successfully built KVM on a current SUSE system.
>>>>
>>>> Signed-off-by: Alexander Graf <agraf@suse.de>
>>>>
>>>> Please check and see if it breaks the build process for anyone  
>>>> else.
>>>> Building with IA64 on SUSE-kernels is still broken due to similar
>>>> but separate problems.
>>>
>>> Hi, Graf
>>> 	what problems did you meet when building kvm/ia64 ?
>>> Xiantao
>>
>>
>> This is the list of stuff I had to modify to get kvm-74 to run. I
>> haven't tried anything more recent yet.
>>
>> 1) -mno-sdata does not work. Qemu CFLAGS need to be -msdata
>
> Which version of GCC are you using ?

# gcc --version
gcc (SUSE Linux) 4.3.2 20080815 (prerelease) [gcc-4_3-branch revision  
139129]

>
>
>> 2) In qemu/ia64.ld SEARCH_DIR is invalid for SUSE. It shouldn't  
>> search
>> in /usr/ia64-linux but in /usr/ia64-suse-linux.
> You can make a patch to fix it.

Right, this one should be really easy to fix.

>> 3) TARGET_PAGE_BITS is wrong
> Currently, we just set TARGET_PAGE_BITS to 16 by hardcoding in Qemu.  
> In
> the future, we should add the logic to make it equal to kernel's
> PAGE_SHIFT.

 From what I read on the KVM IA64 ML, this is not the first time  
someone ran into this issue. Sounds like a really good idea to add the  
logic ;-).

>> 4) kernel/ia64/Makefile.pre assumes to find .S files in $KERNELDIR.
>> They are in $SOURCEDIR.
>
> There is no issue here, I think.  You have to set corrent $KERNELDIR
> through ./configure --kerneldir=xxxx once target machine and build
> machine is not same.

No, since $KERNELDIR is the build dir, as that one contains all the  
configurations. The source dir only contains the sources.

>> 5) kernel/ia64/Makefile.pre suffers from the same problem my original
>> fix for x86 addressed.
>
> Ditto.
>
>> 6) The IA64 kernel build system is broken. The file asm-ia64/nr- 
>> irqs.h
>> gets generated during the kernel build, but is not exported as obj,  
>> so
>> it won't be in the split obj directory. Since the source-dir is clean
>> though, the file simply does not exist on SUSE rpms.
>
> You mean the source-dir is clean ?  So the generated header files
> including (linux/autoconf.h, asm-offsets.h...)should be removed  
> wholely?
> I don't think they can be built with a clean source for most modules.

This is what I'm talking about all the time :-). For SUSE kernels  
there are two distinct directories:

# ls -l /usr/src
drwxr-xr-x 24 root root 4096 Aug 27 14:51 linux-2.6.27-rc4- 
HEAD_20080825213702
drwxr-xr-x  3 root root 4096 Aug 25 23:28 linux-2.6.27-rc4- 
HEAD_20080825213702-obj

whereas the normal directory contains the sources and the -obj  
directory contains all files that were generated during the build.  
Scratch that with the 2 rpm files - they are all in kernel-sources.

# ls -l /usr/src/linux-2.6.27-rc4-HEAD_20080825213702-obj/ia64/default/
total 568
-rw-r--r-- 1 root root  76287 Aug 25 23:27 .config
-rw-r--r-- 2 root root    554 Aug 25 23:28 Makefile
-rw-r--r-- 1 root root 468615 Aug 25 23:51 Module.symvers
drwxr-xr-x 3 root root   4096 Aug 26 18:01 arch
drwxr-xr-x 5 root root   4096 Aug 26 18:01 include
drwxr-xr-x 2 root root   4096 Aug 26 18:01 include2
drwxr-xr-x 2 root root   4096 Aug 25 23:28 kernel
drwxr-xr-x 6 root root   4096 Aug 26 18:01 scripts

The problem is now that a lot of people assume these two directories  
to be one. This is the default mode when you compile a kernel. SUSE  
does not take this route though, and hasn't for quite a while now. I  
don't know about other distributions, but I guess we're not the only  
ones ;-).

Alex

RE: [PATCH] Make KVM compile on split source/object kernel configurations

[Zhang, Xiantao]

Alexander Graf wrote:
> On Sep 16, 2008, at 12:22 PM, Zhang, Xiantao wrote:
> 
>> Alexander Graf wrote:
>>> On Sep 16, 2008, at 9:22 AM, Zhang, Xiantao wrote:
>>> 
>>>> Alexander Graf wrote:
>>>>> KVM as is assumes that the kernel obj dir and the kernel source
>>>>> dir are at the same location. This is true for most self-built
>>>>> vanilla kernels, but some distributions split these up (e.g.
>>>>> SUSE). 
>>>>> To keep compatible and have users have a good experience on
>>>>> building KVM on any distribution, this patch attempts to rebuild
>>>>> the logic from the kernel Makefile as closely as possible. With
>>>>> it I successfully built KVM on a current SUSE system.
>>>>> 
>>>>> Signed-off-by: Alexander Graf <agraf@suse.de>
>>>>> 
>>>>> Please check and see if it breaks the build process for anyone
>>>>> else. Building with IA64 on SUSE-kernels is still broken due to
>>>>> similar but separate problems.
>>>> 
>>>> Hi, Graf
>>>> 	what problems did you meet when building kvm/ia64 ?
>>>> Xiantao
>>> 
>>> 
>>> This is the list of stuff I had to modify to get kvm-74 to run. I
>>> haven't tried anything more recent yet.
>>> 
>>> 1) -mno-sdata does not work. Qemu CFLAGS need to be -msdata
>> 
>> Which version of GCC are you using ?
> 
> # gcc --version
> gcc (SUSE Linux) 4.3.2 20080815 (prerelease) [gcc-4_3-branch revision
> 139129]
> 
>> 
>> 
>>> 2) In qemu/ia64.ld SEARCH_DIR is invalid for SUSE. It shouldn't
>>> search in /usr/ia64-linux but in /usr/ia64-suse-linux.
>> You can make a patch to fix it.
> 
> Right, this one should be really easy to fix.
> 
>>> 3) TARGET_PAGE_BITS is wrong
>> Currently, we just set TARGET_PAGE_BITS to 16 by hardcoding in Qemu.
>> In the future, we should add the logic to make it equal to kernel's
>> PAGE_SHIFT.
> 
>  From what I read on the KVM IA64 ML, this is not the first time
> someone ran into this issue. Sounds like a really good idea to add the
> logic ;-).
> 
>>> 4) kernel/ia64/Makefile.pre assumes to find .S files in $KERNELDIR.
>>> They are in $SOURCEDIR.
>> 
>> There is no issue here, I think.  You have to set corrent $KERNELDIR
>> through ./configure --kerneldir=xxxx once target machine and build
>> machine is not same.
> 
> No, since $KERNELDIR is the build dir, as that one contains all the
> configurations. The source dir only contains the sources.
> 
>>> 5) kernel/ia64/Makefile.pre suffers from the same problem my
>>> original fix for x86 addressed.
>> 
>> Ditto.
>> 
>>> 6) The IA64 kernel build system is broken. The file asm-ia64/nr-
>>> irqs.h gets generated during the kernel build, but is not exported
>>> as obj, so it won't be in the split obj directory. Since the
>>> source-dir is clean though, the file simply does not exist on SUSE
>>> rpms. 
>> 
>> You mean the source-dir is clean ?  So the generated header files
>> including (linux/autoconf.h, asm-offsets.h...)should be removed
>> wholely? I don't think they can be built with a clean source for
>> most modules. 
> 
> This is what I'm talking about all the time :-). For SUSE kernels
> there are two distinct directories:
> 
> # ls -l /usr/src
> drwxr-xr-x 24 root root 4096 Aug 27 14:51 linux-2.6.27-rc4-
> HEAD_20080825213702
> drwxr-xr-x  3 root root 4096 Aug 25 23:28 linux-2.6.27-rc4-
> HEAD_20080825213702-obj
> 
> whereas the normal directory contains the sources and the -obj
> directory contains all files that were generated during the build.
> Scratch that with the 2 rpm files - they are all in kernel-sources.
> 
> # ls -l
> /usr/src/linux-2.6.27-rc4-HEAD_20080825213702-obj/ia64/default/ total
> 568 
> -rw-r--r-- 1 root root  76287 Aug 25 23:27 .config
> -rw-r--r-- 2 root root    554 Aug 25 23:28 Makefile
> -rw-r--r-- 1 root root 468615 Aug 25 23:51 Module.symvers
> drwxr-xr-x 3 root root   4096 Aug 26 18:01 arch
> drwxr-xr-x 5 root root   4096 Aug 26 18:01 include
> drwxr-xr-x 2 root root   4096 Aug 26 18:01 include2
> drwxr-xr-x 2 root root   4096 Aug 25 23:28 kernel
> drwxr-xr-x 6 root root   4096 Aug 26 18:01 scripts
> 
> The problem is now that a lot of people assume these two directories
> to be one. This is the default mode when you compile a kernel. SUSE
> does not take this route though, and hasn't for quite a while now. I
> don't know about other distributions, but I guess we're not the only
> ones ;-).
Okay, I see now. But due to shortage of Suse OS at hand, I can't try it
. Anyway once you found any issues related to the compability with Suse
OS, welcome to post patches to fix them. :)
Xiantao

Re: [PATCH] Make KVM compile on split source/object kernel configurations

[Alexander Graf]

Am 18.09.2008 um 04:12 schrieb "Zhang, Xiantao"  
<xiantao.zhang@intel.com>:

> Alexander Graf wrote:
>> On Sep 16, 2008, at 12:22 PM, Zhang, Xiantao wrote:
>>
>>> Alexander Graf wrote:
>>>> On Sep 16, 2008, at 9:22 AM, Zhang, Xiantao wrote:
>>>>
>>>>> Alexander Graf wrote:
>>>>>> KVM as is assumes that the kernel obj dir and the kernel source
>>>>>> dir are at the same location. This is true for most self-built
>>>>>> vanilla kernels, but some distributions split these up (e.g.
>>>>>> SUSE).
>>>>>> To keep compatible and have users have a good experience on
>>>>>> building KVM on any distribution, this patch attempts to rebuild
>>>>>> the logic from the kernel Makefile as closely as possible. With
>>>>>> it I successfully built KVM on a current SUSE system.
>>>>>>
>>>>>> Signed-off-by: Alexander Graf <agraf@suse.de>
>>>>>>
>>>>>> Please check and see if it breaks the build process for anyone
>>>>>> else. Building with IA64 on SUSE-kernels is still broken due to
>>>>>> similar but separate problems.
>>>>>
>>>>> Hi, Graf
>>>>>    what problems did you meet when building kvm/ia64 ?
>>>>> Xiantao
>>>>
>>>>
>>>> This is the list of stuff I had to modify to get kvm-74 to run. I
>>>> haven't tried anything more recent yet.
>>>>
>>>> 1) -mno-sdata does not work. Qemu CFLAGS need to be -msdata
>>>
>>> Which version of GCC are you using ?
>>
>> # gcc --version
>> gcc (SUSE Linux) 4.3.2 20080815 (prerelease) [gcc-4_3-branch revision
>> 139129]
>>
>>>
>>>
>>>> 2) In qemu/ia64.ld SEARCH_DIR is invalid for SUSE. It shouldn't
>>>> search in /usr/ia64-linux but in /usr/ia64-suse-linux.
>>> You can make a patch to fix it.
>>
>> Right, this one should be really easy to fix.
>>
>>>> 3) TARGET_PAGE_BITS is wrong
>>> Currently, we just set TARGET_PAGE_BITS to 16 by hardcoding in Qemu.
>>> In the future, we should add the logic to make it equal to kernel's
>>> PAGE_SHIFT.
>>
>> From what I read on the KVM IA64 ML, this is not the first time
>> someone ran into this issue. Sounds like a really good idea to add  
>> the
>> logic ;-).
>>
>>>> 4) kernel/ia64/Makefile.pre assumes to find .S files in $KERNELDIR.
>>>> They are in $SOURCEDIR.
>>>
>>> There is no issue here, I think.  You have to set corrent $KERNELDIR
>>> through ./configure --kerneldir=xxxx once target machine and build
>>> machine is not same.
>>
>> No, since $KERNELDIR is the build dir, as that one contains all the
>> configurations. The source dir only contains the sources.
>>
>>>> 5) kernel/ia64/Makefile.pre suffers from the same problem my
>>>> original fix for x86 addressed.
>>>
>>> Ditto.
>>>
>>>> 6) The IA64 kernel build system is broken. The file asm-ia64/nr-
>>>> irqs.h gets generated during the kernel build, but is not exported
>>>> as obj, so it won't be in the split obj directory. Since the
>>>> source-dir is clean though, the file simply does not exist on SUSE
>>>> rpms.
>>>
>>> You mean the source-dir is clean ?  So the generated header files
>>> including (linux/autoconf.h, asm-offsets.h...)should be removed
>>> wholely? I don't think they can be built with a clean source for
>>> most modules.
>>
>> This is what I'm talking about all the time :-). For SUSE kernels
>> there are two distinct directories:
>>
>> # ls -l /usr/src
>> drwxr-xr-x 24 root root 4096 Aug 27 14:51 linux-2.6.27-rc4-
>> HEAD_20080825213702
>> drwxr-xr-x  3 root root 4096 Aug 25 23:28 linux-2.6.27-rc4-
>> HEAD_20080825213702-obj
>>
>> whereas the normal directory contains the sources and the -obj
>> directory contains all files that were generated during the build.
>> Scratch that with the 2 rpm files - they are all in kernel-sources.
>>
>> # ls -l
>> /usr/src/linux-2.6.27-rc4-HEAD_20080825213702-obj/ia64/default/ total
>> 568
>> -rw-r--r-- 1 root root  76287 Aug 25 23:27 .config
>> -rw-r--r-- 2 root root    554 Aug 25 23:28 Makefile
>> -rw-r--r-- 1 root root 468615 Aug 25 23:51 Module.symvers
>> drwxr-xr-x 3 root root   4096 Aug 26 18:01 arch
>> drwxr-xr-x 5 root root   4096 Aug 26 18:01 include
>> drwxr-xr-x 2 root root   4096 Aug 26 18:01 include2
>> drwxr-xr-x 2 root root   4096 Aug 25 23:28 kernel
>> drwxr-xr-x 6 root root   4096 Aug 26 18:01 scripts
>>
>> The problem is now that a lot of people assume these two directories
>> to be one. This is the default mode when you compile a kernel. SUSE
>> does not take this route though, and hasn't for quite a while now. I
>> don't know about other distributions, but I guess we're not the only
>> ones ;-).
> Okay, I see now. But due to shortage of Suse OS at hand, I can't try  
> it
> . Anyway once you found any issues related to the compability with  
> Suse
> OS, welcome to post patches to fix them. :)

Oh I will :). I just wanted to make sure x86 compiles for everyone  
first, as that probably hurts more users :).

Thanks for caring!

Alex

Re: [PATCH] Make KVM compile on split source/object kernel configurations

[Alexander Graf]

Any further comments on this? I would really like to see KVM build out- 
of-the-box on SUSE kernels (and kernels built with O= at the same time).

Alex

On 15.09.2008, at 16:19, Alexander Graf wrote:

> KVM as is assumes that the kernel obj dir and the kernel source dir  
> are at the same location. This is true for most self-built vanilla  
> kernels, but some distributions split these up (e.g. SUSE).
> To keep compatible and have users have a good experience on building  
> KVM on any distribution, this patch attempts to rebuild the logic  
> from the kernel Makefile as closely as possible. With it I  
> successfully built KVM on a current SUSE system.
>
> Signed-off-by: Alexander Graf <agraf@suse.de>
>
> Please check and see if it breaks the build process for anyone else.  
> Building with IA64 on SUSE-kernels is still broken due to similar  
> but separate problems.
>
>
> diff --git a/configure b/configure
> index 3bb10ce..5e9cbab 100755
> --- a/configure
> +++ b/configure
> @@ -1,6 +1,7 @@
> #!/bin/bash
>
> prefix=/usr/local
> +kernelsourcedir=
> kerneldir=/lib/modules/$(uname -r)/build
> cc=gcc
> ld=ld
> @@ -102,6 +103,11 @@ if [ "$arch" = "powerpc" ]; then
>     qemu_ldflags="$qemu_ldflags -L $PWD/libfdt"
> fi
>
> +# see if we have split build and source directories
> +if [ -d "$kerneldir/include2" ]; then
> +    kernelsourcedir=${kerneldir%/*}/source
> +fi
> +
> #configure user dir
> (cd user; ./configure --prefix="$prefix" -- 
> kerneldir="$libkvm_kerneldir" \
>           --arch="$arch" \
> @@ -124,6 +130,7 @@ cat <<EOF > config.mak
> ARCH=$arch
> PREFIX=$prefix
> KERNELDIR=$kerneldir
> +KERNELSOURCEDIR=$kernelsourcedir
> LIBKVM_KERNELDIR=$libkvm_kerneldir
> WANT_MODULE=$want_module
> CROSS_COMPILE=$cross_prefix
> diff --git a/kernel/Makefile b/kernel/Makefile
> index 3f5f6da..d2cf89c 100644
> --- a/kernel/Makefile
> +++ b/kernel/Makefile
> @@ -36,7 +36,9 @@ hack-files = $(hack-files-$(ARCH_DIR))
> all:: header-link prerequisite
> #	include header priority 1) $LINUX 2) $KERNELDIR 3) include-compat
> 	$(MAKE) -C $(KERNELDIR) M=`pwd` \
> -		LINUXINCLUDE="-I`pwd`/include -Iinclude -Iarch/${ARCH_DIR}/ 
> include -I`pwd`/include-compat \
> +		LINUXINCLUDE="-I`pwd`/include -Iinclude \
> +		$(if $(KERNELSOURCEDIR),-Iinclude2 -I$(KERNELSOURCEDIR)/include) \
> +		-Iarch/${ARCH_DIR}/include -I`pwd`/include-compat \
> 		-include include/linux/autoconf.h \
> 		-include `pwd`/$(ARCH_DIR)/external-module-compat.h"
> 		"$$@"

Re: [PATCH] Make KVM compile on split source/object kernel configurations

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 3030 bytes --]

Alexander Graf wrote:
> Any further comments on this? I would really like to see KVM build
> out-of-the-box on SUSE kernels (and kernels built with O= at the same
> time).
>
> Alex

/me, too. And my colleagues @work. We carry around an old hack to build
against SuSE, now waiting for your clean solution to be merged.

Jan

> 
> On 15.09.2008, at 16:19, Alexander Graf wrote:
> 
>> KVM as is assumes that the kernel obj dir and the kernel source dir
>> are at the same location. This is true for most self-built vanilla
>> kernels, but some distributions split these up (e.g. SUSE).
>> To keep compatible and have users have a good experience on building
>> KVM on any distribution, this patch attempts to rebuild the logic from
>> the kernel Makefile as closely as possible. With it I successfully
>> built KVM on a current SUSE system.
>>
>> Signed-off-by: Alexander Graf <agraf@suse.de>
>>
>> Please check and see if it breaks the build process for anyone else.
>> Building with IA64 on SUSE-kernels is still broken due to similar but
>> separate problems.
>>
>>
>> diff --git a/configure b/configure
>> index 3bb10ce..5e9cbab 100755
>> --- a/configure
>> +++ b/configure
>> @@ -1,6 +1,7 @@
>> #!/bin/bash
>>
>> prefix=/usr/local
>> +kernelsourcedir=
>> kerneldir=/lib/modules/$(uname -r)/build
>> cc=gcc
>> ld=ld
>> @@ -102,6 +103,11 @@ if [ "$arch" = "powerpc" ]; then
>>     qemu_ldflags="$qemu_ldflags -L $PWD/libfdt"
>> fi
>>
>> +# see if we have split build and source directories
>> +if [ -d "$kerneldir/include2" ]; then
>> +    kernelsourcedir=${kerneldir%/*}/source
>> +fi
>> +
>> #configure user dir
>> (cd user; ./configure --prefix="$prefix"
>> --kerneldir="$libkvm_kerneldir" \
>>           --arch="$arch" \
>> @@ -124,6 +130,7 @@ cat <<EOF > config.mak
>> ARCH=$arch
>> PREFIX=$prefix
>> KERNELDIR=$kerneldir
>> +KERNELSOURCEDIR=$kernelsourcedir
>> LIBKVM_KERNELDIR=$libkvm_kerneldir
>> WANT_MODULE=$want_module
>> CROSS_COMPILE=$cross_prefix
>> diff --git a/kernel/Makefile b/kernel/Makefile
>> index 3f5f6da..d2cf89c 100644
>> --- a/kernel/Makefile
>> +++ b/kernel/Makefile
>> @@ -36,7 +36,9 @@ hack-files = $(hack-files-$(ARCH_DIR))
>> all:: header-link prerequisite
>> #    include header priority 1) $LINUX 2) $KERNELDIR 3) include-compat
>>     $(MAKE) -C $(KERNELDIR) M=`pwd` \
>> -        LINUXINCLUDE="-I`pwd`/include -Iinclude
>> -Iarch/${ARCH_DIR}/include -I`pwd`/include-compat \
>> +        LINUXINCLUDE="-I`pwd`/include -Iinclude \
>> +        $(if $(KERNELSOURCEDIR),-Iinclude2
>> -I$(KERNELSOURCEDIR)/include) \
>> +        -Iarch/${ARCH_DIR}/include -I`pwd`/include-compat \
>>         -include include/linux/autoconf.h \
>>         -include `pwd`/$(ARCH_DIR)/external-module-compat.h"
>>         "$$@"
> 
> -- 
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 258 bytes --]

Re: [PATCH] Make KVM compile on split source/object kernel configurations

[Brian Jackson]

[-- Attachment #1: Type: text/plain, Size: 3132 bytes --]

With your patch, I still can't build with a split kernel source/object dir.

Don't know if it's a difference of configure options or what. Here's mine just 
in case:
./configure --disable-gfx-check --disable-sdl --with-patched-kernel --prefix=/usr

I needed the attached patch as well.




On Monday 29 September 2008 1:13:52 pm Alexander Graf wrote:
> Any further comments on this? I would really like to see KVM build out-
> of-the-box on SUSE kernels (and kernels built with O= at the same time).
>
> Alex
>
> On 15.09.2008, at 16:19, Alexander Graf wrote:
> > KVM as is assumes that the kernel obj dir and the kernel source dir
> > are at the same location. This is true for most self-built vanilla
> > kernels, but some distributions split these up (e.g. SUSE).
> > To keep compatible and have users have a good experience on building
> > KVM on any distribution, this patch attempts to rebuild the logic
> > from the kernel Makefile as closely as possible. With it I
> > successfully built KVM on a current SUSE system.
> >
> > Signed-off-by: Alexander Graf <agraf@suse.de>
> >
> > Please check and see if it breaks the build process for anyone else.
> > Building with IA64 on SUSE-kernels is still broken due to similar
> > but separate problems.
> >
> >
> > diff --git a/configure b/configure
> > index 3bb10ce..5e9cbab 100755
> > --- a/configure
> > +++ b/configure
> > @@ -1,6 +1,7 @@
> > #!/bin/bash
> >
> > prefix=/usr/local
> > +kernelsourcedir=
> > kerneldir=/lib/modules/$(uname -r)/build
> > cc=gcc
> > ld=ld
> > @@ -102,6 +103,11 @@ if [ "$arch" = "powerpc" ]; then
> >     qemu_ldflags="$qemu_ldflags -L $PWD/libfdt"
> > fi
> >
> > +# see if we have split build and source directories
> > +if [ -d "$kerneldir/include2" ]; then
> > +    kernelsourcedir=${kerneldir%/*}/source
> > +fi
> > +
> > #configure user dir
> > (cd user; ./configure --prefix="$prefix" --
> > kerneldir="$libkvm_kerneldir" \
> >           --arch="$arch" \
> > @@ -124,6 +130,7 @@ cat <<EOF > config.mak
> > ARCH=$arch
> > PREFIX=$prefix
> > KERNELDIR=$kerneldir
> > +KERNELSOURCEDIR=$kernelsourcedir
> > LIBKVM_KERNELDIR=$libkvm_kerneldir
> > WANT_MODULE=$want_module
> > CROSS_COMPILE=$cross_prefix
> > diff --git a/kernel/Makefile b/kernel/Makefile
> > index 3f5f6da..d2cf89c 100644
> > --- a/kernel/Makefile
> > +++ b/kernel/Makefile
> > @@ -36,7 +36,9 @@ hack-files = $(hack-files-$(ARCH_DIR))
> > all:: header-link prerequisite
> > #	include header priority 1) $LINUX 2) $KERNELDIR 3) include-compat
> > 	$(MAKE) -C $(KERNELDIR) M=`pwd` \
> > -		LINUXINCLUDE="-I`pwd`/include -Iinclude -Iarch/${ARCH_DIR}/
> > include -I`pwd`/include-compat \
> > +		LINUXINCLUDE="-I`pwd`/include -Iinclude \
> > +		$(if $(KERNELSOURCEDIR),-Iinclude2 -I$(KERNELSOURCEDIR)/include) \
> > +		-Iarch/${ARCH_DIR}/include -I`pwd`/include-compat \
> > 		-include include/linux/autoconf.h \
> > 		-include `pwd`/$(ARCH_DIR)/external-module-compat.h"
> > 		"$$@"
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


[-- Attachment #2: split-kerneldir.patch --]
[-- Type: text/x-diff, Size: 511 bytes --]

diff --git a/kernel/Makefile b/kernel/Makefile
index f2a71fa..46948cc 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -58,8 +58,8 @@ T = $(subst -sync,,$@)-tmp
 header-sync:
 	rm -rf $T
 	rsync -R \
-	     "$(LINUX)"/./include/linux/kvm*.h \
-	     "$(LINUX)"/./include/asm-*/kvm*.h \
+	     "$(subst build,source,$(LINUX))"/./include/linux/kvm*.h \
+	     "$(subst build,source,$(LINUX))"/./include/asm-*/kvm*.h \
              $T/
 	-rsync -R \
 	     "$(LINUX)"/arch/$(ARCH_DIR)/include/asm/./kvm*.h \

Re: [PATCH] Make KVM compile on split source/object kernel configurations

[Alexander Graf]

On 29.09.2008, at 23:50, Brian Jackson wrote:

> With your patch, I still can't build with a split kernel source/ 
> object dir.
>
> Don't know if it's a difference of configure options or what. Here's  
> mine just
> in case:
> ./configure --disable-gfx-check --disable-sdl --with-patched-kernel  
> --prefix=/usr
>
> I needed the attached patch as well.

Does --with-patched-kernel work again? I thought that's broken anyways.

Alex

Re: [PATCH] Make KVM compile on split source/object kernel configurations

[Brian Jackson]

Works for me with that patch... I don't know if it's working the way it's 
supposed to. I just have a build script that's always used that.

If it doesn't work, maybe we should disable or remove it.



On Monday 29 September 2008 5:00:29 pm Alexander Graf wrote:
> On 29.09.2008, at 23:50, Brian Jackson wrote:
> > With your patch, I still can't build with a split kernel source/
> > object dir.
> >
> > Don't know if it's a difference of configure options or what. Here's
> > mine just
> > in case:
> > ./configure --disable-gfx-check --disable-sdl --with-patched-kernel
> > --prefix=/usr
> >
> > I needed the attached patch as well.
>
> Does --with-patched-kernel work again? I thought that's broken anyways.
>
> Alex
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html