compiler errors in latest unstable from 20051027

compiler errors in latest unstable from 20051027

[Adam Heath]

If CONFIG_SMP_ALTERNATIVES is selected, arch/xen/kernel/smpboot.c attempts to
include <asm/smp_boot.h>, which does not exist anywhere.  Which means
selecting that is not allowed.

However, there are several additional unprotected includes of that same file
in other places; include/asm-xen/asm-i386/system.h and
include/asm-xen/asm-i386/spinlock.h.

Plus, there is a complaint about LOCK not being defined in
include/asm-xen/asm/system.h.  If I add include <asm/atomic.h>, it gets by the
first such error, but then I get this later on:

==
In file included from include/asm-xen/asm/processor.h:18,
                 from include/asm/atomic.h:6,
                 from kernel/audit.c:45:
include/asm-xen/asm/system.h: In function `__cmpxchg':
include/asm-xen/asm/system.h:259: error: `LOCK' undeclared (first use in this
function)
==

This is because asm/system.h is being included recursively, so none of the
defines in asm/atomic.h are defined yet.

The steps I took to repeat this:

1: apt-get install lkinux-tree-2.6.12
2: /usr/src/kernel-patches/all/2.6.12/unpatch/debian 2.6.12
   # this gives a real 2.6.12 tree
3: cd linux-2.6-sparse
4: ./mkbuildtree $path_to_linux
   # this applies the 2.6.12.6 patch, then overlays the xen code on top of it.
5: create diff between xen-patched linux and plain 2.6.12.6
6: Go to another machine
7: extract 2.6.12, apply 2.6.12.6; no warnings from diff during this
8: apply xen patch; no warnings from diff during this
9: make menuconfig ARCH=xen
10: error as above

RE: compiler errors in latest unstable from 20051027

[Ian Pratt]

 

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com 
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Adam Heath
> Sent: 09 November 2005 17:23
> To: xen-devel@lists.xensource.com
> Subject: [Xen-devel] compiler errors in latest unstable from 20051027

Why are you using unstable from 20051027 rather than today? A week is a
long time in the unstable tree, and there have definitely been patches
in this area.

Ian

> If CONFIG_SMP_ALTERNATIVES is selected, 
> arch/xen/kernel/smpboot.c attempts to include 
> <asm/smp_boot.h>, which does not exist anywhere.  Which means 
> selecting that is not allowed.
> 
> However, there are several additional unprotected includes of 
> that same file in other places; 
> include/asm-xen/asm-i386/system.h and 
> include/asm-xen/asm-i386/spinlock.h.
> 
> Plus, there is a complaint about LOCK not being defined in 
> include/asm-xen/asm/system.h.  If I add include 
> <asm/atomic.h>, it gets by the first such error, but then I 
> get this later on:
> 
> ==
> In file included from include/asm-xen/asm/processor.h:18,
>                  from include/asm/atomic.h:6,
>                  from kernel/audit.c:45:
> include/asm-xen/asm/system.h: In function `__cmpxchg':
> include/asm-xen/asm/system.h:259: error: `LOCK' undeclared 
> (first use in this
> function)
> ==
> 
> This is because asm/system.h is being included recursively, 
> so none of the defines in asm/atomic.h are defined yet.
> 
> The steps I took to repeat this:
> 
> 1: apt-get install lkinux-tree-2.6.12
> 2: /usr/src/kernel-patches/all/2.6.12/unpatch/debian 2.6.12
>    # this gives a real 2.6.12 tree
> 3: cd linux-2.6-sparse
> 4: ./mkbuildtree $path_to_linux
>    # this applies the 2.6.12.6 patch, then overlays the xen 
> code on top of it.
> 5: create diff between xen-patched linux and plain 2.6.12.6
> 6: Go to another machine
> 7: extract 2.6.12, apply 2.6.12.6; no warnings from diff during this
> 8: apply xen patch; no warnings from diff during this
> 9: make menuconfig ARCH=xen
> 10: error as above
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 

RE: compiler errors in latest unstable from 20051027

[Adam Heath]

On Wed, 9 Nov 2005, Ian Pratt wrote:

>
>
> > -----Original Message-----
> > From: xen-devel-bounces@lists.xensource.com
> > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Adam Heath
> > Sent: 09 November 2005 17:23
> > To: xen-devel@lists.xensource.com
> > Subject: [Xen-devel] compiler errors in latest unstable from 20051027
>
> Why are you using unstable from 20051027 rather than today? A week is a
> long time in the unstable tree, and there have definitely been patches
> in this area.

Er, no.  I verified that smp_alt.h doesn't exist in the latest before sending
this mail.  As for the other problems, that I didn't verify.

A. Because it breaks up the logical flow of conversation.
Q. Why is "top-posting" bad?

RE: compiler errors in latest unstable from 20051027

[Adam Heath]

On Wed, 9 Nov 2005, Adam Heath wrote:

> On Wed, 9 Nov 2005, Ian Pratt wrote:
>
> >
> >
> > > -----Original Message-----
> > > From: xen-devel-bounces@lists.xensource.com
> > > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Adam Heath
> > > Sent: 09 November 2005 17:23
> > > To: xen-devel@lists.xensource.com
> > > Subject: [Xen-devel] compiler errors in latest unstable from 20051027
> >
> > Why are you using unstable from 20051027 rather than today? A week is a
> > long time in the unstable tree, and there have definitely been patches
> > in this area.
>
> Er, no.  I verified that smp_alt.h doesn't exist in the latest before sending
> this mail.  As for the other problems, that I didn't verify.

And I just verified that it doesn't work with latest unstable either.

==
purple2:~/kernel/xen/linux-2.6.12.6-xen0# cd ..
purple2:~/kernel/xen# mv linux-2.6.12.6-xen0/ foo
purple2:~/kernel/xen# rm foo -rf &
purple2:~/kernel/xen# cp linux-2.6.12.6 linux-2.6.12.6-xen0 -a
purple2:~/kernel/xen# wget http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads/xen-unstable-src.tgz
purple2:~/kernel/xen# tar zxf xen-unstable-src.tgz
purple2:~/kernel/xen# cd xen-unstable/linux-2.6-xen-sparse/
purple2:~/kernel/xen/xen-unstable/linux-2.6-xen-sparse# ./mkbuildtree ~/kernel/xen/linux-2.6.12.6-xen0/
purple2:~/kernel/xen/xen-unstable/linux-2.6-xen-sparse# cd ~/kernel/xen/linux-2.6.12.6-xen0/
purple2:~/kernel/xen/linux-2.6.12.6-xen0# cp ../config-xen0 .config
purple2:~/kernel/xen/linux-2.6.12.6-xen0# make oldconfig ARCH=xen
purple2:~/kernel/xen/linux-2.6.12.6-xen0# make bzImage modules ARCH=xen CC=gcc-3.3
....
 CC      arch/i386/kernel/asm-offsets.s
In file included from include/asm-xen/asm/processor.h:18,
                 from include/asm/thread_info.h:17,
                 from include/linux/thread_info.h:21,
                 from include/linux/spinlock.h:12,
                 from include/linux/capability.h:45,
                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:
include/asm-xen/asm/system.h:11:25: asm/smp_alt.h: No such file or directory
In file included from include/asm-xen/asm/processor.h:18,
                 from include/asm/thread_info.h:17,
                 from include/linux/thread_info.h:21,
                 from include/linux/spinlock.h:12,
                 from include/linux/capability.h:45,
                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:
include/asm-xen/asm/system.h: In function `__cmpxchg':
include/asm-xen/asm/system.h:256: error: `LOCK' undeclared (first use in this function)
include/asm-xen/asm/system.h:256: error: (Each undeclared identifier is reported only once
include/asm-xen/asm/system.h:256: error: for each function it appears in.)
include/asm-xen/asm/system.h:256: error: syntax error before string constant
include/asm-xen/asm/system.h:262: error: syntax error before string constant
include/asm-xen/asm/system.h:268: error: syntax error before string constant
In file included from include/linux/spinlock.h:43,
                 from include/linux/capability.h:45,
                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:
include/asm-xen/asm/spinlock.h:9:25: asm/smp_alt.h: No such file or directory
make[1]: *** [arch/i386/kernel/asm-offsets.s] Error 1
make: *** [arch/i386/kernel/asm-offsets.s] Error 2
==

RE: compiler errors in latest unstable from 20051027

[Jeremy Katz]

[-- Attachment #1: Type: text/plain, Size: 406 bytes --]

On Wed, 2005-11-09 at 15:59 -0600, Adam Heath wrote:
> > Er, no.  I verified that smp_alt.h doesn't exist in the latest before sending
> > this mail.  As for the other problems, that I didn't verify.
> 
> And I just verified that it doesn't work with latest unstable either.

It looks like a few places aren't making sure that the smp_alternatives
patch is applied... does the attached fix things?

Jeremy

[-- Attachment #2: sparse-smp-alt.patch --]
[-- Type: text/x-patch, Size: 949 bytes --]

diff -r abbe3df33774 linux-2.6-xen-sparse/include/asm-xen/asm-i386/spinlock.h
--- a/linux-2.6-xen-sparse/include/asm-xen/asm-i386/spinlock.h	Tue Nov  8 17:39:58 2005
+++ b/linux-2.6-xen-sparse/include/asm-xen/asm-i386/spinlock.h	Wed Nov  9 17:08:18 2005
@@ -6,7 +6,10 @@
 #include <asm/page.h>
 #include <linux/config.h>
 #include <linux/compiler.h>
+
+#ifdef CONFIG_SMP_ALTERNATIVES
 #include <asm/smp_alt.h>
+#endif
 
 asmlinkage int printk(const char * fmt, ...)
 	__attribute__ ((format (printf, 1, 2)));
diff -r abbe3df33774 linux-2.6-xen-sparse/include/asm-xen/asm-i386/system.h
--- a/linux-2.6-xen-sparse/include/asm-xen/asm-i386/system.h	Tue Nov  8 17:39:58 2005
+++ b/linux-2.6-xen-sparse/include/asm-xen/asm-i386/system.h	Wed Nov  9 17:08:18 2005
@@ -8,7 +8,10 @@
 #include <asm/segment.h>
 #include <asm/cpufeature.h>
 #include <asm/hypervisor.h>
+
+#ifdef CONFIG_SMP_ALTERNATIVES
 #include <asm/smp_alt.h>
+#endif
 
 #ifdef __KERNEL__
 

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: compiler errors in latest unstable from 20051027

[Adam Heath]

On Wed, 9 Nov 2005, Jeremy Katz wrote:

> On Wed, 2005-11-09 at 15:59 -0600, Adam Heath wrote:
> > > Er, no.  I verified that smp_alt.h doesn't exist in the latest before sending
> > > this mail.  As for the other problems, that I didn't verify.
> >
> > And I just verified that it doesn't work with latest unstable either.
>
> It looks like a few places aren't making sure that the smp_alternatives
> patch is applied... does the attached fix things?

==
purple2:~/kernel/xen/linux-2.6.12.6-xen0# patch -p2 < ~/sparse-smp-alt.patch
patching file include/asm-xen/asm-i386/spinlock.h
patching file include/asm-xen/asm-i386/system.h
purple2:~/kernel/xen/linux-2.6.12.6-xen0# make bzImage modules ARCH=xen
CC=gcc-3.3
  CHK     include/linux/version.h
  CC      arch/i386/kernel/asm-offsets.s
In file included from include/asm-xen/asm/processor.h:18,
                 from include/asm/thread_info.h:17,
                 from include/linux/thread_info.h:21,
                 from include/linux/spinlock.h:12,
                 from include/linux/capability.h:45,
                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:
include/asm-xen/asm/system.h: In function `__cmpxchg':
include/asm-xen/asm/system.h:259: error: `LOCK' undeclared (first use in this function)
include/asm-xen/asm/system.h:259: error: (Each undeclared identifier is reported only once
include/asm-xen/asm/system.h:259: error: for each function it appears in.)
include/asm-xen/asm/system.h:259: error: syntax error before string constant
include/asm-xen/asm/system.h:265: error: syntax error before string constant
include/asm-xen/asm/system.h:271: error: syntax error before string constant
make[1]: *** [arch/i386/kernel/asm-offsets.s] Error 1
make: *** [arch/i386/kernel/asm-offsets.s] Error 2
==

It may fix the one error, but not the big one.

Re: compiler errors in latest unstable from 20051027

[Keir Fraser]

On 9 Nov 2005, at 23:04, Adam Heath wrote:

> It may fix the one error, but not the big one.

You *must* apply the patches in patches/linux-2.6.x before building. 
They are not optional.

  -- Keir

Re: compiler errors in latest unstable from 20051027

[Adam Heath]

On Thu, 10 Nov 2005, Keir Fraser wrote:

>
> On 9 Nov 2005, at 23:04, Adam Heath wrote:
>
> > It may fix the one error, but not the big one.
>
> You *must* apply the patches in patches/linux-2.6.x before building.
> They are not optional.

Well, see, that's the funny thing.

I was testing out the debs of unstable I made.  The xen patch that is produced
calls the existing makefile targets.  I take the patched 2.6.12 debian source,
revert that back to plain 2.6.12, then name the directory to what the makefile
expects.  I then generate a diff from the patched tree against the original.

And this patch still failed.

However, I'll dig more into it.

Re: compiler errors in latest unstable from 20051027

[Adam Heath]

On Thu, 10 Nov 2005, Adam Heath wrote:

> On Thu, 10 Nov 2005, Keir Fraser wrote:
>
> >
> > On 9 Nov 2005, at 23:04, Adam Heath wrote:
> >
> > > It may fix the one error, but not the big one.
> >
> > You *must* apply the patches in patches/linux-2.6.x before building.
> > They are not optional.
>
> Well, see, that's the funny thing.
>
> I was testing out the debs of unstable I made.  The xen patch that is produced
> calls the existing makefile targets.  I take the patched 2.6.12 debian source,
> revert that back to plain 2.6.12, then name the directory to what the makefile
> expects.  I then generate a diff from the patched tree against the original.
>
> And this patch still failed.
>
> However, I'll dig more into it.

Hmm.  I see something a bit odd.  In buildconfigs/Rules.mk, the shell snippet
that applies the patches in a loop, contains a || exit 1 inside a ()
construct.  This doesn't work.  Please observe the following:

==
adam@gradall:~$ cat /tmp/test.sh
#!/bin/sh
set -x

if [ -d /tmp ]; then
        for i in $(seq 1 10); do ( ! [ $i -eq 9 ] || exit 1 ); done
fi
exit 0
adam@gradall:~$ /tmp/test.sh
+ '[' -d /tmp ']'
++ seq 1 10
+ for i in '$(seq 1 10)'
+ '[' 1 -eq 9 ']'
+ for i in '$(seq 1 10)'
+ '[' 2 -eq 9 ']'
+ for i in '$(seq 1 10)'
+ '[' 3 -eq 9 ']'
+ for i in '$(seq 1 10)'
+ '[' 4 -eq 9 ']'
+ for i in '$(seq 1 10)'
+ '[' 5 -eq 9 ']'
+ for i in '$(seq 1 10)'
+ '[' 6 -eq 9 ']'
+ for i in '$(seq 1 10)'
+ '[' 7 -eq 9 ']'
+ for i in '$(seq 1 10)'
+ '[' 8 -eq 9 ']'
+ for i in '$(seq 1 10)'
+ '[' 9 -eq 9 ']'
+ exit 1
+ for i in '$(seq 1 10)'
+ '[' 10 -eq 9 ']'
+ exit 0
==

Note how the inner exit 1 ran, but it didn't actually abort the loop.  This
means that the patch might fail to apply, or half apply, or have fuzz/offsets,
but the other patches will still be applied, and the makefile won't abort.

The fix is to just remove the ().

This explains why I had part of the smpalts patch applied.

Re: compiler errors in latest unstable from 20051027

[Adam Heath]

On Thu, 10 Nov 2005, Adam Heath wrote:

> On Thu, 10 Nov 2005, Adam Heath wrote:
>
> > On Thu, 10 Nov 2005, Keir Fraser wrote:
> >
> > >
> > > On 9 Nov 2005, at 23:04, Adam Heath wrote:
> > >
> > > > It may fix the one error, but not the big one.
> > >
> > > You *must* apply the patches in patches/linux-2.6.x before building.
> > > They are not optional.
> >
> > Well, see, that's the funny thing.
> >
> > I was testing out the debs of unstable I made.  The xen patch that is produced
> > calls the existing makefile targets.  I take the patched 2.6.12 debian source,
> > revert that back to plain 2.6.12, then name the directory to what the makefile
> > expects.  I then generate a diff from the patched tree against the original.
> >
> > And this patch still failed.
> >
> > However, I'll dig more into it.
>
> Hmm.  I see something a bit odd.  In buildconfigs/Rules.mk, the shell snippet
> that applies the patches in a loop, contains a || exit 1 inside a ()
> construct.  This doesn't work.  Please observe the following:

Even more problems with the build system.

When building a patch, the makefiles create a pristine-linux-$ver.  Then they
create a new ref-linux-$ver, apply everything in patches/linux-$ver/*.  Then,
that is copied tmp-linux-$ver.patch.  Finally, a diff is generated between
ref-linux-$ver and tmp-linux-$ver.patch.

This is bad, as the patch does not contain anything applied between
pristine-linux-$ver and ref-linux-$ver.  That is my problem.  I then took the
generated patch, placed it into a deb, installed the deb, then attempted to
apply the patch.  That's how I got the error from before.

However, just generated a full patch between pristine-* and tmp-* seems wrong,
as that would include things that are not actually xen.  I'll probably end up
taking everything out of patches/linux-$ver, and placing them all into the deb
directly.