Re: 2.6.9-rc1-mm4

Re: 2.6.9-rc1-mm4

[Martin J. Bligh]

Well, the good news is that it compiles now, and without forcing ACPI on.
Yay!

On the downside, it seems to have a new error:

make[1]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.

which appears partway through make install, but only if you do "make -j32",
not make -j.

  CC      fs/reiser4/plugin/file/pseudo.o
  CC      fs/reiser4/plugin/file/file.o
  CC      fs/reiser4/plugin/file/tail_conversion.o
  CC      fs/reiser4/sys_reiser4.o
  LD      fs/reiser4/reiser4.o
  LD      fs/reiser4/built-in.o
  LD      fs/built-in.o
  GEN     .version
make[1]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule
.
  CHK     include/linux/compile.h
  UPD     include/linux/compile.h
  CC      init/version.o
  LD      init/built-in.o
  LD      .tmp_vmlinux1
  KSYM    .tmp_kallsyms1.S
  AS      .tmp_kallsyms1.o
  LD      .tmp_vmlinux2
  KSYM    .tmp_kallsyms2.S
  AS      .tmp_kallsyms2.o
  LD      vmlinux
  SYSMAP  System.map
  SYSMAP  .tmp_System.map
  AS      arch/i386/boot/bootsect.o
  AS      arch/i386/boot/setup.o
  HOSTCC  arch/i386/boot/tools/build
  AS      arch/i386/boot/compressed/head.o
  CC      arch/i386/boot/compressed/misc.o
  OBJCOPY arch/i386/boot/compressed/vmlinux.bin
  LD      arch/i386/boot/bootsect
  LD      arch/i386/boot/setup
  GZIP    arch/i386/boot/compressed/vmlinux.bin.gz
  LD      arch/i386/boot/compressed/piggy.o
  LD      arch/i386/boot/compressed/vmlinux
  OBJCOPY arch/i386/boot/vmlinux.bin
  BUILD   arch/i386/boot/bzImage

Re: 2.6.9-rc1-mm4

[Sam Ravnborg]

On Tue, Sep 07, 2004 at 09:45:02AM -0700, Martin J. Bligh wrote:
> Well, the good news is that it compiles now, and without forcing ACPI on.
> Yay!
> 
> On the downside, it seems to have a new error:
> 
> make[1]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.

Just an information - I will silent this.
[Hmm, -j 32 on my UP...]

	Sam

Re: 2.6.9-rc1-mm4

[Andrew Morton]

"Martin J. Bligh" <mbligh@aracnet.com> wrote:
>
> Well, the good news is that it compiles now, and without forcing ACPI on.
>  Yay!

Does it boot?

>  On the downside, it seems to have a new error:
> 
>  make[1]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.
> 
>  which appears partway through make install, but only if you do "make -j32",
>  not make -j.

Yes, I get them too, with make -j6(ish).  I used to get tons of these
warnings, but it stopped happening maybe a year ago.  It looks like Sam
found a way to bring it back ;)

Re: 2.6.9-rc1-mm4

[Sam Ravnborg]

On Tue, Sep 07, 2004 at 09:45:02AM -0700, Martin J. Bligh wrote:
> Well, the good news is that it compiles now, and without forcing ACPI on.
> Yay!
> 
> On the downside, it seems to have a new error:
> 
> make[1]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.
> 
> which appears partway through make install, but only if you do "make -j32",
> not make -j.
Fixed by following patch:

	Sam

# This is a BitKeeper generated diff -Nru style patch.
#
# ChangeSet
#   2004/09/07 23:20:11+02:00 sam@mars.ravnborg.org 
#   kbuild: fix make -j N build
#   
#   Make did say:
#   make[1]: warning: jobserver unavailable: using -j1.
#   
#   Added '+' flag in relevant places to supress this warning.
#   Also removed some trailing tabs in same area spotted by Adrian Bunk <bunk@fs.tum.de>
#   
#   Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
# 
# Makefile
#   2004/09/07 23:19:54+02:00 sam@mars.ravnborg.org +6 -5
#   Add '+' to avoid '-j1' warning from make
#   Removed trailing tabs
# 
diff -Nru a/Makefile b/Makefile
--- a/Makefile	2004-09-07 23:28:03 +02:00
+++ b/Makefile	2004-09-07 23:28:03 +02:00
@@ -590,7 +590,7 @@
 	. $(srctree)/scripts/mkversion > .tmp_version;	\
 	mv -f .tmp_version .version;			\
 	$(MAKE) $(build)=init
-	
+
 # Generate System.map
 quiet_cmd_sysmap = SYSMAP 
       cmd_sysmap = $(CONFIG_SHELL) $(srctree)/scripts/mksysmap
@@ -600,11 +600,11 @@
 # Generate System.map and verify that the content is consistent
 
 define rule_vmlinux__
-	$(if $(CONFIG_KALLSYMS),,$(call cmd,vmlinux_version))
-	
+	$(if $(CONFIG_KALLSYMS),,+$(call cmd,vmlinux_version))
+
 	$(call cmd,vmlinux__)
 	$(Q)echo 'cmd_$@ := $(cmd_vmlinux__)' > $(@D)/.$(@F).cmd
-	
+
 	$(Q)$(if $($(quiet)cmd_sysmap),                 \
 	  echo '  $($(quiet)cmd_sysmap) System.map' &&) \
 	$(cmd_sysmap) $@ System.map;                    \
@@ -653,9 +653,10 @@
 endef
 
 # Update vmlinux version before link
+# Use + in front of this rule to silent warning about make -j1
 cmd_ksym_ld = $(cmd_vmlinux__)
 define rule_ksym_ld
-	$(call cmd,vmlinux_version)
+	+$(call cmd,vmlinux_version)
 	$(call cmd,vmlinux__)
 	$(Q)echo 'cmd_$@ := $(cmd_vmlinux__)' > $(@D)/.$(@F).cmd
 endef

[scripts] pass %{_smp_mflags} to make(1) in scripts/package/mkspec

[William Lee Irwin III]

On Tue, Sep 07, 2004 at 02:17:41PM -0700, Andrew Morton wrote:
> Yes, I get them too, with make -j6(ish).  I used to get tons of these
> warnings, but it stopped happening maybe a year ago.  It looks like Sam
> found a way to bring it back ;)

This appears to have a specific effect, which is that make -j$N rpm in
fact runs single-threaded. I've been using the following patch on SuSE
and RedHat systems for a while.

This patch passes %{_smp_mflags} to various build phases in
scripts/package/mkspec so that -j$N is honored by make rpm.


-- wli

Index: mm2-2.6.8-rc2/scripts/package/mkspec
===================================================================
--- mm2-2.6.8-rc2.orig/scripts/package/mkspec	2004-08-02 03:03:55.000000000 -0700
+++ mm2-2.6.8-rc2/scripts/package/mkspec	2004-08-03 06:23:08.000000000 -0700
@@ -57,14 +57,14 @@
 echo "%build"
 
 if ! $PREBUILT; then
-echo "make clean && make"
+echo "make clean && make %{_smp_mflags}"
 echo ""
 fi
 
 echo "%install"
 echo 'mkdir -p $RPM_BUILD_ROOT/boot $RPM_BUILD_ROOT/lib $RPM_BUILD_ROOT/lib/modules'
 
-echo 'INSTALL_MOD_PATH=$RPM_BUILD_ROOT make modules_install'
+echo 'INSTALL_MOD_PATH=$RPM_BUILD_ROOT make %{_smp_mflags} modules_install'
 echo 'cp $KBUILD_IMAGE $RPM_BUILD_ROOT'"/boot/vmlinuz-$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"
 
 echo 'cp System.map $RPM_BUILD_ROOT'"/boot/System.map-$VERSION.$PATCHLEVEL.$SUBLEVEL$EXTRAVERSION"

Re: [scripts] pass %{_smp_mflags} to make(1) in scripts/package/mkspec

[Sam Ravnborg]

On Tue, Sep 07, 2004 at 02:58:46PM -0700, William Lee Irwin III wrote:
> On Tue, Sep 07, 2004 at 02:17:41PM -0700, Andrew Morton wrote:
> > Yes, I get them too, with make -j6(ish).  I used to get tons of these
> > warnings, but it stopped happening maybe a year ago.  It looks like Sam
> > found a way to bring it back ;)
> 
> This appears to have a specific effect, which is that make -j$N rpm in
> fact runs single-threaded. I've been using the following patch on SuSE
> and RedHat systems for a while.
> 
> This patch passes %{_smp_mflags} to various build phases in
> scripts/package/mkspec so that -j$N is honored by make rpm.

Finally applied this.
I still plan to look into this pkg stuff - but this may be the best
way for the rpm-pkg target.

	Sam

Re: 2.6.9-rc1-mm4

[Martin J. Bligh]

>> Well, the good news is that it compiles now, and without forcing ACPI on.
>>  Yay!
> 
> Does it boot?

Yup. Performance is the same as other -mm's (scheduler changes bring it
down from mainline quite a bit, but otherwise OK).

Kernbench: (make -j N vmlinux, where N = 16 x num_cpus)
                              Elapsed      System        User         CPU
                2.6.9-rc1       44.97       98.66      576.77     1501.33
            2.6.9-rc1-mm1       46.92      107.27      594.10     1493.67
            2.6.9-rc1-mm2       46.95      107.80      593.65     1493.33
            2.6.9-rc1-mm4       46.93      108.91      593.19     1495.00

M.

Re: 2.6.9-rc1-mm4

[Nick Piggin]

Martin J. Bligh wrote:
>>>Well, the good news is that it compiles now, and without forcing ACPI on.
>>> Yay!
>>
>>Does it boot?
> 
> 
> Yup. Performance is the same as other -mm's (scheduler changes bring it
> down from mainline quite a bit, but otherwise OK).
> 
> Kernbench: (make -j N vmlinux, where N = 16 x num_cpus)
>                               Elapsed      System        User         CPU
>                 2.6.9-rc1       44.97       98.66      576.77     1501.33
>             2.6.9-rc1-mm1       46.92      107.27      594.10     1493.67
>             2.6.9-rc1-mm2       46.95      107.80      593.65     1493.33
>             2.6.9-rc1-mm4       46.93      108.91      593.19     1495.00
> 

I'm looking into this performance thing a bit now (although I think
Andrew is going to drop nicksched from the next mm).

It doesn't seem to be from lack of timeslice: with the default
timeslice, nicksched gives average timeslices for a make -j vmlinux
roughly the same size as those for the 2.6 scheduler (22ms).

Increasing base_timeslice can get it up to more than 25% (28ms) larger,
but it still isn't as quick.

I suspect it may be unfairness in the 2.6 scheduler improving cache
utilisation. Need to find a way to measure that though :P