* [PATCH 00/34] Make kernel build deterministic
@ 2011-04-05 14:58 Michal Marek
2011-04-05 14:58 ` [PATCH 03/34] powerpc: Call gzip with -n Michal Marek
` (4 more replies)
0 siblings, 5 replies; 14+ messages in thread
From: Michal Marek @ 2011-04-05 14:58 UTC (permalink / raw)
To: linux-kbuild
Cc: anil_ravindranath, mchehab, x86, mac, aacraid, linux-mtd,
allan.stephens, hpa, netdev, t.sailer, gwingerde, IvDoorn, elf,
cluster-devel, ccaulfie, mingo, dougthompson, linux-usb,
linux-media, arnaud.giersch, teigland, tony.olech, apw,
linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
jon.maloy, linux-scsi, linuxppc-dev, gregkh, linux-wireless,
linux-kernel, bluesmoke-devel, James.Bottomley, tipc-discussion,
chuanxiao.dong, paulus, dwmw2, davem
Hi,
this series makes it possible to build bit-identical kernel image and
modules from identical sources. Of course the build is already
deterministic in terms of behavior of the code, but the various
timestamps embedded in the object files make it hard to compare two
builds, for instance to verify that a makefile cleanup didn't
accidentally change something. A prime example is /proc/config.gz, which
has both a timestamp in the gzip header and a timestamp in the payload
data. With this series applied, a script like this will produce
identical kernels each time:
#!/bin/bash
rm -rf /dev/shm/{source,build}{,1,2}
export KCONFIG_NOTIMESTAMP=1
export KBUILD_BUILD_TIMESTAMP='Sun May 1 12:00:00 CEST 2011'
export KBUILD_BUILD_USER=user
export KBUILD_BUILD_HOST=host
export ROOT_DEV=FLOPPY
for i in 1 2; do
mkdir /dev/shm/source
# randomize the inode order just for fun
git ls-tree -r -z --name-only HEAD | sort -R -z | xargs -0 \
cp --parents --target=/dev/shm/source
pushd /dev/shm/source
mkdir /dev/shm/build
>/dev/shm/build/all.config
for opt in GCOV_KERNEL UBIFS_FS_DEBUG; do
echo "# CONFIG_$opt is not set" >>"/dev/shm/build"/all.config
done
make O="/dev/shm/build" KCONFIG_ALLCONFIG=1 allmodconfig
make O="/dev/shm/build" -j64
popd
mv /dev/shm/source /dev/shm/source$i
mv /dev/shm/build /dev/shm/build$i
done
diff -rq /dev/shm/build{1,2}
Unfortunatelly, this cannot be used to validate indentation-only
patches, even if CONFIG_DEBUG_INFO is turned off. This is because of the
__FILE__ and __LINE__ macros used in many places. For the same reason,
the source and build directory needs to be the same, otherwise the
results will differ. This was tested on
x86_64/{defconfig,allmodconfig,allyesconfig} and ppc64/defconfig. The
series is also available at
git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild-2.6.git deterministic-build-v1
Michal
Michal Marek (34):
kconfig: Do not record timestamp in auto.conf and autoconf.h
kbuild: Call gzip with -n
powerpc: Call gzip with -n
kbuild: Use the deterministic mode of ar
powerpc: Use the deterministic mode of ar
kbuild: Drop unused LINUX_COMPILE_TIME and LINUX_COMPILE_DOMAIN
macros
kbuild: Allow to override LINUX_COMPILE_BY and LINUX_COMPILE_HOST
macros
initramfs: Use KBUILD_BUILD_TIMESTAMP for generated entries
x86: Allow to override the ROOT_DEV variable
cyclades: Drop __TIME__ usage
nozomi: Drop __TIME__ usage
isdn/diva: Drop __TIME__ usage
media/radio-maxiradio: Drop __TIME__ usage
media/cx231xx: Drop __TIME__ usage
baycom: Drop __TIME__ usage
nand/denali: Drop __TIME__ usage
hdlcdrv: Drop __TIME__ usage
wan/pc300: Drop __TIME__ usage
rt2x00: Drop __TIME__ usage
parport: Drop __TIME__ usage
aacraid: Drop __TIME__ usage
scsi/in2000: Drop __TIME__ usage
scsi/wd33c93: Drop __TIME__ usage
usb/u132-hcd: Drop __TIME__ usage
usb/ftdi-elan: Drop __TIME__ usage
dlm: Drop __TIME__ usage
gfs2: Drop __TIME__ usage
atm: Drop __TIME__ usage
tipc: Drop __TIME__ usage
rio: Drop __DATE__ usage
edac: Drop __DATE__ usage
pmcraid: Drop __DATE__ usage
usb/lh7a40x_udc: Drop __DATE__ usage
checkpatch: Warn about usage of __DATE__, __TIME__ and __TIMESTAMP__
Documentation/kbuild/kbuild.txt | 12 ++++++++++
arch/powerpc/boot/Makefile | 2 +-
arch/powerpc/boot/wrapper | 6 ++--
arch/x86/boot/Makefile | 2 +-
drivers/char/cyclades.c | 3 +-
drivers/char/nozomi.c | 3 +-
drivers/char/rio/rioinit.c | 2 +-
drivers/edac/amd76x_edac.c | 2 +-
drivers/edac/amd8111_edac.c | 2 +-
drivers/edac/amd8131_edac.c | 2 +-
drivers/edac/cpc925_edac.c | 2 +-
drivers/edac/e752x_edac.c | 2 +-
drivers/edac/e7xxx_edac.c | 2 +-
drivers/edac/edac_module.c | 2 +-
drivers/edac/i5000_edac.c | 2 +-
drivers/edac/i5400_edac.c | 2 +-
drivers/edac/i7300_edac.c | 2 +-
drivers/edac/i7core_edac.c | 2 +-
drivers/edac/i82860_edac.c | 2 +-
drivers/edac/i82875p_edac.c | 2 +-
drivers/edac/i82975x_edac.c | 2 +-
drivers/edac/mpc85xx_edac.h | 2 +-
drivers/edac/mv64x60_edac.h | 2 +-
drivers/edac/ppc4xx_edac.c | 2 +-
drivers/edac/r82600_edac.c | 2 +-
drivers/isdn/hardware/eicon/divasfunc.c | 5 +--
drivers/media/radio/radio-maxiradio.c | 3 +-
drivers/media/video/cx231xx/cx231xx-avcore.c | 2 +-
drivers/mtd/nand/denali.c | 3 +-
drivers/net/hamradio/baycom_epp.c | 2 +-
drivers/net/hamradio/baycom_par.c | 2 +-
drivers/net/hamradio/baycom_ser_fdx.c | 2 +-
drivers/net/hamradio/baycom_ser_hdx.c | 2 +-
drivers/net/hamradio/hdlcdrv.c | 2 +-
drivers/net/wan/pc300_drv.c | 3 +-
drivers/net/wireless/rt2x00/rt2x00debug.c | 1 -
drivers/parport/parport_ip32.c | 1 -
drivers/scsi/aacraid/linit.c | 3 +-
drivers/scsi/in2000.c | 2 +-
drivers/scsi/pmcraid.c | 9 +++----
drivers/scsi/pmcraid.h | 1 -
drivers/scsi/wd33c93.c | 7 ++---
drivers/usb/gadget/lh7a40x_udc.c | 7 ++---
drivers/usb/host/u132-hcd.c | 3 +-
drivers/usb/misc/ftdi-elan.c | 3 +-
fs/dlm/main.c | 2 +-
fs/gfs2/main.c | 2 +-
net/atm/lec.c | 2 +-
net/atm/mpc.c | 2 +-
net/tipc/core.c | 3 +-
scripts/Makefile.build | 4 +-
scripts/Makefile.lib | 2 +-
scripts/checkpatch.pl | 5 ++++
scripts/gen_initramfs_list.sh | 8 +++++-
scripts/kconfig/confdata.c | 8 +-----
scripts/mkcompile_h | 30 ++++++++++++-------------
usr/gen_init_cpio.c | 17 +++++++++-----
57 files changed, 109 insertions(+), 105 deletions(-)
--
1.7.4.1
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH 03/34] powerpc: Call gzip with -n
2011-04-05 14:58 [PATCH 00/34] Make kernel build deterministic Michal Marek
@ 2011-04-05 14:58 ` Michal Marek
2011-04-05 14:58 ` [PATCH 05/34] powerpc: Use the deterministic mode of ar Michal Marek
` (3 subsequent siblings)
4 siblings, 0 replies; 14+ messages in thread
From: Michal Marek @ 2011-04-05 14:58 UTC (permalink / raw)
To: linux-kbuild; +Cc: linuxppc-dev, Paul Mackerras, linux-kernel
The timestamps recorded in the .gz files add no value.
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: linuxppc-dev@lists.ozlabs.org
Signed-off-by: Michal Marek <mmarek@suse.cz>
---
arch/powerpc/boot/wrapper | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/boot/wrapper b/arch/powerpc/boot/wrapper
index cb97e75..854797b 100755
--- a/arch/powerpc/boot/wrapper
+++ b/arch/powerpc/boot/wrapper
@@ -251,7 +251,7 @@ if [ -z "$cacheit" -o ! -f "$vmz$gzip" -o "$vmz$gzip" -ot "$kernel" ]; then
${CROSS}objcopy $objflags "$kernel" "$vmz.$$"
if [ -n "$gzip" ]; then
- gzip -f -9 "$vmz.$$"
+ gzip -n -f -9 "$vmz.$$"
fi
if [ -n "$cacheit" ]; then
@@ -336,7 +336,7 @@ coff)
$objbin/hack-coff "$ofile"
;;
cuboot*)
- gzip -f -9 "$ofile"
+ gzip -n -f -9 "$ofile"
${MKIMAGE} -A ppc -O linux -T kernel -C gzip -a "$base" -e "$entry" \
$uboot_version -d "$ofile".gz "$ofile"
;;
@@ -383,6 +383,6 @@ ps3)
odir="$(dirname "$ofile.bin")"
rm -f "$odir/otheros.bld"
- gzip --force -9 --stdout "$ofile.bin" > "$odir/otheros.bld"
+ gzip -n --force -9 --stdout "$ofile.bin" > "$odir/otheros.bld"
;;
esac
--
1.7.4.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 05/34] powerpc: Use the deterministic mode of ar
2011-04-05 14:58 [PATCH 00/34] Make kernel build deterministic Michal Marek
2011-04-05 14:58 ` [PATCH 03/34] powerpc: Call gzip with -n Michal Marek
@ 2011-04-05 14:58 ` Michal Marek
2011-04-05 15:49 ` [PATCH 00/34] Make kernel build deterministic Greg KH
` (2 subsequent siblings)
4 siblings, 0 replies; 14+ messages in thread
From: Michal Marek @ 2011-04-05 14:58 UTC (permalink / raw)
To: linux-kbuild; +Cc: linuxppc-dev, Paul Mackerras, linux-kernel
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: linuxppc-dev@lists.ozlabs.org
Signed-off-by: Michal Marek <mmarek@suse.cz>
---
arch/powerpc/boot/Makefile | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/arch/powerpc/boot/Makefile b/arch/powerpc/boot/Makefile
index 8917816..d028a65 100644
--- a/arch/powerpc/boot/Makefile
+++ b/arch/powerpc/boot/Makefile
@@ -127,7 +127,7 @@ quiet_cmd_bootas = BOOTAS $@
cmd_bootas = $(CROSS32CC) -Wp,-MD,$(depfile) $(BOOTAFLAGS) -c -o $@ $<
quiet_cmd_bootar = BOOTAR $@
- cmd_bootar = $(CROSS32AR) -cr $@.$$$$ $(filter-out FORCE,$^); mv $@.$$$$ $@
+ cmd_bootar = $(CROSS32AR) -crD $@.$$$$ $(filter-out FORCE,$^); mv $@.$$$$ $@
$(obj-libfdt): $(obj)/%.o: $(srctree)/scripts/dtc/libfdt/%.c FORCE
$(call if_changed_dep,bootcc)
--
1.7.4.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH 00/34] Make kernel build deterministic
2011-04-05 14:58 [PATCH 00/34] Make kernel build deterministic Michal Marek
2011-04-05 14:58 ` [PATCH 03/34] powerpc: Call gzip with -n Michal Marek
2011-04-05 14:58 ` [PATCH 05/34] powerpc: Use the deterministic mode of ar Michal Marek
@ 2011-04-05 15:49 ` Greg KH
2011-04-05 18:16 ` James Bottomley
` (2 more replies)
2011-04-06 9:01 ` Ingo Molnar
2011-05-05 15:22 ` [PATCH v2] powerpc: Use the deterministic mode of ar Michal Marek
4 siblings, 3 replies; 14+ messages in thread
From: Greg KH @ 2011-04-05 15:49 UTC (permalink / raw)
To: Michal Marek
Cc: anil_ravindranath, mchehab, x86, mac, aacraid, linux-mtd,
allan.stephens, hpa, netdev, t.sailer, gwingerde, IvDoorn, elf,
cluster-devel, ccaulfie, mingo, dougthompson, linux-media,
arnaud.giersch, linux-kbuild, teigland, tony.olech, apw,
linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
jon.maloy, linux-scsi, linuxppc-dev, linux-usb, linux-wireless,
linux-kernel, bluesmoke-devel, James.Bottomley, tipc-discussion,
chuanxiao.dong, paulus, dwmw2, davem
On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
>
> Hi,
>
> this series makes it possible to build bit-identical kernel image and
> modules from identical sources. Of course the build is already
> deterministic in terms of behavior of the code, but the various
> timestamps embedded in the object files make it hard to compare two
> builds, for instance to verify that a makefile cleanup didn't
> accidentally change something. A prime example is /proc/config.gz, which
> has both a timestamp in the gzip header and a timestamp in the payload
> data. With this series applied, a script like this will produce
> identical kernels each time:
Very nice stuff. Do you want to take the individual patches through one
of your trees, or do you mind if the subsystem maintainers take them
through theirs?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 00/34] Make kernel build deterministic
2011-04-05 15:49 ` [PATCH 00/34] Make kernel build deterministic Greg KH
@ 2011-04-05 18:16 ` James Bottomley
2011-04-05 18:29 ` Mauro Carvalho Chehab
2011-04-05 19:24 ` Artem Bityutskiy
2011-04-06 9:23 ` Michal Marek
2 siblings, 1 reply; 14+ messages in thread
From: James Bottomley @ 2011-04-05 18:16 UTC (permalink / raw)
To: Greg KH
Cc: anil_ravindranath, mchehab, x86, mac, aacraid, linux-mtd,
allan.stephens, hpa, netdev, t.sailer, gwingerde, IvDoorn, elf,
cluster-devel, ccaulfie, mingo, dougthompson, linux-media,
jon.maloy, arnaud.giersch, linux-kbuild, teigland, tony.olech,
apw, linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
Michal Marek, linux-scsi, linuxppc-dev, linux-usb, linux-wireless,
linux-kernel, bluesmoke-devel, tipc-discussion, chuanxiao.dong,
paulus, dwmw2, davem
On Tue, 2011-04-05 at 08:49 -0700, Greg KH wrote:
> On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
> >
> > Hi,
> >
> > this series makes it possible to build bit-identical kernel image and
> > modules from identical sources. Of course the build is already
> > deterministic in terms of behavior of the code, but the various
> > timestamps embedded in the object files make it hard to compare two
> > builds, for instance to verify that a makefile cleanup didn't
> > accidentally change something. A prime example is /proc/config.gz, which
> > has both a timestamp in the gzip header and a timestamp in the payload
> > data. With this series applied, a script like this will produce
> > identical kernels each time:
>
> Very nice stuff. Do you want to take the individual patches through one
> of your trees, or do you mind if the subsystem maintainers take them
> through theirs?
I'm happy for this to go through a single tree.
James
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 00/34] Make kernel build deterministic
2011-04-05 18:16 ` James Bottomley
@ 2011-04-05 18:29 ` Mauro Carvalho Chehab
2011-04-05 18:44 ` Greg KH
0 siblings, 1 reply; 14+ messages in thread
From: Mauro Carvalho Chehab @ 2011-04-05 18:29 UTC (permalink / raw)
To: James Bottomley
Cc: anil_ravindranath, x86, mac, aacraid, linux-mtd, allan.stephens,
hpa, netdev, t.sailer, gwingerde, IvDoorn, elf, cluster-devel,
ccaulfie, mingo, dougthompson, linux-media, jon.maloy,
arnaud.giersch, linux-kbuild, teigland, tony.olech, apw,
linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
Michal Marek, linux-scsi, linuxppc-dev, Greg KH, linux-wireless,
linux-kernel, bluesmoke-devel, linux-usb, tipc-discussion,
chuanxiao.dong, paulus, dwmw2, davem
Em 05-04-2011 15:16, James Bottomley escreveu:
> On Tue, 2011-04-05 at 08:49 -0700, Greg KH wrote:
>> On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
>>>
>>> Hi,
>>>
>>> this series makes it possible to build bit-identical kernel image and
>>> modules from identical sources. Of course the build is already
>>> deterministic in terms of behavior of the code, but the various
>>> timestamps embedded in the object files make it hard to compare two
>>> builds, for instance to verify that a makefile cleanup didn't
>>> accidentally change something. A prime example is /proc/config.gz, which
>>> has both a timestamp in the gzip header and a timestamp in the payload
>>> data. With this series applied, a script like this will produce
>>> identical kernels each time:
>>
>> Very nice stuff. Do you want to take the individual patches through one
>> of your trees, or do you mind if the subsystem maintainers take them
>> through theirs?
>
> I'm happy for this to go through a single tree.
Me too.
With respect to the patches I was c/c (patches 13, 14, 31):
Acked-by: Mauro Carvalho Chehab <mchehab@redhat.com>
Thanks,
Mauro.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 00/34] Make kernel build deterministic
2011-04-05 18:29 ` Mauro Carvalho Chehab
@ 2011-04-05 18:44 ` Greg KH
0 siblings, 0 replies; 14+ messages in thread
From: Greg KH @ 2011-04-05 18:44 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: anil_ravindranath, x86, mac, aacraid, linux-mtd, allan.stephens,
hpa, netdev, t.sailer, gwingerde, IvDoorn, elf, cluster-devel,
ccaulfie, mingo, dougthompson, linux-media, jon.maloy,
arnaud.giersch, linux-kbuild, teigland, tony.olech, apw,
linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
Michal Marek, linux-scsi, linuxppc-dev, linux-usb, linux-wireless,
linux-kernel, bluesmoke-devel, James Bottomley, tipc-discussion,
chuanxiao.dong, paulus, dwmw2, davem
On Tue, Apr 05, 2011 at 03:29:19PM -0300, Mauro Carvalho Chehab wrote:
> Em 05-04-2011 15:16, James Bottomley escreveu:
> > On Tue, 2011-04-05 at 08:49 -0700, Greg KH wrote:
> >> On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
> >>>
> >>> Hi,
> >>>
> >>> this series makes it possible to build bit-identical kernel image and
> >>> modules from identical sources. Of course the build is already
> >>> deterministic in terms of behavior of the code, but the various
> >>> timestamps embedded in the object files make it hard to compare two
> >>> builds, for instance to verify that a makefile cleanup didn't
> >>> accidentally change something. A prime example is /proc/config.gz, which
> >>> has both a timestamp in the gzip header and a timestamp in the payload
> >>> data. With this series applied, a script like this will produce
> >>> identical kernels each time:
> >>
> >> Very nice stuff. Do you want to take the individual patches through one
> >> of your trees, or do you mind if the subsystem maintainers take them
> >> through theirs?
> >
> > I'm happy for this to go through a single tree.
>
> Me too.
>
> With respect to the patches I was c/c (patches 13, 14, 31):
>
> Acked-by: Mauro Carvalho Chehab <mchehab@redhat.com>
Me too.
Acked-by: Greg Kroah-Hartman <gregkh@suse.de>
on the patches I was copied on.
thanks,
greg k-h
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 00/34] Make kernel build deterministic
2011-04-05 15:49 ` [PATCH 00/34] Make kernel build deterministic Greg KH
2011-04-05 18:16 ` James Bottomley
@ 2011-04-05 19:24 ` Artem Bityutskiy
2011-04-06 8:57 ` Ingo Molnar
2011-04-06 9:07 ` Michal Marek
2011-04-06 9:23 ` Michal Marek
2 siblings, 2 replies; 14+ messages in thread
From: Artem Bityutskiy @ 2011-04-05 19:24 UTC (permalink / raw)
To: Greg KH
Cc: anil_ravindranath, mchehab, mac, aacraid, linux-mtd,
allan.stephens, hpa, t.sailer, gwingerde, x86, elf, cluster-devel,
ccaulfie, mingo, dougthompson, dwmw2, linux-media, jon.maloy,
arnaud.giersch, linux-kbuild, IvDoorn, teigland, tony.olech, apw,
linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
Michal Marek, linux-scsi, netdev, linux-usb, linux-wireless,
linux-kernel, davem, James.Bottomley, tipc-discussion,
chuanxiao.dong, paulus, linuxppc-dev, bluesmoke-devel
On Tue, 2011-04-05 at 08:49 -0700, Greg KH wrote:
> On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
> >
> > Hi,
> >
> > this series makes it possible to build bit-identical kernel image and
> > modules from identical sources. Of course the build is already
> > deterministic in terms of behavior of the code, but the various
> > timestamps embedded in the object files make it hard to compare two
> > builds, for instance to verify that a makefile cleanup didn't
> > accidentally change something. A prime example is /proc/config.gz, which
> > has both a timestamp in the gzip header and a timestamp in the payload
> > data. With this series applied, a script like this will produce
> > identical kernels each time:
>
> Very nice stuff. Do you want to take the individual patches through one
> of your trees, or do you mind if the subsystem maintainers take them
> through theirs?
But unfortunately, it is very easy to break this and for sure it'll be
broken very soon.
So additionally, I'd suggest:
1. Instrument checkpatch.pl and make it err or warn on timestamps.
2. Probably instrument linux-next to rise a warning when people break
this.
--
Best Regards,
Artem Bityutskiy (Битюцкий Артём)
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 00/34] Make kernel build deterministic
2011-04-05 19:24 ` Artem Bityutskiy
@ 2011-04-06 8:57 ` Ingo Molnar
2011-04-06 9:07 ` Michal Marek
1 sibling, 0 replies; 14+ messages in thread
From: Ingo Molnar @ 2011-04-06 8:57 UTC (permalink / raw)
To: Artem Bityutskiy
Cc: anil_ravindranath, mchehab, mac, aacraid, linux-mtd,
allan.stephens, hpa, t.sailer, gwingerde, x86, elf, cluster-devel,
ccaulfie, mingo, dougthompson, tipc-discussion, dwmw2,
linux-media, jon.maloy, arnaud.giersch, linux-kbuild, IvDoorn,
teigland, tony.olech, apw, linux-hams, tglx, swhiteho,
linux-arm-kernel, linux-edac, Michal Marek, linux-scsi, netdev,
Greg KH, linux-wireless, linux-kernel, davem, linux-usb,
James.Bottomley, chuanxiao.dong, paulus, linuxppc-dev,
bluesmoke-devel
* Artem Bityutskiy <dedekind1@gmail.com> wrote:
> On Tue, 2011-04-05 at 08:49 -0700, Greg KH wrote:
> > On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
> > >
> > > Hi,
> > >
> > > this series makes it possible to build bit-identical kernel image and
> > > modules from identical sources. Of course the build is already
> > > deterministic in terms of behavior of the code, but the various
> > > timestamps embedded in the object files make it hard to compare two
> > > builds, for instance to verify that a makefile cleanup didn't
> > > accidentally change something. A prime example is /proc/config.gz, which
> > > has both a timestamp in the gzip header and a timestamp in the payload
> > > data. With this series applied, a script like this will produce
> > > identical kernels each time:
> >
> > Very nice stuff. Do you want to take the individual patches through one
> > of your trees, or do you mind if the subsystem maintainers take them
> > through theirs?
>
> But unfortunately, it is very easy to break this and for sure it'll be
> broken very soon.
>
> So additionally, I'd suggest:
> 1. Instrument checkpatch.pl and make it err or warn on timestamps.
See the grandparent mail:
checkpatch: Warn about usage of __DATE__, __TIME__ and __TIMESTAMP__
Thanks,
Ingo
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 00/34] Make kernel build deterministic
2011-04-05 14:58 [PATCH 00/34] Make kernel build deterministic Michal Marek
` (2 preceding siblings ...)
2011-04-05 15:49 ` [PATCH 00/34] Make kernel build deterministic Greg KH
@ 2011-04-06 9:01 ` Ingo Molnar
2011-05-05 15:22 ` [PATCH v2] powerpc: Use the deterministic mode of ar Michal Marek
4 siblings, 0 replies; 14+ messages in thread
From: Ingo Molnar @ 2011-04-06 9:01 UTC (permalink / raw)
To: Michal Marek
Cc: anil_ravindranath, mchehab, x86, mac, aacraid, linux-mtd,
allan.stephens, hpa, netdev, t.sailer, gwingerde, IvDoorn, elf,
cluster-devel, ccaulfie, mingo, dougthompson, linux-usb,
linux-media, arnaud.giersch, linux-kbuild, teigland, tony.olech,
apw, linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
jon.maloy, linux-scsi, linuxppc-dev, gregkh, linux-wireless,
linux-kernel, bluesmoke-devel, James.Bottomley, tipc-discussion,
chuanxiao.dong, paulus, dwmw2, davem
* Michal Marek <mmarek@suse.cz> wrote:
>
> Hi,
>
> this series makes it possible to build bit-identical kernel image and
> modules from identical sources. Of course the build is already
> deterministic in terms of behavior of the code, but the various
> timestamps embedded in the object files make it hard to compare two
> builds, for instance to verify that a makefile cleanup didn't
> accidentally change something. A prime example is /proc/config.gz, which
> has both a timestamp in the gzip header and a timestamp in the payload
> data. With this series applied, a script like this will produce
> identical kernels each time:
>
> #!/bin/bash
> rm -rf /dev/shm/{source,build}{,1,2}
> export KCONFIG_NOTIMESTAMP=1
> export KBUILD_BUILD_TIMESTAMP='Sun May 1 12:00:00 CEST 2011'
> export KBUILD_BUILD_USER=user
> export KBUILD_BUILD_HOST=host
> export ROOT_DEV=FLOPPY
> for i in 1 2; do
> mkdir /dev/shm/source
> # randomize the inode order just for fun
> git ls-tree -r -z --name-only HEAD | sort -R -z | xargs -0 \
> cp --parents --target=/dev/shm/source
> pushd /dev/shm/source
> mkdir /dev/shm/build
> >/dev/shm/build/all.config
> for opt in GCOV_KERNEL UBIFS_FS_DEBUG; do
> echo "# CONFIG_$opt is not set" >>"/dev/shm/build"/all.config
> done
> make O="/dev/shm/build" KCONFIG_ALLCONFIG=1 allmodconfig
> make O="/dev/shm/build" -j64
> popd
> mv /dev/shm/source /dev/shm/source$i
> mv /dev/shm/build /dev/shm/build$i
> done
> diff -rq /dev/shm/build{1,2}
Nice!
Acked-by: Ingo Molnar <mingo@elte.hu>
> Unfortunatelly, this cannot be used to validate indentation-only
> patches, even if CONFIG_DEBUG_INFO is turned off. This is because of the
> __FILE__ and __LINE__ macros used in many places. For the same reason,
> the source and build directory needs to be the same, otherwise the
> results will differ.
Nor can it be used to validate untrusted patches in general: a subtle change
might be introduced in a piece of code that is dependent on a .config detail
that is off for that particular build.
But in the common case it's nice to be able to have deterministic/reproducable
builds.
Thanks,
Ingo
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 00/34] Make kernel build deterministic
2011-04-05 19:24 ` Artem Bityutskiy
2011-04-06 8:57 ` Ingo Molnar
@ 2011-04-06 9:07 ` Michal Marek
2011-04-06 9:25 ` Artem Bityutskiy
1 sibling, 1 reply; 14+ messages in thread
From: Michal Marek @ 2011-04-06 9:07 UTC (permalink / raw)
To: dedekind1
Cc: anil_ravindranath, mchehab, mac, aacraid, linux-mtd,
allan.stephens, hpa, t.sailer, gwingerde, x86, elf, cluster-devel,
ccaulfie, mingo, dougthompson, tipc-discussion, dwmw2,
linux-media, arnaud.giersch, linux-kbuild, IvDoorn, teigland,
tony.olech, apw, linux-hams, tglx, swhiteho, linux-arm-kernel,
linux-edac, jon.maloy, linux-scsi, netdev, Greg KH,
linux-wireless, linux-kernel, davem, linux-usb, James.Bottomley,
chuanxiao.dong, paulus, linuxppc-dev, bluesmoke-devel
On 5.4.2011 21:24, Artem Bityutskiy wrote:
> On Tue, 2011-04-05 at 08:49 -0700, Greg KH wrote:
>> On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
>>>
>>> Hi,
>>>
>>> this series makes it possible to build bit-identical kernel image and
>>> modules from identical sources. Of course the build is already
>>> deterministic in terms of behavior of the code, but the various
>>> timestamps embedded in the object files make it hard to compare two
>>> builds, for instance to verify that a makefile cleanup didn't
>>> accidentally change something. A prime example is /proc/config.gz, which
>>> has both a timestamp in the gzip header and a timestamp in the payload
>>> data. With this series applied, a script like this will produce
>>> identical kernels each time:
>>
>> Very nice stuff. Do you want to take the individual patches through one
>> of your trees, or do you mind if the subsystem maintainers take them
>> through theirs?
>
> But unfortunately, it is very easy to break this and for sure it'll be
> broken very soon.
I'm not so pessimistic. 34 patches and 57 files might sound like a lot,
but given that this has been accumulating since day one, the cleanup
should last for some time.
> So additionally, I'd suggest:
> 1. Instrument checkpatch.pl and make it err or warn on timestamps.
This is patch 34/34 in this series: https://lkml.org/lkml/2011/4/5/198
> 2. Probably instrument linux-next to rise a warning when people break
> this.
I'm not sure if Stephen has that much spare time, and I don't think it
is necessary. I think the checkpatch check is sufficient and I'll check
myself occasionally.
Michal
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 00/34] Make kernel build deterministic
2011-04-05 15:49 ` [PATCH 00/34] Make kernel build deterministic Greg KH
2011-04-05 18:16 ` James Bottomley
2011-04-05 19:24 ` Artem Bityutskiy
@ 2011-04-06 9:23 ` Michal Marek
2 siblings, 0 replies; 14+ messages in thread
From: Michal Marek @ 2011-04-06 9:23 UTC (permalink / raw)
To: Greg KH
Cc: anil_ravindranath, mchehab, x86, mac, aacraid, linux-mtd,
allan.stephens, hpa, netdev, t.sailer, gwingerde, IvDoorn, elf,
cluster-devel, ccaulfie, mingo, dougthompson, linux-media,
arnaud.giersch, linux-kbuild, teigland, tony.olech, apw,
linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
jon.maloy, linux-scsi, linuxppc-dev, linux-usb, linux-wireless,
linux-kernel, bluesmoke-devel, James.Bottomley, tipc-discussion,
chuanxiao.dong, paulus, dwmw2, davem
On 5.4.2011 17:49, Greg KH wrote:
> Very nice stuff. Do you want to take the individual patches through one
> of your trees, or do you mind if the subsystem maintainers take them
> through theirs?
I'd leave it up to the subsystem maintainers. I'll check once the merge
window opens and send the remaining patches to Linus directly.
Michal
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 00/34] Make kernel build deterministic
2011-04-06 9:07 ` Michal Marek
@ 2011-04-06 9:25 ` Artem Bityutskiy
0 siblings, 0 replies; 14+ messages in thread
From: Artem Bityutskiy @ 2011-04-06 9:25 UTC (permalink / raw)
To: Michal Marek
Cc: anil_ravindranath, mchehab, mac, aacraid, linux-mtd,
allan.stephens, hpa, t.sailer, gwingerde, x86, elf, cluster-devel,
ccaulfie, mingo, dougthompson, tipc-discussion, dwmw2,
linux-media, arnaud.giersch, linux-kbuild, IvDoorn, teigland,
tony.olech, apw, linux-hams, tglx, swhiteho, linux-arm-kernel,
linux-edac, jon.maloy, linux-scsi, netdev, Greg KH,
linux-wireless, linux-kernel, davem, linux-usb, James.Bottomley,
chuanxiao.dong, paulus, linuxppc-dev, bluesmoke-devel
On Wed, 2011-04-06 at 11:07 +0200, Michal Marek wrote:
> > So additionally, I'd suggest:
> > 1. Instrument checkpatch.pl and make it err or warn on timestamps.
>
> This is patch 34/34 in this series: https://lkml.org/lkml/2011/4/5/198
Yeah, great, did not notice, thanks!
> > 2. Probably instrument linux-next to rise a warning when people break
> > this.
>
> I'm not sure if Stephen has that much spare time, and I don't think it
> is necessary. I think the checkpatch check is sufficient and I'll check
> myself occasionally.
Yes, that's fine, I just wanted to speak this out - there is a
probability that someone gets excited and creates some instrumentation
to kbuild to automatically detect bad things and then Stephen could
easily use that.
WRT "not necessary" - well, I think it is always better to cut bad
things before they are merged, rather than afterwards.
But anyway, let's forget about this.
--
Best Regards,
Artem Bityutskiy (Артём Битюцкий)
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH v2] powerpc: Use the deterministic mode of ar
2011-04-05 14:58 [PATCH 00/34] Make kernel build deterministic Michal Marek
` (3 preceding siblings ...)
2011-04-06 9:01 ` Ingo Molnar
@ 2011-05-05 15:22 ` Michal Marek
4 siblings, 0 replies; 14+ messages in thread
From: Michal Marek @ 2011-05-05 15:22 UTC (permalink / raw)
To: linux-kbuild; +Cc: linuxppc-dev, Paul Mackerras, linux-kernel
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: linuxppc-dev@lists.ozlabs.org
Signed-off-by: Michal Marek <mmarek@suse.cz>
---
Hi,
this version fixes build with binutils <= 2.19 by first checking if ar(1)
supports the D option. It depends on a previous patch in the kbuild tree,
commit 40df759e (kbuild: Fix build with binutils <= 2.19) in linux-next,
so it will need to go through the kbuild tree. But feedback from the PPC
maintainers would be appreciated.
Thanks
Michal
---
arch/powerpc/boot/Makefile | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/arch/powerpc/boot/Makefile b/arch/powerpc/boot/Makefile
index 8917816..f0543f8 100644
--- a/arch/powerpc/boot/Makefile
+++ b/arch/powerpc/boot/Makefile
@@ -127,7 +127,7 @@ quiet_cmd_bootas = BOOTAS $@
cmd_bootas = $(CROSS32CC) -Wp,-MD,$(depfile) $(BOOTAFLAGS) -c -o $@ $<
quiet_cmd_bootar = BOOTAR $@
- cmd_bootar = $(CROSS32AR) -cr $@.$$$$ $(filter-out FORCE,$^); mv $@.$$$$ $@
+ cmd_bootar = $(CROSS32AR) -cr$(KBUILD_ARFLAGS) $@.$$$$ $(filter-out FORCE,$^); mv $@.$$$$ $@
$(obj-libfdt): $(obj)/%.o: $(srctree)/scripts/dtc/libfdt/%.c FORCE
$(call if_changed_dep,bootcc)
--
1.7.4.1
^ permalink raw reply related [flat|nested] 14+ messages in thread
end of thread, other threads:[~2011-05-05 15:23 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-04-05 14:58 [PATCH 00/34] Make kernel build deterministic Michal Marek
2011-04-05 14:58 ` [PATCH 03/34] powerpc: Call gzip with -n Michal Marek
2011-04-05 14:58 ` [PATCH 05/34] powerpc: Use the deterministic mode of ar Michal Marek
2011-04-05 15:49 ` [PATCH 00/34] Make kernel build deterministic Greg KH
2011-04-05 18:16 ` James Bottomley
2011-04-05 18:29 ` Mauro Carvalho Chehab
2011-04-05 18:44 ` Greg KH
2011-04-05 19:24 ` Artem Bityutskiy
2011-04-06 8:57 ` Ingo Molnar
2011-04-06 9:07 ` Michal Marek
2011-04-06 9:25 ` Artem Bityutskiy
2011-04-06 9:23 ` Michal Marek
2011-04-06 9:01 ` Ingo Molnar
2011-05-05 15:22 ` [PATCH v2] powerpc: Use the deterministic mode of ar Michal Marek
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).