public inbox for linux-hams@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH 00/34] Make kernel build deterministic
@ 2011-04-05 14:58 Michal Marek
  2011-04-05 14:59 ` [PATCH 15/34] baycom: Drop __TIME__ usage Michal Marek
                   ` (3 more replies)
  0 siblings, 4 replies; 17+ messages in thread
From: Michal Marek @ 2011-04-05 14:58 UTC (permalink / raw)
  To: linux-kbuild
  Cc: anil_ravindranath, mchehab, benh, x86, mac, aacraid, linux-mtd,
	allan.stephens, hpa, netdev, t.sailer, gwingerde, IvDoorn, elf,
	cluster-devel, ccaulfie, mingo, dougthompson, linux-usb,
	linux-media, arnaud.giersch, teigland, tony.olech, apw,
	linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
	jon.maloy, linux-scsi, linuxppc-dev, gregkh, linux-wireless,
	linux-kernel, bluesmoke-devel


Hi,

this series makes it possible to build bit-identical kernel image and
modules from identical sources. Of course the build is already
deterministic in terms of behavior of the code, but the various
timestamps embedded in the object files make it hard to compare two
builds, for instance to verify that a makefile cleanup didn't
accidentally change something. A prime example is /proc/config.gz, which
has both a timestamp in the gzip header and a timestamp in the payload
data. With this series applied, a script like this will produce
identical kernels each time:

#!/bin/bash
rm -rf /dev/shm/{source,build}{,1,2}
export KCONFIG_NOTIMESTAMP=1
export KBUILD_BUILD_TIMESTAMP='Sun May  1 12:00:00 CEST 2011'
export KBUILD_BUILD_USER=user
export KBUILD_BUILD_HOST=host
export ROOT_DEV=FLOPPY
for i in 1 2; do
	mkdir /dev/shm/source
	# randomize the inode order just for fun
	git ls-tree -r -z --name-only HEAD | sort -R -z | xargs -0 \
		cp --parents --target=/dev/shm/source
	pushd /dev/shm/source
	mkdir /dev/shm/build
	>/dev/shm/build/all.config
	for opt in GCOV_KERNEL UBIFS_FS_DEBUG; do
		echo "# CONFIG_$opt is not set" >>"/dev/shm/build"/all.config
	done
	make O="/dev/shm/build" KCONFIG_ALLCONFIG=1 allmodconfig
	make O="/dev/shm/build" -j64
	popd
	mv /dev/shm/source /dev/shm/source$i
	mv /dev/shm/build /dev/shm/build$i
done
diff -rq /dev/shm/build{1,2}

Unfortunatelly, this cannot be used to validate indentation-only
patches, even if CONFIG_DEBUG_INFO is turned off. This is because of the
__FILE__ and __LINE__ macros used in many places. For the same reason,
the source and build directory needs to be the same, otherwise the
results will differ. This was tested on
x86_64/{defconfig,allmodconfig,allyesconfig} and ppc64/defconfig. The
series is also available at

git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild-2.6.git deterministic-build-v1

Michal


Michal Marek (34):
  kconfig: Do not record timestamp in auto.conf and autoconf.h
  kbuild: Call gzip with -n
  powerpc: Call gzip with -n
  kbuild: Use the deterministic mode of ar
  powerpc: Use the deterministic mode of ar
  kbuild: Drop unused LINUX_COMPILE_TIME and LINUX_COMPILE_DOMAIN
    macros
  kbuild: Allow to override LINUX_COMPILE_BY and LINUX_COMPILE_HOST
    macros
  initramfs: Use KBUILD_BUILD_TIMESTAMP for generated entries
  x86: Allow to override the ROOT_DEV variable
  cyclades: Drop __TIME__ usage
  nozomi: Drop __TIME__ usage
  isdn/diva: Drop __TIME__ usage
  media/radio-maxiradio: Drop __TIME__ usage
  media/cx231xx: Drop __TIME__ usage
  baycom: Drop __TIME__ usage
  nand/denali: Drop __TIME__ usage
  hdlcdrv: Drop __TIME__ usage
  wan/pc300: Drop __TIME__ usage
  rt2x00: Drop __TIME__ usage
  parport: Drop __TIME__ usage
  aacraid: Drop __TIME__ usage
  scsi/in2000: Drop __TIME__ usage
  scsi/wd33c93: Drop __TIME__ usage
  usb/u132-hcd: Drop __TIME__ usage
  usb/ftdi-elan: Drop __TIME__ usage
  dlm: Drop __TIME__ usage
  gfs2: Drop __TIME__ usage
  atm: Drop __TIME__ usage
  tipc: Drop __TIME__ usage
  rio: Drop __DATE__ usage
  edac: Drop __DATE__ usage
  pmcraid: Drop __DATE__ usage
  usb/lh7a40x_udc: Drop __DATE__ usage
  checkpatch: Warn about usage of __DATE__, __TIME__ and __TIMESTAMP__

 Documentation/kbuild/kbuild.txt              |   12 ++++++++++
 arch/powerpc/boot/Makefile                   |    2 +-
 arch/powerpc/boot/wrapper                    |    6 ++--
 arch/x86/boot/Makefile                       |    2 +-
 drivers/char/cyclades.c                      |    3 +-
 drivers/char/nozomi.c                        |    3 +-
 drivers/char/rio/rioinit.c                   |    2 +-
 drivers/edac/amd76x_edac.c                   |    2 +-
 drivers/edac/amd8111_edac.c                  |    2 +-
 drivers/edac/amd8131_edac.c                  |    2 +-
 drivers/edac/cpc925_edac.c                   |    2 +-
 drivers/edac/e752x_edac.c                    |    2 +-
 drivers/edac/e7xxx_edac.c                    |    2 +-
 drivers/edac/edac_module.c                   |    2 +-
 drivers/edac/i5000_edac.c                    |    2 +-
 drivers/edac/i5400_edac.c                    |    2 +-
 drivers/edac/i7300_edac.c                    |    2 +-
 drivers/edac/i7core_edac.c                   |    2 +-
 drivers/edac/i82860_edac.c                   |    2 +-
 drivers/edac/i82875p_edac.c                  |    2 +-
 drivers/edac/i82975x_edac.c                  |    2 +-
 drivers/edac/mpc85xx_edac.h                  |    2 +-
 drivers/edac/mv64x60_edac.h                  |    2 +-
 drivers/edac/ppc4xx_edac.c                   |    2 +-
 drivers/edac/r82600_edac.c                   |    2 +-
 drivers/isdn/hardware/eicon/divasfunc.c      |    5 +--
 drivers/media/radio/radio-maxiradio.c        |    3 +-
 drivers/media/video/cx231xx/cx231xx-avcore.c |    2 +-
 drivers/mtd/nand/denali.c                    |    3 +-
 drivers/net/hamradio/baycom_epp.c            |    2 +-
 drivers/net/hamradio/baycom_par.c            |    2 +-
 drivers/net/hamradio/baycom_ser_fdx.c        |    2 +-
 drivers/net/hamradio/baycom_ser_hdx.c        |    2 +-
 drivers/net/hamradio/hdlcdrv.c               |    2 +-
 drivers/net/wan/pc300_drv.c                  |    3 +-
 drivers/net/wireless/rt2x00/rt2x00debug.c    |    1 -
 drivers/parport/parport_ip32.c               |    1 -
 drivers/scsi/aacraid/linit.c                 |    3 +-
 drivers/scsi/in2000.c                        |    2 +-
 drivers/scsi/pmcraid.c                       |    9 +++----
 drivers/scsi/pmcraid.h                       |    1 -
 drivers/scsi/wd33c93.c                       |    7 ++---
 drivers/usb/gadget/lh7a40x_udc.c             |    7 ++---
 drivers/usb/host/u132-hcd.c                  |    3 +-
 drivers/usb/misc/ftdi-elan.c                 |    3 +-
 fs/dlm/main.c                                |    2 +-
 fs/gfs2/main.c                               |    2 +-
 net/atm/lec.c                                |    2 +-
 net/atm/mpc.c                                |    2 +-
 net/tipc/core.c                              |    3 +-
 scripts/Makefile.build                       |    4 +-
 scripts/Makefile.lib                         |    2 +-
 scripts/checkpatch.pl                        |    5 ++++
 scripts/gen_initramfs_list.sh                |    8 +++++-
 scripts/kconfig/confdata.c                   |    8 +-----
 scripts/mkcompile_h                          |   30 ++++++++++++-------------
 usr/gen_init_cpio.c                          |   17 +++++++++-----
 57 files changed, 109 insertions(+), 105 deletions(-)

-- 
1.7.4.1

^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH 15/34] baycom: Drop __TIME__ usage
  2011-04-05 14:58 [PATCH 00/34] Make kernel build deterministic Michal Marek
@ 2011-04-05 14:59 ` Michal Marek
       [not found]   ` <1302015780.4124.53.camel@xbox360.hq.axsem.com>
  2011-04-05 14:59 ` [PATCH 17/34] hdlcdrv: " Michal Marek
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 17+ messages in thread
From: Michal Marek @ 2011-04-05 14:59 UTC (permalink / raw)
  To: linux-kbuild; +Cc: linux-kernel, Thomas Sailer, linux-hams, netdev

The kernel already prints its build timestamp during boot, no need to
repeat it in random drivers and produce different object files each
time.

Cc: Thomas Sailer <t.sailer@alumni.ethz.ch>
Cc: linux-hams@vger.kernel.org
Cc: netdev@vger.kernel.org
Signed-off-by: Michal Marek <mmarek@suse.cz>
---
 drivers/net/hamradio/baycom_epp.c     |    2 +-
 drivers/net/hamradio/baycom_par.c     |    2 +-
 drivers/net/hamradio/baycom_ser_fdx.c |    2 +-
 drivers/net/hamradio/baycom_ser_hdx.c |    2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/net/hamradio/baycom_epp.c b/drivers/net/hamradio/baycom_epp.c
index a3c0dc9..9537aaa 100644
--- a/drivers/net/hamradio/baycom_epp.c
+++ b/drivers/net/hamradio/baycom_epp.c
@@ -69,7 +69,7 @@ static const char paranoia_str[] = KERN_ERR
 
 static const char bc_drvname[] = "baycom_epp";
 static const char bc_drvinfo[] = KERN_INFO "baycom_epp: (C) 1998-2000 Thomas Sailer, HB9JNX/AE4WA\n"
-"baycom_epp: version 0.7 compiled " __TIME__ " " __DATE__ "\n";
+"baycom_epp: version 0.7\n";
 
 /* --------------------------------------------------------------------- */
 
diff --git a/drivers/net/hamradio/baycom_par.c b/drivers/net/hamradio/baycom_par.c
index 5f5af9a..279d229 100644
--- a/drivers/net/hamradio/baycom_par.c
+++ b/drivers/net/hamradio/baycom_par.c
@@ -102,7 +102,7 @@
 
 static const char bc_drvname[] = "baycom_par";
 static const char bc_drvinfo[] = KERN_INFO "baycom_par: (C) 1996-2000 Thomas Sailer, HB9JNX/AE4WA\n"
-"baycom_par: version 0.9 compiled " __TIME__ " " __DATE__ "\n";
+"baycom_par: version 0.9\n";
 
 /* --------------------------------------------------------------------- */
 
diff --git a/drivers/net/hamradio/baycom_ser_fdx.c b/drivers/net/hamradio/baycom_ser_fdx.c
index 3e25f10..99cdce3 100644
--- a/drivers/net/hamradio/baycom_ser_fdx.c
+++ b/drivers/net/hamradio/baycom_ser_fdx.c
@@ -92,7 +92,7 @@
 
 static const char bc_drvname[] = "baycom_ser_fdx";
 static const char bc_drvinfo[] = KERN_INFO "baycom_ser_fdx: (C) 1996-2000 Thomas Sailer, HB9JNX/AE4WA\n"
-"baycom_ser_fdx: version 0.10 compiled " __TIME__ " " __DATE__ "\n";
+"baycom_ser_fdx: version 0.10\n";
 
 /* --------------------------------------------------------------------- */
 
diff --git a/drivers/net/hamradio/baycom_ser_hdx.c b/drivers/net/hamradio/baycom_ser_hdx.c
index 1686f6d..d92fe6c 100644
--- a/drivers/net/hamradio/baycom_ser_hdx.c
+++ b/drivers/net/hamradio/baycom_ser_hdx.c
@@ -80,7 +80,7 @@
 
 static const char bc_drvname[] = "baycom_ser_hdx";
 static const char bc_drvinfo[] = KERN_INFO "baycom_ser_hdx: (C) 1996-2000 Thomas Sailer, HB9JNX/AE4WA\n"
-"baycom_ser_hdx: version 0.10 compiled " __TIME__ " " __DATE__ "\n";
+"baycom_ser_hdx: version 0.10\n";
 
 /* --------------------------------------------------------------------- */
 
-- 
1.7.4.1

^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH 17/34] hdlcdrv: Drop __TIME__ usage
  2011-04-05 14:58 [PATCH 00/34] Make kernel build deterministic Michal Marek
  2011-04-05 14:59 ` [PATCH 15/34] baycom: Drop __TIME__ usage Michal Marek
@ 2011-04-05 14:59 ` Michal Marek
  2011-04-19 10:17   ` walter harms
       [not found]   ` <1302015790.4124.54.camel@xbox360.hq.axsem.com>
  2011-04-05 15:49 ` [PATCH 00/34] Make kernel build deterministic Greg KH
  2011-04-06  9:01 ` Ingo Molnar
  3 siblings, 2 replies; 17+ messages in thread
From: Michal Marek @ 2011-04-05 14:59 UTC (permalink / raw)
  To: linux-kbuild; +Cc: linux-kernel, Thomas Sailer, linux-hams, netdev

The kernel already prints its build timestamp during boot, no need to
repeat it in random drivers and produce different object files each
time.

Cc: Thomas Sailer <t.sailer@alumni.ethz.ch>
Cc: linux-hams@vger.kernel.org
Cc: netdev@vger.kernel.org
Signed-off-by: Michal Marek <mmarek@suse.cz>
---
 drivers/net/hamradio/hdlcdrv.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/net/hamradio/hdlcdrv.c b/drivers/net/hamradio/hdlcdrv.c
index 5b37579..a4a3516 100644
--- a/drivers/net/hamradio/hdlcdrv.c
+++ b/drivers/net/hamradio/hdlcdrv.c
@@ -749,7 +749,7 @@ EXPORT_SYMBOL(hdlcdrv_unregister);
 static int __init hdlcdrv_init_driver(void)
 {
 	printk(KERN_INFO "hdlcdrv: (C) 1996-2000 Thomas Sailer HB9JNX/AE4WA\n");
-	printk(KERN_INFO "hdlcdrv: version 0.8 compiled " __TIME__ " " __DATE__ "\n");
+	printk(KERN_INFO "hdlcdrv: version 0.8\n");
 	return 0;
 }
 
-- 
1.7.4.1


^ permalink raw reply related	[flat|nested] 17+ messages in thread

* Re: [PATCH 00/34] Make kernel build deterministic
  2011-04-05 14:58 [PATCH 00/34] Make kernel build deterministic Michal Marek
  2011-04-05 14:59 ` [PATCH 15/34] baycom: Drop __TIME__ usage Michal Marek
  2011-04-05 14:59 ` [PATCH 17/34] hdlcdrv: " Michal Marek
@ 2011-04-05 15:49 ` Greg KH
  2011-04-05 18:16   ` James Bottomley
                     ` (2 more replies)
  2011-04-06  9:01 ` Ingo Molnar
  3 siblings, 3 replies; 17+ messages in thread
From: Greg KH @ 2011-04-05 15:49 UTC (permalink / raw)
  To: Michal Marek
  Cc: anil_ravindranath, mchehab, benh, x86, mac, aacraid, linux-mtd,
	allan.stephens, hpa, netdev, t.sailer, gwingerde, IvDoorn, elf,
	cluster-devel, ccaulfie, mingo, dougthompson, linux-media,
	arnaud.giersch, linux-kbuild, teigland, tony.olech, apw,
	linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
	jon.maloy, linux-scsi, linuxppc-dev, linux-usb, linux-wireless,
	linux-kernel, bluesmoke-

On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
> 
> Hi,
> 
> this series makes it possible to build bit-identical kernel image and
> modules from identical sources. Of course the build is already
> deterministic in terms of behavior of the code, but the various
> timestamps embedded in the object files make it hard to compare two
> builds, for instance to verify that a makefile cleanup didn't
> accidentally change something. A prime example is /proc/config.gz, which
> has both a timestamp in the gzip header and a timestamp in the payload
> data. With this series applied, a script like this will produce
> identical kernels each time:

Very nice stuff.  Do you want to take the individual patches through one
of your trees, or do you mind if the subsystem maintainers take them
through theirs?

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 00/34] Make kernel build deterministic
  2011-04-05 15:49 ` [PATCH 00/34] Make kernel build deterministic Greg KH
@ 2011-04-05 18:16   ` James Bottomley
  2011-04-05 18:29     ` Mauro Carvalho Chehab
  2011-04-05 19:24   ` Artem Bityutskiy
  2011-04-06  9:23   ` Michal Marek
  2 siblings, 1 reply; 17+ messages in thread
From: James Bottomley @ 2011-04-05 18:16 UTC (permalink / raw)
  To: Greg KH
  Cc: anil_ravindranath, mchehab, benh, x86, mac, aacraid, linux-mtd,
	allan.stephens, hpa, netdev, t.sailer, gwingerde, IvDoorn, elf,
	cluster-devel, ccaulfie, mingo, dougthompson, linux-media,
	jon.maloy, arnaud.giersch, linux-kbuild, teigland, tony.olech,
	apw, linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
	Michal Marek, linux-scsi, linuxppc-dev, linux-usb, linux-wireless,
	linux-ker

On Tue, 2011-04-05 at 08:49 -0700, Greg KH wrote:
> On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
> > 
> > Hi,
> > 
> > this series makes it possible to build bit-identical kernel image and
> > modules from identical sources. Of course the build is already
> > deterministic in terms of behavior of the code, but the various
> > timestamps embedded in the object files make it hard to compare two
> > builds, for instance to verify that a makefile cleanup didn't
> > accidentally change something. A prime example is /proc/config.gz, which
> > has both a timestamp in the gzip header and a timestamp in the payload
> > data. With this series applied, a script like this will produce
> > identical kernels each time:
> 
> Very nice stuff.  Do you want to take the individual patches through one
> of your trees, or do you mind if the subsystem maintainers take them
> through theirs?

I'm happy for this to go through a single tree.

James

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 00/34] Make kernel build deterministic
  2011-04-05 18:16   ` James Bottomley
@ 2011-04-05 18:29     ` Mauro Carvalho Chehab
  2011-04-05 18:44       ` Greg KH
  0 siblings, 1 reply; 17+ messages in thread
From: Mauro Carvalho Chehab @ 2011-04-05 18:29 UTC (permalink / raw)
  To: James Bottomley
  Cc: anil_ravindranath, benh, x86, mac, aacraid, linux-mtd,
	allan.stephens, hpa, netdev, t.sailer, gwingerde, IvDoorn, elf,
	cluster-devel, ccaulfie, mingo, dougthompson, linux-media,
	jon.maloy, arnaud.giersch, linux-kbuild, teigland, tony.olech,
	apw, linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
	Michal Marek, linux-scsi, linuxppc-dev, Greg KH, linux-wireless,
	linux-kernel

Em 05-04-2011 15:16, James Bottomley escreveu:
> On Tue, 2011-04-05 at 08:49 -0700, Greg KH wrote:
>> On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
>>>
>>> Hi,
>>>
>>> this series makes it possible to build bit-identical kernel image and
>>> modules from identical sources. Of course the build is already
>>> deterministic in terms of behavior of the code, but the various
>>> timestamps embedded in the object files make it hard to compare two
>>> builds, for instance to verify that a makefile cleanup didn't
>>> accidentally change something. A prime example is /proc/config.gz, which
>>> has both a timestamp in the gzip header and a timestamp in the payload
>>> data. With this series applied, a script like this will produce
>>> identical kernels each time:
>>
>> Very nice stuff.  Do you want to take the individual patches through one
>> of your trees, or do you mind if the subsystem maintainers take them
>> through theirs?
> 
> I'm happy for this to go through a single tree.

Me too.

With respect to the patches I was c/c (patches 13, 14, 31):

Acked-by: Mauro Carvalho Chehab <mchehab@redhat.com>

Thanks,
Mauro.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 00/34] Make kernel build deterministic
  2011-04-05 18:29     ` Mauro Carvalho Chehab
@ 2011-04-05 18:44       ` Greg KH
  0 siblings, 0 replies; 17+ messages in thread
From: Greg KH @ 2011-04-05 18:44 UTC (permalink / raw)
  To: Mauro Carvalho Chehab
  Cc: anil_ravindranath, benh, x86, mac, aacraid, linux-mtd,
	allan.stephens, hpa, netdev, t.sailer, gwingerde, IvDoorn, elf,
	cluster-devel, ccaulfie, mingo, dougthompson, linux-media,
	jon.maloy, arnaud.giersch, linux-kbuild, teigland, tony.olech,
	apw, linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
	Michal Marek, linux-scsi, linuxppc-dev, linux-usb, linux-wireless,
	linux-kernel

On Tue, Apr 05, 2011 at 03:29:19PM -0300, Mauro Carvalho Chehab wrote:
> Em 05-04-2011 15:16, James Bottomley escreveu:
> > On Tue, 2011-04-05 at 08:49 -0700, Greg KH wrote:
> >> On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
> >>>
> >>> Hi,
> >>>
> >>> this series makes it possible to build bit-identical kernel image and
> >>> modules from identical sources. Of course the build is already
> >>> deterministic in terms of behavior of the code, but the various
> >>> timestamps embedded in the object files make it hard to compare two
> >>> builds, for instance to verify that a makefile cleanup didn't
> >>> accidentally change something. A prime example is /proc/config.gz, which
> >>> has both a timestamp in the gzip header and a timestamp in the payload
> >>> data. With this series applied, a script like this will produce
> >>> identical kernels each time:
> >>
> >> Very nice stuff.  Do you want to take the individual patches through one
> >> of your trees, or do you mind if the subsystem maintainers take them
> >> through theirs?
> > 
> > I'm happy for this to go through a single tree.
> 
> Me too.
> 
> With respect to the patches I was c/c (patches 13, 14, 31):
> 
> Acked-by: Mauro Carvalho Chehab <mchehab@redhat.com>

Me too.

Acked-by: Greg Kroah-Hartman <gregkh@suse.de>

on the patches I was copied on.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 00/34] Make kernel build deterministic
  2011-04-05 15:49 ` [PATCH 00/34] Make kernel build deterministic Greg KH
  2011-04-05 18:16   ` James Bottomley
@ 2011-04-05 19:24   ` Artem Bityutskiy
  2011-04-06  8:57     ` Ingo Molnar
  2011-04-06  9:07     ` Michal Marek
  2011-04-06  9:23   ` Michal Marek
  2 siblings, 2 replies; 17+ messages in thread
From: Artem Bityutskiy @ 2011-04-05 19:24 UTC (permalink / raw)
  To: Greg KH
  Cc: anil_ravindranath, mchehab, benh, mac, aacraid, linux-mtd,
	allan.stephens, hpa, t.sailer, gwingerde, x86, elf, cluster-devel,
	ccaulfie, mingo, dougthompson, dwmw2, linux-media, jon.maloy,
	arnaud.giersch, linux-kbuild, IvDoorn, teigland, tony.olech, apw,
	linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
	Michal Marek, linux-scsi, netdev, linux-usb, linux-wireless,
	linux-kernel

On Tue, 2011-04-05 at 08:49 -0700, Greg KH wrote:
> On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
> > 
> > Hi,
> > 
> > this series makes it possible to build bit-identical kernel image and
> > modules from identical sources. Of course the build is already
> > deterministic in terms of behavior of the code, but the various
> > timestamps embedded in the object files make it hard to compare two
> > builds, for instance to verify that a makefile cleanup didn't
> > accidentally change something. A prime example is /proc/config.gz, which
> > has both a timestamp in the gzip header and a timestamp in the payload
> > data. With this series applied, a script like this will produce
> > identical kernels each time:
> 
> Very nice stuff.  Do you want to take the individual patches through one
> of your trees, or do you mind if the subsystem maintainers take them
> through theirs?

But unfortunately, it is very easy to break this and for sure it'll be
broken very soon.

So additionally, I'd suggest:
1. Instrument checkpatch.pl and make it err or warn on timestamps.
2. Probably instrument linux-next to rise a warning when people break
   this.

-- 
Best Regards,
Artem Bityutskiy (Битюцкий Артём)


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 00/34] Make kernel build deterministic
  2011-04-05 19:24   ` Artem Bityutskiy
@ 2011-04-06  8:57     ` Ingo Molnar
  2011-04-06  9:07     ` Michal Marek
  1 sibling, 0 replies; 17+ messages in thread
From: Ingo Molnar @ 2011-04-06  8:57 UTC (permalink / raw)
  To: Artem Bityutskiy
  Cc: anil_ravindranath, mchehab, benh, mac, aacraid, linux-mtd,
	allan.stephens, hpa, t.sailer, gwingerde, x86, elf, cluster-devel,
	ccaulfie, mingo, dougthompson, tipc-discussion, dwmw2,
	linux-media, jon.maloy, arnaud.giersch, linux-kbuild, IvDoorn,
	teigland, tony.olech, apw, linux-hams, tglx, swhiteho,
	linux-arm-kernel, linux-edac, Michal Marek, linux-scsi, netdev,
	Greg KH, linux-wireless


* Artem Bityutskiy <dedekind1@gmail.com> wrote:

> On Tue, 2011-04-05 at 08:49 -0700, Greg KH wrote:
> > On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
> > > 
> > > Hi,
> > > 
> > > this series makes it possible to build bit-identical kernel image and
> > > modules from identical sources. Of course the build is already
> > > deterministic in terms of behavior of the code, but the various
> > > timestamps embedded in the object files make it hard to compare two
> > > builds, for instance to verify that a makefile cleanup didn't
> > > accidentally change something. A prime example is /proc/config.gz, which
> > > has both a timestamp in the gzip header and a timestamp in the payload
> > > data. With this series applied, a script like this will produce
> > > identical kernels each time:
> > 
> > Very nice stuff.  Do you want to take the individual patches through one
> > of your trees, or do you mind if the subsystem maintainers take them
> > through theirs?
> 
> But unfortunately, it is very easy to break this and for sure it'll be
> broken very soon.
> 
> So additionally, I'd suggest:
> 1. Instrument checkpatch.pl and make it err or warn on timestamps.

See the grandparent mail:

  checkpatch: Warn about usage of __DATE__, __TIME__ and __TIMESTAMP__

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 00/34] Make kernel build deterministic
  2011-04-05 14:58 [PATCH 00/34] Make kernel build deterministic Michal Marek
                   ` (2 preceding siblings ...)
  2011-04-05 15:49 ` [PATCH 00/34] Make kernel build deterministic Greg KH
@ 2011-04-06  9:01 ` Ingo Molnar
  3 siblings, 0 replies; 17+ messages in thread
From: Ingo Molnar @ 2011-04-06  9:01 UTC (permalink / raw)
  To: Michal Marek
  Cc: anil_ravindranath, mchehab, benh, x86, mac, aacraid, linux-mtd,
	allan.stephens, hpa, netdev, t.sailer, gwingerde, IvDoorn, elf,
	cluster-devel, ccaulfie, mingo, dougthompson, linux-usb,
	linux-media, arnaud.giersch, linux-kbuild, teigland, tony.olech,
	apw, linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
	jon.maloy, linux-scsi, linuxppc-dev, gregkh, linux-wireless,
	linux-kernel


* Michal Marek <mmarek@suse.cz> wrote:

> 
> Hi,
> 
> this series makes it possible to build bit-identical kernel image and
> modules from identical sources. Of course the build is already
> deterministic in terms of behavior of the code, but the various
> timestamps embedded in the object files make it hard to compare two
> builds, for instance to verify that a makefile cleanup didn't
> accidentally change something. A prime example is /proc/config.gz, which
> has both a timestamp in the gzip header and a timestamp in the payload
> data. With this series applied, a script like this will produce
> identical kernels each time:
> 
> #!/bin/bash
> rm -rf /dev/shm/{source,build}{,1,2}
> export KCONFIG_NOTIMESTAMP=1
> export KBUILD_BUILD_TIMESTAMP='Sun May  1 12:00:00 CEST 2011'
> export KBUILD_BUILD_USER=user
> export KBUILD_BUILD_HOST=host
> export ROOT_DEV=FLOPPY
> for i in 1 2; do
> 	mkdir /dev/shm/source
> 	# randomize the inode order just for fun
> 	git ls-tree -r -z --name-only HEAD | sort -R -z | xargs -0 \
> 		cp --parents --target=/dev/shm/source
> 	pushd /dev/shm/source
> 	mkdir /dev/shm/build
> 	>/dev/shm/build/all.config
> 	for opt in GCOV_KERNEL UBIFS_FS_DEBUG; do
> 		echo "# CONFIG_$opt is not set" >>"/dev/shm/build"/all.config
> 	done
> 	make O="/dev/shm/build" KCONFIG_ALLCONFIG=1 allmodconfig
> 	make O="/dev/shm/build" -j64
> 	popd
> 	mv /dev/shm/source /dev/shm/source$i
> 	mv /dev/shm/build /dev/shm/build$i
> done
> diff -rq /dev/shm/build{1,2}

Nice!

Acked-by: Ingo Molnar <mingo@elte.hu>

> Unfortunatelly, this cannot be used to validate indentation-only
> patches, even if CONFIG_DEBUG_INFO is turned off. This is because of the
> __FILE__ and __LINE__ macros used in many places. For the same reason,
> the source and build directory needs to be the same, otherwise the
> results will differ.

Nor can it be used to validate untrusted patches in general: a subtle change 
might be introduced in a piece of code that is dependent on a .config detail 
that is off for that particular build.

But in the common case it's nice to be able to have deterministic/reproducable 
builds.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 00/34] Make kernel build deterministic
  2011-04-05 19:24   ` Artem Bityutskiy
  2011-04-06  8:57     ` Ingo Molnar
@ 2011-04-06  9:07     ` Michal Marek
  2011-04-06  9:25       ` Artem Bityutskiy
  1 sibling, 1 reply; 17+ messages in thread
From: Michal Marek @ 2011-04-06  9:07 UTC (permalink / raw)
  To: dedekind1
  Cc: anil_ravindranath, mchehab, benh, mac, aacraid, linux-mtd,
	allan.stephens, hpa, t.sailer, gwingerde, x86, elf, cluster-devel,
	ccaulfie, mingo, dougthompson, tipc-discussion, dwmw2,
	linux-media, arnaud.giersch, linux-kbuild, IvDoorn, teigland,
	tony.olech, apw, linux-hams, tglx, swhiteho, linux-arm-kernel,
	linux-edac, jon.maloy, linux-scsi, netdev, Greg KH,
	linux-wireless

On 5.4.2011 21:24, Artem Bityutskiy wrote:
> On Tue, 2011-04-05 at 08:49 -0700, Greg KH wrote:
>> On Tue, Apr 05, 2011 at 04:58:47PM +0200, Michal Marek wrote:
>>>
>>> Hi,
>>>
>>> this series makes it possible to build bit-identical kernel image and
>>> modules from identical sources. Of course the build is already
>>> deterministic in terms of behavior of the code, but the various
>>> timestamps embedded in the object files make it hard to compare two
>>> builds, for instance to verify that a makefile cleanup didn't
>>> accidentally change something. A prime example is /proc/config.gz, which
>>> has both a timestamp in the gzip header and a timestamp in the payload
>>> data. With this series applied, a script like this will produce
>>> identical kernels each time:
>>
>> Very nice stuff.  Do you want to take the individual patches through one
>> of your trees, or do you mind if the subsystem maintainers take them
>> through theirs?
> 
> But unfortunately, it is very easy to break this and for sure it'll be
> broken very soon.

I'm not so pessimistic. 34 patches and 57 files might sound like a lot,
but given that this has been accumulating since day one, the cleanup
should last for some time.


> So additionally, I'd suggest:
> 1. Instrument checkpatch.pl and make it err or warn on timestamps.

This is patch 34/34 in this series: https://lkml.org/lkml/2011/4/5/198


> 2. Probably instrument linux-next to rise a warning when people break
>    this.

I'm not sure if Stephen has that much spare time, and I don't think it
is necessary. I think the checkpatch check is sufficient and I'll check
myself occasionally.

Michal

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 00/34] Make kernel build deterministic
  2011-04-05 15:49 ` [PATCH 00/34] Make kernel build deterministic Greg KH
  2011-04-05 18:16   ` James Bottomley
  2011-04-05 19:24   ` Artem Bityutskiy
@ 2011-04-06  9:23   ` Michal Marek
  2 siblings, 0 replies; 17+ messages in thread
From: Michal Marek @ 2011-04-06  9:23 UTC (permalink / raw)
  To: Greg KH
  Cc: anil_ravindranath, mchehab, benh, x86, mac, aacraid, linux-mtd,
	allan.stephens, hpa, netdev, t.sailer, gwingerde, IvDoorn, elf,
	cluster-devel, ccaulfie, mingo, dougthompson, linux-media,
	arnaud.giersch, linux-kbuild, teigland, tony.olech, apw,
	linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
	jon.maloy, linux-scsi, linuxppc-dev, linux-usb, linux-wireless,
	linux-kernel, bluesmoke-

On 5.4.2011 17:49, Greg KH wrote:
> Very nice stuff.  Do you want to take the individual patches through one
> of your trees, or do you mind if the subsystem maintainers take them
> through theirs?

I'd leave it up to the subsystem maintainers. I'll check once the merge
window opens and send the remaining patches to Linus directly.

Michal

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 00/34] Make kernel build deterministic
  2011-04-06  9:07     ` Michal Marek
@ 2011-04-06  9:25       ` Artem Bityutskiy
  0 siblings, 0 replies; 17+ messages in thread
From: Artem Bityutskiy @ 2011-04-06  9:25 UTC (permalink / raw)
  To: Michal Marek
  Cc: anil_ravindranath, mchehab, benh, mac, aacraid, linux-mtd,
	allan.stephens, hpa, t.sailer, gwingerde, x86, elf, cluster-devel,
	ccaulfie, mingo, dougthompson, tipc-discussion, dwmw2,
	linux-media, arnaud.giersch, linux-kbuild, IvDoorn, teigland,
	tony.olech, apw, linux-hams, tglx, swhiteho, linux-arm-kernel,
	linux-edac, jon.maloy, linux-scsi, netdev, Greg KH,
	linux-wireless

On Wed, 2011-04-06 at 11:07 +0200, Michal Marek wrote:
> > So additionally, I'd suggest:
> > 1. Instrument checkpatch.pl and make it err or warn on timestamps.
> 
> This is patch 34/34 in this series: https://lkml.org/lkml/2011/4/5/198

Yeah, great, did not notice, thanks!

> > 2. Probably instrument linux-next to rise a warning when people break
> >    this.
> 
> I'm not sure if Stephen has that much spare time, and I don't think it
> is necessary. I think the checkpatch check is sufficient and I'll check
> myself occasionally.

Yes, that's fine, I just wanted to speak this out - there is a
probability that someone gets excited and creates some instrumentation
to kbuild to automatically detect bad things and then Stephen could
easily use that.

WRT "not necessary" - well, I think it is always better to cut bad
things before they are merged, rather than afterwards.

But anyway, let's forget about this.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 17/34] hdlcdrv: Drop __TIME__ usage
  2011-04-05 14:59 ` [PATCH 17/34] hdlcdrv: " Michal Marek
@ 2011-04-19 10:17   ` walter harms
  2011-04-19 10:56     ` Michal Marek
       [not found]   ` <1302015790.4124.54.camel@xbox360.hq.axsem.com>
  1 sibling, 1 reply; 17+ messages in thread
From: walter harms @ 2011-04-19 10:17 UTC (permalink / raw)
  To: Michal Marek; +Cc: linux-kbuild, Thomas Sailer, linux-hams, netdev



Am 05.04.2011 16:59, schrieb Michal Marek:
> The kernel already prints its build timestamp during boot, no need to
> repeat it in random drivers and produce different object files each
> time.
> 
> Cc: Thomas Sailer <t.sailer@alumni.ethz.ch>
> Cc: linux-hams@vger.kernel.org
> Cc: netdev@vger.kernel.org
> Signed-off-by: Michal Marek <mmarek@suse.cz>
> ---
>  drivers/net/hamradio/hdlcdrv.c |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
> 
> diff --git a/drivers/net/hamradio/hdlcdrv.c b/drivers/net/hamradio/hdlcdrv.c
> index 5b37579..a4a3516 100644
> --- a/drivers/net/hamradio/hdlcdrv.c
> +++ b/drivers/net/hamradio/hdlcdrv.c
> @@ -749,7 +749,7 @@ EXPORT_SYMBOL(hdlcdrv_unregister);
>  static int __init hdlcdrv_init_driver(void)
>  {
>  	printk(KERN_INFO "hdlcdrv: (C) 1996-2000 Thomas Sailer HB9JNX/AE4WA\n");
> -	printk(KERN_INFO "hdlcdrv: version 0.8 compiled " __TIME__ " " __DATE__ "\n");
> +	printk(KERN_INFO "hdlcdrv: version 0.8\n");
>  	return 0;
>  }
>  

Its a bit late but i am not a big fan of hard coded version id's.
Does GIT provide something like that ? If not
IMHO the line
 printk(KERN_INFO "hdlcdrv: version 0.8\n");
can be removed.

re,
 wh

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 17/34] hdlcdrv: Drop __TIME__ usage
  2011-04-19 10:17   ` walter harms
@ 2011-04-19 10:56     ` Michal Marek
  0 siblings, 0 replies; 17+ messages in thread
From: Michal Marek @ 2011-04-19 10:56 UTC (permalink / raw)
  To: wharms; +Cc: linux-kbuild, Thomas Sailer, linux-hams, netdev

On 19.4.2011 12:17, walter harms wrote:
> Am 05.04.2011 16:59, schrieb Michal Marek:
>> --- a/drivers/net/hamradio/hdlcdrv.c
>> +++ b/drivers/net/hamradio/hdlcdrv.c
>> @@ -749,7 +749,7 @@ EXPORT_SYMBOL(hdlcdrv_unregister);
>>  static int __init hdlcdrv_init_driver(void)
>>  {
>>  	printk(KERN_INFO "hdlcdrv: (C) 1996-2000 Thomas Sailer HB9JNX/AE4WA\n");
>> -	printk(KERN_INFO "hdlcdrv: version 0.8 compiled " __TIME__ " " __DATE__ "\n");
>> +	printk(KERN_INFO "hdlcdrv: version 0.8\n");
>>  	return 0;
>>  }
>>  
> 
> Its a bit late but i am not a big fan of hard coded version id's.
> Does GIT provide something like that ? If not
> IMHO the line
>  printk(KERN_INFO "hdlcdrv: version 0.8\n");
> can be removed.

OK, submit a separate patch then. I personally don't care about version
id's, as long as they do not embed any timestamp ;-).

Michal

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 15/34] baycom: Drop __TIME__ usage
       [not found]   ` <1302015780.4124.53.camel@xbox360.hq.axsem.com>
@ 2011-05-25 15:23     ` Michal Marek
  0 siblings, 0 replies; 17+ messages in thread
From: Michal Marek @ 2011-05-25 15:23 UTC (permalink / raw)
  To: Thomas Sailer
  Cc: linux-kbuild, linux-hams, Linux Kernel Network Developers, lkml

On 5.4.2011 17:03, Thomas Sailer wrote:
> On Tue, 2011-04-05 at 16:59 +0200, Michal Marek wrote:
>> The kernel already prints its build timestamp during boot, no need to
>> repeat it in random drivers and produce different object files each
>> time.
>>
>> Cc: Thomas Sailer<t.sailer@alumni.ethz.ch>
>> Cc: linux-hams@vger.kernel.org
>> Cc: netdev@vger.kernel.org
>> Signed-off-by: Michal Marek<mmarek@suse.cz>
> Acked-By: Thomas Sailer<t.sailer@alumni.ethz.ch>

I didn't found the commit in today's linux-next, so I applied this to 
kbuild-2.6.git#trivial.

Michal

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 17/34] hdlcdrv: Drop __TIME__ usage
       [not found]   ` <1302015790.4124.54.camel@xbox360.hq.axsem.com>
@ 2011-05-25 15:24     ` Michal Marek
  0 siblings, 0 replies; 17+ messages in thread
From: Michal Marek @ 2011-05-25 15:24 UTC (permalink / raw)
  To: Thomas Sailer
  Cc: linux-kbuild, lkml, linux-hams, Linux Kernel Network Developers

On 5.4.2011 17:03, Thomas Sailer wrote:
> On Tue, 2011-04-05 at 16:59 +0200, Michal Marek wrote:
>> The kernel already prints its build timestamp during boot, no need to
>> repeat it in random drivers and produce different object files each
>> time.
>>
>> Cc: Thomas Sailer<t.sailer@alumni.ethz.ch>
>> Cc: linux-hams@vger.kernel.org
>> Cc: netdev@vger.kernel.org
>> Signed-off-by: Michal Marek<mmarek@suse.cz>
> Acked-By: Thomas Sailer<t.sailer@alumni.ethz.ch>
>
> Ok with me

I didn't find the commit in today's linux-next, so I applied this to
kbuild-2.6.git#trivial.

Michal

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2011-05-25 15:24 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-04-05 14:58 [PATCH 00/34] Make kernel build deterministic Michal Marek
2011-04-05 14:59 ` [PATCH 15/34] baycom: Drop __TIME__ usage Michal Marek
     [not found]   ` <1302015780.4124.53.camel@xbox360.hq.axsem.com>
2011-05-25 15:23     ` Michal Marek
2011-04-05 14:59 ` [PATCH 17/34] hdlcdrv: " Michal Marek
2011-04-19 10:17   ` walter harms
2011-04-19 10:56     ` Michal Marek
     [not found]   ` <1302015790.4124.54.camel@xbox360.hq.axsem.com>
2011-05-25 15:24     ` Michal Marek
2011-04-05 15:49 ` [PATCH 00/34] Make kernel build deterministic Greg KH
2011-04-05 18:16   ` James Bottomley
2011-04-05 18:29     ` Mauro Carvalho Chehab
2011-04-05 18:44       ` Greg KH
2011-04-05 19:24   ` Artem Bityutskiy
2011-04-06  8:57     ` Ingo Molnar
2011-04-06  9:07     ` Michal Marek
2011-04-06  9:25       ` Artem Bityutskiy
2011-04-06  9:23   ` Michal Marek
2011-04-06  9:01 ` Ingo Molnar

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox