All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 00/34] Make kernel build deterministic
@ 2011-04-05 14:58 ` Michal Marek
  0 siblings, 0 replies; 174+ messages in thread
From: Michal Marek @ 2011-04-05 14:58 UTC (permalink / raw)
  To: linux-kbuild
  Cc: anil_ravindranath, mchehab, benh, x86, mac, aacraid, linux-mtd,
	allan.stephens, hpa, netdev, t.sailer, gwingerde, IvDoorn, elf,
	cluster-devel, ccaulfie, mingo, dougthompson, linux-usb,
	linux-media, arnaud.giersch, teigland, tony.olech, apw,
	linux-hams, tglx, swhiteho, linux-arm-kernel, linux-edac,
	jon.maloy, linux-scsi, linuxppc-dev, gregkh, linux-wireless,
	linux-kernel, bluesmoke-devel


Hi,

this series makes it possible to build bit-identical kernel image and
modules from identical sources. Of course the build is already
deterministic in terms of behavior of the code, but the various
timestamps embedded in the object files make it hard to compare two
builds, for instance to verify that a makefile cleanup didn't
accidentally change something. A prime example is /proc/config.gz, which
has both a timestamp in the gzip header and a timestamp in the payload
data. With this series applied, a script like this will produce
identical kernels each time:

#!/bin/bash
rm -rf /dev/shm/{source,build}{,1,2}
export KCONFIG_NOTIMESTAMP=1
export KBUILD_BUILD_TIMESTAMP='Sun May  1 12:00:00 CEST 2011'
export KBUILD_BUILD_USER=user
export KBUILD_BUILD_HOST=host
export ROOT_DEV=FLOPPY
for i in 1 2; do
	mkdir /dev/shm/source
	# randomize the inode order just for fun
	git ls-tree -r -z --name-only HEAD | sort -R -z | xargs -0 \
		cp --parents --target=/dev/shm/source
	pushd /dev/shm/source
	mkdir /dev/shm/build
	>/dev/shm/build/all.config
	for opt in GCOV_KERNEL UBIFS_FS_DEBUG; do
		echo "# CONFIG_$opt is not set" >>"/dev/shm/build"/all.config
	done
	make O="/dev/shm/build" KCONFIG_ALLCONFIG=1 allmodconfig
	make O="/dev/shm/build" -j64
	popd
	mv /dev/shm/source /dev/shm/source$i
	mv /dev/shm/build /dev/shm/build$i
done
diff -rq /dev/shm/build{1,2}

Unfortunatelly, this cannot be used to validate indentation-only
patches, even if CONFIG_DEBUG_INFO is turned off. This is because of the
__FILE__ and __LINE__ macros used in many places. For the same reason,
the source and build directory needs to be the same, otherwise the
results will differ. This was tested on
x86_64/{defconfig,allmodconfig,allyesconfig} and ppc64/defconfig. The
series is also available at

git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild-2.6.git deterministic-build-v1

Michal


Michal Marek (34):
  kconfig: Do not record timestamp in auto.conf and autoconf.h
  kbuild: Call gzip with -n
  powerpc: Call gzip with -n
  kbuild: Use the deterministic mode of ar
  powerpc: Use the deterministic mode of ar
  kbuild: Drop unused LINUX_COMPILE_TIME and LINUX_COMPILE_DOMAIN
    macros
  kbuild: Allow to override LINUX_COMPILE_BY and LINUX_COMPILE_HOST
    macros
  initramfs: Use KBUILD_BUILD_TIMESTAMP for generated entries
  x86: Allow to override the ROOT_DEV variable
  cyclades: Drop __TIME__ usage
  nozomi: Drop __TIME__ usage
  isdn/diva: Drop __TIME__ usage
  media/radio-maxiradio: Drop __TIME__ usage
  media/cx231xx: Drop __TIME__ usage
  baycom: Drop __TIME__ usage
  nand/denali: Drop __TIME__ usage
  hdlcdrv: Drop __TIME__ usage
  wan/pc300: Drop __TIME__ usage
  rt2x00: Drop __TIME__ usage
  parport: Drop __TIME__ usage
  aacraid: Drop __TIME__ usage
  scsi/in2000: Drop __TIME__ usage
  scsi/wd33c93: Drop __TIME__ usage
  usb/u132-hcd: Drop __TIME__ usage
  usb/ftdi-elan: Drop __TIME__ usage
  dlm: Drop __TIME__ usage
  gfs2: Drop __TIME__ usage
  atm: Drop __TIME__ usage
  tipc: Drop __TIME__ usage
  rio: Drop __DATE__ usage
  edac: Drop __DATE__ usage
  pmcraid: Drop __DATE__ usage
  usb/lh7a40x_udc: Drop __DATE__ usage
  checkpatch: Warn about usage of __DATE__, __TIME__ and __TIMESTAMP__

 Documentation/kbuild/kbuild.txt              |   12 ++++++++++
 arch/powerpc/boot/Makefile                   |    2 +-
 arch/powerpc/boot/wrapper                    |    6 ++--
 arch/x86/boot/Makefile                       |    2 +-
 drivers/char/cyclades.c                      |    3 +-
 drivers/char/nozomi.c                        |    3 +-
 drivers/char/rio/rioinit.c                   |    2 +-
 drivers/edac/amd76x_edac.c                   |    2 +-
 drivers/edac/amd8111_edac.c                  |    2 +-
 drivers/edac/amd8131_edac.c                  |    2 +-
 drivers/edac/cpc925_edac.c                   |    2 +-
 drivers/edac/e752x_edac.c                    |    2 +-
 drivers/edac/e7xxx_edac.c                    |    2 +-
 drivers/edac/edac_module.c                   |    2 +-
 drivers/edac/i5000_edac.c                    |    2 +-
 drivers/edac/i5400_edac.c                    |    2 +-
 drivers/edac/i7300_edac.c                    |    2 +-
 drivers/edac/i7core_edac.c                   |    2 +-
 drivers/edac/i82860_edac.c                   |    2 +-
 drivers/edac/i82875p_edac.c                  |    2 +-
 drivers/edac/i82975x_edac.c                  |    2 +-
 drivers/edac/mpc85xx_edac.h                  |    2 +-
 drivers/edac/mv64x60_edac.h                  |    2 +-
 drivers/edac/ppc4xx_edac.c                   |    2 +-
 drivers/edac/r82600_edac.c                   |    2 +-
 drivers/isdn/hardware/eicon/divasfunc.c      |    5 +--
 drivers/media/radio/radio-maxiradio.c        |    3 +-
 drivers/media/video/cx231xx/cx231xx-avcore.c |    2 +-
 drivers/mtd/nand/denali.c                    |    3 +-
 drivers/net/hamradio/baycom_epp.c            |    2 +-
 drivers/net/hamradio/baycom_par.c            |    2 +-
 drivers/net/hamradio/baycom_ser_fdx.c        |    2 +-
 drivers/net/hamradio/baycom_ser_hdx.c        |    2 +-
 drivers/net/hamradio/hdlcdrv.c               |    2 +-
 drivers/net/wan/pc300_drv.c                  |    3 +-
 drivers/net/wireless/rt2x00/rt2x00debug.c    |    1 -
 drivers/parport/parport_ip32.c               |    1 -
 drivers/scsi/aacraid/linit.c                 |    3 +-
 drivers/scsi/in2000.c                        |    2 +-
 drivers/scsi/pmcraid.c                       |    9 +++----
 drivers/scsi/pmcraid.h                       |    1 -
 drivers/scsi/wd33c93.c                       |    7 ++---
 drivers/usb/gadget/lh7a40x_udc.c             |    7 ++---
 drivers/usb/host/u132-hcd.c                  |    3 +-
 drivers/usb/misc/ftdi-elan.c                 |    3 +-
 fs/dlm/main.c                                |    2 +-
 fs/gfs2/main.c                               |    2 +-
 net/atm/lec.c                                |    2 +-
 net/atm/mpc.c                                |    2 +-
 net/tipc/core.c                              |    3 +-
 scripts/Makefile.build                       |    4 +-
 scripts/Makefile.lib                         |    2 +-
 scripts/checkpatch.pl                        |    5 ++++
 scripts/gen_initramfs_list.sh                |    8 +++++-
 scripts/kconfig/confdata.c                   |    8 +-----
 scripts/mkcompile_h                          |   30 ++++++++++++-------------
 usr/gen_init_cpio.c                          |   17 +++++++++-----
 57 files changed, 109 insertions(+), 105 deletions(-)

-- 
1.7.4.1

^ permalink raw reply	[flat|nested] 174+ messages in thread

end of thread, other threads:[~2011-05-26 12:42 UTC | newest]

Thread overview: 174+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-04-05 14:58 [PATCH 00/34] Make kernel build deterministic Michal Marek
2011-04-05 14:58 ` Michal Marek
2011-04-05 14:58 ` Michal Marek
2011-04-05 14:58 ` Michal Marek
2011-04-05 14:58 ` [PATCH 01/34] kconfig: Do not record timestamp in auto.conf and autoconf.h Michal Marek
2011-04-05 18:57   ` Arnaud Lacombe
2011-04-06  7:37     ` Michal Marek
2011-04-06 16:25       ` Arnaud Lacombe
2011-04-06 20:39         ` Michal Marek
2011-04-18 13:04   ` Michal Marek
2011-04-05 14:58 ` [PATCH 02/34] kbuild: Call gzip with -n Michal Marek
2011-04-18 13:05   ` Michal Marek
2011-04-05 14:58 ` [PATCH 03/34] powerpc: " Michal Marek
2011-04-05 14:58   ` Michal Marek
2011-04-05 14:58 ` [PATCH 04/34] kbuild: Use the deterministic mode of ar Michal Marek
2011-04-18 13:05   ` Michal Marek
2011-04-05 14:58 ` [PATCH 05/34] powerpc: " Michal Marek
2011-04-05 14:58   ` Michal Marek
2011-04-05 14:58 ` [PATCH 06/34] kbuild: Drop unused LINUX_COMPILE_TIME and LINUX_COMPILE_DOMAIN macros Michal Marek
2011-04-18 13:05   ` Michal Marek
2011-04-05 14:58 ` [PATCH 07/34] kbuild: Allow to override LINUX_COMPILE_BY and LINUX_COMPILE_HOST macros Michal Marek
2011-04-05 15:52   ` Jack Stone
2011-04-05 18:09     ` Michal Marek
2011-04-11 15:04     ` [PATCH v2] " Michal Marek
2011-04-12  3:25       ` Valdis.Kletnieks
2011-04-12  9:01         ` Michal Marek
2011-04-18 13:05           ` Michal Marek
2011-04-05 14:58 ` [PATCH 08/34] initramfs: Use KBUILD_BUILD_TIMESTAMP for generated entries Michal Marek
2011-04-05 17:31   ` [08/34] " Milton Miller
2011-04-05 18:12     ` Michal Marek
2011-04-11 15:06     ` [PATCH v2] " Michal Marek
2011-04-18 13:06       ` Michal Marek
2011-04-05 14:58 ` [PATCH 09/34] x86: Allow to override the ROOT_DEV variable Michal Marek
2011-04-05 15:30   ` H. Peter Anvin
2011-04-06  7:47     ` Michal Marek
2011-04-11 15:08       ` Michal Marek
2011-04-12  3:25         ` Valdis.Kletnieks
2011-04-12  7:11           ` Paul Bolle
2011-04-12  3:39         ` H. Peter Anvin
2011-04-12  5:01         ` Paul Bolle
2011-04-12  9:57           ` Michal Marek
2011-04-12 10:40             ` Paul Bolle
2011-04-12 11:30         ` [PATCH v2] x86: Do not set the root_dev field in bzImage Michal Marek
2011-05-11 15:22           ` Michal Marek
2011-05-25 20:32             ` Michal Marek
2011-05-25 20:49           ` [tip:x86/build] x86, build: " tip-bot for Michal Marek
2011-04-05 14:58 ` [PATCH 10/34] cyclades: Drop __TIME__ usage Michal Marek
2011-04-18 22:17   ` Michal Marek
2011-04-05 14:58 ` [PATCH 11/34] nozomi: " Michal Marek
2011-04-05 14:58 ` [PATCH 12/34] isdn/diva: " Michal Marek
2011-04-05 15:10   ` Armin Schindler
2011-04-05 15:37     ` Ben Hutchings
2011-04-05 16:07       ` Armin Schindler
2011-04-05 20:01         ` Sam Ravnborg
2011-04-05 19:17     ` Valdis.Kletnieks
2011-04-06  8:06       ` Michal Marek
2011-05-25 20:38     ` Michal Marek
2011-05-26  8:15       ` Armin Schindler
2011-05-26  8:29         ` Michal Marek
2011-04-05 14:59 ` [PATCH 13/34] media/radio-maxiradio: " Michal Marek
2011-04-18 22:18   ` Michal Marek
2011-04-05 14:59 ` [PATCH 14/34] media/cx231xx: " Michal Marek
2011-04-18 22:18   ` Michal Marek
2011-04-05 14:59 ` [PATCH 15/34] baycom: " Michal Marek
     [not found]   ` <1302015780.4124.53.camel@xbox360.hq.axsem.com>
2011-05-25 15:23     ` Michal Marek
2011-04-05 14:59 ` [PATCH 16/34] nand/denali: " Michal Marek
2011-04-05 14:59   ` Michal Marek
2011-04-05 15:02   ` Artem Bityutskiy
2011-04-05 15:02     ` Artem Bityutskiy
2011-04-05 15:10     ` Michal Marek
2011-04-05 15:10       ` Michal Marek
2011-04-05 15:15   ` Artem Bityutskiy
2011-04-05 15:15     ` Artem Bityutskiy
2011-04-05 14:59 ` [PATCH 17/34] hdlcdrv: " Michal Marek
2011-04-19 10:17   ` walter harms
2011-04-19 10:56     ` Michal Marek
     [not found]   ` <1302015790.4124.54.camel@xbox360.hq.axsem.com>
2011-05-25 15:24     ` Michal Marek
2011-04-05 14:59 ` [PATCH 18/34] wan/pc300: " Michal Marek
2011-05-25 20:43   ` Michal Marek
2011-05-25 20:44     ` David Miller
2011-04-05 14:59 ` [PATCH 19/34] rt2x00: " Michal Marek
2011-04-05 15:28   ` Gertjan van Wingerde
2011-04-05 18:37   ` Ivo Van Doorn
2011-04-05 18:37     ` Ivo Van Doorn
2011-04-05 14:59 ` [PATCH 20/34] parport: " Michal Marek
2011-04-05 20:00   ` Arnaud Giersch
2011-05-25 20:44     ` Michal Marek
2011-04-05 14:59 ` [PATCH 21/34] aacraid: " Michal Marek
2011-04-18 22:20   ` Michal Marek
2011-04-05 14:59 ` [PATCH 22/34] scsi/in2000: " Michal Marek
2011-04-18 22:20   ` Michal Marek
2011-04-05 14:59 ` [PATCH 23/34] scsi/wd33c93: " Michal Marek
2011-04-18 22:20   ` Michal Marek
2011-04-05 14:59 ` [PATCH 24/34] usb/u132-hcd: " Michal Marek
2011-04-05 14:59 ` [PATCH 25/34] usb/ftdi-elan: " Michal Marek
2011-04-05 17:50   ` Matthew Dharm
2011-04-05 18:13     ` Greg KH
2011-04-05 14:59 ` [PATCH 26/34] dlm: " Michal Marek
2011-05-25 20:47   ` Michal Marek
2011-05-25 20:47     ` [Cluster-devel] " David Teigland
2011-05-25 20:47       ` David Teigland
2011-04-05 14:59 ` [PATCH 27/34] gfs2: " Michal Marek
2011-05-25 20:48   ` Michal Marek
2011-05-26  8:43     ` [Cluster-devel] " Steven Whitehouse
2011-05-26  8:43       ` Steven Whitehouse
2011-05-26  9:08       ` Michal Marek
2011-04-05 14:59 ` [PATCH 28/34] atm: " Michal Marek
2011-05-25 20:49   ` Michal Marek
2011-05-25 21:39     ` David Miller
2011-04-05 14:59 ` [PATCH 29/34] tipc: " Michal Marek
2011-04-05 22:25   ` Paul Gortmaker
2011-04-05 22:25     ` Paul Gortmaker
2011-04-05 14:59 ` [PATCH 30/34] rio: Drop __DATE__ usage Michal Marek
2011-04-18 22:21   ` Michal Marek
2011-04-05 14:59 ` [PATCH 31/34] edac: " Michal Marek
2011-04-18 22:21   ` Michal Marek
2011-04-05 14:59 ` [PATCH 32/34] pmcraid: " Michal Marek
2011-04-18 22:21   ` Michal Marek
2011-04-05 14:59 ` [PATCH 33/34] usb/lh7a40x_udc: " Michal Marek
2011-04-05 14:59   ` Michal Marek
2011-04-05 16:25   ` Russell King - ARM Linux
2011-04-05 16:25     ` Russell King - ARM Linux
2011-04-06  8:23     ` Michal Marek
2011-04-06  8:23       ` Michal Marek
2011-04-05 14:59 ` [PATCH 34/34] checkpatch: Warn about usage of __DATE__, __TIME__ and __TIMESTAMP__ Michal Marek
2011-05-26 11:39   ` Michal Marek
2011-05-26 12:42   ` Andy Whitcroft
2011-04-05 15:49 ` [PATCH 00/34] Make kernel build deterministic Greg KH
2011-04-05 15:49   ` Greg KH
2011-04-05 15:49   ` Greg KH
2011-04-05 15:49   ` Greg KH
2011-04-05 18:16   ` James Bottomley
2011-04-05 18:16     ` James Bottomley
2011-04-05 18:16     ` James Bottomley
2011-04-05 18:16     ` James Bottomley
2011-04-05 18:29     ` Mauro Carvalho Chehab
2011-04-05 18:29       ` Mauro Carvalho Chehab
2011-04-05 18:29       ` Mauro Carvalho Chehab
2011-04-05 18:29       ` Mauro Carvalho Chehab
2011-04-05 18:29       ` Mauro Carvalho Chehab
2011-04-05 18:44       ` Greg KH
2011-04-05 18:44         ` Greg KH
2011-04-05 18:44         ` Greg KH
2011-04-05 18:44         ` Greg KH
2011-04-05 18:44         ` Greg KH
2011-04-05 19:24   ` Artem Bityutskiy
2011-04-05 19:24     ` Artem Bityutskiy
2011-04-05 19:24     ` Artem Bityutskiy
2011-04-05 19:24     ` Artem Bityutskiy
2011-04-06  8:57     ` Ingo Molnar
2011-04-06  8:57       ` Ingo Molnar
2011-04-06  8:57       ` Ingo Molnar
2011-04-06  8:57       ` Ingo Molnar
2011-04-06  9:04       ` Artem Bityutskiy
2011-04-06  9:07     ` Michal Marek
2011-04-06  9:07       ` Michal Marek
2011-04-06  9:07       ` Michal Marek
2011-04-06  9:07       ` Michal Marek
2011-04-06  9:07       ` Michal Marek
2011-04-06  9:25       ` Artem Bityutskiy
2011-04-06  9:25         ` Artem Bityutskiy
2011-04-06  9:25         ` Artem Bityutskiy
2011-04-06  9:25         ` Artem Bityutskiy
2011-04-06  9:25         ` Artem Bityutskiy
2011-04-06  9:23   ` Michal Marek
2011-04-06  9:23     ` Michal Marek
2011-04-06  9:23     ` Michal Marek
2011-04-06  9:23     ` Michal Marek
2011-04-06  9:01 ` Ingo Molnar
2011-04-06  9:01   ` Ingo Molnar
2011-04-06  9:01   ` Ingo Molnar
2011-04-06  9:01   ` Ingo Molnar
2011-05-05 15:22 ` [PATCH v2] powerpc: Use the deterministic mode of ar Michal Marek
2011-05-05 15:22   ` Michal Marek

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.