From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:57686) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UhHDH-0008Ma-OI for qemu-devel@nongnu.org; Tue, 28 May 2013 06:33:16 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UhHD8-0001X9-1a for qemu-devel@nongnu.org; Tue, 28 May 2013 06:33:07 -0400 Received: from mx1.redhat.com ([209.132.183.28]:17765) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UhHD7-0001X3-PV for qemu-devel@nongnu.org; Tue, 28 May 2013 06:32:57 -0400 Date: Tue, 28 May 2013 13:33:13 +0300 From: "Michael S. Tsirkin" Message-ID: <20130528103313.GA5467@redhat.com> References: <20130526073540.GA32691@redhat.com> <20130526123142.GA6865@redhat.com> <20130526134045.GA26267@redhat.com> <20130526182447.GA3427@redhat.com> <20130526201502.GB4305@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [Qemu-devel] [PATCH] makefile: detect corrupted elf files List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Blue Swirl Cc: Peter Maydell , Anthony Liguori , Markus Armbruster , qemu-devel@nongnu.org, Luiz Capitulino , Paolo Bonzini On Sun, May 26, 2013 at 08:29:35PM +0000, Blue Swirl wrote: > On Sun, May 26, 2013 at 8:15 PM, Michael S. Tsirkin wrote: > > On Sun, May 26, 2013 at 07:28:40PM +0000, Blue Swirl wrote: > >> On Sun, May 26, 2013 at 6:24 PM, Michael S. Tsirkin wrote: > >> > On Sun, May 26, 2013 at 06:20:17PM +0000, Blue Swirl wrote: > >> >> On Sun, May 26, 2013 at 1:40 PM, Michael S. Tsirkin wrote: > >> >> > On Sun, May 26, 2013 at 02:36:28PM +0100, Peter Maydell wrote: > >> >> >> On 26 May 2013 13:31, Michael S. Tsirkin wrote: > >> >> >> > On Sun, May 26, 2013 at 10:12:21AM +0100, Peter Maydell wrote: > >> >> >> >> I definitely think individual project makefiles are the wrong place > >> >> >> >> to fix this. If create-as-temp-and-rename is useful functionality > >> >> >> >> it needs to go in the compiler so that everybody benefits. > >> >> >> > > >> >> >> > This will not help users on existing systems. > >> >> >> > Also it's not just compiler. We'd have to do it in linker, > >> >> >> > asm, ... lots of work. > >> >> >> > >> >> >> This is clearly less work than implementing it in the makefile > >> >> >> of every single open source project in the world (or even every > >> >> >> single open source project in Debian). > >> >> > > >> >> > You seem to have removed the part that explained that > >> >> > 1. we run scripts in our makefiles so need to handle that anyway > >> >> > 2. we care about users on existing systems > >> >> > >> >> A generic hook (default none, or maybe "test -s") after object > >> >> production and before linkage should be enough but would scale to SHA > >> >> producing/verifying tools. > >> >> > >> >> > > >> >> > This means that we would need the fix in our makefiles even > >> >> > if compiler and linker gain this feature. > >> >> > >> >> Depends on the feature. If the object files have robust checksums > >> >> which are checked after output and before input, this should be > >> >> transparent to the build system. > >> >> > >> >> > > >> >> >> > You are wellcome to implement this in compiler/linker/etc if you like > >> >> >> > but we will still want to handle it in our makefile as well. > >> >> >> > >> >> >> I specifically don't want it handled in our makefiles because > >> >> >> it's the wrong place to fix the problem and it will make > >> >> >> our build system more complicated. > >> >> > >> >> +1 > >> >> > >> >> Also, what is the worst case scenario? The link fails and you have to > >> >> clean up and rebuild? An automated build system can't produce the > >> >> expected output if the build machine is unreliable? > >> > > >> > It's a simple issue. > >> > Each time I reboot during build, I have to make clean and rebuild. > >> > This wastes my time so I looked for ways to save the time. > >> > >> Compile under a stable kernel and test the bleeding edge kernel only > >> as KVM guest? Get a different box for testing or compiling? Run 'sync' > >> every time gcc finishes? > > > > What's the question here? > > The question is if any of the suggestions solves the problem? > > Also how about something this: post boot, find -name '*.o' | xargs -iF > sh -c 'if test ! -s F; then rm F;fi' On Linux, even easier: find -name '*.o' -empty -exec rm '{}' ';' Seems to be enough here. Thanks, I'll use this hack and leave makefiles alone for now. > > > >> Don't you have bigger problems with file systems due to the crashes? > > > > As it happens, no. Maybe because I'm using ext4. > > Maybe I'm lucky. > > > >> > On my system at least, it has no measureable cost, > >> > likely also because size only looks at headers and metadata. > >> > >> For example on OpenBSD, 'size' does not seem to come from binutils, so > >> there could be portability issues. > > > > True, I'm not saying it's perfect. > > > >> > > >> > If others are not interested, I can keep it out of tree. > >> > >> I've had problems with disk close to full, so I'm semi-interested if > >> the solution does not slow down others and it's not too ugly. > > > > I think the simplest way to do it is to change makefile to unlike, create > > then rename. Then you are safe against abrupt killing or crashing make. > > And with a journaled fs, if you also have e.g. linux ext4 and mount with > > data=ordered, you are safe against power failures. > > > > It shouldn't be hard to do and I don't expect this to have any > > measureable speed impact. What do you think? > > I'd prefer a more generic solution, like the hook. What you propose > wouldn't protect from the disk full scenario. > > > > >> > > >> >> >> > >> >> >> -- PMM > >> >> > > >> >> >