Re: [Qemu-devel] [PATCH] makefile: detect corrupted elf files

["Michael S. Tsirkin" <mst@redhat.com>]

On Sat, May 25, 2013 at 05:32:24PM +0000, Blue Swirl wrote:
> On Wed, May 22, 2013 at 11:35 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
> > On Wed, May 22, 2013 at 01:12:15PM +0200, Paolo Bonzini wrote:
> >> Il 22/05/2013 13:09, Michael S. Tsirkin ha scritto:
> >> > > Usually I do the same---I just do slightly more thorough testing for
> >> > > configure patches.
> >> >
> >> > I've no idea what happens with ccache on a crash by the way.
> >> > It's possible that it's careful to do renames in order to not leave
> >> > corrupted output files behind.
> >>
> >> It doesn't, it leave 0-sized files.  (Or at least it didn't last time
> >> power failed during a compilation. :))
> >>
> >> Paolo
> >
> > Well looking at the source, there's quite a bit of
> > handling of renames, so maybe ccache hackers will be
> > interested in fixing this.
> >
> > Manpage says:
> >        It should be noted that ccache is susceptible to general storage
> >         problems. If a bad object file sneaks into the cache for some reason, it
> >         will of course stay bad. Some possible reasons for erroneous object
> >         files are bad hardware (disk drive, disk controller, memory, etc), buggy
> >         drivers or file systems, a bad CCACHE_PREFIX command or compiler
> >         wrapper.
> >
> >         ...
> >
> >
> >        There are no reported issues about ccache producing broken object
> >        files reproducibly. That doesn’t mean it can’t happen, so if you find
> >        a repeatable case, please report it.
> >
> > power failure is not listed ...
> 
> Neither is kill -9 issued by evil BOFH.

So presumably ccache will not fill with junk if you do this.

> IIRC I've also had bad builds
> and weird errors because the disk was almost completely full (not
> necessarily due to ccache). Once I overclocked a machine but I had to
> reduce the speed because of random compile errors.

This could be a parallel build, and possibly we have some
missing dependencies in the makefile.

> 
> But I think your patch is way too simple to cover possible failure
> cases when you can't trust the compile environment.

That's not the intent. Merely to address the common failure
which I personally observe all the time.

> Maybe you should
> build in two separate directories and compare the resulting objects or
> just the final executables. For added paranoia, build using two
> machines which have different set of components from different
> manufacturers, but identical userland.
> 
> Another way to handle this would be to enhance GCC and linker to use
> atomic operations when producing or combining object files. The tools
> could also print a SHA of the object which the next user should
> verify. Even better, the object files should include a robust checksum
> to ensure integrity.

I think we can make the makefile more robust. It can create a temporary
file in same directory and rename when ready. This will prevent
corrupted files from appearing in the first place.


> >
> > --
> > MST