Re: reproducible builds with btrfs seed feature

[Anand Jain <anand.jain@oracle.com>]

On 10/14/2018 06:28 AM, Chris Murphy wrote:
> Is it practical and desirable to make Btrfs based OS installation
> images reproducible? Or is Btrfs simply too complex and
> non-deterministic? [1]
> 
> The main three problems with Btrfs right now for reproducibility are:
> a. many objects have uuids other than the volume uuid; and mkfs only
> lets us set the volume uuid
> b. atime, ctime, mtime, otime; and no way to make them all the same
> c. non-deterministic allocation of file extents, compression, inode
> assignment, logical and physical address allocation
> 
> I'm imagining reproducible image creation would be a mkfs feature that
> builds on Btrfs seed and --rootdir concepts to constrain Btrfs
> features to maybe make reproducible Btrfs volumes possible:
> 
> - No raid
> - Either all objects needing uuids can have those uuids specified by
> switch, or possibly a defined set of uuids expressly for this use
> case, or possibly all of them can just be zeros (eek? not sure)
> - A flag to set all times the same
> - Possibly require that target block device is zero filled before
> creation of the Btrfs
> - Possibly disallow subvolumes and snapshots
> - Require the resulting image is seed/ro and maybe also a new
> compat_ro flag to enforce that such Btrfs file systems cannot be
> modified after the fact.
> - Enforce a consistent means of allocation and compression
> 
> The end result is creating two Btrfs volumes would yield image files
> with matching hashes.

> If I had to guess, the biggest challenge would be allocation. But it's
> also possible that such an image may have problems with "sprouts". A
> non-removable sprout seems fairly straightforward and safe; but if a
> "reproducible build" type of seed is removed, it seems like removal
> needs to be smart enough to refresh *all* uuids found in the sprout: a
> hard break from the seed.

Right. The seed fsid will be gone in a detached sprout.

> Competing file systems, ext4 with make_ext4 fork, and squashfs. At the
> moment I'm thinking it might be easier to teach squashfs integrity
> checking than to make Btrfs reproducible.  But then I also think
> restricting Btrfs features, and applying some requirements to
> constrain Btrfs to make it reproducible, really enhances the Btrfs
> seed-sprout feature.

 > Any thoughts? Useful? Difficult to implement?

Recently Nikolay sent a patch to change fsid on a mounted btrfs. However 
for a reproducible builds it also needs neutralized uuids, time, 
bytenr(s) further more though the ondisk layout won't change without 
notice but block-bytenr might.

One question why not reproducible builds get the file data extents from 
the image and stitch the hashes together to verify the hash. And there 
could be a vfs ioctl to import and export filesystem images for a better 
support-ability of the use-case similar to the reproducible builds.

For the seed sprout feature one thing I have in mind is to make it image 
and subvolume granular rather than the disk and fsid granular, and 
ability to transpire golden image (seed) updates, but I haven't checked 
the feasibility yet.

Thanks, Anand

> 
> Squashfs might be a better fit for this use case *if* it can be taught
> about integrity checking.

> It does per file checksums for the purpose
> of deduplication but those checksums aren't retained for later
> integrity checking.
> 
> [1] problems of reproducible system images
> https://reproducible-builds.org/docs/system-images/
> 
> [2] purpose and motivation for reproducible builds
> https://reproducible-builds.org/
> 
> [3] who is involved?
> https://reproducible-builds.org/who/#Qubes%20OS
> 
> 
> 
>