Re: [Buildroot] [RFC PATCH 1/2] support/scripts/genimage.sh: support creating a bmap image

[Peter Korsgaard <peter@korsgaard.com>]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

Hi,

 >> I would suggest to drop the gzip step and only create the .bmap files
 >> (which presumably is quite fast?)
 >> 
 >> What do you say?

 > I understand your concern, but compression is also what helps in making
 > those sparse images small.

NIT: No, it is the sparseness (E.G. the holes). Compression in fact
breaks the holes (replaces them by zeros):

truncate -s 1G disk.img
du -hs disk.img
0       disk.img

gzip disk.img
du -hs disk.img.gz
1020K   disk.img.gz

gunzip disk.img.gz
du -hs disk.img
1.0G    disk.img

To transfer a file with holes without losing them you need an archival
format that understands sparse files, E.G. tar -S:

truncate -s 1G disk.img
du -hs disk.img
0       disk.img

tar cfSz disk.tar.gz disk.img
du -hs disk.tar.gz
4.0K    disk.tar.gz

rm disk.img
tar zxf disk.tar.gz
du -hs disk.img
0       disk.img

Notice also that the .tar.gz is a lot smaller than the img.gz


 > Of course, as long as you keep them where Buildroot generated them,
 > they are sparse, so they don't take much space. But as soon as you
 > "distribute" them, they usually loose their sparse-ness, and this is
 > where compressing the image + the bmap metadata file helps in lot in
 > reducing the amount of data.

Correct, unless an archival format supporting sparse files is used, so
if anything we should make a tarball of the images - But that is also
not so simple, E.G.:

- Busybox tar cannot extract sparse tar files
- One tarball for everything or a tarball for each image?
- What compression algorithm and parameters to use?

So it very fast ends up in a very use case specific setup, hence my
suggestion to NOT do it and leave such customizations to a post-image
script.

 > But fair enough, I assume this compression can be done as a custom
 > post-processing step by whoever needs it.

Indeed.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot