Re: [docs] [PATCH] doc/wic,README.hardware: add dd option conv=sparse

["Antonin Godard" <antonin.godard@bootlin.com>]

Hi all,

On Sun Jan 26, 2025 at 5:56 PM CET, Yoann Congal via lists.yoctoproject.org wrote:
> Le dim. 26 janv. 2025 à 16:25, Jörg Sommer via lists.yoctoproject.org
> <joerg.sommer=navimatix.de@lists.yoctoproject.org> a écrit :
>
>> Richard Purdie schrieb am So 26. Jan, 12:53 (+0000):
>> > On Sun, 2025-01-26 at 13:37 +0100, Jörg Sommer via
>> lists.yoctoproject.org wrote:
>> > > From: Jörg Sommer <joerg.sommer@navimatix.de>
>> > >
>> > > The GNU implementation of dd supports the skipping of sparse blocks
>> which
>> > > can speed-up the write, esp. of wic images with big holes:
>> > >
>> > >     $ sudo dd if=image.wic of=/dev/sdb bs=2M
>> > >     2683+1 Datensätze ein
>> > >     2683+1 Datensätze aus
>> > >     5628735488 Bytes (5,6 GB, 5,2 GiB) kopiert, 84,2773 s, 66,8 MB/s
>> > >
>> > >     $ sudo dd if=image.wic of=/dev/sdb bs=2M conv=sparse
>> > >     2683+1 Datensätze ein
>> > >     2683+1 Datensätze aus
>> > >     5628735488 Bytes (5,6 GB, 5,2 GiB) kopiert, 11,3314 s, 497 MB/s
>> > >
>> > > Signed-off-by: Jörg Sommer <joerg.sommer@navimatix.de>
>> > > ---
>> > >  documentation/dev-manual/wic.rst  |  8 +++++++-
>> > >  meta-yocto-bsp/README.hardware.md | 20 ++++++++++++++------
>> > >  2 files changed, 21 insertions(+), 7 deletions(-)
>> >
>> > I'm torn on this as it is potentially dangerous. It depends whether the
>> > blocks really need to be zero or their contents doesn't matter. You
>> > could leave existing data behind if you use the sparse option.
>>
>> Isn't this the same what bmaptool does?
>> https://github.com/yoctoproject/bmaptool?tab=readme-ov-file#concept
>>
>
> It look it does not. bmaptool readme is very particular on what a "unmapped
> block" is. And is not a zero block.
> TL;DR: (a gross reinterpretation) that is a block not referenced anywhere
> in your filesystem.
>
> Imagine a file filled with zeroes on your filesystem. Those zeros are
> referenced so must be flashed. bmaptool will flash them.
> "dd conv=sparse" will not and this is only right in a corner case : if you
> already know that your disk/memory is zero filled.
> But, in the general case, you'll get a "zero file" filled with garbage.
>
> Sorry, but I'm against this change since it will be a source of weird/hard
> bugs.
> Maybe instead of changing the dd command to speedup we can push users to
> "bmaptool copy" (we will get speedup without compromising on the validity
> of the filesystem)?

I agree with Yoann, I'm not for providing conv=sparse as a default option for
dd.

I also vote for redirecting to bmaptool. This can be done with:

  :ref:`bmaptool <dev-manual/bmaptool:flashing images using \`bmaptool\`>`

I think in a `..note ::` like you did Jörg, would be good.


Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com