Untar on empty partition returns ENOSPACE

Untar on empty partition returns ENOSPACE

[Jean-Denis Girard]

Hi list,

My goal is to duplicate some SD cards, to prepare 50 similar Raspberry Pi.

First, I made a tar of my master SD card (unmounted). Then I made a
script, which creates 2 partitions (50 MB for boot, 14 GB for /),
creates the file-systems (vfat and btrfs, default options), mounts the
two partitions:

mount $part2 $mnt
-ocompress=zstd,space_cache=v2,autodefrag,noatime,nodiratime
mkdir $mnt/boot
mount $part1 $mnt/boot

When untarring, I get many errors, like:
tar:
./usr/lib/libreoffice/share/gallery/arrows/A45-TrendArrow-Red-GoUp.png :
open impossible: No space left on device
tar:
./usr/lib/libreoffice/share/gallery/arrows/A53-TrendArrow-LightBlue-TwoDirections.svg
: open impossible: No space left on device
tar:
./usr/lib/libreoffice/share/gallery/arrows/A27-CurvedArrow-DarkRed.png :
open impossible: No space left on device
tar:
./usr/lib/libreoffice/share/gallery/arrows/A41-CurvedArrow-Gray-Left.svg
: open impossible: No space left on device

Which usually results in unusable SD card.

When the first errors occur, less than 1 GB has been written. I tried to
change mount options, especially commit interval, but still got ENOSPACE.

What I ended up doing is limiting write speed with:
xzcat $archive | pv -L 4M | tar x
(with -L 5M I start to get a couple of errors)

Is there a better work around? Or a patch to test that could help?

The machine that runs the script has:
[jdg@tiare tar-install]$ uname -r
4.19.0-rc7-snx
[jdg@tiare tar-install]$ btrfs version
btrfs-progs v4.17.1


Thanks,
-- 
Jean-Denis Girard

SysNux                   Systèmes   Linux   en   Polynésie  française
https://www.sysnux.pf/   Tél: +689 40.50.10.40 / GSM: +689 87.797.527

Re: Untar on empty partition returns ENOSPACE

[Henk Slager]

On Thu, Oct 18, 2018 at 6:04 AM Jean-Denis Girard <jd.girard@sysnux.pf> wrote:
>
> Hi list,
>
> My goal is to duplicate some SD cards, to prepare 50 similar Raspberry Pi.
>
> First, I made a tar of my master SD card (unmounted). Then I made a
> script, which creates 2 partitions (50 MB for boot, 14 GB for /),
> creates the file-systems (vfat and btrfs, default options), mounts the
> two partitions:
>
> mount $part2 $mnt
> -ocompress=zstd,space_cache=v2,autodefrag,noatime,nodiratime
> mkdir $mnt/boot
> mount $part1 $mnt/boot
>
> When untarring, I get many errors, like:
> tar:
> ./usr/lib/libreoffice/share/gallery/arrows/A45-TrendArrow-Red-GoUp.png :
> open impossible: No space left on device
> tar:
> ./usr/lib/libreoffice/share/gallery/arrows/A53-TrendArrow-LightBlue-TwoDirections.svg
> : open impossible: No space left on device
> tar:
> ./usr/lib/libreoffice/share/gallery/arrows/A27-CurvedArrow-DarkRed.png :
> open impossible: No space left on device
> tar:
> ./usr/lib/libreoffice/share/gallery/arrows/A41-CurvedArrow-Gray-Left.svg
> : open impossible: No space left on device
>
> Which usually results in unusable SD card.
>
> When the first errors occur, less than 1 GB has been written. I tried to
> change mount options, especially commit interval, but still got ENOSPACE.
>
> What I ended up doing is limiting write speed with:
> xzcat $archive | pv -L 4M | tar x
> (with -L 5M I start to get a couple of errors)
>
> Is there a better work around? Or a patch to test that could help?
>
> The machine that runs the script has:
> [jdg@tiare tar-install]$ uname -r
> 4.19.0-rc7-snx
> [jdg@tiare tar-install]$ btrfs version
> btrfs-progs v4.17.1
This looks like to have a similar root cause as this:
https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg81536.html

For 50 clones, you could prepare 1 master (sparse) image and change
the UUID with btrfstune for every SD-card you have after written the
image (optionally sparse write with e.g. dd_rescue).

But it looks like some change is needed in the kernel code, although I
have no clue at the moment where exactly.

Re: Untar on empty partition returns ENOSPACE

[Jean-Denis Girard]

Le 17/10/2018 à 20:43, Henk Slager a écrit :
> On Thu, Oct 18, 2018 at 6:04 AM Jean-Denis Girard <jd.girard@sysnux.pf> wrote:
> This looks like to have a similar root cause as this:
> https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg81536.html

Yes, seems similar.

> For 50 clones, you could prepare 1 master (sparse) image and change
> the UUID with btrfstune for every SD-card you have after written the
> image (optionally sparse write with e.g. dd_rescue).

Thanks for suggestion, I may try this another time, but I can live with
my workaround.

> But it looks like some change is needed in the kernel code, although I
> have no clue at the moment where exactly.

Right, this is why I send this message, try to motivate a dev to look at
this issue ;)


Thanks,
-- 
Jean-Denis Girard

SysNux                   Systèmes   Linux   en   Polynésie  française
https://www.sysnux.pf/   Tél: +689 40.50.10.40 / GSM: +689 87.797.527

Re: Untar on empty partition returns ENOSPACE

[Cerem Cem ASLAN]

You don't even need dd_rescue or something for RaspberryPi creation,
you may use https://github.com/aktos-io/dcs-tools#cloning-a-target-into-a-new-target
Jean-Denis Girard <jd.girard@sysnux.pf>, 20 Eki 2018 Cmt, 22:00
tarihinde şunu yazdı:
>
> Le 17/10/2018 à 20:43, Henk Slager a écrit :
> > On Thu, Oct 18, 2018 at 6:04 AM Jean-Denis Girard <jd.girard@sysnux.pf> wrote:
> > This looks like to have a similar root cause as this:
> > https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg81536.html
>
> Yes, seems similar.
>
> > For 50 clones, you could prepare 1 master (sparse) image and change
> > the UUID with btrfstune for every SD-card you have after written the
> > image (optionally sparse write with e.g. dd_rescue).
>
> Thanks for suggestion, I may try this another time, but I can live with
> my workaround.
>
> > But it looks like some change is needed in the kernel code, although I
> > have no clue at the moment where exactly.
>
> Right, this is why I send this message, try to motivate a dev to look at
> this issue ;)
>
>
> Thanks,
> --
> Jean-Denis Girard
>
> SysNux                   Systèmes   Linux   en   Polynésie  française
> https://www.sysnux.pf/   Tél: +689 40.50.10.40 / GSM: +689 87.797.527
>
>

Re: Untar on empty partition returns ENOSPACE

[Jean-Denis Girard]

Le 20/10/2018 à 10:39, Cerem Cem ASLAN a écrit :
> You don't even need dd_rescue or something for RaspberryPi creation,
> you may use https://github.com/aktos-io/dcs-tools#cloning-a-target-into-a-new-target

That's interesting, but a bit overkill for my needs : I'll stay with my
simple script for this project.


Thanks,
-- 
Jean-Denis Girard

SysNux                   Systèmes   Linux   en   Polynésie  française
https://www.sysnux.pf/   Tél: +689 40.50.10.40 / GSM: +689 87.797.527