[HFS] generic/740 failure details

[HFS] generic/740 failure details

[Viacheslav Dubeyko]

Hi Adrian, Yangtao,

We have failure for generic/740 test:

./check generic/740
FSTYP         -- hfs
PLATFORM      -- Linux/x86_64 hfsplus-testing-0001 6.15.0-rc4+ #8 SMP
PREEMPT_DYNAMIC Thu May  1 16:43:22 PDT 2025
MKFS_OPTIONS  -- /dev/loop51
MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch

generic/740       - output mismatch (see /home/slavad/XFSTESTS-2/xfstests-
dev/results//generic/740.out.bad)
    --- tests/generic/740.out	2025-04-24 12:48:45.964286739 -0700
    +++ /home/slavad/XFSTESTS-2/xfstests-
dev/results//generic/740.out.bad	2025-06-05 15:25:18.071217224 -0700
    @@ -1,2 +1,16 @@
     QA output created by 740
     Silence is golden.
    +Failed - overwrote fs type bfs!
    +Failed - overwrote fs type cramfs!
    +Failed - overwrote fs type exfat!
    +Failed - overwrote fs type ext2!
    +Failed - overwrote fs type ext3!
    ...
    (Run 'diff -u /home/slavad/XFSTESTS-2/xfstests-dev/tests/generic/740.out
/home/slavad/XFSTESTS-2/xfstests-dev/results//generic/740.out.bad'  to see the
entire diff)
Ran: generic/740
Failures: generic/740
Failed 1 of 1 tests

As far as I can see, the workflow of the test is to reformat the existing file
system by using the forcing option of mkfs tool (for example, -F of mkfs.ext4).
And, then, it tries to reformat the partition with existing file system (ext4,
xfs, btrfs, etc) by HFS/HFS+ mkfs tool with default option. By default, it is
expected that mkfs tool should refuse the reformat of partition with existing
file system. However, HFS/HFS+ mkfs tool easily reformat the partition without
any concerns or questions:

sudo mkfs.ext4 /dev/loop51
mke2fs 1.47.0 (5-Feb-2023)
/dev/loop51 contains a hfs file system labelled 'untitled'
Proceed anyway? (y,N) n

sudo mkfs.ext4 -F /dev/loop51
mke2fs 1.47.0 (5-Feb-2023)
/dev/loop51 contains a hfs file system labelled 'untitled'
Discarding device blocks: done                            
Creating filesystem with 2621440 4k blocks and 655360 inodes
Filesystem UUID: 2b65062e-d8d5-4731-9f3d-dddcf1aa73ee
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

sudo mkfs.hfs /dev/loop51
Initialized /dev/loop51 as a 10240 MB HFS volume

It looks like we need to modify the HFS/HFS+ mkfs tool to refuse the reformat of
existing file system and to add the forcing option.

Adrian, How does it feasible such modification?

Thanks,
Slava.

Re: [HFS] generic/740 failure details

[John Paul Adrian Glaubitz]

Hi Slava,

On Thu, 2025-06-05 at 22:41 +0000, Viacheslav Dubeyko wrote:
> It looks like we need to modify the HFS/HFS+ mkfs tool to refuse the reformat of
> existing file system and to add the forcing option.
> 
> Adrian, How does it feasible such modification?

It would be certainly possible although we would deviate from what Apple
does upstream with their hfs utilities. I'll add an issue in my repository
for hfs for Linux [1].

Adrian

> [1] https://github.com/glaubitz/hfs/tree/linux

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

RE: [HFS] generic/740 failure details

[Viacheslav Dubeyko]

Hi Adrian,

On Fri, 2025-06-06 at 08:31 +0200, John Paul Adrian Glaubitz wrote:
> Hi Slava,
> 
> On Thu, 2025-06-05 at 22:41 +0000, Viacheslav Dubeyko wrote:
> > It looks like we need to modify the HFS/HFS+ mkfs tool to refuse the reformat of
> > existing file system and to add the forcing option.
> > 
> > Adrian, How does it feasible such modification?
> 
> It would be certainly possible although we would deviate from what Apple
> does upstream with their hfs utilities. I'll add an issue in my repository
> for hfs for Linux [1].
> 

Could we suggest this option for Apple's repository? Or is HFS tool's code
already obsolete for Apple?

Anyway, we need to have this option. Otherwise, it's too easy to destroy the
existing file system by mistake.

Feel free to assign [1,2] issues for yourself. :)

Thanks,
Slava.

> Adrian
> 
> > [1] https://github.com/glaubitz/hfs/tree/linux  

[1] https://github.com/hfs-linux-kernel/hfs-linux-kernel/issues/74
[2] https://github.com/hfs-linux-kernel/hfs-linux-kernel/issues/215

Re: [HFS] generic/740 failure details

[Yangtao Li]

Hi Slava and Adrian,

在 2025/6/6 06:41, Viacheslav Dubeyko 写道:
> Hi Adrian, Yangtao,
> 
> We have failure for generic/740 test:
> 
> ./check generic/740
> FSTYP         -- hfs
> PLATFORM      -- Linux/x86_64 hfsplus-testing-0001 6.15.0-rc4+ #8 SMP
> PREEMPT_DYNAMIC Thu May  1 16:43:22 PDT 2025
> MKFS_OPTIONS  -- /dev/loop51
> MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch
> 
> generic/740       - output mismatch (see /home/slavad/XFSTESTS-2/xfstests-
> dev/results//generic/740.out.bad)
>      --- tests/generic/740.out	2025-04-24 12:48:45.964286739 -0700
>      +++ /home/slavad/XFSTESTS-2/xfstests-
> dev/results//generic/740.out.bad	2025-06-05 15:25:18.071217224 -0700
>      @@ -1,2 +1,16 @@
>       QA output created by 740
>       Silence is golden.
>      +Failed - overwrote fs type bfs!
>      +Failed - overwrote fs type cramfs!
>      +Failed - overwrote fs type exfat!
>      +Failed - overwrote fs type ext2!
>      +Failed - overwrote fs type ext3!
>      ...
>      (Run 'diff -u /home/slavad/XFSTESTS-2/xfstests-dev/tests/generic/740.out
> /home/slavad/XFSTESTS-2/xfstests-dev/results//generic/740.out.bad'  to see the
> entire diff)
> Ran: generic/740
> Failures: generic/740
> Failed 1 of 1 tests
> 
> As far as I can see, the workflow of the test is to reformat the existing file
> system by using the forcing option of mkfs tool (for example, -F of mkfs.ext4).
> And, then, it tries to reformat the partition with existing file system (ext4,
> xfs, btrfs, etc) by HFS/HFS+ mkfs tool with default option. By default, it is
> expected that mkfs tool should refuse the reformat of partition with existing
> file system. However, HFS/HFS+ mkfs tool easily reformat the partition without
> any concerns or questions:
> 
> sudo mkfs.ext4 /dev/loop51
> mke2fs 1.47.0 (5-Feb-2023)
> /dev/loop51 contains a hfs file system labelled 'untitled'
> Proceed anyway? (y,N) n
> 
> sudo mkfs.ext4 -F /dev/loop51
> mke2fs 1.47.0 (5-Feb-2023)
> /dev/loop51 contains a hfs file system labelled 'untitled'
> Discarding device blocks: done
> Creating filesystem with 2621440 4k blocks and 655360 inodes
> Filesystem UUID: 2b65062e-d8d5-4731-9f3d-dddcf1aa73ee
> Superblock backups stored on blocks:
> 	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
> 
> Allocating group tables: done
> Writing inode tables: done
> Creating journal (16384 blocks): done
> Writing superblocks and filesystem accounting information: done
> 
> sudo mkfs.hfs /dev/loop51
> Initialized /dev/loop51 as a 10240 MB HFS volume
> 
> It looks like we need to modify the HFS/HFS+ mkfs tool to refuse the reformat of
> existing file system and to add the forcing option.

I wonder if this is a good time to reimplement hfs-progs in rust.

Thx,
Yangtao

RE: [HFS] generic/740 failure details

[Viacheslav Dubeyko]

Hi Yangtao,

On Mon, 2025-06-09 at 18:52 +0800, Yangtao Li wrote:
> Hi Slava and Adrian,
> 
> 在 2025/6/6 06:41, Viacheslav Dubeyko 写道:
> > Hi Adrian, Yangtao,
> > 
> > We have failure for generic/740 test:
> > 
> > ./check generic/740
> > FSTYP         -- hfs
> > PLATFORM      -- Linux/x86_64 hfsplus-testing-0001 6.15.0-rc4+ #8 SMP
> > PREEMPT_DYNAMIC Thu May  1 16:43:22 PDT 2025
> > MKFS_OPTIONS  -- /dev/loop51
> > MOUNT_OPTIONS -- /dev/loop51 /mnt/scratch
> > 
> > generic/740       - output mismatch (see /home/slavad/XFSTESTS-2/xfstests-
> > dev/results//generic/740.out.bad)
> >      --- tests/generic/740.out	2025-04-24 12:48:45.964286739 -0700
> >      +++ /home/slavad/XFSTESTS-2/xfstests-
> > dev/results//generic/740.out.bad	2025-06-05 15:25:18.071217224 -0700
> >      @@ -1,2 +1,16 @@
> >       QA output created by 740
> >       Silence is golden.
> >      +Failed - overwrote fs type bfs!
> >      +Failed - overwrote fs type cramfs!
> >      +Failed - overwrote fs type exfat!
> >      +Failed - overwrote fs type ext2!
> >      +Failed - overwrote fs type ext3!
> >      ...
> >      (Run 'diff -u /home/slavad/XFSTESTS-2/xfstests-dev/tests/generic/740.out
> > /home/slavad/XFSTESTS-2/xfstests-dev/results//generic/740.out.bad'  to see the
> > entire diff)
> > Ran: generic/740
> > Failures: generic/740
> > Failed 1 of 1 tests
> > 
> > As far as I can see, the workflow of the test is to reformat the existing file
> > system by using the forcing option of mkfs tool (for example, -F of mkfs.ext4).
> > And, then, it tries to reformat the partition with existing file system (ext4,
> > xfs, btrfs, etc) by HFS/HFS+ mkfs tool with default option. By default, it is
> > expected that mkfs tool should refuse the reformat of partition with existing
> > file system. However, HFS/HFS+ mkfs tool easily reformat the partition without
> > any concerns or questions:
> > 
> > sudo mkfs.ext4 /dev/loop51
> > mke2fs 1.47.0 (5-Feb-2023)
> > /dev/loop51 contains a hfs file system labelled 'untitled'
> > Proceed anyway? (y,N) n
> > 
> > sudo mkfs.ext4 -F /dev/loop51
> > mke2fs 1.47.0 (5-Feb-2023)
> > /dev/loop51 contains a hfs file system labelled 'untitled'
> > Discarding device blocks: done
> > Creating filesystem with 2621440 4k blocks and 655360 inodes
> > Filesystem UUID: 2062e-d8d5-4731-9f3d-dddcf1aa73ee
> > Superblock backups stored on blocks:
> > 	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
> > 
> > Allocating group tables: done
> > Writing inode tables: done
> > Creating journal (16384 blocks): done
> > Writing superblocks and filesystem accounting information: done
> > 
> > sudo mkfs.hfs /dev/loop51
> > Initialized /dev/loop51 as a 10240 MB HFS volume
> > 
> > It looks like we need to modify the HFS/HFS+ mkfs tool to refuse the reformat of
> > existing file system and to add the forcing option.
> 
> I wonder if this is a good time to reimplement hfs-progs in rust.
> 

Frankly speaking, I don't see the point to re-write the hfs-progs in Rust for
multiple reasons:
(1) mostly, the main use-case that HFS/HFS+ partition is created under Mac OS
and somebody tries to mount it under Linux to access data;
(2) Apple is the owner of the code on Mac OS side and it's not good to
significantly deviate from the Apple's state of the code;
(3) I believe that Apple considers hfs-progs as obsolete code and they don't
want any significant changes in it;
(4) the hfs-progs is user-space tool, it is not frequently used, and even it
fails, then there is no much harm.

The point for Linux kernel driver is completely different:
(1) it's completely independent from Apple implementation and we can modify in
any reasonable way;
(2) the memory safety and stability of kernel driver is more important;
(3) using Rust for HFS/HFS+ driver is a practically new technology adoption;
(4) potentially, re-writing HFS/HFS+ drivers in Rust could remove multiple
existing (and unknown yet) bugs and improve efficiency of it;
(5) re-writing HFS/HFS+ kernel driver could be potentially interesting for
involving new guys into HFS/HFS+ activity but hfs-progs is not so attractive,
from my point of view.

Thanks,
Slava.

Re: [HFS] generic/740 failure details

[John Paul Adrian Glaubitz]

Hi,

On Mon, 2025-06-09 at 19:41 +0000, Viacheslav Dubeyko wrote:
> Frankly speaking, I don't see the point to re-write the hfs-progs in Rust for
> multiple reasons:
> (1) mostly, the main use-case that HFS/HFS+ partition is created under Mac OS
> and somebody tries to mount it under Linux to access data;
> (2) Apple is the owner of the code on Mac OS side and it's not good to
> significantly deviate from the Apple's state of the code;
> (3) I believe that Apple considers hfs-progs as obsolete code and they don't
> want any significant changes in it;
> (4) the hfs-progs is user-space tool, it is not frequently used, and even it
> fails, then there is no much harm.

Writing hfsprogs from scratch for Linux would mean though that a proper license
could be chosen that is not APSL which some distributions consider non-free.

On the other hands, the current hfsprogs utility is based on Apple's original
HFS/HFS+ code which is battle-tested and known to be reliable and robust. I'm
not sure whether we would be able to reach this level of stability within a
reasonable amount of time.

I do have to resume my work on my Linux patches again so that we can synchronize
hfsprogs with the latest upstream version.


Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913