HFS/HFS+ maintainership action items

HFS/HFS+ maintainership action items

[Viacheslav Dubeyko]

Hi Adrian,

I am trying to elaborate the HFS/HFS+ maintainership action items:
(1) We need to prepare a Linux kernel tree fork to collect patches.
(2) I think it needs to prepare the list of current known issues (TODO list).
(3) Let me prepare environment and start to run xfstests for HFS/HFS+ (to check
the current status).
(4) Which use-cases do we need to consider for regular testing?

Anything else? What am I missing?

Thanks,
Slava.

Re: HFS/HFS+ maintainership action items

[Theodore Ts'o]

On Mon, Apr 21, 2025 at 09:52:14PM +0000, Viacheslav Dubeyko wrote:
> Hi Adrian,
> 
> I am trying to elaborate the HFS/HFS+ maintainership action items:
> (1) We need to prepare a Linux kernel tree fork to collect patches.
> (2) I think it needs to prepare the list of current known issues (TODO list).
> (3) Let me prepare environment and start to run xfstests for HFS/HFS+ (to check

One potential problem is that the userspace utilities to format,
check, repair HFS/HFS+ utilities don't really exist.  There is the HFS
Utilities[1] which is packaged in Debian as hfsutils, but it only
supports HFS, not HFS+, and it can only format an HFS file system; it
doesn't have a fsck analog.  This is going to very limit the ability
to run xfstests for HFS or HFS+.

[1] https://www.mars.org/home/rob/proj/hfs/

						- Ted

Re: HFS/HFS+ maintainership action items

[Darrick J. Wong]

On Mon, Apr 21, 2025 at 09:43:33PM -0500, Theodore Ts'o wrote:
> On Mon, Apr 21, 2025 at 09:52:14PM +0000, Viacheslav Dubeyko wrote:
> > Hi Adrian,
> > 
> > I am trying to elaborate the HFS/HFS+ maintainership action items:
> > (1) We need to prepare a Linux kernel tree fork to collect patches.
> > (2) I think it needs to prepare the list of current known issues (TODO list).
> > (3) Let me prepare environment and start to run xfstests for HFS/HFS+ (to check
> 
> One potential problem is that the userspace utilities to format,
> check, repair HFS/HFS+ utilities don't really exist.  There is the HFS
> Utilities[1] which is packaged in Debian as hfsutils, but it only
> supports HFS, not HFS+, and it can only format an HFS file system; it
> doesn't have a fsck analog.  This is going to very limit the ability
> to run xfstests for HFS or HFS+.
> 
> [1] https://www.mars.org/home/rob/proj/hfs/

How about hfsprogs, it has mkfs and fsck tools.  Though it /is/ APSL
licensed...

--D

> 						- Ted
> 

Re: HFS/HFS+ maintainership action items

[John Paul Adrian Glaubitz]

Hi Ted,

On Mon, 2025-04-21 at 21:43 -0500, Theodore Ts'o wrote:
> On Mon, Apr 21, 2025 at 09:52:14PM +0000, Viacheslav Dubeyko wrote:
> > Hi Adrian,
> > 
> > I am trying to elaborate the HFS/HFS+ maintainership action items:
> > (1) We need to prepare a Linux kernel tree fork to collect patches.
> > (2) I think it needs to prepare the list of current known issues (TODO list).
> > (3) Let me prepare environment and start to run xfstests for HFS/HFS+ (to check
> 
> One potential problem is that the userspace utilities to format,
> check, repair HFS/HFS+ utilities don't really exist.  There is the HFS
> Utilities[1] which is packaged in Debian as hfsutils, but it only
> supports HFS, not HFS+, and it can only format an HFS file system; it
> doesn't have a fsck analog.  This is going to very limit the ability
> to run xfstests for HFS or HFS+.

There is actually hfsprogs from Apple themselves which supports both HFS
and HFS+ works without any problems. I'm maintaining it in Debian [1] and
openSUSE. It's available in Fedora as hfsplus-tools. I have hacked on it
for a while, so I can also provide updated versions.

In the future, I'm planning to split the package into a normal and -legacy
version as Apple dropped legacy HFS support from the utility somewhere around
version 500.

Adrian

> [1] https://tracker.debian.org/pkg/hfsprogs

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: HFS/HFS+ maintainership action items

[John Paul Adrian Glaubitz]

On Mon, 2025-04-21 at 21:16 -0700, Darrick J. Wong wrote:
> > One potential problem is that the userspace utilities to format,
> > check, repair HFS/HFS+ utilities don't really exist.  There is the HFS
> > Utilities[1] which is packaged in Debian as hfsutils, but it only
> > supports HFS, not HFS+, and it can only format an HFS file system; it
> > doesn't have a fsck analog.  This is going to very limit the ability
> > to run xfstests for HFS or HFS+.
> > 
> > [1] https://www.mars.org/home/rob/proj/hfs/
> 
> How about hfsprogs, it has mkfs and fsck tools.  Though it /is/ APSL
> licensed...

That's what I use and maintain in Debian. I have been trying to reach out
to Apple to get it relicensed under Apache or MIT but I unfortunately got
a reply.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: HFS/HFS+ maintainership action items

[John Paul Adrian Glaubitz]

Hi Slava,

On Mon, 2025-04-21 at 21:52 +0000, Viacheslav Dubeyko wrote:
> I am trying to elaborate the HFS/HFS+ maintainership action items:
> (1) We need to prepare a Linux kernel tree fork to collect patches.

Yes. I suggest creating a tree on git.kernel.org.

> (2) I think it needs to prepare the list of current known issues (TODO list).

Shall we use the kernel wiki for that? I suggest starting with the collection
of known CVEs as well as possible patches. I know of at least one CVE that
Ubuntu has fixed locally.

I can send an email to the author of that patch and ask them to send their
patch upstream.

From my memory, there are some occasional filesystem corruptions reported
on HFS partitions which might be a result of a bug in the kernel driver.

They can be easily fixed with fsck_hfs from hfsprogs though.

> (3) Let me prepare environment and start to run xfstests for HFS/HFS+ (to check
> the current status).

I suggest a Debian VM for that as it has hfsprogs which allows creating both
HFS and HFS+ filesystems. It's also easily possible to test on PowerPC inside
QEMU if necessary.

> (4) Which use-cases do we need to consider for regular testing?

Definitely testing both legacy HFS and HFS+ with creating new filesystems, writing
and reading random files from it as well as running fsck on these.

I'm not a Linux kernel filesystem expert, so I don't know what the recommend tests
for CI are, but I suggest everything that is commonly used, both with HFS and HFS+.

> Anything else? What am I missing?

No, I think that should get us going for the time being.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

RE: HFS/HFS+ maintainership action items

[Viacheslav Dubeyko]

Hi Adrian,

On Tue, 2025-04-22 at 14:35 +0200, John Paul Adrian Glaubitz wrote:
> Hi Slava,
> 
> On Mon, 2025-04-21 at 21:52 +0000, Viacheslav Dubeyko wrote:
> > I am trying to elaborate the HFS/HFS+ maintainership action items:
> > (1) We need to prepare a Linux kernel tree fork to collect patches.
> 
> Yes. I suggest creating a tree on git.kernel.org.
> 

Makes sense. Whom do we need to ask to make it happened?

> > (2) I think it needs to prepare the list of current known issues (TODO list).
> 
> Shall we use the kernel wiki for that? I suggest starting with the collection
> of known CVEs as well as possible patches. I know of at least one CVE that
> Ubuntu has fixed locally.
> 

We can do this, but, as far as I can see, the kernel wiki's pages are marked as
obsolete content. Also, Bugzilla could be more suitable for this. And, yes, we
need to collect all known CVEs somewhere.

> I can send an email to the author of that patch and ask them to send their
> patch upstream.
> 

Let's prepare the kernel tree and WiKi or/and Bugzilla at first.

> From my memory, there are some occasional filesystem corruptions reported
> on HFS partitions which might be a result of a bug in the kernel driver.
> 
> They can be easily fixed with fsck_hfs from hfsprogs though.
> 

Yeah, I need to check the email list for the issue reports. Even if some issues
can be fixed by the fsck_hfs, potentially, some issues could be harmful enough.
So, it makes sense to fix it.

> > (3) Let me prepare environment and start to run xfstests for HFS/HFS+ (to check
> > the current status).
> 
> I suggest a Debian VM for that as it has hfsprogs which allows creating both
> HFS and HFS+ filesystems. It's also easily possible to test on PowerPC inside
> QEMU if necessary.
> 

Sounds good! Do you mean a particular link with ready-made Debian VM images?

> > (4) Which use-cases do we need to consider for regular testing?
> 
> Definitely testing both legacy HFS and HFS+ with creating new filesystems, writing
> and reading random files from it as well as running fsck on these.
> 
> I'm not a Linux kernel filesystem expert, so I don't know what the recommend tests
> for CI are, but I suggest everything that is commonly used, both with HFS and HFS+.
> 

Makes sense. First of all, xfstests is a good tool for file system's
functionality regression testing. Probably, I need to take a deeper look into
fstests, finally. Maybe, we need to consider fio tool for the testing too. I
think that it makes sense to check all supported logical block sizes and some
set of volume sizes. I need to double check the HFS/HFS+ features, and we will
need to test that supported features are not broken. And, of course, we will
need to be sure that file system volume is consistent after been used by x86 and
PowerPC platforms. 

So, let me spend some time for the testing strategy elaboration. 

> 
Thanks,
Slava.

Re: HFS/HFS+ maintainership action items

[John Paul Adrian Glaubitz]

Hi Slava,

On Tue, 2025-04-22 at 21:12 +0000, Viacheslav Dubeyko wrote:
> On Tue, 2025-04-22 at 14:35 +0200, John Paul Adrian Glaubitz wrote:
> > Hi Slava,
> > 
> > On Mon, 2025-04-21 at 21:52 +0000, Viacheslav Dubeyko wrote:
> > > I am trying to elaborate the HFS/HFS+ maintainership action items:
> > > (1) We need to prepare a Linux kernel tree fork to collect patches.
> > 
> > Yes. I suggest creating a tree on git.kernel.org.
> > 
> 
> Makes sense. Whom do we need to ask to make it happened?

Reducing the discussion on the repository for now.

To get a repository on git.kernel.org, you need to have an entry in MAINTAINERS.
Thus, first we would need to get ourselves added to MAINTAINERS for the hfs code.

I have already an entry there as I'm maintaining arch/sh.

You can just send a patch to the LKML, get me and you added to MAINTAINERS, see
[1] and [2] for an example how to do that. Once Linus has merged your patch, you
can request a kernel.org and git tree here [3].

Adrian

> [1] https://lore.kernel.org/lkml/87v8k7rrnf.wl-ysato@users.sourceforge.jp/T/
> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/MAINTAINERS?id=80510b63f7b6bdd30e07b3a42115d0a324e20cd6
> [3] https://korg.docs.kernel.org/accounts.html

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: HFS/HFS+ maintainership action items

[John Paul Adrian Glaubitz]

Hi Slava,

On Tue, 2025-04-22 at 23:47 +0200, John Paul Adrian Glaubitz wrote:
> Reducing the discussion on the repository for now.
> 
> To get a repository on git.kernel.org, you need to have an entry in MAINTAINERS.
> Thus, first we would need to get ourselves added to MAINTAINERS for the hfs code.
> 
> I have already an entry there as I'm maintaining arch/sh.
> 
> You can just send a patch to the LKML, get me and you added to MAINTAINERS, see
> [1] and [2] for an example how to do that. Once Linus has merged your patch, you
> can request a kernel.org and git tree here [3].
> 
> Adrian
> 
> > [1] https://lore.kernel.org/lkml/87v8k7rrnf.wl-ysato@users.sourceforge.jp/T/
> > [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/MAINTAINERS?id=80510b63f7b6bdd30e07b3a42115d0a324e20cd6
> > [3] https://korg.docs.kernel.org/accounts.html

Please see also: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/MAINTAINERS?id=bf8f5de17442bba5f811e7e724980730e079ee11

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

RE: HFS/HFS+ maintainership action items

[Viacheslav Dubeyko]

Hi Adrian, Yangtao,

As far as I can see, xfstests runs have such results:

HFS case

sudo ./check -g quick
FSTYP         -- hfs
PLATFORM      -- Linux/x86_64 hfsplus-testing-0001 6.15.0-rc2+ #2 SMP
PREEMPT_DYNAMIC Mon Apr 21 15:25:18 PDT 2025
MKFS_OPTIONS  -- /dev/sda2
MOUNT_OPTIONS -- /dev/sda2 /mnt/scratch

<skipped>

Failed 100 of 614 tests

Failed test cases:
generic/001
generic/003
generic/006
generic/007
generic/011
generic/013
generic/028
generic/034
generic/035
generic/073
generic/075
generic/080
generic/084
generic/087
generic/088
generic/091
generic/095
generic/101
generic/112
generic/113
generic/120
generic/124
generic/131
generic/198
generic/207
generic/210
generic/211
generic/212
generic/215
generic/221
generic/245
generic/246
generic/247
generic/248
generic/249
generic/257
generic/258
generic/263
generic/308
generic/309
generic/313
generic/321
generic/322
generic/335
generic/338
generic/341
generic/342
generic/362
generic/364
generic/376
generic/377
generic/394
generic/409
generic/410
generic/411
generic/412
generic/428
generic/437
generic/443
generic/448
generic/451
generic/452
generic/471
generic/478
generic/481
generic/490
generic/504
generic/510
generic/532
generic/534
generic/535
generic/547
generic/552
generic/557
generic/563
generic/571
generic/591
generic/609
generic/632
generic/637
generic/638
generic/639
generic/640
generic/647
generic/676
generic/678
generic/704
generic/706
generic/708
generic/729
generic/730
generic/731
generic/732
generic/736
generic/740
generic/741
generic/755
generic/759
generic/760
generic/763
generic/764

HFS+ case

sudo ./check -g quick
FSTYP         -- hfsplus
PLATFORM      -- Linux/x86_64 hfsplus-testing-0001 6.15.0-rc2+ #2 SMP
PREEMPT_DYNAMIC Mon Apr 21 15:25:18 PDT 2025
MKFS_OPTIONS  -- /dev/sda2
MOUNT_OPTIONS -- /dev/sda2 /mnt/scratch

<skipped>

Failed 142 of 614 tests

Failed test cases:
generic/003
generic/013
generic/020
generic/023
generic/024
generic/028
generic/034
generic/035
generic/039
generic/040
generic/041
generic/056
generic/057
generic/062
generic/065
generic/066
generic/067
generic/069
generic/070
generic/073
generic/075
generic/079
generic/080
generic/087
generic/088
generic/090
generic/091
generic/095
generic/097
generic/101
generic/104
generic/106
generic/107
generic/112
generic/113
generic/124
generic/126
generic/131
generic/184
generic/198
generic/207
generic/210
generic/211
generic/212
generic/215
generic/221
generic/236
generic/245
generic/246
generic/247
generic/248
generic/249
generic/257
generic/258
generic/263
generic/306
generic/308
generic/309
generic/313
generic/321
generic/322
generic/335
generic/336
generic/337
generic/341
generic/342
generic/343
generic/348
generic/360
generic/362
generic/364
generic/376
generic/377
generic/394
generic/409
generic/410
generic/411
generic/412
generic/423
generic/424
generic/428
generic/437
generic/443
generic/448
generic/450
generic/451
generic/471
generic/478
generic/479
generic/480
generic/481
generic/489
generic/490
generic/498
generic/502
generic/504
generic/510
generic/523
generic/525
generic/526
generic/527
generic/528
generic/532
generic/533
generic/534
generic/535
generic/538
generic/545
generic/547
generic/552
generic/555
generic/557
generic/563
generic/571
generic/591
generic/609
generic/632
generic/637
generic/638
generic/639
generic/640
generic/647
generic/676
generic/678
generic/690
generic/704
generic/706
generic/708
generic/728
generic/729
generic/730
generic/731
generic/732
generic/736
generic/740
generic/741
generic/755
generic/759
generic/760
generic/763
generic/764

So, we need at first to check these issues. And it's a lot of work. :)

Thanks,
Slava.

回复: HFS/HFS+ maintainership action items

[李扬韬]

Hi Slava,

> So, we need at first to check these issues. And it's a lot of work. :)

That's a lot of test failures, probably many for the same reason.

Are there any changes to xfstest? Has this been sent to the fstest mailing list?

I'm also planning to start deploying a local xfstest environment. : )

MBR,
Yangtao

Re: 回复: HFS/HFS+ maintainership action items

[John Paul Adrian Glaubitz]

Hi Slava and Yangtao,

On Fri, 2025-04-25 at 10:17 +0000, 李扬韬 wrote:
> > So, we need at first to check these issues. And it's a lot of work. :)
> 
> That's a lot of test failures, probably many for the same reason.

I agree, there is probably one bug causing most of these failures.

> Are there any changes to xfstest? Has this been sent to the fstest mailing list?
> 
> I'm also planning to start deploying a local xfstest environment. : )

Would you guys mind help me create an xfstest test environment as well?

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

回复: 回复: HFS/HFS+ maintainership action items

[李扬韬]

Hi Adrian,

> Would you guys mind help me create an xfstest test environment as well?

I am currently running arch linux on wsl 2 in a windows laptop.

I'm following the README steps here (ing, not sure if I'll run into problems).

https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/tree/README?h=for-next
git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git

Maybe slava and some other additions too?

Thx,
Yangtao

Re: 回复: HFS/HFS+ maintainership action items

[Viacheslav Dubeyko]

Hi Yangtao,

On Fri, 2025-04-25 at 10:17 +0000, 李扬韬 wrote:
> Hi Slava,
> 
> > So, we need at first to check these issues. And it's a lot of work. :)
> 
> That's a lot of test failures, probably many for the same reason.
> 

Probably, yes, but we need to check it. If it is one or several root causes,
then we are lucky. :)

> Are there any changes to xfstest? Has this been sent to the fstest mailing list?
> 

This is a generic test cases and other file systems can pass it. For example, I
can pass xfstests for CephFS kernel client. So, we must think now that it is
HFS/HFS+ issues until we can prove the opposite one.

> I'm also planning to start deploying a local xfstest environment. : )
> 

Sounds great! I think we will need to distribute test-cases check to avoid the
duplication of efforts.

Thanks,
Slava.

Re: 回复: 回复: HFS/HFS+ maintainership action items

[Viacheslav Dubeyko]

On Fri, 2025-04-25 at 11:39 +0000, 李扬韬 wrote:
> Hi Adrian,
> 
> > Would you guys mind help me create an xfstest test environment as well?
> 
> I am currently running arch linux on wsl 2 in a windows laptop.
> 
> I'm following the README steps here (ing, not sure if I'll run into problems).
> 
> https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/tree/README?h=for-next  
> git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
> 
> Maybe slava and some other additions too?

Yes, I used this way to prepare the xfstests environment. So, the main steps:
(1) Clone git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
(2) Install all necessary packages + hfsprog tools. This step could depends from
a particular environment. 
(3) Do make and make install. Usually, it works well but last time I had some
issues that I managed to resolve.
(4) Prepare the local.config file:

HFS case:

export TEST_DEV=/dev/sda1
export TEST_DIR=/mnt/test
export SCRATCH_DEV=/dev/sda2
export SCRATCH_MNT=/mnt/scratch

export FSTYP=hfs

HFS+ case:

export TEST_DEV=/dev/sda1
export TEST_DIR=/mnt/test
export SCRATCH_DEV=/dev/sda2
export SCRATCH_MNT=/mnt/scratch

export FSTYP=hfsplus

(5) Run the check command:

Group test (for example quick group):
sudo ./check -g quick

Particular test:
sudo ./check generic/001

Mostly, these steps should prepare the xfstests environment. Of course, it could
run on physical machine or inside of virtual machine. It needs to prepare the
two  partitions or drives (test and scratch) for testing.

If you will have some troubles, please, let me know and I will try to help. :)

Thanks,
Slava.

回复: 回复: 回复: HFS/HFS+ maintainership action items

[李扬韬]

Hi Slava,

> If you will have some troubles, please, let me know and I will try to help. :)

There are some strange issues indeed,. : )

One has been solved, and the other may be solved by changing the environment.

1). Header files installed, but linux/blkdev.h is still missing.

Currently I copied the header files manually and it works.

/usr/include/liburing/compat.h:11:10: fatal error: linux/blkdev.h: No such file or directory
   11 | #include <linux/blkdev.h>
      |          ^~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [Makefile:51: fsstress] Error 1
make[2]: *** [include/buildrules:31: ltp] Error 2
make[1]: *** [Makefile:65: default] Error 2
make: *** [Makefile:63: default] Error 2

2). Arch Linux has the hfsprogs installation package, but after installation there is only mkfs.hfsplus, missing mkfs.hfs

I think if I switch to Ubuntu or something, this problem should go away.

I used the following command to test and got these failure cases.

sudo ./check -g quick

Failures: generic/001 generic/002 generic/003 generic/005 generic/006 generic/007 generic/011 generic/013 generic/020 generic/023 generic/024 generic/028 generic/029 generic/030 generic/035 generic/037 generic/062 generic/067 generic/069 generic/070 generic/075 generic/076 generic/079 generic/080 generic/084 generic/087 generic/088 generic/091 generic/095 generic/097 generic/098 generic/112 generic/113 generic/117 generic/120 generic/124 generic/126 generic/131 generic/135 generic/141 generic/169 generic/184 generic/198 generic/207 generic/210 generic/211 generic/212 generic/215 generic/221 generic/236 generic/245 generic/246 generic/247 generic/248 generic/249 generic/257 generic/258 generic/263 generic/294 generic/306 generic/308 generic/309 generic/313 generic/337 generic/338 generic/346 generic/360 generic/362 generic/364 generic/366 generic/377 generic/393 generic/394 generic/401 generic/403 generic/406 generic/409 generic/410 generic/411 generic/412 generic/423 generic/424 generic/428 generic/437 generic/441 generic/443 generic/448 generic/450 generic/451 generic/452 generic/460 generic/465 generic/471 generic/472 generic/478 generic/484 generic/486 generic/490 generic/504 generic/519 generic/523 generic/524 generic/525 generic/528 generic/532 generic/533 generic/538 generic/545 generic/555 generic/571 generic/591 generic/604 generic/609 generic/611 generic/615 generic/618 generic/632 generic/634 generic/636 generic/637 generic/638 generic/639 generic/647 generic/676 generic/706 generic/708 generic/728 generic/729 generic/732 generic/736 generic/740 generic/755 generic/759 generic/760 generic/761 generic/763
Failed 136 of 614 tests

I have to go to bed right now. : )

Thanks,
Yangtao

Re: 回复: 回复: 回复: HFS/HFS+ maintainership action items

[Viacheslav Dubeyko]

Hi Yangtao,

On Fri, 2025-04-25 at 19:36 +0000, 李扬韬 wrote:
> Hi Slava,
> 
> > If you will have some troubles, please, let me know and I will try to help. :)
> 
> There are some strange issues indeed,. : )
> 
> One has been solved, and the other may be solved by changing the environment.
> 
> 1). Header files installed, but linux/blkdev.h is still missing.
> 
> Currently I copied the header files manually and it works.
> 
> /usr/include/liburing/compat.h:11:10: fatal error: linux/blkdev.h: No such file or directory
>    11 | #include <linux/blkdev.h>
>       |          ^~~~~~~~~~~~~~~~
> compilation terminated.
> make[3]: *** [Makefile:51: fsstress] Error 1
> make[2]: *** [include/buildrules:31: ltp] Error 2
> make[1]: *** [Makefile:65: default] Error 2
> make: *** [Makefile:63: default] Error 2
> 

Do you have libblkid-dev installed?

> 2). Arch Linux has the hfsprogs installation package, but after installation there is only mkfs.hfsplus, missing mkfs.hfs
> 

As far as I know, you need to use -h option to create HFS volume:

usage: mkfs.hfsplus [-N [partition-size]] [hfsplus-options] special-device
  options:
	-h create an HFS format filesystem (HFS Plus is the default)
	-N do not create file system, just print out parameters
	-s use case-sensitive filenames (default is case-insensitive)
	-w add a HFS wrapper (i.e. Native Mac OS 9 bootable)
  where hfsplus-options are:
	-J [journal-size] make this HFS+ volume journaled
	-D journal-dev use 'journal-dev' for an external journal
	-G group-id (for root directory)
	-U user-id (for root directory)
	-M octal access-mask (for root directory)
	-b allocation block size (4096 optimal)
	-c clump size list (comma separated)
		a=blocks (attributes file)
		b=blocks (bitmap file)
		c=blocks (catalog file)
		d=blocks (user data fork)
		e=blocks (extents file)
		r=blocks (user resource fork)
	-i starting catalog node id
	-n b-tree node size list (comma separated)
		e=size (extents b-tree)
		c=size (catalog b-tree)
		a=size (attributes b-tree)
	-v volume name (in ascii or UTF-8)
  examples:
	mkfs.hfsplus -v Untitled /dev/rdisk0s7 
	mkfs.hfsplus -v Untitled -n c=4096,e=1024 /dev/rdisk0s7 
	mkfs.hfsplus -v Untitled -c b=64,c=1024 /dev/rdisk0s7

> I think if I switch to Ubuntu or something, this problem should go away.
> 
> I used the following command to test and got these failure cases.
> 
> sudo ./check -g quick
> 
> Failures: generic/001 generic/002 generic/003 generic/005 generic/006 generic/007 generic/011 generic/013 generic/020 generic/023 generic/024 generic/028 generic/029 generic/030 generic/035 generic/037 generic/062 generic/067 generic/069 generic/070 generic/075 generic/076 generic/079 generic/080 generic/084 generic/087 generic/088 generic/091 generic/095 generic/097 generic/098 generic/112 generic/113 generic/117 generic/120 generic/124 generic/126 generic/131 generic/135 generic/141 generic/169 generic/184 generic/198 generic/207 generic/210 generic/211 generic/212 generic/215 generic/221 generic/236 generic/245 generic/246 generic/247 generic/248 generic/249 generic/257 generic/258 generic/263 generic/294 generic/306 generic/308 generic/309 generic/313 generic/337 generic/338 generic/346 generic/360 generic/362 generic/364 generic/366 generic/377 generic/393 generic/394 generic/401 generic/403 generic/406 generic/409 generic/410 generic/411 generic/412 generic/423 generic/424 generic/428 generic/437 generic/441 generic/443 generic/448 generic/450 generic/451 generic/452 generic/460 generic/465 generic/471 generic/472 generic/478 generic/484 generic/486 generic/490 generic/504 generic/519 generic/523 generic/524 generic/525 generic/528 generic/532 generic/533 generic/538 generic/545 generic/555 generic/571 generic/591 generic/604 generic/609 generic/611 generic/615 generic/618 generic/632 generic/634 generic/636 generic/637 generic/638 generic/639 generic/647 generic/676 generic/706 generic/708 generic/728 generic/729 generic/732 generic/736 generic/740 generic/755 generic/759 generic/760 generic/761 generic/763
> Failed 136 of 614 tests
> 

Yeah, it is reproducible.


Thanks,
Slava.

Re: 回复: 回复: 回复: HFS/HFS+ maintainership action items

[John Paul Adrian Glaubitz]

Hi Yangtao,

On Fri, 2025-04-25 at 19:36 +0000, 李扬韬 wrote:
> 2). Arch Linux has the hfsprogs installation package, but
> after installation there is only mkfs.hfsplus, missing mkfs.hfs
> 
> I think if I switch to Ubuntu or something, this problem should go away.

Apple removed support for creating legacy HFS filesystems around version
500 of the hfs source code but kept the code for checking legacy HFS
filesystems.

For the Debian package, I just forward-ported support for creating legacy
HFS filesystems such that Debian's hfsprogs supports both HFS and HFS+.

The patch can be found in [1].

In the future, I'm planning to split the Debian packages into a hfsprogs-legacy
and a hfsprogs package. The -legacy package is supposed to stay at the old
package version and provide mkfs.hfs while the hfsprogs package will track the
current upstream sources as Apple is still actively maintaining hfsprogs [2].

I am also working on creating a patch set that all Linux distributions can use
on top of Apple's vanilla upstream sources. The current WIP can be found in [3].

Adrian

> [1] https://salsa.debian.org/debian/hfsprogs/-/blob/master/debian/patches/0005-Re-add-support-for-creating-legacy-HFS-filesystems.patch?ref_type=heads
> [2] https://github.com/apple-oss-distributions/hfs
> [3] https://github.com/glaubitz/hfs/tree/linux

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

回复: 回复: 回复: 回复: HFS/HFS+ maintainership action items

[李扬韬]

Hi Slava and Adrian,

> Do you have libblkid-dev installed?

Arch Linux doesn't seem to have this package. Anyway, since it works now, I think I can ignore this problem.

> As far as I know, you need to use -h option to create HFS volume:

This seems to be the point mentioned by Adrian, which is caused by the hfsprogs code removing mkfs.hfs.

> I am also working on creating a patch set that all Linux distributions can use on top of Apple's vanilla upstream sources. The current WIP can be found in [3].

Adrian, Would you mind adding a branch that supports mkfs.hfs in your git repository now?

Maybe the hfsprogs-related code will be worth rewriting in the future, 
but this is probably not the focus at the moment (at least for me).

I plan on looking at a few hfsplus failures first, or if there is something else planned I'll go for that too.

MBR,
Yangtao

Re: 回复: 回复: 回复: 回复: HFS/HFS+ maintainership action items

[John Paul Adrian Glaubitz]

Hi Yangtao,

On Sat, 2025-04-26 at 06:17 +0000, 李扬韬 wrote:
> > I am also working on creating a patch set that all Linux distributions can use on top of
> > Apple's vanilla upstream sources. The current WIP can be found in [3].
> 
> Adrian, Would you mind adding a branch that supports mkfs.hfs in your git repository now?

Yes, I can do that. Will let you know once that's ready.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: 回复: 回复: 回复: 回复: HFS/HFS+ maintainership action items

[Viacheslav Dubeyko]

On Sat, 2025-04-26 at 06:17 +0000, 李扬韬 wrote:
> Hi Slava and Adrian,

> I plan on looking at a few hfsplus failures first, or if there is something else planned I'll go for that too.
> 

Sounds good! I think I can share some resources [1, 2] if you need to take a
deeper look into HFS/HFS+ on-disk layout.

I already started to take a look into HFS issues. Currently, I am investigating
generic/001 case.

fsck.hfs -d -n ./test-image.bin 
** ./test-image.bin (NO WRITE)
	Using cacheBlockSize=32K cacheTotalBlock=1024 cacheSize=32768K.
   Executing fsck_hfs (version 540.1-Linux).
** Checking HFS volume.
   The volume name is untitled
** Checking extents overflow file.
** Checking catalog file.
   Unused node is not erased (node = 2)
   Unused node is not erased (node = 4)
   Unused node is not erased (node = 5)
   Unused node is not erased (node = 6)
   Unused node is not erased (node = 7)
   Unused node is not erased (node = 8)
   Unused node is not erased (node = 9)
   Unused node is not erased (node = 10)
   Unused node is not erased (node = 11)
   Unused node is not erased (node = 12)
   Unused node is not erased (node = 13)
   Unused node is not erased (node = 14)
   Unused node is not erased (node = 16)
   Unused node is not erased (node = 17)
   Unused node is not erased (node = 18)
   Unused node is not erased (node = 19)
   Unused node is not erased (node = 20)
   Unused node is not erased (node = 21)
   Unused node is not erased (node = 22)
   Unused node is not erased (node = 23)
   Unused node is not erased (node = 24)
   Unused node is not erased (node = 25)
   Unused node is not erased (node = 26)
   Unused node is not erased (node = 27)
   Unused node is not erased (node = 28)
   Unused node is not erased (node = 29)
   Unused node is not erased (node = 30)
   Unused node is not erased (node = 31)
   Unused node is not erased (node = 32)
   Unused node is not erased (node = 33)
   Unused node is not erased (node = 34)
   Unused node is not erased (node = 35)
   Unused node is not erased (node = 36)
   Unused node is not erased (node = 37)
   Unused node is not erased (node = 38)
   Unused node is not erased (node = 39)
   Unused node is not erased (node = 40)
   Unused node is not erased (node = 41)
   Unused node is not erased (node = 42)
   Unused node is not erased (node = 43)
   Unused node is not erased (node = 44)
   Unused node is not erased (node = 45)
   Unused node is not erased (node = 46)
   Unused node is not erased (node = 47)
   Unused node is not erased (node = 48)
   Unused node is not erased (node = 49)
   Unused node is not erased (node = 50)
   Unused node is not erased (node = 51)
   Unused node is not erased (node = 52)
   Unused node is not erased (node = 53)
   Unused node is not erased (node = 54)
   Unused node is not erased (node = 55)
   Unused node is not erased (node = 56)
   Unused node is not erased (node = 57)
   Unused node is not erased (node = 58)
   Unused node is not erased (node = 59)
   Unused node is not erased (node = 60)
   Unused node is not erased (node = 61)
   Unused node is not erased (node = 62)
   Unused node is not erased (node = 63)
   Unused node is not erased (node = 64)
   Unused node is not erased (node = 65)
   Unused node is not erased (node = 66)
   Unused node is not erased (node = 67)
   Unused node is not erased (node = 68)
   Unused node is not erased (node = 69)
   Unused node is not erased (node = 70)
   Unused node is not erased (node = 71)
   Unused node is not erased (node = 72)
   Unused node is not erased (node = 73)
   Unused node is not erased (node = 74)
   Unused node is not erased (node = 75)
   Unused node is not erased (node = 76)
   Unused node is not erased (node = 77)
   Unused node is not erased (node = 78)
   Unused node is not erased (node = 79)
   Unused node is not erased (node = 80)
   Unused node is not erased (node = 81)
   Unused node is not erased (node = 82)
   Unused node is not erased (node = 83)
   Unused node is not erased (node = 84)
   Unused node is not erased (node = 85)
   Unused node is not erased (node = 86)
   Unused node is not erased (node = 87)
   Unused node is not erased (node = 88)
   Unused node is not erased (node = 89)
   Unused node is not erased (node = 90)
   Unused node is not erased (node = 91)
   Unused node is not erased (node = 92)
   Unused node is not erased (node = 93)
   Unused node is not erased (node = 95)
   Unused node is not erased (node = 96)
   Unused node is not erased (node = 97)
   Unused node is not erased (node = 98)
   Unused node is not erased (node = 99)
   Unused node is not erased (node = 100)
   Unused node is not erased (node = 101)
   Unused node is not erased (node = 102)
   Unused node is not erased (node = 103)
   Unused node is not erased (node = 104)
   Unused node is not erased (node = 105)
   Unused node is not erased (node = 106)
   Unused node is not erased (node = 107)
   Unused node is not erased (node = 108)
   Unused node is not erased (node = 109)
   Unused node is not erased (node = 110)
   Unused node is not erased (node = 111)
   Unused node is not erased (node = 112)
   Unused node is not erased (node = 113)
   Unused node is not erased (node = 114)
   Unused node is not erased (node = 115)
   Unused node is not erased (node = 116)
   Unused node is not erased (node = 117)
   Unused node is not erased (node = 118)
   Unused node is not erased (node = 119)
   Unused node is not erased (node = 120)
   Unused node is not erased (node = 121)
   Unused node is not erased (node = 122)
   Unused node is not erased (node = 123)
   Unused node is not erased (node = 124)
   Unused node is not erased (node = 125)
   Unused node is not erased (node = 126)
   Unused node is not erased (node = 127)
   Unused node is not erased (node = 128)
   Unused node is not erased (node = 129)
   Unused node is not erased (node = 130)
   Unused node is not erased (node = 131)
   Unused node is not erased (node = 132)
   Unused node is not erased (node = 133)
   Unused node is not erased (node = 134)
   Unused node is not erased (node = 135)
   Unused node is not erased (node = 136)
   Unused node is not erased (node = 137)
   Unused node is not erased (node = 138)
   Unused node is not erased (node = 139)
   Unused node is not erased (node = 140)
   Unused node is not erased (node = 141)
   Unused node is not erased (node = 142)
   Unused node is not erased (node = 143)
   Unused node is not erased (node = 144)
   Unused node is not erased (node = 145)
   Unused node is not erased (node = 146)
   Unused node is not erased (node = 147)
   Unused node is not erased (node = 148)
   Unused node is not erased (node = 149)
   Unused node is not erased (node = 150)
   Unused node is not erased (node = 151)
   Unused node is not erased (node = 152)
   Unused node is not erased (node = 153)
   Unused node is not erased (node = 154)
   Unused node is not erased (node = 155)
   Unused node is not erased (node = 156)
   Unused node is not erased (node = 157)
   Unused node is not erased (node = 158)
   Unused node is not erased (node = 159)
   Unused node is not erased (node = 160)
   Unused node is not erased (node = 161)
   Unused node is not erased (node = 162)
   Unused node is not erased (node = 163)
   Unused node is not erased (node = 164)
   Unused node is not erased (node = 165)
   Unused node is not erased (node = 166)
   Unused node is not erased (node = 167)
   Unused node is not erased (node = 168)
   Unused node is not erased (node = 169)
   Unused node is not erased (node = 170)
   Unused node is not erased (node = 171)
   Unused node is not erased (node = 172)
   Unused node is not erased (node = 173)
   Unused node is not erased (node = 174)
   Unused node is not erased (node = 175)
   Unused node is not erased (node = 176)
   Unused node is not erased (node = 177)
   Unused node is not erased (node = 178)
   Unused node is not erased (node = 179)
   Unused node is not erased (node = 180)
   Unused node is not erased (node = 181)
   Unused node is not erased (node = 182)
   Unused node is not erased (node = 183)
   Unused node is not erased (node = 184)
   Unused node is not erased (node = 185)
   Unused node is not erased (node = 186)
   Unused node is not erased (node = 187)
   Unused node is not erased (node = 188)
   Unused node is not erased (node = 189)
   Unused node is not erased (node = 190)
   Unused node is not erased (node = 191)
   Unused node is not erased (node = 192)
   Unused node is not erased (node = 193)
   Unused node is not erased (node = 194)
   Unused node is not erased (node = 195)
   Unused node is not erased (node = 196)
   Unused node is not erased (node = 197)
   Unused node is not erased (node = 198)
   Unused node is not erased (node = 199)
   Unused node is not erased (node = 200)
   Unused node is not erased (node = 201)
   Unused node is not erased (node = 202)
   Unused node is not erased (node = 203)
   Unused node is not erased (node = 204)
   Unused node is not erased (node = 205)
   Unused node is not erased (node = 206)
   Unused node is not erased (node = 207)
   Unused node is not erased (node = 208)
   Unused node is not erased (node = 209)
   Unused node is not erased (node = 210)
   Unused node is not erased (node = 211)
   Unused node is not erased (node = 212)
   Unused node is not erased (node = 213)
   Unused node is not erased (node = 214)
   Unused node is not erased (node = 215)
   Unused node is not erased (node = 216)
   Unused node is not erased (node = 217)
   Unused node is not erased (node = 218)
   Unused node is not erased (node = 219)
   Unused node is not erased (node = 220)
   Unused node is not erased (node = 221)
   Unused node is not erased (node = 222)
   Unused node is not erased (node = 223)
   Unused node is not erased (node = 224)
   Unused node is not erased (node = 225)
   Unused node is not erased (node = 226)
   Unused node is not erased (node = 227)
   Unused node is not erased (node = 228)
   Unused node is not erased (node = 229)
   Unused node is not erased (node = 230)
   Unused node is not erased (node = 231)
   Unused node is not erased (node = 232)
   Unused node is not erased (node = 233)
   Unused node is not erased (node = 234)
   Unused node is not erased (node = 235)
   Unused node is not erased (node = 236)
   Unused node is not erased (node = 237)
   Unused node is not erased (node = 238)
   Unused node is not erased (node = 239)
   Unused node is not erased (node = 240)
   Unused node is not erased (node = 241)
   Unused node is not erased (node = 242)
   Unused node is not erased (node = 243)
   Unused node is not erased (node = 244)
   Unused node is not erased (node = 245)
   Unused node is not erased (node = 246)
   Unused node is not erased (node = 247)
   Unused node is not erased (node = 248)
   Unused node is not erased (node = 249)
   Unused node is not erased (node = 250)
   Unused node is not erased (node = 251)
   Unused node is not erased (node = 252)
   Unused node is not erased (node = 253)
   Unused node is not erased (node = 254)
   Unused node is not erased (node = 255)
   Unused node is not erased (node = 256)
** Checking catalog hierarchy.
** Checking volume bitmap.
** Checking volume information.
   Verify Status: VIStat = 0x0000, ABTStat = 0x0000 EBTStat = 0x0000
                  CBTStat = 0x0004 CatStat = 0x00000000
** The volume untitled was found corrupt and needs to be repaired.
	volume type is HFS 
	primary MDB is at block 2 0x02 
	alternate MDB is at block 20971518 0x13ffffe 
	primary VHB is at block 0 0x00 
	alternate VHB is at block 0 0x00 
	sector size = 512 0x200 
	VolumeObject flags = 0x19 
	total sectors for volume = 20971520 0x1400000 
	total sectors for embedded volume = 0 0x00

It looks like that HFS driver doesn't erase deleted nodes of catalog file. So, I
need to refresh my knowledge of HFS on-disk layout.

Somehow, scratch volume becomes corrupted too. But I don't quite follow yet
which operation makes it happened. We lost magic signature in the first MDB.

00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000400  00 00 e4 31 b7 03 e4 31  b7 03 01 00 00 00 00 03  |...1...1........|  
00000410  00 00 ff 33 00 02 82 00  00 0a 08 00 00 13 00 00  |...3............|
00000420  00 10 ff 27 08 75 6e 74  69 74 6c 65 64 00 00 00  |...'.untitled...|
00000430  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000440  00 00 00 00 00 00 00 00  00 03 00 0f 0c 00 00 0f  |................|
00000450  0c 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000460  00 00 00 00 00 00 00 00  00 00 00 00 00 63 6e 65  |.............cne|
00000470  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000480  00 00 00 0f 0c 00 00 00  00 06 00 00 00 00 00 00  |................|
00000490  00 00 00 0f 0c 00 00 06  00 06 00 00 00 00 00 00  |................|
000004a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000600  ff f0 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000610  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00002600  00 00 00 00 00 00 00 00  01 00 00 03 00 00 00 00  |................|
00002610  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00002620  02 00 00 07 00 00 07 86  00 00 07 85 00 00 00 0f  |................|
00002630  0c 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00002640  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000026f0  00 00 00 00 00 00 00 00  80 00 00 00 00 00 00 00  |................|
00002700  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000027f0  00 00 00 00 00 00 00 00  01 f8 00 f8 00 78 00 0e  |.............x..|
00002800  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000f3200  00 00 00 00 00 00 00 00  01 00 00 03 00 00 00 01  |................|
000f3210  00 00 00 01 00 00 00 02  00 00 00 01 00 00 00 01  |................|
000f3220  02 00 00 25 00 00 07 86  00 00 07 84 00 00 00 0f  |...%............|
000f3230  0c 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
000f3240  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000f32f0  00 00 00 00 00 00 00 00  c0 00 00 00 00 00 00 00  |................|
000f3300  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000f33f0  00 00 00 00 00 00 00 00  01 f8 00 f8 00 78 00 0e  |.............x..|
000f3400  00 00 00 00 00 00 00 00  ff 01 00 02 00 00 0f 00  |................|
000f3410  00 00 00 01 08 75 6e 74  69 74 6c 65 64 00 01 00  |.....untitled...|
000f3420  00 00 00 00 00 00 00 02  e4 31 b7 03 e4 31 b7 03  |.........1...1..|
000f3430  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000f3460  00 00 00 00 06 00 00 00  00 02 00 00 03 00 00 00  |................|
000f3470  00 00 00 00 00 00 00 00  00 01 08 75 6e 74 69 74  |...........untit|
000f3480  6c 65 64 00 00 00 00 00  00 00 00 00 00 00 00 00  |led.............|
000f3490  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
000f35f0  00 00 00 00 00 00 00 00  00 00 00 9a 00 64 00 0e  |.............d..|
000f3600  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
27ffffc00  42 44 e4 31 b7 03 e4 31  b7 03 01 00 00 00 00 03  |BD.1...1........|
27ffffc10  00 00 ff 33 00 02 82 00  00 0a 08 00 00 13 00 00  |...3............|
27ffffc20  00 10 ff 27 08 75 6e 74  69 74 6c 65 64 00 00 00  |...'.untitled...|
27ffffc30  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
27ffffc40  00 00 00 00 00 00 00 00  00 02 00 0f 0c 00 00 0f  |................|
27ffffc50  0c 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
27ffffc60  00 00 00 00 00 00 00 00  00 00 00 00 00 63 6e 65  |.............cne|
27ffffc70  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
27ffffc80  00 00 00 0f 0c 00 00 00  00 06 00 00 00 00 00 00  |................|
27ffffc90  00 00 00 0f 0c 00 00 06  00 06 00 00 00 00 00 00  |................|
27ffffca0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
280000000

And it looks like that mkfs.hfs simply ignores the -v option. I requested the
test volume label, but it's untitled, finally. 

mkfs.hfs -v test ./scratch-image.bin 
Initialized ./scratch-image.bin as a 10240 MB HFS volume

hexdump -C ./scratch-image.bin
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00000400  42 44 e4 34 41 f8 e4 34  41 f8 01 00 00 00 00 03  |BD.4A..4A.......|
00000410  00 00 ff 33 00 02 82 00  00 0a 08 00 00 13 00 00  |...3............|
00000420  00 10 ff 27 08 75 6e 74  69 74 6c 65 64 00 00 00  |...'.untitled...|
00000430  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000440  00 00 00 00 00 00 00 00  00 02 00 0f 0c 00 00 0f  |................|
00000450  0c 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000460  00 00 00 00 00 00 00 00  00 00 00 00 00 63 6e 65  |.............cne|
00000470  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000480  00 00 00 0f 0c 00 00 00  00 06 00 00 00 00 00 00  |................|
00000490  00 00 00 0f 0c 00 00 06  00 06 00 00 00 00 00 00  |................|
000004a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

mkfs.hfs -h
usage: mkfs.hfs [-N [partition-size]] [hfsplus-options] special-device
  options:
	-h create an HFS format filesystem (HFS Plus is the default)
	-N do not create file system, just print out parameters
	-s use case-sensitive filenames (default is case-insensitive)
	-w add a HFS wrapper (i.e. Native Mac OS 9 bootable)
  where hfsplus-options are:
	-J [journal-size] make this HFS+ volume journaled
	-D journal-dev use 'journal-dev' for an external journal
	-G group-id (for root directory)
	-U user-id (for root directory)
	-M octal access-mask (for root directory)
	-b allocation block size (4096 optimal)
	-c clump size list (comma separated)
		a=blocks (attributes file)
		b=blocks (bitmap file)
		c=blocks (catalog file)
		d=blocks (user data fork)
		e=blocks (extents file)
		r=blocks (user resource fork)
	-i starting catalog node id
	-n b-tree node size list (comma separated)
		e=size (extents b-tree)
		c=size (catalog b-tree)
		a=size (attributes b-tree)
	-v volume name (in ascii or UTF-8)
  examples:
	mkfs.hfs -v Untitled /dev/rdisk0s7 
	mkfs.hfs -v Untitled -n c=4096,e=1024 /dev/rdisk0s7 
	mkfs.hfs -v Untitled -c b=64,c=1024 /dev/rdisk0s

Thanks,
Slava.

[1] https://dubeyko.com/development/FileSystems/hfs.html
[2] https://dubeyko.com/development/FileSystems/hfsplus.html

Re: 回复: 回复: 回复: 回复: HFS/HFS+ maintainership action items

[Theodore Ts'o]

Hey, in case it would be helpfui, I've added hfs support to the
kvm-xfstests/gce-xfstests[1] test appliance.  Following the
instructions at [2], you can now run "kvm-xfstests -c hfs -g auto" to
run all of the tests in the auto group.  If you want to replicate the
failure in generic/001, you could run "kvm-fstests -c hfs generic/001".

[1] http://thunk.org/gce-xfstests
[2] https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md

Your IBM colleages Ritesh Harjani and Ojaswin Mujoo use this framework
for testing ext4, and have contributed towards this test framework.
So if you have any questions, you could reach out to them.  I'm quite
willing to help as well, of course!

					- Ted

RE: 回复: 回复: 回复: 回复: HFS/HFS+ maintainership action items

[Viacheslav Dubeyko]

On Thu, 2025-05-01 at 23:01 -0400, Theodore Ts'o wrote:
> Hey, in case it would be helpfui, I've added hfs support to the
> kvm-xfstests/gce-xfstests[1] test appliance.  Following the
> instructions at [2], you can now run "kvm-xfstests -c hfs -g auto" to
> run all of the tests in the auto group.  If you want to replicate the
> failure in generic/001, you could run "kvm-fstests -c hfs generic/001".
> 

Yes, it is really helpful! Sounds great! Let me try this framework for HFS/HFS+.
Thanks a lot.

> [1] http://thunk.org/gce-xfstests  
> [2] https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md  
> 
> Your IBM colleages Ritesh Harjani and Ojaswin Mujoo use this framework
> for testing ext4, and have contributed towards this test framework.
> So if you have any questions, you could reach out to them.  I'm quite
> willing to help as well, of course!
> 

Make sense. I will do. Thanks again.

Thanks,
Slava.

Re: 回复: 回复: 回复: 回复: HFS/HFS+ maintainership action items

[Theodore Ts'o]

On Fri, May 02, 2025 at 07:14:26PM +0000, Viacheslav Dubeyko wrote:
> On Thu, 2025-05-01 at 23:01 -0400, Theodore Ts'o wrote:
> > Hey, in case it would be helpfui, I've added hfs support to the
> > kvm-xfstests/gce-xfstests[1] test appliance.  Following the
> > instructions at [2], you can now run "kvm-xfstests -c hfs -g auto" to
> > run all of the tests in the auto group.  If you want to replicate the
> > failure in generic/001, you could run "kvm-fstests -c hfs generic/001".
> > 
> 
> Yes, it is really helpful! Sounds great! Let me try this framework for HFS/HFS+.
> Thanks a lot.

FYI, I'm using the hfsprogs from Debian, which at the moment only
supports HFS+.  The prebuilt test appliance for {kvm,gce}-xfstests are
based on Debian Stable (Bookworm), but I am building test appliances
using Debian Testing (Trixie).  However, for the purposes of hfsprogs,
both Debian Bookwrm and Trixie are based on the 540.1 version of
hfsprogs.

But there are plenty of bugs to fix until we can manage to get a
version of hfsprogs that supports HFS --- also I'd argue that for many
users support of HFS+ is probably more useful.

If you find some test failures which are more about test bugs than
kernel bug, so we can add them to exclude files.  For example, in
/root/fs/ext4/exclude I have things like:

// generic/04[456] tests how truncate and delayed allocation works
// ext4 uses the data=ordered to avoid exposing stale data, and
// so it uses a different mechanism than xfs.  So these tests will fail
generic/044
generic/045
generic/046

Since I aso test LTS kernels, and sometimes it's not practcal to
backport fixes to older kernels we can also do versioned excludes.
For example, I have in /root/fs/global_exclude entries like:

#if LINUX_VERSION_CODE < KERNEL_VERSION(6,6,30)
// This test failure is fixed by commit 631426ba1d45q ("mm/madvise:
// make MADV_POPULATE_(READ|WRITE) handle VM_FAULT_RETRY properly"),
// which first landed in v6.9, and was backported to 6.6.30 as commit
// 631426ba1d45.  Unfortunately, it's too involved to backport it and its
// dependencies to the 6.1 or earlier LTS kernels
generic/743
#endif

Finally, I have things set up to automatically run tests when a branch
on a git tree that I'm watching changes.  For exmaple:

gce-xfstests ltm -c ext4/all,xfs/all,btrfs/all,f2fs/all -g auto --repo https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next --watch fs-next

gce-xfstests ltm -c ext4/all,xfs/all -g auto --repo stable-rc.git --watch linux-6.12.y

gce-xfstests ltm -c ext4/all,xfs/all -g auto --repo stable-rc.git --watch linux-6.6.y

If it's helpful, I can set up watchers for hfs and send them to you or
some mailing list once the number of failures are reduced toa
manageable number.

Cheers,

					- Ted

P.S.   An example e-mail report:


TESTRUNID: ltm-20250416105110-0009
KERNEL:    kernel 6.12.25-rc1-xfstests-g56d2398227a2 #6 SMP PREEMPT_DYNAMIC Wed Apr 23 10:48:50 EDT 2025 x86_64
CMDLINE:   -c ext4/all,xfs/all -g auto --repo stable-rc.git --watch linux-6.12.y
CPUS:      2
MEM:       7680

ext4/4k: 587 tests, 1 failures, 55 skipped, 5596 seconds
  Flaky: generic/234: 20% (1/5)
ext4/1k: 581 tests, 2 failures, 59 skipped, 7807 seconds
  Failures: generic/363 generic/758
ext4/ext3: 579 tests, 1 failures, 149 skipped, 4646 seconds
  Failures: ext4/028
ext4/encrypt: 562 tests, 175 skipped, 3795 seconds
ext4/nojournal: 579 tests, 127 skipped, 4038 seconds
ext4/ext3conv: 584 tests, 57 skipped, 5044 seconds
ext4/adv: 580 tests, 2 failures, 63 skipped, 4933 seconds
  Failures: generic/757 generic/764
ext4/dioread_nolock: 585 tests, 55 skipped, 5135 seconds
ext4/data_journal: 580 tests, 1 failures, 135 skipped, 5776 seconds
  Failures: generic/127
ext4/bigalloc_4k: 558 tests, 58 skipped, 4870 seconds
ext4/bigalloc_1k: 559 tests, 2 failures, 69 skipped, 5141 seconds
  Failures: generic/363 generic/758
ext4/dax: 571 tests, 1 failures, 160 skipped, 3196 seconds
  Failures: generic/363
xfs/4k: 1181 tests, 2 failures, 173 skipped, 12295 seconds
  Failures: xfs/273 xfs/820
xfs/1k: 1181 tests, 2 failures, 173 skipped, 12407 seconds
  Failures: xfs/273 xfs/820
xfs/adv: 1181 tests, 4 failures, 173 skipped, 12416 seconds
  Failures: xfs/273 xfs/820
  Flaky: generic/234: 20% (1/5)   generic/370: 20% (1/5)
xfs/quota: 1181 tests, 3 failures, 174 skipped, 11305 seconds
  Failures: xfs/273 xfs/820
  Flaky: generic/370: 60% (3/5)
xfs/quota_1k: 1181 tests, 4 failures, 177 skipped, 21927 seconds
  Failures: xfs/273 xfs/629 xfs/820
  Flaky: generic/627: 60% (3/5)
xfs/dirblock_8k: 1181 tests, 2 failures, 175 skipped, 15638 seconds
  Failures: xfs/273 xfs/820
xfs/realtime: 1180 tests, 4 failures, 531 skipped, 9820 seconds
  Failures: generic/455 xfs/013 xfs/273 xfs/609
xfs/realtime_28k_logdev: 1216 tests, 41 failures, 1 errors, 613 skipped, 12164 seconds
  Failures: generic/757 xfs/253 xfs/273 xfs/598 xfs/629 xfs/630 xfs/631 
    xfs/632
  Flaky: xfs/609: 20% (1/5)
  Errors: xfs/137
xfs/realtime_logdev: 1200 tests, 21 failures, 1 errors, 586 skipped, 11650 seconds
  Failures: generic/757 xfs/253 xfs/273 xfs/598
  Flaky: xfs/609: 20% (1/5)
  Errors: xfs/137
xfs/logdev: 1209 tests, 31 failures, 1 errors, 245 skipped, 15002 seconds
  Failures: generic/757 xfs/234 xfs/253 xfs/273 xfs/598 xfs/820
  Flaky: generic/234: 20% (1/5)
  Errors: xfs/137
xfs/dax: 1191 tests, 7 failures, 344 skipped, 8396 seconds
  Failures: xfs/550 xfs/551 xfs/552 xfs/629 xfs/632 xfs/820
  Flaky: generic/363: 40% (2/5)
Totals: 20139 tests, 4526 skipped, 264 failures, 3 errors, 192766s

FSTESTIMG: gce-xfstests/xfstests-amd64-202504110828
FSTESTPRJ: gce-xfstests
FSTESTVER: blktests 236edfd (Tue, 18 Mar 2025 12:56:26 +0900)
FSTESTVER: fio  fio-3.39 (Tue, 18 Feb 2025 08:36:57 -0700)
FSTESTVER: fsverity v1.6-2-gee7d74d (Mon, 17 Feb 2025 11:41:58 -0800)
FSTESTVER: ima-evm-utils v1.5 (Mon, 6 Mar 2023 07:40:07 -0500)
FSTESTVER: libaio   libaio-0.3.108-82-gb8eadc9 (Thu, 2 Jun 2022 13:33:11 +0200)
FSTESTVER: ltp  20250130-195-ge2bbba0c1 (Fri, 11 Apr 2025 18:06:15 +0800)
FSTESTVER: quota  v4.05-69-g68952f1 (Mon, 7 Oct 2024 15:45:56 -0400)
FSTESTVER: util-linux v2.41 (Tue, 18 Mar 2025 13:50:51 +0100)
FSTESTVER: xfsprogs v6.13.0-2-gf0d16c9e (Tue, 1 Apr 2025 20:23:42 -0400)
FSTESTVER: xfstests-bld 42bcd9aa (Wed, 9 Apr 2025 07:51:57 -0400)
FSTESTVER: xfstests v2025.03.30-11-g344015670 (Mon, 31 Mar 2025 13:50:06 -0400)
FSTESTVER: zz_build-distro bookworm
FSTESTSET: -g auto
FSTESTOPT: aex

RE: 回复: 回复: 回复: 回复: HFS/HFS+ maintainership action items

[Viacheslav Dubeyko]

On Sat, 2025-05-03 at 01:39 -0400, Theodore Ts'o wrote:
> On Fri, May 02, 2025 at 07:14:26PM +0000, Viacheslav Dubeyko wrote:
> > On Thu, 2025-05-01 at 23:01 -0400, Theodore Ts'o wrote:
> > > Hey, in case it would be helpfui, I've added hfs support to the
> > > kvm-xfstests/gce-xfstests[1] test appliance.  Following the
> > > instructions at [2], you can now run "kvm-xfstests -c hfs -g auto" to
> > > run all of the tests in the auto group.  If you want to replicate the
> > > failure in generic/001, you could run "kvm-fstests -c hfs generic/001".
> > > 
> > 
> > Yes, it is really helpful! Sounds great! Let me try this framework for HFS/HFS+.
> > Thanks a lot.
> 
> FYI, I'm using the hfsprogs from Debian, which at the moment only
> supports HFS+.  The prebuilt test appliance for {kvm,gce}-xfstests are
> based on Debian Stable (Bookworm), but I am building test appliances
> using Debian Testing (Trixie).  However, for the purposes of hfsprogs,
> both Debian Bookwrm and Trixie are based on the 540.1 version of
> hfsprogs.
> 
> But there are plenty of bugs to fix until we can manage to get a
> version of hfsprogs that supports HFS --- also I'd argue that for many
> users support of HFS+ is probably more useful.
> 

Yeah, HFS+ is more important. And, yes, we need to manage a lot of bugs yet.

> If you find some test failures which are more about test bugs than
> kernel bug, so we can add them to exclude files.  For example, in
> /root/fs/ext4/exclude I have things like:
> 
> // generic/04[456] tests how truncate and delayed allocation works
> // ext4 uses the data=ordered to avoid exposing stale data, and
> // so it uses a different mechanism than xfs.  So these tests will fail
> generic/044
> generic/045
> generic/046
> 

Yes, makes sense. Let us identify such cases at first.

> Since I aso test LTS kernels, and sometimes it's not practcal to
> backport fixes to older kernels we can also do versioned excludes.
> For example, I have in /root/fs/global_exclude entries like:
> 
> #if LINUX_VERSION_CODE < KERNEL_VERSION(6,6,30)
> // This test failure is fixed by commit 631426ba1d45q ("mm/madvise:
> // make MADV_POPULATE_(READ|WRITE) handle VM_FAULT_RETRY properly"),
> // which first landed in v6.9, and was backported to 6.6.30 as commit
> // 631426ba1d45.  Unfortunately, it's too involved to backport it and its
> // dependencies to the 6.1 or earlier LTS kernels
> generic/743
> #endif
> 
> Finally, I have things set up to automatically run tests when a branch
> on a git tree that I'm watching changes.  For exmaple:
> 
> gce-xfstests ltm -c ext4/all,xfs/all,btrfs/all,f2fs/all -g auto --repo https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next   --watch fs-next
> 
> gce-xfstests ltm -c ext4/all,xfs/all -g auto --repo stable-rc.git --watch linux-6.12.y
> 
> gce-xfstests ltm -c ext4/all,xfs/all -g auto --repo stable-rc.git --watch linux-6.6.y
> 
> If it's helpful, I can set up watchers for hfs and send them to you or
> some mailing list once the number of failures are reduced toa
> manageable number.
> 

Sounds great! But we definitely have to reduce the number of bugs to manageable
level at first. :)

Thanks,
Slava.