hfsplus journalling support revisited, netgear, and fsck.hfsplus's observation.

hfsplus journalling support revisited, netgear, and fsck.hfsplus's observation.

[Hin-Tak Leung]

After a whole year, I am revisiting the HFS+ journalling work from last summer - rebasing it to 3.4.4 (current-ish production-use kernel), a couple of patches from Matthew Garrett which clean up all the ifdefs CONFIG_HFSPLUS_JOURNAL and makes the code far more readable; and actually have a disk I don't mind trashing, formatted with two HFS+ journalled partition (one case-sensitive, one not) to try writing, deleting. At a minimum, I am hoping that normal usage should work: i.e. partitions which are unmounted cleanly should pass 100% for fsck.hfsplus (port of apple's official diskdev_cmds). We are not yet talking about unplugging a disk in the middle of a write/sync yet. The result is rather surprising:

- I found that delete/unlink from linux almost always results in fsck.hfsplus complaining about unused nodes not being zero'ed, and needing minor repair. So this is somewhat expected - the typical linux design for performance, and not doing more than necessary; but this also indicates a somewhat significant difference in behavior with HFS+ under Mac OS X - Mac OS X probably zero's deleted/unlinked data nodes, maybe for security reasons. Should this be changed in the linux driver for hfsplus? Just to make fsck.hfsplus happy?

- I also found that deleting/unlinking Mac-generated files from linux results in complaints about inconsistent counts of extended attributes and catalog files. This is again not surprising - some of Mac OS X's file attributes have no linux equivalents, and are stored elsewhere anyway. But one might want to try to look up and clean up those associated extended attributes. This is a bug.

Neither of these are specific to journalling support - just problems with hfsplus that have always been there for years. The good news is that read/write to journalled HFS+ from linux is okay - just do not try to delete, and especially files which were not created by linux in the first place. (i.e. write from Mac and delete/move in linux is bad).

I also just found that since last summer, Netgear had released a few updated tar balls for the GPL part of their network storage appliances (from which the HFS+ journalling support came from). I downloaded the new tar balls but haven't looked at what changed yet, and will do so in due course.

Perhaps this is a question for the file system maintainer: how strict is one in enforcing the 80-character line length and no-multi-line-string warning? there are a lot of long debug messages in the netgear code.
 

Re: hfsplus journalling support revisited, netgear, and fsck.hfsplus's observation.

[Vyacheslav Dubeyko]

Hi,

On Jul 20, 2012, at 3:56 AM, Hin-Tak Leung wrote:

> After a whole year, I am revisiting the HFS+ journalling work from last summer - rebasing it to 3.4.4 (current-ish production-use kernel), a couple of patches from Matthew Garrett which clean up all the ifdefs CONFIG_HFSPLUS_JOURNAL and makes the code far more readable; and actually have a disk I don't mind trashing, formatted with two HFS+ journalled partition (one case-sensitive, one not) to try writing, deleting. At a minimum, I am hoping that normal usage should work: i.e. partitions which are unmounted cleanly should pass 100% for fsck.hfsplus (port of apple's official diskdev_cmds). We are not yet talking about unplugging a disk in the middle of a write/sync yet. The result is rather surprising:
> 
> - I found that delete/unlink from linux almost always results in fsck.hfsplus complaining about unused nodes not being zero'ed, and needing minor repair. So this is somewhat expected - the typical linux design for performance, and not doing more than necessary; but this also indicates a somewhat significant difference in behavior with HFS+ under Mac OS X - Mac OS X probably zero's deleted/unlinked data nodes, maybe for security reasons. Should this be changed in the linux driver for hfsplus? Just to make fsck.hfsplus happy?
> 
> - I also found that deleting/unlinking Mac-generated files from linux results in complaints about inconsistent counts of extended attributes and catalog files. This is again not surprising - some of Mac OS X's file attributes have no linux equivalents, and are stored elsewhere anyway. But one might want to try to look up and clean up those associated extended attributes. This is a bug.

As I can understand, you report about found bugs. Am I correct? Could you please describe these bugs with reproduction path and error output of fsck.hfsplus?

> Neither of these are specific to journalling support - just problems with hfsplus that have always been there for years. The good news is that read/write to journalled HFS+ from linux is okay - just do not try to delete, and especially files which were not created by linux in the first place. (i.e. write from Mac and delete/move in linux is bad).
> 
> I also just found that since last summer, Netgear had released a few updated tar balls for the GPL part of their network storage appliances (from which the HFS+ journalling support came from). I downloaded the new tar balls but haven't looked at what changed yet, and will do so in due course.
> 
> Perhaps this is a question for the file system maintainer: how strict is one in enforcing the 80-character line length and no-multi-line-string warning? there are a lot of long debug messages in the netgear code.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

With the best regards,
Vyacheslav Dubeyko.

Re: hfsplus journalling support revisited, netgear, and fsck.hfsplus's observation.

[Hin-Tak Leung]

--- On Sat, 21/7/12, Vyacheslav Dubeyko <slava@dubeyko.com> wrote:

> Hi,
> 
> On Jul 20, 2012, at 3:56 AM, Hin-Tak Leung wrote:
> 
<snipped>
> > 
> > - I found that delete/unlink from linux almost always
> results in fsck.hfsplus complaining about unused nodes not
> being zero'ed, and needing minor repair. So this is somewhat
> expected - the typical linux design for performance, and not
> doing more than necessary; but this also indicates a
> somewhat significant difference in behavior with HFS+ under
> Mac OS X - Mac OS X probably zero's deleted/unlinked data
> nodes, maybe for security reasons. Should this be changed in
> the linux driver for hfsplus? Just to make fsck.hfsplus
> happy?
> > 
> > - I also found that deleting/unlinking Mac-generated
> files from linux results in complaints about inconsistent
> counts of extended attributes and catalog files. This is
> again not surprising - some of Mac OS X's file attributes
> have no linux equivalents, and are stored elsewhere anyway.
> But one might want to try to look up and clean up those
> associated extended attributes. This is a bug.
> 
> As I can understand, you report about found bugs. Am I
> correct? Could you please describe these bugs with
> reproduction path and error output of fsck.hfsplus?

<snipped>

Correct - two interesting bugs. I thought I had supplied enough details for others to try to reproduce? The first is simply about deleting files, the latter about deleting files with extended attributes. So, to reproduce:

1. Have a hfs+ volume (created under Mac OS X would be better). Make sure it passed fsck.hfsplus (on linux).
2. copy the system Fonts folder (I think it is under /Library/Fonts) to it from Mac OS X. Font files have extended attributes, but other system files might do too.
3. Try to delete some of those files from Linux.
4. umount, run fsck.hfsplus on the volume. One would see:
    
   Executing fsck_hfs (version 540.1-Linux).
   ...
** Checking extents overflow file.
   Unused node is not erased (node = 1)
** Checking catalog file.
   Unused node is not erased (node = 18)
   ...
** Checking extended attributes file.
   Incorrect number of extended attributes
   (It should be 13 instead of 6)
   ...
** Repairing volume.
	RepairAttributesCheckABT: No matching catalog record found for id=438
   ...
** Rechecking volume.
   ...
** The volume journalled was repaired successfully.

   
'Unused node is not erased' & 'Incorrect number of extended attributes' were essentially what I wrote earlier. BTW, I ran fsck.hfsplus with:

fsck.hfsplus -d -D 0x0033 -f 

- i.e. maximum amount of information, and have a look even if it appears to be clean. (there are -l, -y and -n switches to control whether fsck.hfsplus would actually fix anything found or leave it alone and just output info).

The font folder is only about a few hundred MB, so one should be able to experiment with a small pen drive and do byte-level comparisons to see what fsk.hfsplus is not happy about and what does it change - although I am sure looking at diskdev_cmds's source would also be useful. I just happen to want the font folder for other things I do.
 

Re: hfsplus journalling support revisited, netgear, and fsck.hfsplus's observation.

[Vyacheslav Dubeyko]

Hi,

Thank you. Your additional description is very helpful. I am going to investigate these issues.

With the best regards,
Vyacheslav Dubeyko.

On Jul 22, 2012, at 1:06 AM, Hin-Tak Leung wrote:

> --- On Sat, 21/7/12, Vyacheslav Dubeyko <slava@dubeyko.com> wrote:
> 
>> Hi,
>> 
>> On Jul 20, 2012, at 3:56 AM, Hin-Tak Leung wrote:
>> 
> <snipped>
>>> 
>>> - I found that delete/unlink from linux almost always
>> results in fsck.hfsplus complaining about unused nodes not
>> being zero'ed, and needing minor repair. So this is somewhat
>> expected - the typical linux design for performance, and not
>> doing more than necessary; but this also indicates a
>> somewhat significant difference in behavior with HFS+ under
>> Mac OS X - Mac OS X probably zero's deleted/unlinked data
>> nodes, maybe for security reasons. Should this be changed in
>> the linux driver for hfsplus? Just to make fsck.hfsplus
>> happy?
>>> 
>>> - I also found that deleting/unlinking Mac-generated
>> files from linux results in complaints about inconsistent
>> counts of extended attributes and catalog files. This is
>> again not surprising - some of Mac OS X's file attributes
>> have no linux equivalents, and are stored elsewhere anyway.
>> But one might want to try to look up and clean up those
>> associated extended attributes. This is a bug.
>> 
>> As I can understand, you report about found bugs. Am I
>> correct? Could you please describe these bugs with
>> reproduction path and error output of fsck.hfsplus?
> 
> <snipped>
> 
> Correct - two interesting bugs. I thought I had supplied enough details for others to try to reproduce? The first is simply about deleting files, the latter about deleting files with extended attributes. So, to reproduce:
> 
> 1. Have a hfs+ volume (created under Mac OS X would be better). Make sure it passed fsck.hfsplus (on linux).
> 2. copy the system Fonts folder (I think it is under /Library/Fonts) to it from Mac OS X. Font files have extended attributes, but other system files might do too.
> 3. Try to delete some of those files from Linux.
> 4. umount, run fsck.hfsplus on the volume. One would see:
> 
>   Executing fsck_hfs (version 540.1-Linux).
>   ...
> ** Checking extents overflow file.
>   Unused node is not erased (node = 1)
> ** Checking catalog file.
>   Unused node is not erased (node = 18)
>   ...
> ** Checking extended attributes file.
>   Incorrect number of extended attributes
>   (It should be 13 instead of 6)
>   ...
> ** Repairing volume.
> 	RepairAttributesCheckABT: No matching catalog record found for id=438
>   ...
> ** Rechecking volume.
>   ...
> ** The volume journalled was repaired successfully.
> 
> 
> 'Unused node is not erased' & 'Incorrect number of extended attributes' were essentially what I wrote earlier. BTW, I ran fsck.hfsplus with:
> 
> fsck.hfsplus -d -D 0x0033 -f 
> 
> - i.e. maximum amount of information, and have a look even if it appears to be clean. (there are -l, -y and -n switches to control whether fsck.hfsplus would actually fix anything found or leave it alone and just output info).
> 
> The font folder is only about a few hundred MB, so one should be able to experiment with a small pen drive and do byte-level comparisons to see what fsk.hfsplus is not happy about and what does it change - although I am sure looking at diskdev_cmds's source would also be useful. I just happen to want the font folder for other things I do.
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: hfsplus journalling support revisited, netgear, and fsck.hfsplus's observation.

[Vyacheslav Dubeyko]

Hi,

On Jul 22, 2012, at 1:06 AM, Hin-Tak Leung wrote:

> 
> Correct - two interesting bugs. I thought I had supplied enough details for others to try to reproduce? The first is simply about deleting files, the latter about deleting files with extended attributes. So, to reproduce:
> 
> 1. Have a hfs+ volume (created under Mac OS X would be better). Make sure it passed fsck.hfsplus (on linux).
> 2. copy the system Fonts folder (I think it is under /Library/Fonts) to it from Mac OS X. Font files have extended attributes, but other system files might do too.
> 3. Try to delete some of those files from Linux.
> 4. umount, run fsck.hfsplus on the volume. One would see:
> 
>   Executing fsck_hfs (version 540.1-Linux).
>   ...
> ** Checking extents overflow file.
>   Unused node is not erased (node = 1)
> ** Checking catalog file.
>   Unused node is not erased (node = 18)
>   ...
> ** Checking extended attributes file.
>   Incorrect number of extended attributes
>   (It should be 13 instead of 6)
>   ...
> ** Repairing volume.
> 	RepairAttributesCheckABT: No matching catalog record found for id=438
>   ...
> ** Rechecking volume.
>   ...
> ** The volume journalled was repaired successfully.

Currently, I can't reproduce these bugs. I think that I have not fully correct reproduction path. So, I have some questions.
What version of Linux kernel do you use? Or maybe do you have hfsplus file system driver code from special branch?

I tried to reproduce these bugs on non-journaled hfs+ volume, firstly, but without success. As I can see you use journaled hfs+ volume. Maybe the forced mount of journaled hfs+ was the reason of these bugs? By the way do you use MBR or GPT partitioned disk?

> 
> 'Unused node is not erased' & 'Incorrect number of extended attributes' were essentially what I wrote earlier. BTW, I ran fsck.hfsplus with:
> 
> fsck.hfsplus -d -D 0x0033 -f 
> 
> - i.e. maximum amount of information, and have a look even if it appears to be clean. (there are -l, -y and -n switches to control whether fsck.hfsplus would actually fix anything found or leave it alone and just output info).
> 
> The font folder is only about a few hundred MB, so one should be able to experiment with a small pen drive and do byte-level comparisons to see what fsk.hfsplus is not happy about and what does it change - although I am sure looking at diskdev_cmds's source would also be useful. I just happen to want the font folder for other things I do.
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

With the best regards,
Vyacheslav Dubeyko.

Re: hfsplus journalling support revisited, netgear, and fsck.hfsplus's observation.

[Vyacheslav Dubeyko]

Hi Hin-Tak,

Sorry, I can see from the hfsplus code about what you are talking. You
are right. The situation is clear for me. It needs to work. :-)

Thank you for bug report.

With the best regards,
Vyacheslav Dubeyko.

On Mon, 2012-07-23 at 23:35 +0400, Vyacheslav Dubeyko wrote:
> Hi,
> 
> On Jul 22, 2012, at 1:06 AM, Hin-Tak Leung wrote:
> 
> > 
> > Correct - two interesting bugs. I thought I had supplied enough details for others to try to reproduce? The first is simply about deleting files, the latter about deleting files with extended attributes. So, to reproduce:
> > 
> > 1. Have a hfs+ volume (created under Mac OS X would be better). Make sure it passed fsck.hfsplus (on linux).
> > 2. copy the system Fonts folder (I think it is under /Library/Fonts) to it from Mac OS X. Font files have extended attributes, but other system files might do too.
> > 3. Try to delete some of those files from Linux.
> > 4. umount, run fsck.hfsplus on the volume. One would see:
> > 
> >   Executing fsck_hfs (version 540.1-Linux).
> >   ...
> > ** Checking extents overflow file.
> >   Unused node is not erased (node = 1)
> > ** Checking catalog file.
> >   Unused node is not erased (node = 18)
> >   ...
> > ** Checking extended attributes file.
> >   Incorrect number of extended attributes
> >   (It should be 13 instead of 6)
> >   ...
> > ** Repairing volume.
> > 	RepairAttributesCheckABT: No matching catalog record found for id=438
> >   ...
> > ** Rechecking volume.
> >   ...
> > ** The volume journalled was repaired successfully.
> 
> Currently, I can't reproduce these bugs. I think that I have not fully correct reproduction path. So, I have some questions.
> What version of Linux kernel do you use? Or maybe do you have hfsplus file system driver code from special branch?
> 
> I tried to reproduce these bugs on non-journaled hfs+ volume, firstly, but without success. As I can see you use journaled hfs+ volume. Maybe the forced mount of journaled hfs+ was the reason of these bugs? By the way do you use MBR or GPT partitioned disk?
> 
> > 
> > 'Unused node is not erased' & 'Incorrect number of extended attributes' were essentially what I wrote earlier. BTW, I ran fsck.hfsplus with:
> > 
> > fsck.hfsplus -d -D 0x0033 -f 
> > 
> > - i.e. maximum amount of information, and have a look even if it appears to be clean. (there are -l, -y and -n switches to control whether fsck.hfsplus would actually fix anything found or leave it alone and just output info).
> > 
> > The font folder is only about a few hundred MB, so one should be able to experiment with a small pen drive and do byte-level comparisons to see what fsk.hfsplus is not happy about and what does it change - although I am sure looking at diskdev_cmds's source would also be useful. I just happen to want the font folder for other things I do.
> > 
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> With the best regards,
> Vyacheslav Dubeyko.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: hfsplus journalling support revisited, netgear, and fsck.hfsplus's observation.

[Hin-Tak Leung]

Hi Vyacheslav,

--- On Wed, 25/7/12, Vyacheslav Dubeyko <slava@dubeyko.com> wrote:

> Hi Hin-Tak,
> 
> Sorry, I can see from the hfsplus code about what you are
> talking. You
> are right. The situation is clear for me. It needs to work.
> :-)
> 
> Thank you for bug report.
> 
> With the best regards,
> Vyacheslav Dubeyko.

I was going to confess that I might be using buggy code - it is rather a long history, but I am testing a port of netgear's HFS+ journalling change to current kernel. The long and short of the story is that Netgear contracted some company to add HFS+ support in 2010, which they released as a NAS product in 2011, and therefore posted a tar ball of modified 2.6.15 kernel source. This was worked on to port to 3.0-ish last summer, and I am moving it forward to 3.4.x-ish. Updating the journal shouldn't cause the kind of things I saw  - even if it is done wrongly, I was hoping. 

I am using 3.4.5/3.4.6, and indeed on special code branch and on journalled hfs+. It is an MBR disk.

I found that I can cause massive amount of problem by just untarring some big thing and then deleting the files one by one. (I did this with the netgear tar ball, which contains a kernel tree, and after comparing against a git checkout of the relevant version, delete identical files one by one from a list with a script).

i.e. try something similar to this:
  tar -jxpvf kernel-tarball.tar.bz2 > somewhere_else/file-list
  cat somewhere_else/file-list | perl -ne 'chomp; if (-f $_) {unlink $_;}'

On a separate but related note,

- there are 6 netgear tarballs, with 2 implementations of hfs+ journalling.  i.e. there is a slightly newer hfs+ journalling from them since we checked last summer. The newer code copied the whole of /jbd/ from ext3 and renamed everything from jbd to hfsplus_jbd, etc and added some small HFS+-related things. Rather ugly, but possibly do something similiar as jbd does for ext3.

- there is a commercial/free-for-personal-use implementation of hfs+ support from a company Paragon, released last summer, for 2.6.38-. Doesn't build with current kernel; it is almost like nvidia/ATI's X 3D drivers - small glue code to build a kernel module against a binary-only archive library. (also Paragon's HFS+ product for windows seems to make a lot of mac users unhappy for corrupting data, but I might be doing some selective reading there :-))

- I have been looking at some random HFS+ disk images I have as well. One - the command line tools from xcode on apple's web site - is interesting in that the HFS+ volume header says it is over 400MB, but the file itself is only 180MB so obviously there is no secondary volume header (which is located at the end of volume). Or at least, not at the expected location. But Mac OS X does not have problem mounting it - there is only about 180MB worth of stuff inside.

I am thinking maybe for disk images, and also read-only mount, one should not need to check the secondary volume header, as this seems to be what Mac OS X is doing.

The interesting disk image is one of the command-line-tools from xcode on apple's web site (free registration). A few of them are compressed, which is at about 150MB, but one is 180MB, and shows what I wrote about, for those who wants to investigate.

Hin-Tak
 
> On Mon, 2012-07-23 at 23:35 +0400, Vyacheslav Dubeyko
> wrote:
> > Hi,
> > 
> > On Jul 22, 2012, at 1:06 AM, Hin-Tak Leung wrote:
> > 
> > > 
> > > Correct - two interesting bugs. I thought I had
> supplied enough details for others to try to reproduce? The
> first is simply about deleting files, the latter about
> deleting files with extended attributes. So, to reproduce:
> > > 
> > > 1. Have a hfs+ volume (created under Mac OS X
> would be better). Make sure it passed fsck.hfsplus (on
> linux).
> > > 2. copy the system Fonts folder (I think it is
> under /Library/Fonts) to it from Mac OS X. Font files have
> extended attributes, but other system files might do too.
> > > 3. Try to delete some of those files from Linux.
> > > 4. umount, run fsck.hfsplus on the volume. One
> would see:
> > > 
> > >   Executing fsck_hfs (version
> 540.1-Linux).
> > >   ...
> > > ** Checking extents overflow file.
> > >   Unused node is not erased (node =
> 1)
> > > ** Checking catalog file.
> > >   Unused node is not erased (node =
> 18)
> > >   ...
> > > ** Checking extended attributes file.
> > >   Incorrect number of extended
> attributes
> > >   (It should be 13 instead of 6)
> > >   ...
> > > ** Repairing volume.
> > >     RepairAttributesCheckABT: No
> matching catalog record found for id=438
> > >   ...
> > > ** Rechecking volume.
> > >   ...
> > > ** The volume journalled was repaired
> successfully.
> > 
> > Currently, I can't reproduce these bugs. I think that I
> have not fully correct reproduction path. So, I have some
> questions.
> > What version of Linux kernel do you use? Or maybe do
> you have hfsplus file system driver code from special
> branch?
> > 
> > I tried to reproduce these bugs on non-journaled hfs+
> volume, firstly, but without success. As I can see you use
> journaled hfs+ volume. Maybe the forced mount of journaled
> hfs+ was the reason of these bugs? By the way do you use MBR
> or GPT partitioned disk?
> > 
> > > 
> > > 'Unused node is not erased' & 'Incorrect
> number of extended attributes' were essentially what I wrote
> earlier. BTW, I ran fsck.hfsplus with:
> > > 
> > > fsck.hfsplus -d -D 0x0033 -f 
> > > 
> > > - i.e. maximum amount of information, and have a
> look even if it appears to be clean. (there are -l, -y and
> -n switches to control whether fsck.hfsplus would actually
> fix anything found or leave it alone and just output info).
> > > 
> > > The font folder is only about a few hundred MB, so
> one should be able to experiment with a small pen drive and
> do byte-level comparisons to see what fsk.hfsplus is not
> happy about and what does it change - although I am sure
> looking at diskdev_cmds's source would also be useful. I
> just happen to want the font folder for other things I do.
> > > 
> > > 
> > > --
> > > To unsubscribe from this list: send the line
> "unsubscribe linux-fsdevel" in
> > > the body of a message to majordomo@vger.kernel.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 
> > With the best regards,
> > Vyacheslav Dubeyko.
> > 
> > --
> > To unsubscribe from this list: send the line
> "unsubscribe linux-fsdevel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

off-spec-ness of HFS+ image from apple, hfsplus 2nd volume header, and mount options

[Hin-Tak Leung]

I think I found another small bug in the kernel's hfsplus support, or more properly, a difference between how the linux kernel deal with HFS+ images versus how Mac OS X does it.

Some of the necessary/boring details are in the bug report https://bugzilla.redhat.com/show_bug.cgi?id=845419 (which isn't about this bug, but some other issues about mount options).

According to Apple's TN1150 technote (the HFS+ specification), the primary volume header is at 1024 bytes into an image, while a duplicate back-up version of it, the secondary volume header, is at 1024 bytes counting from the end of the image/storage. (or for 512-sector devices, the 3rd sector from the beginning and the 2nd sector from the last).

Apple's xcode images (I just happen to have those in my hard disc, as I am looking at any/every HFS+ disk images I might have), and all the apple tools images I have, have an anomaly: the secondary volume header is located one logical block (a logical block being 4k in this case from the HFS+ header) from the end.

Indeed, apple's own fsck.hfsplus itself complains about the xcode image not of the right size, if you set up the loopback device as I wrote in the bug report. OTOH, since Apple developers all over the world download the xcode images and access them as virtual disk images on their computers on a daily basis, and Apple can change what is considered "correct behavior" about HFS+ without notice, this anomaly probably should be supported.

The wording of TN1150 is also a bit ambiguous about the 2nd volume header - it says it is for disk repair utilities only, and may not be where it said it is.

So I am wondering whether the linux kernel should as it currently does, enforce the 2nd volume header being at 1024 bytes from the end and agree with the primary volume header, and refuse to mount, as correct. It also seems unnecessary for read-only mounts, or image-loop-back mounts. I have two suggestions:

- for read-only/look-back mounts, allows the 2nd volume header to not agree with the primary, with a warning.

- or, for read-only/look-back mounts, allows the 2nd volume header to locate one logical block from the end rather than 2 x 512-sectors from the end, if the logical block according to the HFS+ header is larger than or different from 512.

Re: off-spec-ness of HFS+ image from apple, hfsplus 2nd volume header, and mount options

[Vyacheslav Dubeyko]

Hi Hin-Tak,

It is very strange. My experience with HFS+ volumes is different. I
worked with HFS+ volumes with different block sizes and different
volumes' sizes but I haven't such anomaly. Maybe, I don't know about
last changes or some specific use-cases.

Could you share more details about your volumes (size in bytes, size in
blocks, block size, offset in bytes of second volume header and so on)?

What utilities did you use for HFS+ volumes creation?

With the best regards,
Vyacheslav Dubeyko.

On Fri, 2012-08-03 at 05:21 +0100, Hin-Tak Leung wrote:
> I think I found another small bug in the kernel's hfsplus support, or more properly, a difference between how the linux kernel deal with HFS+ images versus how Mac OS X does it.
> 
> Some of the necessary/boring details are in the bug report https://bugzilla.redhat.com/show_bug.cgi?id=845419 (which isn't about this bug, but some other issues about mount options).
> 
> According to Apple's TN1150 technote (the HFS+ specification), the primary volume header is at 1024 bytes into an image, while a duplicate back-up version of it, the secondary volume header, is at 1024 bytes counting from the end of the image/storage. (or for 512-sector devices, the 3rd sector from the beginning and the 2nd sector from the last).
> 
> Apple's xcode images (I just happen to have those in my hard disc, as I am looking at any/every HFS+ disk images I might have), and all the apple tools images I have, have an anomaly: the secondary volume header is located one logical block (a logical block being 4k in this case from the HFS+ header) from the end.
> 
> Indeed, apple's own fsck.hfsplus itself complains about the xcode image not of the right size, if you set up the loopback device as I wrote in the bug report. OTOH, since Apple developers all over the world download the xcode images and access them as virtual disk images on their computers on a daily basis, and Apple can change what is considered "correct behavior" about HFS+ without notice, this anomaly probably should be supported.
> 
> The wording of TN1150 is also a bit ambiguous about the 2nd volume header - it says it is for disk repair utilities only, and may not be where it said it is.
> 
> So I am wondering whether the linux kernel should as it currently does, enforce the 2nd volume header being at 1024 bytes from the end and agree with the primary volume header, and refuse to mount, as correct. It also seems unnecessary for read-only mounts, or image-loop-back mounts. I have two suggestions:
> 
> - for read-only/look-back mounts, allows the 2nd volume header to not agree with the primary, with a warning.
> 
> - or, for read-only/look-back mounts, allows the 2nd volume header to locate one logical block from the end rather than 2 x 512-sectors from the end, if the logical block according to the HFS+ header is larger than or different from 512.
> 

Re: off-spec-ness of HFS+ image from apple, hfsplus 2nd volume header, and mount options

[Hin-Tak Leung]

--- On Fri, 3/8/12, Vyacheslav Dubeyko <slava@dubeyko.com> wrote:

> Hi Hin-Tak,
> 
> It is very strange. My experience with HFS+ volumes is
> different. I
> worked with HFS+ volumes with different block sizes and
> different
> volumes' sizes but I haven't such anomaly. Maybe, I don't
> know about
> last changes or some specific use-cases.
> 
> Could you share more details about your volumes (size in
> bytes, size in
> blocks, block size, offset in bytes of second volume header
> and so on)?
> 
> What utilities did you use for HFS+ volumes creation?
> 
> With the best regards,
> Vyacheslav Dubeyko.

I did not make those volumes - they were downloaded them from Apple's developer web site (http://developer.apple.com/xcode/ - sorry, free registration required). Some of them have additional compression built-in - there are 3rd-party tools on linux for decompressing them - but these 4 appear to be plain disk images, of sorts (I have a small random selection, from helping Mac-user friends to set up their dev environment from time to time).

cltools_lion_latemarch12.dmg
command_line_tools_for_xcode.dmg
xcode_4.3.1_for_lion.dmg
xcode_4.3.3_for_lion.dmg

These have an Apple partition map, an empty ATA driver partition and and HFS+ partition. The HFS+ partition's header says it uses block size 4k, and the 2nd volume header is located at 4k from the end of the HFS+ partition rather than 1024 byte from the end. The two command_line_tools* also have an extra issue - its HFS+ header says it is about 400MB in total, half of that free, but the actual size is only 180MB and the secondary volume header is located on the last 4k block (i.e. freespace is not included), and not 1024 byte from the end.

These are the "official" Apple development tool distributions, and quite obviously millions of Apple-centric developers are using them without problem. when they are slightly off-spec, I think one should interpret this as Apple extending/changing/adjusting the HFS+ format, relaxing/changing some of the constraints in the case of disk images.

The "sizelimit" I wrote in the redhat bug report is modified to fool the linux kernel into finding the 2nd volume header in its "expected" place to loop-back mount that image i.e. the true end of the HFS+ volume is 3072 bytes further, from the HFS+ header. (the redhat bug report is about sizelimit not working from mount -o, but need to be specified separately on losetup).

> On Fri, 2012-08-03 at 05:21 +0100, Hin-Tak Leung wrote:
> > I think I found another small bug in the kernel's
> hfsplus support, or more properly, a difference between how
> the linux kernel deal with HFS+ images versus how Mac OS X
> does it.
> > 
> > Some of the necessary/boring details are in the bug
> report https://bugzilla.redhat.com/show_bug.cgi?id=845419
> (which isn't about this bug, but some other issues about
> mount options).
> > 
> > According to Apple's TN1150 technote (the HFS+
> specification), the primary volume header is at 1024 bytes
> into an image, while a duplicate back-up version of it, the
> secondary volume header, is at 1024 bytes counting from the
> end of the image/storage. (or for 512-sector devices, the
> 3rd sector from the beginning and the 2nd sector from the
> last).
> > 
> > Apple's xcode images (I just happen to have those in my
> hard disc, as I am looking at any/every HFS+ disk images I
> might have), and all the apple tools images I have, have an
> anomaly: the secondary volume header is located one logical
> block (a logical block being 4k in this case from the HFS+
> header) from the end.
> > 
> > Indeed, apple's own fsck.hfsplus itself complains about
> the xcode image not of the right size, if you set up the
> loopback device as I wrote in the bug report. OTOH, since
> Apple developers all over the world download the xcode
> images and access them as virtual disk images on their
> computers on a daily basis, and Apple can change what is
> considered "correct behavior" about HFS+ without notice,
> this anomaly probably should be supported.
> > 
> > The wording of TN1150 is also a bit ambiguous about the
> 2nd volume header - it says it is for disk repair utilities
> only, and may not be where it said it is.
> > 
> > So I am wondering whether the linux kernel should as it
> currently does, enforce the 2nd volume header being at 1024
> bytes from the end and agree with the primary volume header,
> and refuse to mount, as correct. It also seems unnecessary
> for read-only mounts, or image-loop-back mounts. I have two
> suggestions:
> > 
> > - for read-only/look-back mounts, allows the 2nd volume
> header to not agree with the primary, with a warning.
> > 
> > - or, for read-only/look-back mounts, allows the 2nd
> volume header to locate one logical block from the end
> rather than 2 x 512-sectors from the end, if the logical
> block according to the HFS+ header is larger than or
> different from 512.
> > 
> 
> 
> 

Re: off-spec-ness of HFS+ image from apple, hfsplus 2nd volume header, and mount options

[Vyacheslav Dubeyko]

Hi,
 
On Fri, 2012-08-03 at 09:01 +0100, Hin-Tak Leung wrote:

> I did not make those volumes - they were downloaded them from Apple's developer web site (http://developer.apple.com/xcode/ - sorry, free registration required). Some of them have additional compression built-in - there are 3rd-party tools on linux for decompressing them - but these 4 appear to be plain disk images, of sorts (I have a small random selection, from helping Mac-user friends to set up their dev environment from time to time).
> 

Could you share dump of the first and secondary volume headers from one
of these volumes, offset of secondary ones in bytes from volume begin
and output any utilities with size of the partition?

I think that from file system points of view all can be OK (without
anomaly). But partition map can have some different value of volume
size.

It is possible to talk about anomaly after real reproduction of the
problem on real hardware, from my point of view.

With the best regards,
Vyacheslav Dubeyko. 


> cltools_lion_latemarch12.dmg
> command_line_tools_for_xcode.dmg
> xcode_4.3.1_for_lion.dmg
> xcode_4.3.3_for_lion.dmg
> 
> These have an Apple partition map, an empty ATA driver partition and and HFS+ partition. The HFS+ partition's header says it uses block size 4k, and the 2nd volume header is located at 4k from the end of the HFS+ partition rather than 1024 byte from the end. The two command_line_tools* also have an extra issue - its HFS+ header says it is about 400MB in total, half of that free, but the actual size is only 180MB and the secondary volume header is located on the last 4k block (i.e. freespace is not included), and not 1024 byte from the end.
> 
> These are the "official" Apple development tool distributions, and quite obviously millions of Apple-centric developers are using them without problem. when they are slightly off-spec, I think one should interpret this as Apple extending/changing/adjusting the HFS+ format, relaxing/changing some of the constraints in the case of disk images.
> 
> The "sizelimit" I wrote in the redhat bug report is modified to fool the linux kernel into finding the 2nd volume header in its "expected" place to loop-back mount that image i.e. the true end of the HFS+ volume is 3072 bytes further, from the HFS+ header. (the redhat bug report is about sizelimit not working from mount -o, but need to be specified separately on losetup).
> 
> > On Fri, 2012-08-03 at 05:21 +0100, Hin-Tak Leung wrote:
> > > I think I found another small bug in the kernel's
> > hfsplus support, or more properly, a difference between how
> > the linux kernel deal with HFS+ images versus how Mac OS X
> > does it.
> > > 
> > > Some of the necessary/boring details are in the bug
> > report https://bugzilla.redhat.com/show_bug.cgi?id=845419
> > (which isn't about this bug, but some other issues about
> > mount options).
> > > 
> > > According to Apple's TN1150 technote (the HFS+
> > specification), the primary volume header is at 1024 bytes
> > into an image, while a duplicate back-up version of it, the
> > secondary volume header, is at 1024 bytes counting from the
> > end of the image/storage. (or for 512-sector devices, the
> > 3rd sector from the beginning and the 2nd sector from the
> > last).
> > > 
> > > Apple's xcode images (I just happen to have those in my
> > hard disc, as I am looking at any/every HFS+ disk images I
> > might have), and all the apple tools images I have, have an
> > anomaly: the secondary volume header is located one logical
> > block (a logical block being 4k in this case from the HFS+
> > header) from the end.
> > > 
> > > Indeed, apple's own fsck.hfsplus itself complains about
> > the xcode image not of the right size, if you set up the
> > loopback device as I wrote in the bug report. OTOH, since
> > Apple developers all over the world download the xcode
> > images and access them as virtual disk images on their
> > computers on a daily basis, and Apple can change what is
> > considered "correct behavior" about HFS+ without notice,
> > this anomaly probably should be supported.
> > > 
> > > The wording of TN1150 is also a bit ambiguous about the
> > 2nd volume header - it says it is for disk repair utilities
> > only, and may not be where it said it is.
> > > 
> > > So I am wondering whether the linux kernel should as it
> > currently does, enforce the 2nd volume header being at 1024
> > bytes from the end and agree with the primary volume header,
> > and refuse to mount, as correct. It also seems unnecessary
> > for read-only mounts, or image-loop-back mounts. I have two
> > suggestions:
> > > 
> > > - for read-only/look-back mounts, allows the 2nd volume
> > header to not agree with the primary, with a warning.
> > > 
> > > - or, for read-only/look-back mounts, allows the 2nd
> > volume header to locate one logical block from the end
> > rather than 2 x 512-sectors from the end, if the logical
> > block according to the HFS+ header is larger than or
> > different from 512.
> > > 
> > 
> > 
> > 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: off-spec-ness of HFS+ image from apple, hfsplus 2nd volume header, and mount options

[Hin-Tak Leung]

[-- Attachment #1: Type: text/plain, Size: 6739 bytes --]

--- On Fri, 3/8/12, Vyacheslav Dubeyko <slava@dubeyko.com> wrote:

> Hi,
>  
> On Fri, 2012-08-03 at 09:01 +0100, Hin-Tak Leung wrote:
> 
> > I did not make those volumes - they were downloaded
> them from Apple's developer web site (http://developer.apple.com/xcode/ - sorry, free
> registration required). Some of them have additional
> compression built-in - there are 3rd-party tools on linux
> for decompressing them - but these 4 appear to be plain disk
> images, of sorts (I have a small random selection, from
> helping Mac-user friends to set up their dev environment
> from time to time).
> > 
> 
> Could you share dump of the first and secondary volume
> headers from one
> of these volumes, offset of secondary ones in bytes from
> volume begin
> and output any utilities with size of the partition?
> 
> I think that from file system points of view all can be OK
> (without
> anomaly). But partition map can have some different value of
> volume
> size.

I know parition map can have different ideas about volume sizes. I was talking about the HFS+ volume header itself - located on the 3rd 512-byte, and *supposedly* located on the 2nd 512-byte from the "end" - the "end" being the size declared by multiplying the blocksize with blockcount in the HFS+ header, and counting from 1024 byte before the primary header. 

The smaller set of files also have blockcount way over the image size itself, and the 2nd header is located at the beginning of the last used 4k block.

Here they are:

dd if=xcode_4.3.3_for_lion.dmg of=/tmp/header1 count=3 skip=65
dd if=xcode_4.3.3_for_lion.dmg of=/tmp/header2 count=1 skip=3858241

("header1" includes the 2x512 bytes before the primary volume header, hence count=3).
The HFS+ header says 482273 blocks of 4 KiB, which means the 2nd header sould be located at 65 + 482273 * 8 - 2, i.e. at skip 3858247 instead of 3858241 as above.

> It is possible to talk about anomaly after real reproduction
> of the
> problem on real hardware, from my point of view.
> 
> With the best regards,
> Vyacheslav Dubeyko. 
> 
> 
> > cltools_lion_latemarch12.dmg
> > command_line_tools_for_xcode.dmg
> > xcode_4.3.1_for_lion.dmg
> > xcode_4.3.3_for_lion.dmg
> > 
> > These have an Apple partition map, an empty ATA driver
> partition and and HFS+ partition. The HFS+ partition's
> header says it uses block size 4k, and the 2nd volume header
> is located at 4k from the end of the HFS+ partition rather
> than 1024 byte from the end. The two command_line_tools*
> also have an extra issue - its HFS+ header says it is about
> 400MB in total, half of that free, but the actual size is
> only 180MB and the secondary volume header is located on the
> last 4k block (i.e. freespace is not included), and not 1024
> byte from the end.
> > 
> > These are the "official" Apple development tool
> distributions, and quite obviously millions of Apple-centric
> developers are using them without problem. when they are
> slightly off-spec, I think one should interpret this as
> Apple extending/changing/adjusting the HFS+ format,
> relaxing/changing some of the constraints in the case of
> disk images.
> > 
> > The "sizelimit" I wrote in the redhat bug report is
> modified to fool the linux kernel into finding the 2nd
> volume header in its "expected" place to loop-back mount
> that image i.e. the true end of the HFS+ volume is 3072
> bytes further, from the HFS+ header. (the redhat bug report
> is about sizelimit not working from mount -o, but need to be
> specified separately on losetup).
> > 
> > > On Fri, 2012-08-03 at 05:21 +0100, Hin-Tak Leung
> wrote:
> > > > I think I found another small bug in the
> kernel's
> > > hfsplus support, or more properly, a difference
> between how
> > > the linux kernel deal with HFS+ images versus how
> Mac OS X
> > > does it.
> > > > 
> > > > Some of the necessary/boring details are in
> the bug
> > > report https://bugzilla.redhat.com/show_bug.cgi?id=845419
> > > (which isn't about this bug, but some other issues
> about
> > > mount options).
> > > > 
> > > > According to Apple's TN1150 technote (the
> HFS+
> > > specification), the primary volume header is at
> 1024 bytes
> > > into an image, while a duplicate back-up version
> of it, the
> > > secondary volume header, is at 1024 bytes counting
> from the
> > > end of the image/storage. (or for 512-sector
> devices, the
> > > 3rd sector from the beginning and the 2nd sector
> from the
> > > last).
> > > > 
> > > > Apple's xcode images (I just happen to have
> those in my
> > > hard disc, as I am looking at any/every HFS+ disk
> images I
> > > might have), and all the apple tools images I
> have, have an
> > > anomaly: the secondary volume header is located
> one logical
> > > block (a logical block being 4k in this case from
> the HFS+
> > > header) from the end.
> > > > 
> > > > Indeed, apple's own fsck.hfsplus itself
> complains about
> > > the xcode image not of the right size, if you set
> up the
> > > loopback device as I wrote in the bug report.
> OTOH, since
> > > Apple developers all over the world download the
> xcode
> > > images and access them as virtual disk images on
> their
> > > computers on a daily basis, and Apple can change
> what is
> > > considered "correct behavior" about HFS+ without
> notice,
> > > this anomaly probably should be supported.
> > > > 
> > > > The wording of TN1150 is also a bit ambiguous
> about the
> > > 2nd volume header - it says it is for disk repair
> utilities
> > > only, and may not be where it said it is.
> > > > 
> > > > So I am wondering whether the linux kernel
> should as it
> > > currently does, enforce the 2nd volume header
> being at 1024
> > > bytes from the end and agree with the primary
> volume header,
> > > and refuse to mount, as correct. It also seems
> unnecessary
> > > for read-only mounts, or image-loop-back mounts. I
> have two
> > > suggestions:
> > > > 
> > > > - for read-only/look-back mounts, allows the
> 2nd volume
> > > header to not agree with the primary, with a
> warning.
> > > > 
> > > > - or, for read-only/look-back mounts, allows
> the 2nd
> > > volume header to locate one logical block from the
> end
> > > rather than 2 x 512-sectors from the end, if the
> logical
> > > block according to the HFS+ header is larger than
> or
> > > different from 512.
> > > > 
> > > 
> > > 
> > > 
> > --
> > To unsubscribe from this list: send the line
> "unsubscribe linux-fsdevel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
>

[-- Attachment #2: header1 --]
[-- Type: application/octet-stream, Size: 1536 bytes --]

[-- Attachment #3: header2 --]
[-- Type: application/octet-stream, Size: 512 bytes --]

Re: off-spec-ness of HFS+ image from apple, hfsplus 2nd volume header, and mount options

[Vyacheslav Dubeyko]

Hi Hin-Tak,

On Aug 3, 2012, at 12:40 PM, Hin-Tak Leung wrote:
> 
> I know parition map can have different ideas about volume sizes. I was talking about the HFS+ volume header itself - located on the 3rd 512-byte, and *supposedly* located on the 2nd 512-byte from the "end" - the "end" being the size declared by multiplying the blocksize with blockcount in the HFS+ header, and counting from 1024 byte before the primary header. 
> 
> The smaller set of files also have blockcount way over the image size itself, and the 2nd header is located at the beginning of the last used 4k block.
> 
> Here they are:
> 
> dd if=xcode_4.3.3_for_lion.dmg of=/tmp/header1 count=3 skip=65
> dd if=xcode_4.3.3_for_lion.dmg of=/tmp/header2 count=1 skip=3858241
> 
> ("header1" includes the 2x512 bytes before the primary volume header, hence count=3).
> The HFS+ header says 482273 blocks of 4 KiB, which means the 2nd header sould be located at 65 + 482273 * 8 - 2, i.e. at skip 3858247 instead of 3858241 as above.
> 

I made some experiments with creation of hfs+ volumes under Mac OS X (10.6.8) Snow Leopard. I can't detect such anomaly in the case of my version of Mac OS. For example, I created  partition with size of 2097155 sectors (sector is 512 bytes in size) and I can see second volume header in the 2097153 sector. It means that second volume header is located on the distance in 1024 bytes from partition's end. In volume header I can see volume size in 0x40000 blocks (block is 4096 bytes) or 2097152 sectors. The detected position of second volume header is coincided with Apple's technical specification.

Unfortunately, I haven't any Apple hardware with installed Mac OS X Lion. So, I think that maybe reported by you anomaly is take place under only Lion. Maybe, it was made as optimization efforts for SSD case.  Do you have opportunity to make similar experiments under Lion?

With the best regards,
Vyacheslav Dubeyko.

Re: off-spec-ness of HFS+ image from apple, hfsplus 2nd volume header, and mount options

[Vyacheslav Dubeyko]

Hi Hin-Tak,

On Aug 3, 2012, at 12:40 PM, Hin-Tak Leung wrote:
> 
> I know parition map can have different ideas about volume sizes. I was talking about the HFS+ volume header itself - located on the 3rd 512-byte, and *supposedly* located on the 2nd 512-byte from the "end" - the "end" being the size declared by multiplying the blocksize with blockcount in the HFS+ header, and counting from 1024 byte before the primary header. 
> 
> The smaller set of files also have blockcount way over the image size itself, and the 2nd header is located at the beginning of the last used 4k block.
> 
> Here they are:
> 
> dd if=xcode_4.3.3_for_lion.dmg of=/tmp/header1 count=3 skip=65
> dd if=xcode_4.3.3_for_lion.dmg of=/tmp/header2 count=1 skip=3858241
> 
> ("header1" includes the 2x512 bytes before the primary volume header, hence count=3).
> The HFS+ header says 482273 blocks of 4 KiB, which means the 2nd header sould be located at 65 + 482273 * 8 - 2, i.e. at skip 3858247 instead of 3858241 as above.
> 

I made some experiments with creation of hfs+ volumes under Mac OS X (10.6.8) Snow Leopard. I can't detect such anomaly in the case of my version of Mac OS. For example, I created  partition with size of 2097155 sectors (sector is 512 bytes in size) and I can see second volume header in the 2097153 sector. It means that second volume header is located on the distance in 1024 bytes from partition's end. In volume header I can see volume size in 0x40000 blocks (block is 4096 bytes) or 2097152 sectors. The detected position of second volume header is coincided with Apple's technical specification.

Unfortunately, I haven't any Apple hardware with installed Mac OS X Lion. So, I think that maybe reported by you anomaly is take place under only Lion. Maybe, it was made as optimization efforts for SSD case.  Do you have opportunity to make similar experiments under Lion?

With the best regards,
Vyacheslav Dubeyko.

Re: off-spec-ness of HFS+ image from apple, hfsplus 2nd volume header, and mount options

[Hin-Tak Leung]

--- On Sun, 5/8/12, Vyacheslav Dubeyko <slava@dubeyko.com> wrote:

> Hi Hin-Tak,
> 
> On Aug 3, 2012, at 12:40 PM, Hin-Tak Leung wrote:
> > 
> > I know parition map can have different ideas about
> volume sizes. I was talking about the HFS+ volume header
> itself - located on the 3rd 512-byte, and *supposedly*
> located on the 2nd 512-byte from the "end" - the "end" being
> the size declared by multiplying the blocksize with
> blockcount in the HFS+ header, and counting from 1024 byte
> before the primary header. 
> > 
> > The smaller set of files also have blockcount way over
> the image size itself, and the 2nd header is located at the
> beginning of the last used 4k block.
> > 
> > Here they are:
> > 
> > dd if=xcode_4.3.3_for_lion.dmg of=/tmp/header1 count=3
> skip=65
> > dd if=xcode_4.3.3_for_lion.dmg of=/tmp/header2 count=1
> skip=3858241
> > 
> > ("header1" includes the 2x512 bytes before the primary
> volume header, hence count=3).
> > The HFS+ header says 482273 blocks of 4 KiB, which
> means the 2nd header sould be located at 65 + 482273 * 8 -
> 2, i.e. at skip 3858247 instead of 3858241 as above.
> > 
> 
> I made some experiments with creation of hfs+ volumes under
> Mac OS X (10.6.8) Snow Leopard. I can't detect such anomaly
> in the case of my version of Mac OS. For example, I
> created  partition with size of 2097155 sectors (sector
> is 512 bytes in size) and I can see second volume header in
> the 2097153 sector. It means that second volume header is
> located on the distance in 1024 bytes from partition's end.
> In volume header I can see volume size in 0x40000 blocks
> (block is 4096 bytes) or 2097152 sectors. The detected
> position of second volume header is coincided with Apple's
> technical specification.
> 
> Unfortunately, I haven't any Apple hardware with installed
> Mac OS X Lion. So, I think that maybe reported by you
> anomaly is take place under only Lion. Maybe, it was made as
> optimization efforts for SSD case.  Do you have
> opportunity to make similar experiments under Lion?

Those images did mount and worked on Lion (as I mentioned I was helping a Mac-user friend to set up his dev environment in March). I can try downloading some of the earlier xcode images (or other dev material images) from Apple's to see if earlier Mac OS'es are supposed to be able to cope with that anomaly.

I should point out again that I did not make those images, nor know how they were made - they are "freely" available on apple's web site. They are the updates/installers of their standard dev tools (contain Apple's patched version of gcc, binutils, autoconf, etc) on Mac OS X, and Mac OS X [some version] obviously have no problem loop-back mounting them, even when they are off-spec.

It is free registration, and apple doesn't really try to bombard one with more info/offers, so everybody is free to take a look around, if interested.
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html