Re: [Bugme-new] [Bug 14021] New: hfsplus caused data loss

[Brad Boyer <flar@allandria.com>]

On Thu, Aug 20, 2009 at 03:02:42PM -0700, Andrew Morton wrote:
> 
> (switched to email.  Please respond via emailed reply-to-all, not via the
> bugzilla web interface).
> 
> On Thu, 20 Aug 2009 02:17:21 GMT
> bugzilla-daemon@bugzilla.kernel.org wrote:
> 
> > http://bugzilla.kernel.org/show_bug.cgi?id=14021
> > 
> >            Summary: hfsplus caused data loss
> >            Product: File System
> >            Version: 2.5
> >     Kernel Version: 2.6.31-rc5
> >           Platform: All
> >         OS/Version: Linux
> >               Tree: Mainline
> >             Status: NEW
> >           Severity: normal
> >           Priority: P1
> >          Component: HFS/HFSPLUS
> >         AssignedTo: zippel@linux-m68k.org
> >         ReportedBy: rbrito@ime.usp.br
> >         Regression: Yes
> > 
> > 
> > Hi.
> > 
> > I am trying to package a new version of Apple's own fsck/mkfs for HFS+
> > filesystems so that they can be used in Linux as a way to transfer data among
> > computers, but I had quite a surprise when I was using the hfsplus module.
> > 
> > I just created a 100MB file with nulls (dd if=/dev/null ...) and I created an
> > HFS+ filesystem on it.
> > 
> > Then, I loop mounted it and tried to use it a little bit (in particular,
> > applying patches with quilt on a source tree). The commands spit some errors
> > about not being able to create links (I never had that problem before) and the
> > directory where I was became empty!
> > 
> > Furthermore, here is a quite, quite strange directory listing:
> > 
> > ,----
> > | rbrito@chagas:/media/usb7$ ls -lAF
> > | ls: hfsprogs_332.14-7.diff.gz: No such file or directory
> > | ls: hfsprogs_332.14-7.dsc: No such file or directory
> > | ls: hfsprogs_332.14.orig.tar.gz: No such file or directory
> > | ls: hfsprogs_332.18-1.diff.gz: No such file or directory
> > | ls: hfsprogs_332.18-1.dsc: No such file or directory
> > | ls: hfsprogs_332.18-1_amd64.changes: No such file or directory
> > | ls: hfsprogs_332.18-1_amd64.deb: No such file or directory
> > | ls: hfsprogs_332.18.orig.tar.gz: No such file or directory
> > | total 1636
> > | drwxr-xr-x 1 rbrito rbrito     39 Aug 17 08:13 hfsprogs-332.14/
> > | drwxr-xr-x 1 rbrito rbrito     39 Aug 17 08:13 hfsprogs-332.14/
> > | drwxr-xr-x 1 rbrito rbrito     45 Aug 17 15:30 hfsprogs-332.18/
> > | -rw-r--r-- 1 rbrito rbrito  35609 Aug 17 08:13 hfsprogs_332.14-7.diff.gz
> > | -rw-r--r-- 1 rbrito rbrito   1193 Aug 17 08:13 hfsprogs_332.14-7.dsc
> > | -rw-r--r-- 1 rbrito rbrito 714035 Aug 17 08:13 hfsprogs_332.14.orig.tar.gz
> > | -rw-r--r-- 1 rbrito rbrito  35342 Aug 17 15:26 hfsprogs_332.18-1.diff.gz
> > | -rw-r--r-- 1 rbrito rbrito    954 Aug 17 15:26 hfsprogs_332.18-1.dsc
> > | -rw-r--r-- 1 rbrito rbrito   2148 Aug 17 15:26
> > hfsprogs_332.18-1_amd64.changes
> > | -rw-r--r-- 1 rbrito rbrito 135398 Aug 17 15:26 hfsprogs_332.18-1_amd64.deb
> > | -rw-r--r-- 1 rbrito rbrito 732449 Aug 17 08:35 hfsprogs_332.18.orig.tar.gz
> > | rbrito@chagas:/media/usb7$
> > `----
> > 
> > I am using kernel 2.6.31-rc5-1rb.pre6 (that is, rc5 with git updates up to one
> > or two days before rc6).
> > 
> > There are no messages in the dmesg, besides these:
> > 
> > ,----
> > | [30991.501804] loop: module loaded
> > | [30991.513337] hfs: create hidden dir...
> > | [39897.867830] hfs: create hidden dir...
> > | [39960.061622] hfs: splitting index node...
> > `----
> > 
> > No stack traces, no nothing. Oh, I still have the disk image that I created, if
> > it is of any interest.
> > 
> > Please let me know if I can provide any further information.
> > 
> 
> Gee.  Nobody really does much maintenance work on hfs/hfsplus any more.
> I cc'ed the ppc guys as I expect that most HFS users are over there.
> 
> It seems like a pretty gross failure - others should be hitting it.
> 
> I wonder if it's a weird interaction with the loop driver.

I can explain the ls output in a very generic sense. This is what you
get if the lookup fails on a name returned by a readdir. I suspect
that creating hard links is failing in some way. In particular, the
"hidden dir" mentioned in the log is used to save the real files for
hard links.

As a quick explanation, HFS+ doesn't have a real concept of hard
links. Apple hacked it in after the fact. The way it works is that
there is a special hidden directory, and the real catalog entries
that match the real names are just references to the hidden file.
Both HFS and HFS+ have no separate notion of directory entries and
inode data. The metadata is stored in records keyed by file name.

Hard links were added after I stopped working on the HFS+ code, so
I don't know it in detail. I know it works for reading hard links,
but I never actually tried creating more links.

	Brad Boyer
	flar@allandria.com