Serious problems with HFS+

Serious problems with HFS+

[Matt Mackall]

I've noticed a few problems with HFS+ support in recent kernels on
another user's machine running Ubuntu (Warty) running
2.6.8.1-3-powerpc. I'm not in a position to extensively test or fix
either of these problem because of the fs tools situation so I'm just
passing this on.

First, it reports inappropriate blocks to stat(2). It uses 4096 byte
blocks rather than 512 byte blocks which stat callers are expecting.
This seriously confuses du(1) (and me, for a bit). Looks like it may
be forgetting to set s_blocksize_bits.

Second, if an HFS+ filesystem mounted via Firewire or USB becomes
detached, the filesystem appears to continue working just fine. I can
find on the entire tree, despite memory pressure. I can even create
new files that continue to appear in directory listings! Writes to
such files succeed (they're async, of course) and the typical app is
none the wiser. It's only when apps attempt to read later that they
encounter problems. It turns out that various apps including scp
ignore IO errors on read and silently copy zero-filled files to the
destination. So I got this report as "why aren't the pictures I took
off my camera visible on my website?"

This is obviously a really nasty failure mode. At the very least, open
of new files should fail with -EIO. Preferably the fs should force a
read-only remount on IO errors. Given that the vast majority of HFS+
filesystems Linux is likely to be used with are on hotpluggable media,
I think this FS should be marked EXPERIMENTAL until such integrity
problems are addressed.

Having the whole directory tree seemingly pinned in memory is probably
something that wants addressing as well.

-- 
Mathematics is the supreme nostalgia of our time.

Re: Serious problems with HFS+

[Rogério Brito]

On Mar 13 2005, Matt Mackall wrote:
> I've noticed a few problems with HFS+ support in recent kernels on
> another user's machine running Ubuntu (Warty) running 2.6.8.1-3-powerpc.

I also saw some of the things that you reported with Debian proper (sarge).
Unfortunately, I haven't been able to use my HFS+ formatted system with
Linux for a while, I may do so, if necessary to fix the bugs.

> I'm not in a position to extensively test or fix either of these problem
> because of the fs tools situation so I'm just passing this on.

I, OTOH, can test fixes on the next weekend, since I have a full backup
scheduled for this week.

> First, it reports inappropriate blocks to stat(2). It uses 4096 byte
> blocks rather than 512 byte blocks which stat callers are expecting.
> This seriously confuses du(1) (and me, for a bit). Looks like it may
> be forgetting to set s_blocksize_bits.

I had not noticed what are the size of the blocks that HFS+ seems to use,
but indeed I saw both very confused du and ls outputs. I don't know what
may be the cause.

> Second, if an HFS+ filesystem mounted via Firewire or USB becomes
> detached, the filesystem appears to continue working just fine.

That's the only way that I am using a HFS+ fs (fia Firewire or USB), since
the drive in question is in an enclosure.

> I can find on the entire tree, despite memory pressure. I can even create
> new files that continue to appear in directory listings! Writes to
> such files succeed (they're async, of course) and the typical app is
> none the wiser. It's only when apps attempt to read later that they
> encounter problems. It turns out that various apps including scp
> ignore IO errors on read and silently copy zero-filled files to the
> destination. So I got this report as "why aren't the pictures I took
> off my camera visible on my website?"

I haven't seen this behaviour for lack of testing, but I will try to
reproduce it with an HFS+ fs on a file mounted via loopback.

> This is obviously a really nasty failure mode. At the very least, open
> of new files should fail with -EIO. Preferably the fs should force a
> read-only remount on IO errors. Given that the vast majority of HFS+
> filesystems Linux is likely to be used with are on hotpluggable media,
> I think this FS should be marked EXPERIMENTAL until such integrity
> problems are addressed.

I agree. This is, indeed, very scary.


Just another data point, Rogério.

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
  Rogério Brito - rbrito@ime.usp.br - http://www.ime.usp.br/~rbrito
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Re: Serious problems with HFS+

[Roman Zippel]

Hi,

On Sun, 13 Mar 2005, Matt Mackall wrote:

> I've noticed a few problems with HFS+ support in recent kernels on
> another user's machine running Ubuntu (Warty) running
> 2.6.8.1-3-powerpc. I'm not in a position to extensively test or fix
> either of these problem because of the fs tools situation so I'm just
> passing this on.
> 
> First, it reports inappropriate blocks to stat(2). It uses 4096 byte
> blocks rather than 512 byte blocks which stat callers are expecting.
> This seriously confuses du(1) (and me, for a bit). Looks like it may
> be forgetting to set s_blocksize_bits.

This should be fixed since 2.6.10.

> Second, if an HFS+ filesystem mounted via Firewire or USB becomes
> detached, the filesystem appears to continue working just fine. I can
> find on the entire tree, despite memory pressure. I can even create
> new files that continue to appear in directory listings! Writes to
> such files succeed (they're async, of course) and the typical app is
> none the wiser. It's only when apps attempt to read later that they
> encounter problems. It turns out that various apps including scp
> ignore IO errors on read and silently copy zero-filled files to the
> destination. So I got this report as "why aren't the pictures I took
> off my camera visible on my website?"

HFS+ metadata is also in the page cache, so as long as everything is 
cached, HFS+ won't notice a problem.

> This is obviously a really nasty failure mode. At the very least, open
> of new files should fail with -EIO. Preferably the fs should force a
> read-only remount on IO errors. Given that the vast majority of HFS+
> filesystems Linux is likely to be used with are on hotpluggable media,
> I think this FS should be marked EXPERIMENTAL until such integrity
> problems are addressed.

Currently nobody tells fs about such events, so even if I check for 
write errors, it can still take a while until the error is detected.
I acknowledge that this is a problem, but I don't think it's a HFS+ 
specific problem, there is currently no standard procedure for fs what to 
do in such situations, so I didn't implement anything.
It would be nice if the fs would be tould about plug/unplug events, e.g. 
HFS+ could check the mount count to see if it was connected to a different 
host in the meantime and react appropriately.

> Having the whole directory tree seemingly pinned in memory is probably
> something that wants addressing as well.

This actually might be a real HFS+ problem :), I have to look into it.

bye, Roman

Re: Serious problems with HFS+

[Matt Mackall]

On Mon, Mar 14, 2005 at 11:18:49AM +0100, Roman Zippel wrote:
> Hi,
> 
> On Sun, 13 Mar 2005, Matt Mackall wrote:
> 
> > I've noticed a few problems with HFS+ support in recent kernels on
> > another user's machine running Ubuntu (Warty) running
> > 2.6.8.1-3-powerpc. I'm not in a position to extensively test or fix
> > either of these problem because of the fs tools situation so I'm just
> > passing this on.
> > 
> > First, it reports inappropriate blocks to stat(2). It uses 4096 byte
> > blocks rather than 512 byte blocks which stat callers are expecting.
> > This seriously confuses du(1) (and me, for a bit). Looks like it may
> > be forgetting to set s_blocksize_bits.
> 
> This should be fixed since 2.6.10.
> 
> > Second, if an HFS+ filesystem mounted via Firewire or USB becomes
> > detached, the filesystem appears to continue working just fine. I can
> > find on the entire tree, despite memory pressure. I can even create
> > new files that continue to appear in directory listings! Writes to
> > such files succeed (they're async, of course) and the typical app is
> > none the wiser. It's only when apps attempt to read later that they
> > encounter problems. It turns out that various apps including scp
> > ignore IO errors on read and silently copy zero-filled files to the
> > destination. So I got this report as "why aren't the pictures I took
> > off my camera visible on my website?"
> 
> HFS+ metadata is also in the page cache, so as long as everything is 
> cached, HFS+ won't notice a problem.

It's failing to notice errors at sync time or when such pages get
flushed due to memory pressure.

> > This is obviously a really nasty failure mode. At the very least, open
> > of new files should fail with -EIO. Preferably the fs should force a
> > read-only remount on IO errors. Given that the vast majority of HFS+
> > filesystems Linux is likely to be used with are on hotpluggable media,
> > I think this FS should be marked EXPERIMENTAL until such integrity
> > problems are addressed.
> 
> Currently nobody tells fs about such events, so even if I check for 
> write errors, it can still take a while until the error is detected.

It should catch up within the flush interval or at the next sync, at
least. And then fail all further writes. Consider the scenario of a
user sitting at their laptop when a power glitch offlines their
external drive. They can copy files onto it for the next hour, delete
the originals and be completely unaware that anything is wrong unless
they happen to check dmesg.

> It would be nice if the fs would be tould about plug/unplug events, e.g. 
> HFS+ could check the mount count to see if it was connected to a different 
> host in the meantime and react appropriately.

An FS-level callback when the underlying block device went for a walk
would be a nice hook just about everywhere. And at least a start on
the problem.

-- 
Mathematics is the supreme nostalgia of our time.