[Q] FAT driver enhancement

[Q] FAT driver enhancement

[Jos Hulzink]

Hi,

A while ago I initiated a thread about mounting a NTFS partition as FAT
partition. The problem is that FAT partitions do not have a real
fingerprint, so the FAT driver mounts almost anything.

The current 2.5 driver only tests if some values in the bootsector are
non-zero. IMHO, this is not strict enough. For example, the number of FATs
is always 1 or 2 (anyone ever seen more ?). Besides, when there are two
FATs, all entries in those FATs should be equal. If they are not, we deal
with a non-FAT or broken FAT partition, and we should not mount.

It's not a real fingerprint, but what are the chances all sectors of what
we think is the FAT are equal on non-FAT filesystems ? Yes, when you just
did a

dd if=/dev/zero of=/dev/partition; mkfs.somefs /dev/partition

there is a chance, but that's an empty filesystem. Data corruption isn't
that bad on an empty disk. We know that a FAT is at the beginning of a
partition and I assume that any other filesystem will fill up those first
sectors very soon.

Questions:

1) How do you think about the checking of the FAT tables ? It definitely
   will slow down the mount.
2) If I implement it, where shoud it go ? At the moment, I hacked
   fat_read_super, for there the FAT fs is validated, but I got the
   feeling this is not the place to be.
3) Anyone seen more than two FATs on a filesystem ? Can I assume there is
   a limit ?
4) Comments, anyone ?

Jos

Re: [Q] FAT driver enhancement

[OGAWA Hirofumi]

Jos Hulzink <josh@stack.nl> writes:

> Hi,
> 
> A while ago I initiated a thread about mounting a NTFS partition as FAT
> partition. The problem is that FAT partitions do not have a real
> fingerprint, so the FAT driver mounts almost anything.
> 
> The current 2.5 driver only tests if some values in the bootsector are
> non-zero. IMHO, this is not strict enough. For example, the number of FATs
> is always 1 or 2 (anyone ever seen more ?). Besides, when there are two
> FATs, all entries in those FATs should be equal. If they are not, we deal
> with a non-FAT or broken FAT partition, and we should not mount.
> 
> It's not a real fingerprint, but what are the chances all sectors of what
> we think is the FAT are equal on non-FAT filesystems ? Yes, when you just
> did a
> 
> dd if=/dev/zero of=/dev/partition; mkfs.somefs /dev/partition
> 
> there is a chance, but that's an empty filesystem. Data corruption isn't
> that bad on an empty disk. We know that a FAT is at the beginning of a
> partition and I assume that any other filesystem will fill up those first
> sectors very soon.
> 
> Questions:
> 
> 1) How do you think about the checking of the FAT tables ? It definitely
>    will slow down the mount.

Unfortunately if FAT table has bad sector, FAT tables may not be the
same.

> 2) If I implement it, where shoud it go ? At the moment, I hacked
>    fat_read_super, for there the FAT fs is validated, but I got the
>    feeling this is not the place to be.
> 3) Anyone seen more than two FATs on a filesystem ? Can I assume there is
>    a limit ?
> 4) Comments, anyone ?

How about writing the mount.xxx/fsck.xxx? The mount.xxx/fsck.xxx can
check many of the ordinary FAT status. If the something occurs, output
message to user. And user can handle it by option etc.

Regards.
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

Re: [Q] FAT driver enhancement

[Helge Hafting]

OGAWA Hirofumi wrote:
> 
> Jos Hulzink <josh@stack.nl> writes:
> 
> > Hi,
> >
> > A while ago I initiated a thread about mounting a NTFS partition as FAT
> > partition. The problem is that FAT partitions do not have a real
> > fingerprint, so the FAT driver mounts almost anything.
> >
> > The current 2.5 driver only tests if some values in the bootsector are
> > non-zero. IMHO, this is not strict enough. For example, the number of FATs
> > is always 1 or 2 (anyone ever seen more ?). Besides, when there are two
> > FATs, all entries in those FATs should be equal. If they are not, we deal
> > with a non-FAT or broken FAT partition, and we should not mount.
> >
> > It's not a real fingerprint, but what are the chances all sectors of what
> > we think is the FAT are equal on non-FAT filesystems ? Yes, when you just
> > did a
> >
> > dd if=/dev/zero of=/dev/partition; mkfs.somefs /dev/partition
> >
> > there is a chance, but that's an empty filesystem. Data corruption isn't
> > that bad on an empty disk. We know that a FAT is at the beginning of a
> > partition and I assume that any other filesystem will fill up those first
> > sectors very soon.
> >
> > Questions:
> >
> > 1) How do you think about the checking of the FAT tables ? It definitely
> >    will slow down the mount.
> 
> Unfortunately if FAT table has bad sector, FAT tables may not be the
> same.

And then you don't want to mount unless you know what you
are doing.  And those knowing what they are doing can be bothered
to use some kind of "force" option in this case.  Or perhaps an
option that selects which FAT to trust.

Helge Hafting

Re: [Q] FAT driver enhancement

[OGAWA Hirofumi]

Helge Hafting <helgehaf@aitel.hist.no> writes:

> OGAWA Hirofumi wrote:
> > 
> > Jos Hulzink <josh@stack.nl> writes:
> > 
> > > Hi,
> > >
> > > A while ago I initiated a thread about mounting a NTFS partition as FAT
> > > partition. The problem is that FAT partitions do not have a real
> > > fingerprint, so the FAT driver mounts almost anything.
> > >
> > > The current 2.5 driver only tests if some values in the bootsector are
> > > non-zero. IMHO, this is not strict enough. For example, the number of FATs
> > > is always 1 or 2 (anyone ever seen more ?). Besides, when there are two
> > > FATs, all entries in those FATs should be equal. If they are not, we deal
> > > with a non-FAT or broken FAT partition, and we should not mount.
> > >
> > > It's not a real fingerprint, but what are the chances all sectors of what
> > > we think is the FAT are equal on non-FAT filesystems ? Yes, when you just
> > > did a
> > >
> > > dd if=/dev/zero of=/dev/partition; mkfs.somefs /dev/partition
> > >
> > > there is a chance, but that's an empty filesystem. Data corruption isn't
> > > that bad on an empty disk. We know that a FAT is at the beginning of a
> > > partition and I assume that any other filesystem will fill up those first
> > > sectors very soon.
> > >
> > > Questions:
> > >
> > > 1) How do you think about the checking of the FAT tables ? It definitely
> > >    will slow down the mount.
> > 
> > Unfortunately if FAT table has bad sector, FAT tables may not be the
> > same.
> 
> And then you don't want to mount unless you know what you
> are doing.  And those knowing what they are doing can be bothered
> to use some kind of "force" option in this case.  Or perhaps an
> option that selects which FAT to trust.

I mean I/O error, not data damage.
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

Re: [Q] FAT driver enhancement

[Mike Fedyk]

On Tue, Apr 02, 2002 at 10:27:52PM +0900, OGAWA Hirofumi wrote:
> Helge Hafting <helgehaf@aitel.hist.no> writes:
> 
> > OGAWA Hirofumi wrote:
> > > 
> > > Jos Hulzink <josh@stack.nl> writes:
> > > > Questions:
> > > >
> > > > 1) How do you think about the checking of the FAT tables ? It definitely
> > > >    will slow down the mount.
> > > 
> > > Unfortunately if FAT table has bad sector, FAT tables may not be the
> > > same.
> > 
> > And then you don't want to mount unless you know what you
> > are doing.  And those knowing what they are doing can be bothered
> > to use some kind of "force" option in this case.  Or perhaps an
> > option that selects which FAT to trust.
> 
> I mean I/O error, not data damage.

It is the block layer's responsibility to retry such soft errors and recover.

Probably the best you can do, is retry the read a few times if there is an error
reported, and then fail if it persists.

Re: [Q] FAT driver enhancement

[Jens Axboe]

On Tue, Apr 02 2002, Mike Fedyk wrote:
> On Tue, Apr 02, 2002 at 10:27:52PM +0900, OGAWA Hirofumi wrote:
> > Helge Hafting <helgehaf@aitel.hist.no> writes:
> > 
> > > OGAWA Hirofumi wrote:
> > > > 
> > > > Jos Hulzink <josh@stack.nl> writes:
> > > > > Questions:
> > > > >
> > > > > 1) How do you think about the checking of the FAT tables ? It definitely
> > > > >    will slow down the mount.
> > > > 
> > > > Unfortunately if FAT table has bad sector, FAT tables may not be the
> > > > same.
> > > 
> > > And then you don't want to mount unless you know what you
> > > are doing.  And those knowing what they are doing can be bothered
> > > to use some kind of "force" option in this case.  Or perhaps an
> > > option that selects which FAT to trust.
> > 
> > I mean I/O error, not data damage.
> 
> It is the block layer's responsibility to retry such soft errors and recover.

No it's the low level driver

> Probably the best you can do, is retry the read a few times if there
> is an error reported, and then fail if it persists.

I/O error retrying from the fs is a bad idea, the low level driver
should already have exhausted the possibilities to recover the data. If
the fs receives an I/O error, that's the final result and retrying
would serve no sane purpose.

-- 
Jens Axboe

Re: [Q] FAT driver enhancement

[OGAWA Hirofumi]

Mike Fedyk <mfedyk@matchmail.com> writes:

> On Tue, Apr 02, 2002 at 10:27:52PM +0900, OGAWA Hirofumi wrote:
> > Helge Hafting <helgehaf@aitel.hist.no> writes:
> > 
> > > OGAWA Hirofumi wrote:
> > > > 
> > > > Jos Hulzink <josh@stack.nl> writes:
> > > > > Questions:
> > > > >
> > > > > 1) How do you think about the checking of the FAT tables ? It definitely
> > > > >    will slow down the mount.
> > > > 
> > > > Unfortunately if FAT table has bad sector, FAT tables may not be the
> > > > same.
> > > 
> > > And then you don't want to mount unless you know what you
> > > are doing.  And those knowing what they are doing can be bothered
> > > to use some kind of "force" option in this case.  Or perhaps an
> > > option that selects which FAT to trust.
> > 
> > I mean I/O error, not data damage.
> 
> It is the block layer's responsibility to retry such soft errors and recover.

Yes.

> Probably the best you can do, is retry the read a few times if there
> is an error reported, and then fail if it persists.

Umm, there is a `copy of FAT table' for this kind of error. If the I/O
error occurs, the FAT driver should use the other FAT table.
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

Re: [Q] FAT driver enhancement

[Jos Hulzink]

On Wed, 3 Apr 2002, OGAWA Hirofumi wrote:

> Mike Fedyk <mfedyk@matchmail.com> writes:
>
> > > I mean I/O error, not data damage.
> >
> > It is the block layer's responsibility to retry such soft errors and recover.
>
> Yes.

But what about the data damage errors ?

> > Probably the best you can do, is retry the read a few times if there
> > is an error reported, and then fail if it persists.
>
> Umm, there is a `copy of FAT table' for this kind of error. If the I/O
> error occurs, the FAT driver should use the other FAT table.

How should the FAT driver know that the first FAT is bad if it doesn't
scan the FAT ? You don't want the second FAT to be used, you want the
mount to fail, and fsck.xxx to fix the mess. Who tells you that the second
copy of the FAT is the correct one, and not the first ?

Jos

Re: [Q] FAT driver enhancement

[OGAWA Hirofumi]

Jos Hulzink <josh@stack.nl> writes:

> On Wed, 3 Apr 2002, OGAWA Hirofumi wrote:
> 
> > Mike Fedyk <mfedyk@matchmail.com> writes:
> >
> > > > I mean I/O error, not data damage.
> > >
> > > It is the block layer's responsibility to retry such soft errors and recover.
> >
> > Yes.
> 
> But what about the data damage errors ?
> 
> > > Probably the best you can do, is retry the read a few times if there
> > > is an error reported, and then fail if it persists.
> >
> > Umm, there is a `copy of FAT table' for this kind of error. If the I/O
> > error occurs, the FAT driver should use the other FAT table.
> 
> How should the FAT driver know that the first FAT is bad if it doesn't
> scan the FAT ? You don't want the second FAT to be used, you want the
> mount to fail, and fsck.xxx to fix the mess. Who tells you that the second
> copy of the FAT is the correct one, and not the first ?

FAT16/FAT32 use the second entry of FAT table for data damage.
The 1 bit of second entry is a clean/dirty unmount flag.

But, it's not perfect. Furthermore, currently not implemented.
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>

Re: [Q] FAT driver enhancement

[David D. Hagood]

Jos Hulzink wrote:
> How should the FAT driver know that the first FAT is bad if it doesn't
> scan the FAT ? You don't want the second FAT to be used, you want the
> mount to fail, and fsck.xxx to fix the mess. Who tells you that the second
> copy of the FAT is the correct one, and not the first ?

Seems to me you would want a mount-time option to the FAT fs code to say 
"use FAT#<n>", defaulting to the first if no parm given. If that copy of 
the FAT has any problems, fail the mount.

Then you'd want the fsck.fat to have a similar option, saying "use 
FAT#<n> for the check" - that way if the FATs are out of sync, you could 
do a dry run check on each FAT, and go with the one that seemed to be 
better. Perhaps even having the tool allow you to pick and choose if 
needed (although this would probably be better as a seperate tool, that 
allowed you to view a file given a selected FAT and copy it to a clean 
file system.)

Re: [Q] FAT driver enhancement

[Thunder from the hill]

Hi,

David D. Hagood wrote:
> Jos Hulzink wrote:
> 
>> How should the FAT driver know that the first FAT is bad if it doesn't
>> scan the FAT ? You don't want the second FAT to be used, you want the
>> mount to fail, and fsck.xxx to fix the mess. Who tells you that the 
>> second
>> copy of the FAT is the correct one, and not the first ?
> 
> 
> Seems to me you would want a mount-time option to the FAT fs code to say 
> "use FAT#<n>", defaulting to the first if no parm given. If that copy of 
> the FAT has any problems, fail the mount.
> 
> Then you'd want the fsck.fat to have a similar option, saying "use 
> FAT#<n> for the check" - that way if the FATs are out of sync, you could 
> do a dry run check on each FAT, and go with the one that seemed to be 
> better. Perhaps even having the tool allow you to pick and choose if 
> needed (although this would probably be better as a seperate tool, that 
> allowed you to view a file given a selected FAT and copy it to a clean 
> file system.)
If I apply the "Think big" patch for a second, I say, check individually 
for each entry with fsck.fat and build a fat of the entries that are 
still ok. This could be a special --rebuild option or whatever.

Regards,
Thunder
-- 
Thunder from the hill.
Citizen of our universe.

Re: [Q] FAT driver enhancement

[Pavel Machek]

Hi!

> A while ago I initiated a thread about mounting a NTFS partition as FAT
> partition. The problem is that FAT partitions do not have a real
> fingerprint, so the FAT driver mounts almost anything.
> 
> The current 2.5 driver only tests if some values in the bootsector are
> non-zero. IMHO, this is not strict enough. For example, the number of FATs
> is always 1 or 2 (anyone ever seen more ?). Besides, when there are two
> FATs, all entries in those FATs should be equal. If they are not, we deal
> with a non-FAT or broken FAT partition, and we should not mount.
> 
> It's not a real fingerprint, but what are the chances all sectors of what
> we think is the FAT are equal on non-FAT filesystems ? Yes, when you just
> did a
> 
> dd if=/dev/zero of=/dev/partition; mkfs.somefs /dev/partition
> 
> there is a chance, but that's an empty filesystem. Data corruption isn't
> that bad on an empty disk. We know that a FAT is at the beginning of a
> partition and I assume that any other filesystem will fill up those first
> sectors very soon.
> 
> Questions:
> 
> 1) How do you think about the checking of the FAT tables ? It definitely
>    will slow down the mount.

Reading FATs is very fast, and they are probably going to be needed,
anyway. I guess its okay.

> 2) If I implement it, where shoud it go ? At the moment, I hacked
>    fat_read_super, for there the FAT fs is validated, but I got the
>    feeling this is not the place to be.
> 3) Anyone seen more than two FATs on a filesystem ? Can I assume there is
>    a limit ?

No. I think you can only have two.

									Pavel
-- 
(about SSSCA) "I don't say this lightly.  However, I really think that the U.S.
no longer is classifiable as a democracy, but rather as a plutocracy." --hpa