Libvolume incorrectly detects FAT32

Libvolume incorrectly detects FAT32

[iSteve]

[-- Attachment #1: Type: text/plain, Size: 836 bytes --]

Hello,
I'm having one silly partition, which has remnants of previous FAT32 filesystem
on it, a LILO bootloader on it and ext2 filesystem spaning all over the
partition.

The first 512B look exactly like a valid FAT partition header would, except the
magic is "LILO". 

I've looked at the sources for FAT detection, and I noted that when a FAT-like
filesystem misses magic, it's still considered valid, since as comment says,
some old floppies may lack it. I consider this point valid, however, this
causes my partition to be detected, incorrectly, as FAT32.

My proposed solution is to compare magic if FAT32 is assumed, and if magic will
not match for FAT32, consider it a failure; floppies that old that they lack
any magic will most likely not be formatted in FAT32.

Trivial patch is attached, comments/complaints?:)
-- 
 -- iSteve

[-- Attachment #2: fat32-magic-required.patch --]
[-- Type: text/x-patch, Size: 318 bytes --]

--- fat.c.old	2006-01-30 08:51:38.000000000 +0100
+++ fat.c	2006-03-06 21:55:29.000000000 +0100
@@ -281,6 +281,9 @@
 	goto found;
 
 fat32:
+	if (memcmp(vs->type.fat32.magic, "FAT32   ", 8) != 0)
+		return -1;
+
 	strcpy(id->type_version, "FAT32");
 
 	/* FAT32 root dir is a cluster chain like any other directory */

Re: Libvolume incorrectly detects FAT32

[Michael Buesch]

[-- Attachment #1: Type: text/plain, Size: 1199 bytes --]

On Monday 06 March 2006 21:57, iSteve wrote:
> Hello,
> I'm having one silly partition, which has remnants of previous FAT32 filesystem
> on it, a LILO bootloader on it and ext2 filesystem spaning all over the
> partition.
> 
> The first 512B look exactly like a valid FAT partition header would, except the
> magic is "LILO". 
> 
> I've looked at the sources for FAT detection, and I noted that when a FAT-like
> filesystem misses magic, it's still considered valid, since as comment says,
> some old floppies may lack it. I consider this point valid, however, this
> causes my partition to be detected, incorrectly, as FAT32.
> 
> My proposed solution is to compare magic if FAT32 is assumed, and if magic will
> not match for FAT32, consider it a failure; floppies that old that they lack
> any magic will most likely not be formatted in FAT32.
> 
> Trivial patch is attached, comments/complaints?:)

Yes, I fear users of stupid digicams may get hit by this.
The Formating option on cheap digicams may sometimes be buggy implemented.
I think you should instead do a
dd if=/dev/zero of=/dev/partition bs=512 count=1
and reformat your partition.

-- 
Greetings Michael.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Libvolume incorrectly detects FAT32

[Kay Sievers]

On Mon, Mar 06, 2006 at 10:59:31PM +0100, Michael Buesch wrote:
> On Monday 06 March 2006 21:57, iSteve wrote:
> > Hello,
> > I'm having one silly partition, which has remnants of previous FAT32 filesystem
> > on it, a LILO bootloader on it and ext2 filesystem spaning all over the
> > partition.
> > 
> > The first 512B look exactly like a valid FAT partition header would, except the
> > magic is "LILO". 
> > 
> > I've looked at the sources for FAT detection, and I noted that when a FAT-like
> > filesystem misses magic, it's still considered valid, since as comment says,
> > some old floppies may lack it. I consider this point valid, however, this
> > causes my partition to be detected, incorrectly, as FAT32.
> > 
> > My proposed solution is to compare magic if FAT32 is assumed, and if magic will
> > not match for FAT32, consider it a failure; floppies that old that they lack
> > any magic will most likely not be formatted in FAT32.
> > 
> > Trivial patch is attached, comments/complaints?:)
> 
> Yes, I fear users of stupid digicams may get hit by this.
> The Formating option on cheap digicams may sometimes be buggy implemented.
> I think you should instead do a
> dd if=/dev/zero of=/dev/partition bsQ2 count=1
> and reformat your partition.

Yes, just use a sane formatting program like everything which is
libparted based. The usual Linux commandline formatters are horribly
broken and I gave up to talk to the ignorant maintainers. Just zero-out
the device at the beginning _and_ the end of the device if you reformat it
with a new filesystem type. Or use a libparted or similar based program.

Thanks,
Kay


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x110944&bid$1720&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Libvolume incorrectly detects FAT32

[iSteve]

On Mon, 6 Mar 2006 23:37:20 +0100
Kay Sievers <kay.sievers@vrfy.org> wrote:

> On Mon, Mar 06, 2006 at 10:59:31PM +0100, Michael Buesch wrote:
> > Yes, I fear users of stupid digicams may get hit by this.
> > The Formating option on cheap digicams may sometimes be buggy implemented.
> > I think you should instead do a
> > dd if=/dev/zero of=/dev/partition bsQ2 count=1
> > and reformat your partition.
> 
> Yes, just use a sane formatting program like everything which is
> libparted based. The usual Linux commandline formatters are horribly
> broken and I gave up to talk to the ignorant maintainers. Just zero-out
> the device at the beginning _and_ the end of the device if you reformat it
> with a new filesystem type. Or use a libparted or similar based program.
> 
> Thanks,
> Kay

I would have another suggestion then; how about a scoring system?

Probes would be given OPTIONAL scoring system, let's say in percents, eg. in
case of fat, lack of magic would make it lose 20%; some other filesystem would
return just 0 or 100% hit (ie. just like it is now). Then the probe with
greatest score wins.

I understand this design is for working with semibroken partitions, and I fear
it won't be welcome warmly; but I still got to propose it:)

I am willing to code it, I've written about ten probes for my own detection
system until I noticed libvolumeid, but I want approval of design first.

Thanks in advance
 -- iSteve


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x110944&bid$1720&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Libvolume incorrectly detects FAT32

[Kay Sievers]

On Tue, Mar 07, 2006 at 06:01:17PM +0100, iSteve wrote:
> On Mon, 6 Mar 2006 23:37:20 +0100
> Kay Sievers <kay.sievers@vrfy.org> wrote:
> 
> > On Mon, Mar 06, 2006 at 10:59:31PM +0100, Michael Buesch wrote:
> > > Yes, I fear users of stupid digicams may get hit by this.
> > > The Formating option on cheap digicams may sometimes be buggy implemented.
> > > I think you should instead do a
> > > dd if=/dev/zero of=/dev/partition bsQ2 count=1
> > > and reformat your partition.
> > 
> > Yes, just use a sane formatting program like everything which is
> > libparted based. The usual Linux commandline formatters are horribly
> > broken and I gave up to talk to the ignorant maintainers. Just zero-out
> > the device at the beginning _and_ the end of the device if you reformat it
> > with a new filesystem type. Or use a libparted or similar based program.
> > 
> > Thanks,
> > Kay
> 
> I would have another suggestion then; how about a scoring system?
> 
> Probes would be given OPTIONAL scoring system, let's say in percents, eg. in
> case of fat, lack of magic would make it lose 20%; some other filesystem would
> return just 0 or 100% hit (ie. just like it is now). Then the probe with
> greatest score wins.
> 
> I understand this design is for working with semibroken partitions, and I fear
> it won't be welcome warmly; but I still got to propose it:)
> 
> I am willing to code it, I've written about ten probes for my own detection
> system until I noticed libvolumeid, but I want approval of design first.

I'm all for fixing the stupid formatters. Seems the problems are really
rare these days anyway, with Windows doing it right now and almost all
distros use HAL today which has the same code and we don't get a lot of
error reports anymore.

The right way I think, is to add more checks to the probing code, if the
filesytem that is recognized is actually "valid". But that is impossible
for some situations like mkswap (which is the silliest formatting tool on
earth that I've ever seen) on top of fat. You can even mount the
corrupted fat volume with the kernel and write to it.

So, I'm not sure if a score will give us something valuable. You basically
push the problem to the user of the lib, which is a slippery road. But if
we detect that we can't be sure what kind of filesystem we detect, we
could throw an error and don't return any type, that would be at least a
safe way to handle that and would fix the real problem, which are in
almost all cases the too simple and broken Linux command line tools.

Thanks,
Kay


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x110944&bid$1720&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Libvolume incorrectly detects FAT32

[iSteve]

On Tue, 7 Mar 2006 18:15:04 +0100
Kay Sievers <kay.sievers@vrfy.org> wrote:

> On Tue, Mar 07, 2006 at 06:01:17PM +0100, iSteve wrote:
> > On Mon, 6 Mar 2006 23:37:20 +0100
> > Kay Sievers <kay.sievers@vrfy.org> wrote:
> > I would have another suggestion then; how about a scoring system?
> > 
> > Probes would be given OPTIONAL scoring system, let's say in percents, eg. in
> > case of fat, lack of magic would make it lose 20%; some other filesystem
> > would return just 0 or 100% hit (ie. just like it is now). Then the probe
> > with greatest score wins.
> > 
> > I understand this design is for working with semibroken partitions, and I
> > fear it won't be welcome warmly; but I still got to propose it:)
> > 
> > I am willing to code it, I've written about ten probes for my own detection
> > system until I noticed libvolumeid, but I want approval of design first.
> 
> I'm all for fixing the stupid formatters. Seems the problems are really
> rare these days anyway, with Windows doing it right now and almost all
> distros use HAL today which has the same code and we don't get a lot of
> error reports anymore.
> 
> The right way I think, is to add more checks to the probing code, if the
> filesytem that is recognized is actually "valid". But that is impossible
> for some situations like mkswap (which is the silliest formatting tool on
> earth that I've ever seen) on top of fat. You can even mount the
> corrupted fat volume with the kernel and write to it.
> 
> So, I'm not sure if a score will give us something valuable. You basically
> push the problem to the user of the lib, which is a slippery road. But if
> we detect that we can't be sure what kind of filesystem we detect, we
> could throw an error and don't return any type, that would be at least a
> safe way to handle that and would fix the real problem, which are in
> almost all cases the too simple and broken Linux command line tools.
> 
> Thanks,
> Kay

So the bottom line is, "no, this idea won't make it"? How about, then, let the
user of the lib (ie. the app using the lib) specify whether it wants a full
match or not? In the effect, it'd only allow two approaches; the lax one it is
now and a very strict one... The obvious approach would be to go strict first
and then if no match, try it lax; but having this option could really be
helpful.

 -- iSteve



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x110944&bid$1720&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Libvolume incorrectly detects FAT32

[Michael Buesch]

[-- Attachment #1: Type: text/plain, Size: 1070 bytes --]

On Tuesday 07 March 2006 18:32, you wrote:
> So the bottom line is, "no, this idea won't make it"? How about, then, let the
> user of the lib (ie. the app using the lib) specify whether it wants a full
> match or not?

I think you misunderstand.
The lib (app) is about properly _creating_ the partition with
all magic. So basically every formating tool should zero out
parts of the partition and _then_ write the new magic there.
Obviously some tools fail to zero out the space. This is what needs
fixing.
Pushing the problem to kernel-space, by implementing a faulty-by-design
scoring system is IMHO not the way to go.

BTW:
I often saw this problem in the past, especially with swap and fat
related partitions. So I now always use dd to zero out the first
part of a partition before I reformat it.
That's a good workaround the broken format utilities.

To summerize: I don't think there is something broken in the kernel,
so I don't think we should go and fix nonbroken code. Fix the
broken code in userspace instead.

-- 
Greetings Michael.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Libvolume incorrectly detects FAT32

[iSteve]

On Tue, 7 Mar 2006 22:47:25 +0100
Michael Buesch <mbuesch@freenet.de> wrote:
> I think you misunderstand.
> The lib (app) is about properly _creating_ the partition with
> all magic. So basically every formating tool should zero out
> parts of the partition and _then_ write the new magic there.
> Obviously some tools fail to zero out the space. This is what needs
> fixing.
> Pushing the problem to kernel-space, by implementing a faulty-by-design
> scoring system is IMHO not the way to go.
> 
> BTW:
> I often saw this problem in the past, especially with swap and fat
> related partitions. So I now always use dd to zero out the first
> part of a partition before I reformat it.
> That's a good workaround the broken format utilities.
> 
> To summerize: I don't think there is something broken in the kernel,
> so I don't think we should go and fix nonbroken code. Fix the
> broken code in userspace instead.

I've never suggested to hack the kernel code; I suggested to hack the
pseudoscoring system (or the strict / lax approach) into the libvolume_id
included in udev, which is userspace.

-- 
 -- iSteve


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x110944&bid$1720&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel