Replacing VFAT as filesystem on removeable media

Replacing VFAT as filesystem on removeable media

[Martin Steigerwald]

[-- Attachment #1: Type: text/plain, Size: 1918 bytes --]


Hi!

Now as TomTom appears to have surrendered to Microsoft and Microsoft seems 
to have accepted this deal probably in order to not find out that their 
patents are void, I think replacing VFAT as standard cross platform 
filesystem on removeable media would be an even better idea than before 
[1].

Granted I believe the patents will be made void one day, but even then I 
think it would make sense to replace VFAT for technical reasons.

Such a filesystem IMHO should have the following features:
- cross platform with implementations for Linux, Windows, Mac OS X at 
least, also the varios BSD variantes come to my mind
- open source
- probably some support for flash media without or with bad wear leveling
- some kind of journaling or other metadata consistency guarentee 

People use ext2 as an alternative, but that lacks journaling.

I could also think of UDF with write support, but I am not sure whether 
Windows and Mac OS X has write support.

Or probably even a new filesystem as long as people appear to write one 
filesystem after another these days.

Or some kind of FAT *without* compatibility hacks, but I think this would 
still be an inferior solution as long as it doesn't provide for metadata 
consistency.

Maybe this could become some kind of Linux Foundation or FSF joint effort? 
Together with advertising and advocacy of free software users this could 
probably really replace VFAT in the long term.

What do you think? What other features would make sense to for such a 
filesystem.

I am willing to test such a filesystem and help with documentation as well 
as advocacy. But for coding I better start with something easier ;-).

[1] 
http://www.h-online.com/open/TomTom-Microsoft-settle-patent-dispute--/news/112964

Ciao,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Replacing VFAT as filesystem on removeable media

[Mark Williamson]

Another approach which occurs to me would be to come up with a "filesystem over 
USB" protocol so that the actual on-disk format is no longer relevant.  With a 
standard or reasonably common FSoUSB implementation, a device maker could 
choose a disk filesystem that's appropriate to their purpose and hide the 
implementation details within the device.  For instance, an MP3 player could 
use a flash-specific disk filesystem but would be able hide the details of this 
from the host PC.

This might not be so desirable for simple devices (like USB card readers, 
perhaps) but I imagine it would fairly straightforward on media players, 
GPSes, etc that run a "proper" OS.  The most pain would arguably be in 
*somebody* having to write a filesystem driver for Windows but at least it 
would only need writing once ;-)

I've seen some talk of this sort of approach online but nothing too specific.  
A Linux USB website suggested that a certain camera protocol might be suitable 
as a basis for files-over-USB (there's even a FUSE module for this, I think).  
Another possibility that occurs would be to write a USB transport for 9P, 
which looked like it might be fairly straightforward...

Just a thought, anyhow.

Cheers,
Mark

On Tuesday 31 March 2009 09:15:28 Martin Steigerwald wrote:
> Hi!
>
> Now as TomTom appears to have surrendered to Microsoft and Microsoft seems
> to have accepted this deal probably in order to not find out that their
> patents are void, I think replacing VFAT as standard cross platform
> filesystem on removeable media would be an even better idea than before
> [1].
>
> Granted I believe the patents will be made void one day, but even then I
> think it would make sense to replace VFAT for technical reasons.
>
> Such a filesystem IMHO should have the following features:
> - cross platform with implementations for Linux, Windows, Mac OS X at
> least, also the varios BSD variantes come to my mind
> - open source
> - probably some support for flash media without or with bad wear leveling
> - some kind of journaling or other metadata consistency guarentee
>
> People use ext2 as an alternative, but that lacks journaling.
>
> I could also think of UDF with write support, but I am not sure whether
> Windows and Mac OS X has write support.
>
> Or probably even a new filesystem as long as people appear to write one
> filesystem after another these days.
>
> Or some kind of FAT *without* compatibility hacks, but I think this would
> still be an inferior solution as long as it doesn't provide for metadata
> consistency.
>
> Maybe this could become some kind of Linux Foundation or FSF joint effort?
> Together with advertising and advocacy of free software users this could
> probably really replace VFAT in the long term.
>
> What do you think? What other features would make sense to for such a
> filesystem.
>
> I am willing to test such a filesystem and help with documentation as well
> as advocacy. But for coding I better start with something easier ;-).
>
> [1]
> http://www.h-online.com/open/TomTom-Microsoft-settle-patent-dispute--/news/
>112964
>
> Ciao,

Re: Replacing VFAT as filesystem on removeable media

[Xavier Bestel]

On Tue, 2009-03-31 at 13:57 +0100, Mark Williamson wrote:
> Another approach which occurs to me would be to come up with a "filesystem over 
> USB" protocol so that the actual on-disk format is no longer relevant.

Look for MTP. It's what you want, made by Microsoft, and enables all
sorts of DRM niceties.

	Xav

Re: Replacing VFAT as filesystem on removeable media

[Mark Williamson]

Hi Xav,

On Tuesday 31 March 2009 14:47:07 Xavier Bestel wrote:
> On Tue, 2009-03-31 at 13:57 +0100, Mark Williamson wrote:
> > Another approach which occurs to me would be to come up with a
> > "filesystem over USB" protocol so that the actual on-disk format is no
> > longer relevant.
>
> Look for MTP. It's what you want, made by Microsoft, and enables all
> sorts of DRM niceties.

Ah, thanks for that.  It looks fairly close to what I had in mind, although I 
think there are some differences (it transfers whole files at a time and 
incorporates device control stuff - neither of which is really necessary in the 
scheme I imagined).

According to Wikipedia  it is now an official USB device class, which sounds 
attractive.  I've not heard of Linux providing an MTP responder ("server"), 
which I find mildly surprising as it sounds like it would be useful, 
particularly given the range of advanced filesystems Linux devices might like 
to leverage under the hood.  It seems odd if an embedded Linux company hasn't 
already coded this up for a device ... ?

I can see how a file-based protocol would be "useful" in enforcing DRM-like 
protections in a device implementation but I don't think that such protocols 
fundamentally enables any nastiness that couldn't be enforced in other ways by 
a motivated pro-DRM device manufacturer.  Sounds like MTP is designed to 
facilitate DRM if the communications endpoints want it; not something I'm keen 
on but also not something that rules it out as a useful standard.

A file-based transfer protocol seems like it would be useful to have for 
"sufficiently intelligent" devices, whether it's MTP or something else.

Cheers,
Mark

Re: Replacing VFAT as filesystem on removeable media

[Xavier Bestel]

On Tue, 2009-03-31 at 16:02 +0100, Mark Williamson wrote:
> I've not heard of Linux providing an MTP responder ("server"), 
> which I find mildly surprising as it sounds like it would be useful, 
> particularly given the range of advanced filesystems Linux devices might like 
> to leverage under the hood.  It seems odd if an embedded Linux company hasn't 
> already coded this up for a device ... ?

I'm speaking out of my ass here, but seeing how Microsoft managed to
sneak some patents into something as trivial as FAT, I'm pretty sure MTP
is a hell of a minefield.

	Xav

Re: Replacing VFAT as filesystem on removeable media

[Mark Williamson]

On Tuesday 31 March 2009 16:21:37 Xavier Bestel wrote:
> I'm speaking out of my ass here, but seeing how Microsoft managed to
> sneak some patents into something as trivial as FAT, I'm pretty sure MTP
> is a hell of a minefield.

That did occur to me too but I was somewhat hopeful that perhaps this would be 
mitigated by the (eventual?) adoption as a usb.org standard.

The spec and an adopters' legal agreement around it is here:
http://www.usb.org/developers/devclass_docs/MTP_1.0.zip

I scanned the agreement and it didn't look like it was obviously evil but I'm 
not really qualified to make that judgement ;-)  It had some promising words in 
it like "zero royalty" but I would be much happier if someone with legal 
knowhow (and preferably experience of the usb.org standards procedures) 
decoded it for me / us!

Assuming the spec is legally "safe" to implement, I would have thought it 
would be generally beneficial for device manufacturers to support both 
"initiator" and "responder" endpoints under Linux.

Cheers,
Mark

Re: Replacing VFAT as filesystem on removeable media

[Bryan Henderson]

> Now as TomTom appears to have surrendered to Microsoft and Microsoft 
seems 
> to have accepted this deal probably in order to not find out that their 
> patents are void, I think replacing VFAT as standard cross platform 
> filesystem on removeable media would be an even better idea than before 
> [1].
 
> [1] 
> 
http://www.h-online.com/open/TomTom-Microsoft-settle-patent-dispute--/news/112964

How do you mean "replace"?

VFAT's standardness today comes from the fact that virtually every 
personal computer in the world today and for the foreseeable future is 
able to read and write VFAT.  Do you envision some other filesystem format 
achieving that status?

--
Bryan Henderson                     IBM Almaden Research Center
San Jose CA                         Storage Systems

Re: Replacing VFAT as filesystem on removeable media

[Martin Steigerwald]

[-- Attachment #1: Type: text/plain, Size: 933 bytes --]

Am Donnerstag 02 April 2009 schrieb Bryan Henderson:
> > Now as TomTom appears to have surrendered to Microsoft and Microsoft
>
> seems
>
> > to have accepted this deal probably in order to not find out that
> > their patents are void, I think replacing VFAT as standard cross
> > platform filesystem on removeable media would be an even better idea
> > than before [1].
> >
> > [1]
>
> http://www.h-online.com/open/TomTom-Microsoft-settle-patent-dispute--/n
>ews/112964
>
> How do you mean "replace"?
>
> VFAT's standardness today comes from the fact that virtually every
> personal computer in the world today and for the foreseeable future is
> able to read and write VFAT.  Do you envision some other filesystem
> format achieving that status?

That was my (possibly crazy) idea yes. ;)

-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Replacing VFAT as filesystem on removeable media

[H. Peter Anvin]

Martin Steigerwald wrote:
>>
>> How do you mean "replace"?
>>
>> VFAT's standardness today comes from the fact that virtually every
>> personal computer in the world today and for the foreseeable future is
>> able to read and write VFAT.  Do you envision some other filesystem
>> format achieving that status?
> 
> That was my (possibly crazy) idea yes. ;)
> 

Ironically enough, one of the better filesystems for being supported by
many OSes is probably ext2.  However, that doesn't mean it is even in
the same rough ballpark as (V)FAT.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: Replacing VFAT as filesystem on removeable media

[Alejandro Riveira Fernández]

El Sat, 04 Apr 2009 23:34:50 -0700
"H. Peter Anvin" <hpa@zytor.com> escribió:

> Martin Steigerwald wrote:
> >>
> >> How do you mean "replace"?
> >>
> >> VFAT's standardness today comes from the fact that virtually every
> >> personal computer in the world today and for the foreseeable future is
> >> able to read and write VFAT.  Do you envision some other filesystem
> >> format achieving that status?
> > 
> > That was my (possibly crazy) idea yes. ;)
> > 
> 
> Ironically enough, one of the better filesystems for being supported by
> many OSes is probably ext2.  However, that doesn't mean it is even in
> the same rough ballpark as (V)FAT.

  What about UDF ?
> 
> 	-hpa
> 

Re: Replacing VFAT as filesystem on removeable media

[H. Peter Anvin]

Alejandro Riveira Fernández wrote:
>>>
>> Ironically enough, one of the better filesystems for being supported by
>> many OSes is probably ext2.  However, that doesn't mean it is even in
>> the same rough ballpark as (V)FAT.
> 
>   What about UDF ?

Good point.  UDF is probably widely supported, but again, not in the
same ballpark.  I don't personally know how complex UDF is to implement
on a small memory device.

	-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Replacing VFAT as filesystem on removeable media

[Nicholas Miell]

On Sun, 2009-04-05 at 13:05 -0700, H. Peter Anvin wrote:
> Alejandro Riveira Fernández wrote:
> >>>
> >> Ironically enough, one of the better filesystems for being supported by
> >> many OSes is probably ext2.  However, that doesn't mean it is even in
> >> the same rough ballpark as (V)FAT.
> > 
> >   What about UDF ?
> 
> Good point.  UDF is probably widely supported, but again, not in the
> same ballpark.  I don't personally know how complex UDF is to implement
> on a small memory device.
> 
> 	-hpa
> 

DVD players are small memory devices and they don't actually implement
UDF.

(DVD movies have UDF filesystems, but they're constructed such that the
files that the DVD player needs are at specific fixed offsets on the
disc, and the DVD player never actually interprets the UDF filesystem.)

Also, there's several different revisions of UDF, which causes some
level of operating system compatibility problems.

Fortunately, the UDF flavor likely to be used on magnetic/flash media is
also the flavor that's the most compatible, so that makes things easier
for everyone.

-- 
Nicholas Miell <nmiell@comcast.net>

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html