Re: File System conversion -- ideas

Hugo Mills
2003-06-29T20:36:42Z

[-- Attachment #1: Type: text/plain, Size: 4500 bytes --]

On Sun, Jun 29, 2003 at 04:29:45PM -0400, rmoser wrote:
> *********** REPLY SEPARATOR ***********
>
> NO! You're not getting the point at all!
>
> You don't need a pair! If you have 10 filesystems, you need 10 sets of
> code in each direction, not 90. You convert from the data/metadata set
> in the first filesystem to a self-contained atom, and then back from the
> atom to the data/metadata set in the new filesystem. The atom is object
> oriented, so anything that can't be moved over--like ACLs or Reiser4's
> extended attributes that nobody else has, or permissions if converting to
> vfat--is just lost.

You will, of course, ensure that your atoms contain the superset of
all filesystem metadata semantics.

> Note that if the data has an attribute like "Compressed"
> or "encrypted", it is expanded/decrypted and thus brought back to its
> natural form before being stuffed into an atom.
[snip]

> So with 10 filesystem types, N*(N-1) or 90 pairs to go directly from one
> filesystem's datastructures to any other's; N*2 or 20 pairs to go from
> Metadata/Data pair -> Self-contained object oriented possibly
> compressed atom -> Metadata/Data pair. That's N sets of code to go
> FS_OBJECT -> atom and N sets to go from atom -> FS_OBJECT, in
> this case 10 and 10.
>
> When we get to 20 filesystems, direct conversion needs 380 pieces of
> code, whereas my method needs only 20 + 20 == 40. I obviously put
> more thought into this than you, but that's okay; it's an obscure idea and
> I don't expect everyone to think before answering.

Actually:

1) I think Viro did mention exactly this method in one of his mails.

2) It's not an obscure idea at all -- it's one of the standard
techniques if you've ever had to consider (let alone write!) a
set of data-conversion routines.

> >If you want your idea to be considered seriously - take reiserfs code,
> >take ext3 code, copy both to userland and put together a conversion
> >between them. Both ways. That, by definition, is easier than doing
> >it in kernel - you have the same code available and none of the
> >limitations/
> >interaction with other stuff. When you have it working, well, time to
> >see what extra PITA will come from making it coexist with other parts
> >of kernel (and with much more poor runtime environment).
> >
>
> That would be much harder to maintain as well. It would have to be altered
> every time the filesystem code in the kernel is changed.

Yes, but the point is it's a much easier thing to implement and
test the concept than diving straight into kernel code. You don't have
to maintain it for very long (if at all) -- just long enough to prove
to everyone that this kind of conversion is possible, and that they
should help you roll it into the kernel.

> >AFAICS, it is _very_ hard to implement. Even outside of the kernel.
> >If you can get it done - well, that might do a lot for having the
> >idea considered seriously. "Might" since you need to do it in a way
> >that would survive transplantation into the kernel _and_ would scale
> >better that O((number of filesystem types)^2).
>
> I've beaten the O((FS_COUNT)^2) already. And by the way, it's
> O((FS_COUNT)*(FS_COUNT - 1_). There's exactly O(2*FS_COUNT)
> and o(2*FS_COUNT) sets of code needed total to be able to convert
> between any two filesystems.

There's no such thing as O(x*(x-1)). This is precisely O(x^2).
Similarly, O(2*x) is precisely the same as O(x). If you're going to
try to use mathematics to demonstrate your point, please at least make
sure that you're using it _right_.

> Now, what's impractical is maintaining two sets of code that do exactly
> the same thing. Why maintain code here to read the filesystems, and
> also in the kernel?

It's not a maintenance thing at all -- it's a matter of
demonstrating that you can walk before you try running.

> Just do it in the kernel. Don't lose sight of the fact
> that the final goal (after all else is done) is to modify VFS to actually
> run this thing as a filesystem. THAT is what's going to be a bitch. The
> conversions are simple enough.

Hugo.

--
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
PGP key: 1C335860 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
--- For months now, we have been making triumphant retreats ---
before a demoralised enemy who is advancing
in utter disorder.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]