Fragmentation / repacker / append-performance

Fragmentation / repacker / append-performance

[Lutz Vieweg]

Hans Reiser wrote:

 > 6% fragmentation is enormous.  You very much need the
 > repacker we have not yet written.....

There has been talk about such a feature for a long time -
did anybody already write a concept on how to do this?
Has an effort been estimated? Is somebody already assigned
to that task?

It's not too difficult for us to implement de-fragmentation
mechanisms on the application level, but if the file
system can do it - even better...

We're currently considering reiser4 for an application
that we know would suffer badly from fragmentation, as
it needs to often append small amounts of data (a few
bytes to a few kB) to lots of files (6M).

Alas, such a task does not seem to be covered by any
of the usual benchmarks, so we have to do measurements
on our own. Any suggestion for optimal configuration
parameters we should use?

As of yet, we are using XFS for this task, which is
significantly faster for that particular application
than reiser3. Alas, XFS/kernel-2.6/SMP is known to
be unstable under high load for almost a year now,
but since the problem is hard to reproduce and those
who only need a file server can live with using just
one CPU it does not seem to get fixed anytime soon.

Regards,

Lutz Vieweg

PS: I'm still glad we sponsored the 64bit-port years
     ago, as we're still using reiser3 on a lot of
     systems that can really use it - and of course
     newly bought systems are 64bit architectures... :-)

Re: Fragmentation / repacker / append-performance

[michael chang]

On 11/18/05, Lutz Vieweg <lkv@isg.de> wrote:
> Hans Reiser wrote:
>
>  > 6% fragmentation is enormous.  You very much need the
>  > repacker we have not yet written.....
>
> There has been talk about such a feature for a long time -
> did anybody already write a concept on how to do this?
> Has an effort been estimated? Is somebody already assigned
> to that task?

I believe there used to be an implementation, although right now a lot
of things have sort of dissapeared and been put on hold for Kernel
Merge. [In any case, the repacker has been gone for a while - there
was mention of it on the Reiser4 Wikipedia page for a while but that
was severely outdated, apparently.]

Hans has it on the top of his list; I believe they're just trying to
get priorities straight (kernel acceptance first).

> It's not too difficult for us to implement de-fragmentation
> mechanisms on the application level, but if the file
> system can do it - even better...

Definately.

[...]

> PS: I'm still glad we sponsored the 64bit-port years
>      ago, as we're still using reiser3 on a lot of
>      systems that can really use it - and of course
>      newly bought systems are 64bit architectures... :-)

I feel sorry for all the unaware consumers being suckered out there
into buying two-year-old 32-bit machines.  I guess they do have to
sell them to somebody, though.

--
~Mike
 - Just my two cents
 - No man is an island, and no man is unable.