Re: Benchmark : ext3 vs reiser4 and effects of fragmentation.

[Hans Reiser <reiser@namesys.com>]

Well, I am going to try being honest and see what happens.

I am more than 170k in debt, and Namesys is doing badly fiscally.  A 
technical great success being stabilized now, but then there is my 
ongoing fiscal disaster.  Once again, we are missing payroll.  My wife 
is divorcing me in part because I keep going deeper into debt, and I 
thank her for divorcing me now rather than later.  Unfortunately she is 
making the divorce messy enough to keep me from pulling Namesys out of 
the fiscal tailspin by consuming all my time with things like proving I 
am not making the fantastic amounts of money she claims I am.  I hope 
next month is better.

The consistent tendency of large corporations to buy service contracts 
only for proprietary software and not buy it for free software of equal 
importance to their business is decisive in making our business not 
viable fiscally.  Knowing that it is irrational of them economically 
does not keep me in business.

So we need something to entice them into cutting a check, to which they 
can then add support.  At the same time we want to avoid harming users 
who cannot pay, or who don't care enough about the FS to bother to pay.

The repacker/resizer seems perfect for that.  Nobody has to have it, and 
yet sysadmins of large corporations will pay for it.  I want them to pay 
5% of their storage hardware costs, which means disk drives plus raid 
cards, and if the machine is primarily a file server it means the whole 
computer.  For this they get a resizer, a repacker, and priority 
support, with cell phone numbers of developers if they pay more than $1000.

I keep experimenting with open source business models, trying to make 
them work.  One of the possibilities for us is a blended model, in which 
all the essentials are free, and the fripperies are for a fee.

There is nothing less essential than an online repacker/resizer.  Yet it 
is desirable enough to pay for.

Reiser4 without an online repacker/resizer is better than ext3 by a 
lot.  The money from the repacker will help us increase that lead 
further.  An online  repacker/resizer could pay for the semantic 
enhancements that are really important to users, and could allow us to 
make those free.    I am worried that because of finances we are not 
going to beat WinFS to market when we otherwise could have done so.

Money is like food.  The difference between a fine restaurant, and just 
having a full refridgerator with ordinary vegetables and meat, is not a 
lot.  The difference between a full refridgerator and an empty one is a lot.

Charity is best if made a part time job of, not because men need money, 
but because families need money, and men need to fulfill that obligation 
well enough that the refridgerator is full.

Hans

PS

We need to have a better understanding of why reiser4 is not doing a 
good enough job with just allocation on flush.  I suspect it should be 
doing better.  Next month I hope to be able to take the time to look 
into it.  I am spending all of my time dealing with divorce and money 
issues and the code is just going unsupervised, sigh....

Will Smith wrote:

>
> Markus Törnqvist wrote:
>
>>> yes, we had similar results.  it is so cool that reiser4 repacker 
>>> effect
>>> is visible outside namesys :)
>>
>>
>> S'cuse me, but why does it have to be run several times?
>
>
> I don't pretend to know the maths/logic of why, but from the reiser4 
> description (http://www.namesys.com/v4/v4.html#repacker)
>
> "This repacker goes through the entire tree ordering, from left to 
> right and then from right to left, alternating each time it runs."
>
> "In the absence of FS activity the effect of this over time is to sort 
> by tree order (defragment), and to pack with perfect efficiency."
>
>
> My testing showed that this is indeed the case -
>
> If I ran the repacker once (say left to right), the fragmentation was 
> not fully reduced.
>
> If I ran the repacker several times (left->right, right->left, and
> repeat) then fragmentation fell down to almost 0.
>
>
> Userspace Tool for Repacker
> ----------------------------
>
> I take it from an earlier comment by Hans that the repacker is not
> intended to be used yet?  If it is indeed ready for primetime,
> is anybody working on userspace tools
> (/etc/cron.daily/repacker.reiser4 or similar?)  I'd be happy to
> write these if nobody else is - but it will be in /bin/bash shell.
>
> My proposed design specs would include:
>
> 1) runs in /etc/cron.daily near the end (since cron causes lots of
>    disk activity)
> 2) the last-run-direction should be persistently stored per filesystem
> 3) parallel operation on multiple filesystems where possible,
>    but filesystems on a single physical device should
>    run in serial (same as /sbin/fsck -A)
> 4) some kind of config file in /etc/, parameters could include
>     o) maximum run time, global and per filesystem
>     o) run/dont run flag, global and per filesystem
>
>
> Any thoughts?  Shall I go ahead and write a first cut?
>
>
>
> Will Smith
>
>
>
>
>
>
>