Less disk space with reiser4?

Less disk space with reiser4?

[Stefan Andersson]

Hi,

I have a disk array wich I used reiserfs(v3) on, and got 932G.
Now I started to test reiser4 on it and get less space, 886G.

Is it suppose to be like this or is there something wrong?

I'am using linux 2.6.11.4 and default settings for mkfs.reiser4 and 
mkfs.reiserfs,
and reiser4 patch from 2.6.11-mm3.

//Stetan

Re: Less disk space with reiser4?

[Łukasz Mierzwa]

Dnia Mon, 11 Apr 2005 16:30:53 +0200, Stefan Andersson <stefan@siwnet.net>  
napisa³:

> Hi,
>
> I have a disk array wich I used reiserfs(v3) on, and got 932G.
> Now I started to test reiser4 on it and get less space, 886G.
>
> Is it suppose to be like this or is there something wrong?
>
> I'am using linux 2.6.11.4 and default settings for mkfs.reiser4 and  
> mkfs.reiserfs,
> and reiser4 patch from 2.6.11-mm3.
>
> //Stetan
>

I got the same problem, I've send two email's to this list about it but  
there was no anwsers, I wish You better luck ;)

£ukasz Mierzwa

Re: Less disk space with reiser4?

[Alex Zarochentsev]

On Mon, Apr 11, 2005 at 04:30:53PM +0200, Stefan Andersson wrote:
> Hi,
> 
> I have a disk array wich I used reiserfs(v3) on, and got 932G.
> Now I started to test reiser4 on it and get less space, 886G.
> 
> Is it suppose to be like this or is there something wrong?

reiser4 reserves 5% of disk space.  

That is so because of delayed block allocation impelented in reiser4.  Actual
block allocation is done near the transaction commit, it means the allocation
has to be reliable -- we can't return ENOSPC because the write(2) call when we
could return an error was already completed successfully.  

The reserved space makes the allocation reliable because it covers possible
insertions (and growing of the internal reiser4 tree) of new extent items
insertions on a highly fragmented fs.

> I'am using linux 2.6.11.4 and default settings for mkfs.reiser4 and 
> mkfs.reiserfs,
> and reiser4 patch from 2.6.11-mm3.
> 
> //Stetan
> 

-- 
Alex.

Re: Less disk space with reiser4?

[Hans Reiser]

Alex Zarochentsev wrote:

>On Mon, Apr 11, 2005 at 04:30:53PM +0200, Stefan Andersson wrote:
>  
>
>>Hi,
>>
>>I have a disk array wich I used reiserfs(v3) on, and got 932G.
>>Now I started to test reiser4 on it and get less space, 886G.
>>
>>Is it suppose to be like this or is there something wrong?
>>    
>>
>
>reiser4 reserves 5% of disk space.  
>
>That is so because of delayed block allocation impelented in reiser4.  Actual
>block allocation is done near the transaction commit, it means the allocation
>has to be reliable -- we can't return ENOSPC because the write(2) call when we
>could return an error was already completed successfully.  
>
>The reserved space makes the allocation reliable because it covers possible
>insertions (and growing of the internal reiser4 tree) of new extent items
>insertions on a highly fragmented fs.
>  
>
It also improves performance to ensure that at least 5% is free.

>  
>
>>I'am using linux 2.6.11.4 and default settings for mkfs.reiser4 and 
>>mkfs.reiserfs,
>>and reiser4 patch from 2.6.11-mm3.
>>
>>//Stetan
>>
>>    
>>
>
>  
>

Re: Less disk space with reiser4?

[Chester R. Hosey]

Many filesystems claim degraded performance as free space decreases 
beyond a certain threshold. However, they often allow the user to exceed 
this point and accept the decline in performance.

While Alex's explanation does well to explain why a certain amount of 
free space is required in the case of a commit depending on a certain 
amount of free space, it doesn't quite show that 45 gigs are required to 
cover this requirement on a terabyte of space. Is this a reasonable 
requirement?

Can we get a FAQ entry on namesys.com which addresses this additional 
overhead imposed by V4 (including whether this is a permanent thing, or 
whether future revisions will likely reduce the overhead)?

Hans Reiser wrote:
 > Alex Zarochentsev wrote:
 >
 >
 >>On Mon, Apr 11, 2005 at 04:30:53PM +0200, Stefan Andersson wrote:
 >>
 >>
 >>
 >>>Hi,
 >>>
 >>>I have a disk array wich I used reiserfs(v3) on, and got 932G.
 >>>Now I started to test reiser4 on it and get less space, 886G.
 >>>
 >>>Is it suppose to be like this or is there something wrong?
 >>>
 >>>
 >>
 >>reiser4 reserves 5% of disk space.
 >>
 >>That is so because of delayed block allocation impelented in reiser4. 
  Actual
 >>block allocation is done near the transaction commit, it means the 
allocation
 >>has to be reliable -- we can't return ENOSPC because the write(2) 
call when we
 >>could return an error was already completed successfully.
 >>
 >>The reserved space makes the allocation reliable because it covers 
possible
 >>insertions (and growing of the internal reiser4 tree) of new extent items
 >>insertions on a highly fragmented fs.
 >>
 >>
 >
 > It also improves performance to ensure that at least 5% is free.
 >
 >
 >>
 >>
 >>
 >>>I'am using linux 2.6.11.4 and default settings for mkfs.reiser4 and
 >>>mkfs.reiserfs,
 >>>and reiser4 patch from 2.6.11-mm3.
 >>>
 >>>//Stetan
 >>>
 >>>
 >>>
 >>
 >>
 >>
 >
 >

Re: Less disk space with reiser4?

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Chester R. Hosey wrote:
> Many filesystems claim degraded performance as free space decreases
> beyond a certain threshold. However, they often allow the user to exceed
> this point and accept the decline in performance.
> 
> While Alex's explanation does well to explain why a certain amount of
> free space is required in the case of a commit depending on a certain
> amount of free space, it doesn't quite show that 45 gigs are required to
> cover this requirement on a terabyte of space. Is this a reasonable
> requirement?

Not in my book.  Let me second this.

It's not all that expensive to build a terabyte array.  It's much more
expensive to build a box with 45 gigs of RAM.  People who can afford to
run Reiser4 on a box with 45 gigs of RAM can afford to hire you guys to
patch it.

I propose that the limit should be configurable, and modifiable online
or offline with some sort of tunefs command.  In fact, I propose that
the default should be 5% or the total RAM at time of creation, whichever
is lower.

What if the user upgrades RAM?  Change this value on mount, if the space
is available, and create a mount option to override this behavior.

Performance concerns should be addressed by printing a warning in dmesg,
somewhat like "flushing like mad".  And most filesystems suffer with
more than 75% full, so it's better to educate the users than just steal
a quarter of their space (or even a twentieth).

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQlsc6HgHNmZLgCUhAQK0Zw/+PLX+CFKx0xPHXBZiyhjykbLcOk1YBfJz
tZ971xf3sE18hDbA3uEbuT6PkSvA5ihcICThk7wwUh9lM0HcN7epCmQqICRDBMl1
m5PJ/SlV4VEkbJEsevHmo1aFTY87c1WW9sk6gPWaHw10d7eGZqMMtuSXhHXOnJ3E
9/M4FZcJsUp/18Ur5e8SJvX5ach0LsYSnKL8vslv0uVaFCdr1lITGVLlztKh7j1n
/UUQiwuGCT1WvOrPjySEUm7Y+eeC0eXea763ll90ZjuhVGsaSCsorIVSmuAm06YO
VJC971hlkMOQJaYp7hduyiF8dskSHyQ4A76K+BArdo/tZdciRNlA/DAE3uG5GFKY
xNxWit5fv4wm96f/X1A1rIPhxOyWK4teP2n9G/1UK+5+uYgLTUqtG8tGZ1WNDysz
na5HgY2MwKXJQwlvLnABCDn4sFZV2amNd/r2//tcSZp6lM2gvXmoo4Velg4YMr2G
TnnlLNf96LOoHQFdZvaDyeXxkFO8WaHbDSA7ffhXbYLkSfNpd8eY/ocGOO8YF9Jn
mRmjecG4iveWg9y7I7DpQ5ofXEGVTYjH2d2Z+UnjN0D7mgm7x454rMgsjrLqHwL+
lsi116nLctIT63e3F9Eq3UQ/fI+QvfiyXXrDHec1oGBl3LFO8OuSGLsRr7GBbHfz
kY4pgD+GIJU=
=VhOT
-----END PGP SIGNATURE-----

Re: Less disk space with reiser4?

[Stefan Andersson]

Thanks for all the answers.

Now would it be possible to change the 5% to something else?
I have been looking at the code but havent been able to figure out what 
it is I need to change to test it.
So far I have only looked at the kernel code, not the reiser4progs or 
libaal.

//Stefan

Re: Less disk space with reiser4?

[Alex Zarochentsev]

On Wed, Apr 13, 2005 at 02:51:13PM +0200, Stefan Andersson wrote:
> Thanks for all the answers.
> 
> Now would it be possible to change the 5% to something else?

even if the fs would be unreliable?

> I have been looking at the code but havent been able to figure out what 
> it is I need to change to test it.
> So far I have only looked at the kernel code, not the reiser4progs or 
> libaal.
> 
> //Stefan
> 

-- 
Alex.

Re: Less disk space with reiser4?

[Stefan Andersson]

Alex Zarochentsev wrote:

>On Wed, Apr 13, 2005 at 02:51:13PM +0200, Stefan Andersson wrote:
>  
>
>>Thanks for all the answers.
>>
>>Now would it be possible to change the 5% to something else?
>>    
>>
>
>even if the fs would be unreliable?
>  
>
That is what I want to test, or if someone knows the answer to that.

>  
>
>>I have been looking at the code but havent been able to figure out what 
>>it is I need to change to test it.
>>So far I have only looked at the kernel code, not the reiser4progs or 
>>libaal.
>>
>>//Stefan
>>
>>    
>>
>
>  
>

Re: Less disk space with reiser4?

[Perry Kundert]

On 4/13/05, Stefan Andersson <stefan@siwnet.net> wrote:
> Alex Zarochentsev wrote:
> 
> >On Wed, Apr 13, 2005 at 02:51:13PM +0200, Stefan Andersson wrote:
> >
> >
> >>Thanks for all the answers.
> >>
> >>Now would it be possible to change the 5% to something else?
> >
> >even if the fs would be unreliable?
> >
> That is what I want to test, or if someone knows the answer to that.
> 

    I think claiming that the FS will become unreliable by choosing
some smaller amount of unused space, or making it a factor of total
RAM size instead of total File System size is a bit extreme...

    There must be some theoretical maximum limit on the number of
dirty blocks that can possibly be outstanding -- and I suspect that it
is more related to the total amount of RAM available, and NOT related
to the total size of the filesystem.  Proof:

    1) The number of dirty blocks is NOT limited to the RAM available
    2) That means that there exists a dirty block X, such that X is
considered dirty, but is not stored in available RAM
    3) Reductio as Absurdium.  Ergo, dirty blocks are limited to the
size of RAM.

    Now, there may be further constraints -- a contiguous segment of
sectors must be available (unlikely), some dirty blocks may require
more than one block on disk be available (unlikely), or some dirty
blocks in RAM may implicitly dirty one or more (additional) blocks in
the filesystem.

    Stilll, the "unused space" requirement would be constrained to
some multiple of RAM, NOT some percentage of the filesystem's size...

    Just a guess; sorry if I'm missing something obvious...

--
    -pjk


"If you will not fight when your victory will be sure and not too
costly, you may come to the moment when you will have to fight with
all the odds against you and only a precarious chance for survival.
There may even be a worse case. You may have to fight when there is no
hope of victory, because it is better to perish than to live as
slaves." -- Winston Churchill

Re: Less disk space with reiser4?

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stefan Andersson wrote:
> Alex Zarochentsev wrote:
> 
>> On Wed, Apr 13, 2005 at 02:51:13PM +0200, Stefan Andersson wrote:
>>  
>>
>>> Thanks for all the answers.
>>>
>>> Now would it be possible to change the 5% to something else?
>>>   
>>
>>
>> even if the fs would be unreliable?
>>  
>>
> That is what I want to test, or if someone knows the answer to that.

The only indication I've heard that it would make the FS unreliable is
if you had less space reserved than you had RAM available to stack up
writes which haven't been allocated.  In your case, it's a bug that you
haven't been able to set this on your own.

Note that I said unreliable, not slow.  Everyone keeps telling us that
you'll get slow with <25% space available, so the 5% is really just a
sanity check, but you're probably fine there, too.

My question to the Namesys guys is, have you actually tested situations
like this?

If you want to artificially create that situation, you can probably just
shrink the values down a bunch -- make an fs on a 1 gig drive on a
virtual machine (qemu + accelerator) with only a couple megs of RAM
available for cache/buffer, then try to fill it.  How close can you get
to 100% full before something blows up?

I'd do it, but I've got problems with school and bad RAM.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQl3y/3gHNmZLgCUhAQKoOQ/9GsES1Wvv6doTuM/f+GX5sEA4z92VkR2s
2LGgnGuuRSKyFoR5UWkPrZt7mpnNVHtD1w0sdXdf6H+p5K4R4bkO0qmodlGyrFIx
OSWmFkyFNjbmftXQPYlWCsYNmHTySrbb60Yt03cQJ6wDy3lmVz2LPDAKXnwWxYpu
mCY3js+DteaVSajzvZMxW4dIAGRcjJn8d/MS0tF5Tkj+621/8HvdQvCK8Xijqfqo
Wdt4X3A7A9npewPFC95jWp/f6RRbS73+7pcWRCV6JCJ99yLW3fvXAn23HrExFSo6
9DcISawlDnS6Wg1QGOgDUN1zgbXccupL4DCbs4FbFh54Dq6GvxR2MosFatdNwjSw
3u3EKO711pc/TplYlCjylUN+ORrsOtHfL3Psz4Yez4LWB427YtMflmv7ONF2IUlU
2Gvssvi5Dtv0P0mLISAgxFlT0i8NR8QMxK8zBTJ9p9e0KYKDmdXEGAVcQXfWllUh
3ufVOr/9Yf+5g4SjxDot1TpAGhImaPbSnNRiWK8dGEiqUU1/Ies/7uPZNLgK0XZL
7D5+++Pu6eoYzu1EZyc8CxRtm3xbunXXaRkdqeNL1TGo36uAlcigHdpOE1BUk8gG
wozu3vWeHDwhWocf2sK/pmixejdxqYIO9zTkbOremFn2R3cbOHcCK9UsG7cd2khd
08CrnNc/xng=
=r8e3
-----END PGP SIGNATURE-----

Re: Less disk space with reiser4?

[Alex Zarochentsev]

On Wed, Apr 13, 2005 at 11:35:12PM -0500, David Masover wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Stefan Andersson wrote:
> > Alex Zarochentsev wrote:
> > 
> >> On Wed, Apr 13, 2005 at 02:51:13PM +0200, Stefan Andersson wrote:
> >>  
> >>
> >>> Thanks for all the answers.
> >>>
> >>> Now would it be possible to change the 5% to something else?
> >>>   
> >>
> >>
> >> even if the fs would be unreliable?
> >>  
> >>
> > That is what I want to test, or if someone knows the answer to that.
> 
> The only indication I've heard that it would make the FS unreliable is
> if you had less space reserved than you had RAM available to stack up
> writes which haven't been allocated.  In your case, it's a bug that you
> haven't been able to set this on your own.
> 
> Note that I said unreliable, not slow.  Everyone keeps telling us that
> you'll get slow with <25% space available, so the 5% is really just a
> sanity check, but you're probably fine there, too.
> 
> My question to the Namesys guys is, have you actually tested situations
> like this?

no, it the 5% is our rough estimation. it would never reached.  It is like
extends of all files address only 1 block each. 

> If you want to artificially create that situation, you can probably just
> shrink the values down a bunch -- make an fs on a 1 gig drive on a
> virtual machine (qemu + accelerator) with only a couple megs of RAM
> available for cache/buffer, then try to fill it.  How close can you get
> to 100% full before something blows up?

Suppose the index part of the tree would take only 0.5%. And what?
Any successful test will not prove that the new estimation is correct
(reliable).

We need to re-design the estimation, make it "adaptive", for example,
to be aware of well-allocated files.  Or we have to change
the current disk space reservation scheme completely.

-- 
Alex.