When is a defragmenting tool expected to be released?

When is a defragmenting tool expected to be released?

[Anders Widman]

   I  am  curious  about  if/when  a  defragmenting  tool that support
   reiserfs  will  be  available.  Please  do  not  say  to  tar/untar
   everything. I simply do not have enough storage for that.

   //Anders


--------
PGP public key: https://tnonline.net/secure/pgp_key.txt

Re: When is a defragmenting tool expected to be released?

[_nasturtium]

On Sat, 26 Apr 2003 05:35 am, Anders Widman wrote:
>    I  am  curious  about  if/when  a  defragmenting  tool that support
>    reiserfs  will  be  available.  Please  do  not  say  to  tar/untar
>    everything. I simply do not have enough storage for that.

No defragmenting tools that I know of, or mentioned on the list.

You might like to check out what Hans Reiser wrote at 
http://www.osnews.com/story.php?news_id=69. 
Basically, reiser4 will have delayed allocation, like XFS, rather than the 
current ext2-like system of preallocating blocks, so it'll be a lot more 
fragmentation-resistant.

However, if you just want to check fragmentation there's a program called 
mongo, it's a benchmarking package and you can see if your filesystem (or 
even individual files) are fragmented.

Regards,
	_nasturtium

Re: When is a defragmenting tool expected to be released?

[Fred -- Speed Up --]

In the article you pointed out I could read this :



3. What is the fragmentation policy of ReiserFS and how is it dealing with
it?

Hans Reiser: This is an area we are still experimenting with. We currently
do what ext2 does, and preallocate blocks. What XFS does is much better,
they allocate blocknrs to blocks at the time they are flushed to disk, and
this allows a much more efficient and optimal allocation to occur. We knew
we couldn't do it the XFS way and make code freeze for 2.4, but reiser4 is
being built around delayed allocation, and I'd like to thank the XFS
developers for taking the time to personally explain to me why delayed
allocation is the way to go.



Hans said Reiser4 would use data preallocate blocks as ext2 does, and the
Reiser4 feature paper is speaking about a "repacker" that should be fully
functionnal at 4.1 release. How do you feel about that ?

Fred

----- Original Message ----- 
From: "_nasturtium" <_nasturtium@bigpond.com>
To: "Anders Widman" <andewid@tnonline.net>; <reiserfs-list@namesys.com>
Sent: Saturday, April 26, 2003 6:14 AM
Subject: Re: When is a defragmenting tool expected to be released?


> On Sat, 26 Apr 2003 05:35 am, Anders Widman wrote:
> >    I  am  curious  about  if/when  a  defragmenting  tool that support
> >    reiserfs  will  be  available.  Please  do  not  say  to  tar/untar
> >    everything. I simply do not have enough storage for that.
>
> No defragmenting tools that I know of, or mentioned on the list.
>
> You might like to check out what Hans Reiser wrote at
> http://www.osnews.com/story.php?news_id=69.
> Basically, reiser4 will have delayed allocation, like XFS, rather than the
> current ext2-like system of preallocating blocks, so it'll be a lot more
> fragmentation-resistant.
>
> However, if you just want to check fragmentation there's a program called
> mongo, it's a benchmarking package and you can see if your filesystem (or
> even individual files) are fragmented.
>
> Regards,
> _nasturtium
>

Re: When is a defragmenting tool expected to be released?

[Oleg Drokin]

Hello!

On Sat, Apr 26, 2003 at 08:14:47AM +0200, Fred -- Speed Up -- wrote:

> Hans said Reiser4 would use data preallocate blocks as ext2 does, and the
> Reiser4 feature paper is speaking about a "repacker" that should be fully
> functionnal at 4.1 release. How do you feel about that ?

reiser4 has delaying allocation implemented.
The "preallocate blocks as ext2 does" reference is reiserfs (v3) thing
(there is no 2.4 version of reiser4)

Bye,
    Oleg

Re: When is a defragmenting tool expected to be released?

[Fred -- Speed Up --]

Why won't there be a 2.4 version of Reiser4 ? I knew Reiser4 would be
available for v2.5, but it is intended to be released on september (you know
better than I, if you could tell me when Reiser4 would be released it would
be nice :-D) and the 2.6 kernel should release at the end of 2003, maybe
first quarter of 2004 : why would you release a stable Reiser4 code only on
2.5 kernels, as 2.4 versions may be used for at least one year before major
distros include the 2.6 version.
Is Reiser4 intended to replace Reiser3 (I've read Reiser4 mounter may be
able to mount Reiser3 volumes), or will Reiser3 be kept in 2.2 and 2.4
kernels, as Reiser4 would be 2.5 (and 2.6 ;)) dedicated ?
There are probably necessary features issues (the new VFS maybe), and mostly
performance issues to that would make you release Reiser4 for 2.5 kernels,
but why did you choose to make Reiser4 2.5-only compliant ?
Why aren't there news on the Namesys website ? Maybe cvs logs, or whatever
completion percentage, implemented features, stable ones, maybe some new
performance tests (there are no tests comparing Reiser4, XFS and JFS anymore
on your website) ?
You seem to be a bit late about the kernel experimental release : the latest
one is 2.5.68, and your patches are only available up to 2.5.60. When do you
decide to move on to a newer experimental kernel version ?
May Reiser4 be included in the -ac series ?


While you're up, I've got some questions to ask you (you don't have to
answer all of them ...), as I'm very interersted in your filesystem, and I'm
writing a french doc about Reiser4 :

- Firstly the trees. I read that the it (the storage one, not the semantic
side) grew on top, which makes the key grow in lenght. But what about fanout
? When a file has been deleted, can its former key be freed and reused so
that the tree keeps balanced ? Does Reiser4 otpimize the tree by storing
small files together in a part of the tree, or directories, or whatever
organisation : when a file is being written, does Reiser4 give it the first
key it finds, or does a segragation exist in order to improve performance ?
For instance, small temp files are created and deleted numbers of times,
does a special part of the tree keep those temp files so that only a little
part of the tree is constantly changing ?
- What do you call a 'graph' when speaking about the structure the semantic
layer uses to resolve paths ? How does this part really work (it is not
formally spcified in the doc) ?
- How about folders : what status do they have ? They may be stored as other
files, as they have to keep their own properties, but their information (the
folder's files and subfolder list) is being kept in the semantic layer's
graph : how do you handle with this ?
- Why do you need to store the locality_id in the key ?
- How do you handle with big files wich do not fit in contiguous bloc space,
so they need more than one extended pointer ? How are those other pointers
stored ?
- B+Trees are simply BTrees that do not use BLOBs, am I right ?
- Dancing trees are simply Balanced Trees wich are only modified in an event
of memory pressure, don't they ?
- How is the developement ? I mean, should we await the Reiser 4.0 release
this summer, or do you need some more time ? What features will be included
in 4.0 and which will be left for 4.1 ? Will the packer be ready for release
as Reiser 4.0 comes out ?


Sorry if I'm bugging you, but as you're Release Manager I'm pretty sure
you're the right person to ask ;)
Thank all the Reiser team for the great job you do, and for releasing such
huge documentation about the project without the help of which I wouldn't be
able to understand what's going on ;)

Fred

----- Original Message ----- 
From: "Oleg Drokin" <green@namesys.com>
To: "Fred -- Speed Up --" <speedup@free.fr>
Cc: <reiserfs-list@namesys.com>
Sent: Saturday, April 26, 2003 9:30 AM
Subject: Re: When is a defragmenting tool expected to be released?


> Hello!
>
> On Sat, Apr 26, 2003 at 08:14:47AM +0200, Fred -- Speed Up -- wrote:
>
> > Hans said Reiser4 would use data preallocate blocks as ext2 does, and
the
> > Reiser4 feature paper is speaking about a "repacker" that should be
fully
> > functionnal at 4.1 release. How do you feel about that ?
>
> reiser4 has delaying allocation implemented.
> The "preallocate blocks as ext2 does" reference is reiserfs (v3) thing
> (there is no 2.4 version of reiser4)
>
> Bye,
>     Oleg

Re: When is a defragmenting tool expected to be released?

[Anders Widman]

> On Sat, 26 Apr 2003 05:35 am, Anders Widman wrote:
>>    I  am  curious  about  if/when  a  defragmenting  tool that support
>>    reiserfs  will  be  available.  Please  do  not  say  to  tar/untar
>>    everything. I simply do not have enough storage for that.

> No defragmenting tools that I know of, or mentioned on the list.

> However, if you just want to check fragmentation there's a program called
> mongo, it's a benchmarking package and you can see if your filesystem (or 
> even individual files) are fragmented.

Thanks,  I  will  look  it  up.  I  know  many  files  are indeed very
fragmented,  and  if I can find out which I might be able to fix a few
manually...

All  I  wanted  to  know  what  if  there  would  be a tool that could
defragment files. There is no way to avoid fragmentation no matter how
smart your filesystem is.

Re: When is a defragmenting tool expected to be released?

[Oleg Drokin]

Hello!

On Sat, Apr 26, 2003 at 09:56:05AM +0200, Fred -- Speed Up -- wrote:
> Why won't there be a 2.4 version of Reiser4 ? I knew Reiser4 would be

All the development is happen on 2.5 kernels.
Changes between 2.4 and 2.5 are big.
So while I cannot say there will never be 2.4 backport, I think it may be not
that easy, anyway. On the other hand, now that OS-dependent layer is clearly abstracted,
I well might be wrong ;)

> available for v2.5, but it is intended to be released on september (you know
> better than I, if you could tell me when Reiser4 would be released it would
> be nice :-D) and the 2.6 kernel should release at the end of 2003, maybe
> first quarter of 2004 : why would you release a stable Reiser4 code only on
> 2.5 kernels, as 2.4 versions may be used for at least one year before major
> distros include the 2.6 version.

The decision was made by Hans. Perhaps he did not want to be limited with 2.4
limitations at the beginning ;)

> Is Reiser4 intended to replace Reiser3 (I've read Reiser4 mounter may be

Yes, we hope that will happen eventually. (I mean the userbase will migrate
to reiser4, the reiserfs v3 will stay in kernel for a long time. It is stable
already, and reiser4 will take some time to stabilize).

> able to mount Reiser3 volumes), or will Reiser3 be kept in 2.2 and 2.4
> kernels, as Reiser4 would be 2.5 (and 2.6 ;)) dedicated ?

We think it is possible to make reiserfs v3 disk format plugin for reiser4,
but still this is not a reason to kill another piece of code from 2.5/2.6
kernels. Not in nearest 5 years, I'd say ;)

> There are probably necessary features issues (the new VFS maybe), and mostly
> performance issues to that would make you release Reiser4 for 2.5 kernels,
> but why did you choose to make Reiser4 2.5-only compliant ?

That's where our development is. Developing for two different kernels is
not very wise thing. Backport might happen later if we'd think this is necessary.

> Why aren't there news on the Namesys website ? Maybe cvs logs, or whatever

Because there no more exciting news for now.
We have a public bitkeeper repository and everyone may lookup the logs
( bk://namesys.com/bk/reiser4 )

> completion percentage, implemented features, stable ones, maybe some new
> performance tests (there are no tests comparing Reiser4, XFS and JFS anymore
> on your website) ?

We are playing with disk allocation algorithms, so if we'd publish benchmark
results now, they won't be of much value as the algorithm (and results) change
on a daily basis.

> You seem to be a bit late about the kernel experimental release : the latest
> one is 2.5.68, and your patches are only available up to 2.5.60. When do you
> decide to move on to a newer experimental kernel version ?

We track kernel releases very closely internally. We have not released any snapshots
recently because we have some stability problems that are not fixed yet.
But our repositories are publically available for those interested.

> May Reiser4 be included in the -ac series ?

It is not ready yet. But when it is ready, we well may submit it to Alan first, just
for testing purposes ;)

> While you're up, I've got some questions to ask you (you don't have to
> answer all of them ...), as I'm very interersted in your filesystem, and I'm
> writing a french doc about Reiser4 :

I think it is better to get key reiser4 developers to answer those questions.
That may happen on Monday (or if Hans will appear in Moscow, then may be even earlier).

Bye,
    Oleg

Re: When is a defragmenting tool expected to be released?

[Hans Reiser]

Fred -- Speed Up -- wrote:

>Why won't there be a 2.4 version of Reiser4 ? I knew Reiser4 would be
>available for v2.5, but it is intended to be released on september (you know
>better than I, if you could tell me when Reiser4 would be released it would
>be nice :-D) and the 2.6 kernel should release at the end of 2003, maybe
>first quarter of 2004 
>
Is this the latest guess for 2.6?  Do you have a URL for it?

>: why would you release a stable Reiser4 code only on
>2.5 kernels, as 2.4 versions may be used for at least one year before major
>distros include the 2.6 version.
>
Ask us again when Reiser4 is stable;-).

>Is Reiser4 intended to replace Reiser3 (I've read Reiser4 mounter may be
>able to mount Reiser3 volumes), or will Reiser3 be kept in 2.2 and 2.4
>kernels, as Reiser4 would be 2.5 (and 2.6 ;)) dedicated ?
>

I expect reiser3 to be in Linux kernel for 10+ more years.

>There are probably necessary features issues (the new VFS maybe), and mostly
>performance issues to that would make you release Reiser4 for 2.5 kernels,
>but why did you choose to make Reiser4 2.5-only compliant ?
>
Limited staff, the need to focus on making it work for something....

>Why aren't there news on the Namesys website ? Maybe cvs logs, or whatever
>completion percentage, implemented features, stable ones, maybe some new
>performance tests (there are no tests comparing Reiser4, XFS and JFS anymore
>on your website) ?
>You seem to be a bit late about the kernel experimental release : the latest
>one is 2.5.68, and your patches are only available up to 2.5.60. When do you
>decide to move on to a newer experimental kernel version ?
>
Oleg?

>May Reiser4 be included in the -ac series ?
>
I think we need to get reiser4 stable first, then worry about what 
kernels it is in.  Eventually, yes, I would hope so....

>
>
>While you're up, I've got some questions to ask you (you don't have to
>answer all of them ...), as I'm very interersted in your filesystem, and I'm
>writing a french doc about Reiser4 :
>
>- Firstly the trees. I read that the it (the storage one, not the semantic
>side) grew on top, which makes the key grow in lenght.
>
I'll answer the below later.  Remind me if I forget....

> But what about fanout
>? When a file has been deleted, can its former key be freed and reused so
>that the tree keeps balanced ? Does Reiser4 otpimize the tree by storing
>small files together in a part of the tree, or directories, or whatever
>organisation : when a file is being written, does Reiser4 give it the first
>key it finds, or does a segragation exist in order to improve performance ?
>For instance, small temp files are created and deleted numbers of times,
>does a special part of the tree keep those temp files so that only a little
>part of the tree is constantly changing ?
>- What do you call a 'graph' when speaking about the structure the semantic
>layer uses to resolve paths ? How does this part really work (it is not
>formally spcified in the doc) ?
>- How about folders : what status do they have ? They may be stored as other
>files, as they have to keep their own properties, but their information (the
>folder's files and subfolder list) is being kept in the semantic layer's
>graph : how do you handle with this ?
>- Why do you need to store the locality_id in the key ?
>- How do you handle with big files wich do not fit in contiguous bloc space,
>so they need more than one extended pointer ? How are those other pointers
>stored ?
>- B+Trees are simply BTrees that do not use BLOBs, am I right ?
>- Dancing trees are simply Balanced Trees wich are only modified in an event
>of memory pressure, don't they ?
>- How is the developement ? I mean, should we await the Reiser 4.0 release
>this summer, or do you need some more time ? What features will be included
>in 4.0 and which will be left for 4.1 ? Will the packer be ready for release
>as Reiser 4.0 comes out ?
>
>
>Sorry if I'm bugging you, but as you're Release Manager I'm pretty sure
>you're the right person to ask ;)
>Thank all the Reiser team for the great job you do, and for releasing such
>huge documentation about the project without the help of which I wouldn't be
>able to understand what's going on ;)
>
>Fred
>
>----- Original Message ----- 
>From: "Oleg Drokin" <green@namesys.com>
>To: "Fred -- Speed Up --" <speedup@free.fr>
>Cc: <reiserfs-list@namesys.com>
>Sent: Saturday, April 26, 2003 9:30 AM
>Subject: Re: When is a defragmenting tool expected to be released?
>
>
>  
>
>>Hello!
>>
>>On Sat, Apr 26, 2003 at 08:14:47AM +0200, Fred -- Speed Up -- wrote:
>>
>>    
>>
>>>Hans said Reiser4 would use data preallocate blocks as ext2 does, and
>>>      
>>>
>the
>  
>
>>>Reiser4 feature paper is speaking about a "repacker" that should be
>>>      
>>>
>fully
>  
>
>>>functionnal at 4.1 release. How do you feel about that ?
>>>      
>>>
>>reiser4 has delaying allocation implemented.
>>The "preallocate blocks as ext2 does" reference is reiserfs (v3) thing
>>(there is no 2.4 version of reiser4)
>>
>>Bye,
>>    Oleg
>>    
>>
>
>
>
>  
>


-- 
Hans