reiser4 resize

reiser4 resize

[Jack Byer]

Short summary: Will a resize program for reiser4 be available within the
next six months?

Long explanation:

I use a 3ware SATA raid card for the main storage on my home network. I
currently have 8 250 GB drives in a raid 5 + hot spare configuration. I
 chose this card because it allows for online capacity expansion. My
plan was to wait until a few generations of hard drives emerged then
upgrade them one at a time in cycles. This way my storage will
periodically expand without any major downtime.

When I first created the filesystem, there was a reiser4 resize program.
This is no longer the case.

I've began my next upgrade cycle with the purchase of two 750 GB drives.
I plan to buy one each month until all eight are replaced. Now I need to
make a decision. Do I backup my raid onto the new drives and reformat my
raid with another filesystem (xfs, reiser3, jfs), or do I put these new
drives into the array and assume that when the time comes to resize the
filesystem that a working program will exist?

Re: reiser4 resize

[Vladimir V. Saveliev]

Hello

On Tuesday 19 September 2006 05:12, Jack Byer wrote:
> Short summary: Will a resize program for reiser4 be available within the
> next six months?
> 

Currently nobody works on that. So, I guess it is not very likely that reiser4.resize will be created within next six months.

> Long explanation:
> 
> I use a 3ware SATA raid card for the main storage on my home network. I
> currently have 8 250 GB drives in a raid 5 + hot spare configuration. I
>  chose this card because it allows for online capacity expansion. My
> plan was to wait until a few generations of hard drives emerged then
> upgrade them one at a time in cycles. This way my storage will
> periodically expand without any major downtime.
> 
> When I first created the filesystem, there was a reiser4 resize program.
> This is no longer the case.

that was not a working program.

> 
> I've began my next upgrade cycle with the purchase of two 750 GB drives.
> I plan to buy one each month until all eight are replaced. Now I need to
> make a decision. Do I backup my raid onto the new drives and reformat my
> raid with another filesystem (xfs, reiser3, jfs), or do I put these new
> drives into the array and assume that when the time comes to resize the
> filesystem that a working program will exist?
> 
I think you should change to a filesystem which has resize.

Re: reiser4 resize

[David Masover]

Vladimir V. Saveliev wrote:
> Hello
> 
> On Tuesday 19 September 2006 05:12, Jack Byer wrote:
>> Short summary: Will a resize program for reiser4 be available within the
>> next six months?
>>
> 
> Currently nobody works on that. So, I guess it is not very likely that reiser4.resize will be created within next six months.

Not even an expand?  I know a shrink depends on a working repacker (even 
an offline one), but I'd think expanding it would be simple enough, so 
long as there's a big warning of "You cannot undo this (can't shrink)!"

>> When I first created the filesystem, there was a reiser4 resize program.
>> This is no longer the case.
> 
> that was not a working program.

Yes, I remember that, it was a stub.

> I think you should change to a filesystem which has resize.

Alternately, how much would it cost to implement basic resizefs.reiser4?

There are other reasons that make me wish I'd stayed away from reiser4 
for awhile.  Mainly, right now, I need a repacker, and the system seems 
to have become absurdly slow when it's fragmented.  When I have over a 
gig of RAM free (not even buffer/cache, but _free_), and am trying to 
download anything over BitTorrent, even if it's less than 200 megs, the 
disk thrashes so badly that the system is really only usable for web and 
email.  Even movies will occasionally stall when this is happening, and 
by "occasionally", I mean every minute or so.

I believe there was a patch to address the thrashing, so I'm eagerly 
awaiting 2.6.18, but the lack of a repacker bothers me.

Re: reiser4 resize

[Jack Byer]

> that was not a working program.
> 
I never looked too far into that issue, because I didn't need it back then.

> I think you should change to a filesystem which has resize.
> 

I guess this means that I'll won't use reiser4 again until 2 TB drives
come out and I upgrade. Maybe by then reiser4 will be included in the
kernel and files-as-directory will work :)

Re: reiser4 resize

[Alexey Polyakov]

On 9/19/06, David Masover <ninja@slaphack.com> wrote:

> When I have over a
> gig of RAM free (not even buffer/cache, but _free_), and am trying to
> download anything over BitTorrent, even if it's less than 200 megs, the
> disk thrashes so badly that the system is really only usable for web and
> email.  Even movies will occasionally stall when this is happening, and
> by "occasionally", I mean every minute or so.

Do you have this problem on plain vanilla + reiser4? I think some
out-of-tree patches (like adaptive readahead?) may result in such
behaviour, independent of file system.
With a free gig of RAM reiser4 shouldn't access disk a lot, only during flushes.

-- 
Alexey Polyakov

Re: reiser4 resize

[Alexey Polyakov]

On 9/20/06, £ukasz Mierzwa <prymitive@pcserwis.net> wrote:

> It's been proven that flushes are doing much more job then they should.
> Not so long ago someone send a trace of block device io accesess during
> reiser4 work and someone anylized it and said that some files or parts of
> file where written over and over 200 times or so.  Bittottent downloads on
> a few months old filesystem that had been used often just shows a week
> spot in reiser4, while downloading files with azureus with only 64KB of
> data per second I got disk lid on almost all the time, swithcing to
> rtorrent helped as it does not seem to call fsync ( I think I disabled
> fsync in azureus).

Ah, I see, if bittorrent calls fsync often, it's no wonder that
reiser4 behaves badly. I had to preload libnosync for some of my
programs that do fsync to avoid this.

-- 
Alexey Polyakov

Re: reiser4 resize

[David Masover]

Alexey Polyakov wrote:
> On 9/20/06, £ukasz Mierzwa <prymitive@pcserwis.net> wrote:
> 
>> It's been proven that flushes are doing much more job then they should.
>> Not so long ago someone send a trace of block device io accesess during
>> reiser4 work and someone anylized it and said that some files or parts of
>> file where written over and over 200 times or so.

Wow.  I should go back and read that -- I assume this is being worked on?

>> a few months old filesystem that had been used often just shows a week
>> spot in reiser4, while downloading files with azureus with only 64KB of
>> data per second I got disk lid on almost all the time, swithcing to
>> rtorrent helped as it does not seem to call fsync ( I think I disabled
>> fsync in azureus).

Hmm, strange.  I am using Azureus, but I don't think it's fsync.  I can 
try rtorrent, but there are several things I like about Azureus that 
nothing else seems to do yet.

But also, Azureus didn't always do this.  In fact, I used it for several 
months before I started having this problem.

> Ah, I see, if bittorrent calls fsync often, it's no wonder that
> reiser4 behaves badly. I had to preload libnosync for some of my
> programs that do fsync to avoid this.

Way ahead of you.  I noticed how much fsync performance sucked when 
using vim, and I was sick of waiting 10 seconds every time I hit :w -- a 
LOT of stuff can pile up in 2 gigs of disk buffer, and at the time, 
Reiser4 fsync effectively just called sync.

I didn't know about libnosync (or it didn't exist yet, or didn't work, 
I'm not entirely sure), so I was faced with either patching vim, which 
had just been patched to _add_ fsync'ing, not to mention all the other 
programs that might fsync too much;  patching glibc (huge, I don't 
update it often, and I'd have no idea where to start);  or patching the 
kernel.

I now keep backups, and I maintain and apply the following (STUPID, 
DON'T TRY THIS AT HOME) patch to my kernel:

--- linux/fs/buffer.c   2006-08-15 20:40:36.504608696 -0500
+++ linux/fs/buffer.c.new       2006-08-15 20:42:35.877461264 -0500
@@ -366,12 +366,12 @@

  asmlinkage long sys_fsync(unsigned int fd)
  {
-       return __do_fsync(fd, 0);
+       return 0;
  }

  asmlinkage long sys_fdatasync(unsigned int fd)
  {
-       return __do_fsync(fd, 1);
+       return 0;
  }

  /*

Re: reiser4 resize

[David Masover]

Alexey Polyakov wrote:
> On 9/19/06, David Masover <ninja@slaphack.com> wrote:
> 
>> When I have over a
>> gig of RAM free (not even buffer/cache, but _free_), and am trying to
>> download anything over BitTorrent, even if it's less than 200 megs, the
>> disk thrashes so badly that the system is really only usable for web and
>> email.  Even movies will occasionally stall when this is happening, and
>> by "occasionally", I mean every minute or so.
> 
> Do you have this problem on plain vanilla + reiser4?

Yes.

Well, no.  My kernel is:

vanilla 2.6.17.13 on amd64

patches:
sk98lin 8.36, latest from the manufacturer
reiser4-for-2.6.17-3
my own patch that disables fsync and fdatasync

external modules, installed via Portage:
ALSA 1.0.11 driver, using snd_emu10k1 and all sorts of support stuff 
(OSS emulation, synth, etc)
nvidia driver, 1.0.8762

I've also been having a bit of instability issues, but only very rarely 
do these seem at all FS-related.  I'm overclocked a bit, and I can 
reliably crash my system by playing Neverball, Doom 3, or Quake 4 for 
several hours.  I strongly suspect this is either my overclocking or the 
nvidia drivers here.

However, I doubt anything I've done beyond vanilla+reiser4 is affecting 
this disk access issue, and I'm pretty much rock solid when I'm not 
playing a game.  I also have a close-to-identical machine nearby which 
is not overclocked, same kernel, same modules, everything except the 
nvidia driver, been rock solid for a year, no performance issues to 
speak of.  The main difference, other than graphics, is that the stable 
machine is using 21 gigs out of 72, whereas the unstable one (the one 
that's sluggish for BitTorrent) is using 279 gigs out of 350, and has 
been up to 320 or 330 at least before I started cleaning things out.

So I think we're down to two possibilities:  Either an update to Azureus 
has found a way to sync that I'm not aware of, or this is the behavior 
someone described where Reiser4 will attempt to find contiguous space to 
allocate, and continue searching and re-searching the same areas of the 
disk almost every write.  To be honest, I hope it's about syncing, 
somehow, because I'd much rather believe my disk isn't horrendously 
fragmented...

Re: reiser4 resize

[Łukasz Mierzwa]

Dnia Thu, 21 Sep 2006 09:30:09 +0200, David Masover <ninja@slaphack.com>  
napisa³:

> --- linux/fs/buffer.c   2006-08-15 20:40:36.504608696 -0500
> +++ linux/fs/buffer.c.new       2006-08-15 20:42:35.877461264 -0500
> @@ -366,12 +366,12 @@
>
>   asmlinkage long sys_fsync(unsigned int fd)
>   {
> -       return __do_fsync(fd, 0);
> +       return 0;
>   }
>
>   asmlinkage long sys_fdatasync(unsigned int fd)
>   {
> -       return __do_fsync(fd, 1);
> +       return 0;
>   }
>
>   /*

I remember that I played a little with disabling sync in reiser4 sources,  
it helped only for amarok (it's uses sqlite for storing statistic data and  
it writes to it on song change, sqlite calls sync and it ends up with  
writeing to disk instead of playing a song, at least on my fs),  
bittorrents clients were generating as lot of disk as previously so I gues  
it's more likely coused by data/tree fragmentation.
Just my 0,6374526$ ;)

Re: reiser4 resize

[Łukasz Mierzwa]

Dnia Thu, 21 Sep 2006 09:30:09 +0200, David Masover <ninja@slaphack.com>  
napisa³:

> On 9/20/06, £ukasz Mierzwa <prymitive@pcserwis.net> wrote:
>  It's been proven that flushes are doing much more job then they should.
> Not so long ago someone send a trace of block device io accesess during
> reiser4 work and someone anylized it and said that some files or parts of
> file where written over and over 200 times or so.
>  Wow.  I should go back and read that -- I assume this is being worked  
> on?

Well it was not a file but a block, but the effect is the same.

http://marc.theaimsgroup.com/?l=reiserfs&m=115488109712570&w=2

Reiser4 resize

[Oleg Osovitskiy]

Hello!

  Does Reiser4 fs support resize as reiserfs ?
If yes which tool I have to use to enlarge my reiser4 fs ?

---
Best regards, Oleg O. Osovitskiy
mailto:o.osovitskiy@rambler.ru, icq# 33366588

Re: Reiser4 resize

[Edward Shishkin]

Oleg Osovitskiy wrote:

>Hello!
>  
>

Hi

>  Does Reiser4 fs support resize as reiserfs ?
>  
>

nop, sorry..

>If yes which tool I have to use to enlarge my reiser4 fs ?
>
>---
>Best regards, Oleg O. Osovitskiy
>mailto:o.osovitskiy@rambler.ru, icq# 33366588
>
>-
>To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>  
>

Re: Reiser4 resize

[John]

Edward Shishkin wrote:
>Oleg Osovitskiy wrote:

>>Hello!
>>  
>>

>Hi

>>  Does Reiser4 fs support resize as reiserfs ?
>>  
>>

>nop, sorry..

>>If yes which tool I have to use to enlarge my reiser4 fs ?
>>
>>---
>>Best regards, Oleg O. Osovitskiy
>>mailto:o.osovitskiy@rambler.ru, icq# 33366588


Hi there,

While playing with LVM today I was wondering about that too.
How hard would that be?
(What will follow are just assumptions from my memories of writing FS, so sorry if I'm way off with R4)

- Growing should not be too hard as only the main table would need to be changed no?
- As for shrinking I see 2 cases:
   a- There is no data beyond my desired new end, it looks like I only need to modify the table
   b- There is some data bothering us, now we need to pull it the same way as a repacker would do I believe.


The first 2 cases look easy. For the last it would need something close to a defrag.

So again, how hard would it be to write a dirty resizer? I have no Linux or R4 dev knowledge, but I have a lot spare time and I'm willing to put some of my data at risk. C is not an issue either. Do you think I can try something if you give me some pointers or am I just day-dreaming?

Thanks,

John