reiserfs rebuild-tree time?

reiserfs rebuild-tree time?

[jlewis]

I've got a several year old Red Hat 7.0 server with a 70gb partition on
[adaptec i2o, 5 18gb drive] hardware raid5.  This morning, it crashed, and
each time someone rebooted it, it would freeze|reboot|panic seconds after
the bootup process was complete.  It seems this reiserfs partition has
corruption, which was not properly handled in the older kernel (it's a Red
Hat 7.0 system).

I booted it single user and installed the latest 7.3 kernel from the
fedorlegacy project.  That kernel didn't crash, but instead began logging
lots of reiserfs corruption warnings, which is actually what alerted us as
to what the problem was.

reiserfsck (from reiserfsprogs-3.6.13) found problems and said it could
not fix them without doing a --rebuild-tree.  So I ran it again with that
option.  That was about 13 hours ago...and its still running.  I didn't
think to redirect output to a file, but AFAIK, it's in Pass 3 now.  Lots
of file names are scrolling by, but only on the last line of my terminal,
so it's difficult to tell what its up to.

At this point, I'm wondering how much longer this could possibly take to
finish?  The partition has about 45gb of files (maildirs, so lots of
files).  My next question was already answered by the FAQ...regardless of
the safety of trying it, mounting it ro at this point to try to get to
some of the files isn't possible.  Am I correct in assuming that stopping
reiserfsck before the rebuild-tree finishes would be "bad"...i.e. we won't
be able to access the data, and would have to start over anyway?

It seems to be taking an awful long time (hours) scrolling path names
inside the same user's maildir...so I'm beginning to wonder if it's caught
in some sort of loop or if this maildir is just huge and its taking time
to get through.

Is there any possibility of getting to the data (most of it anyway) on
this partition if we have to stop the rebuild before it completes?...i.e.
if it really is caught in a loop.

The system is a dual PIII-733.  Other than the rebuild and a few admins
ssh'd in to keep an eye on it, the system is idle.  I'm not seeing a whole
lot of activity (1MB/s in, 1MB/5s or so out according to vmstat) and CPU
idle 98%.  If CPU and disk i/o aren't limiting it, what's causing the
rebuild-tree to take so long?

----------------------------------------------------------------------
 Jon Lewis *jlewis@lewis.org*|  I route
 Senior Network Engineer     |  therefore you are
 Atlantic Net                |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

Re: reiserfs rebuild-tree time?

[Ryan Underwood]

[-- Attachment #1: Type: text/plain, Size: 603 bytes --]

On Thu, Mar 11, 2004 at 01:27:47AM -0500, jlewis@lewis.org wrote:
> 
> The system is a dual PIII-733.  Other than the rebuild and a few admins
> ssh'd in to keep an eye on it, the system is idle.  I'm not seeing a whole
> lot of activity (1MB/s in, 1MB/5s or so out according to vmstat) and CPU
> idle 98%.  If CPU and disk i/o aren't limiting it, what's causing the
> rebuild-tree to take so long?

Maybe it's time to test the HDD?  A quick run through with the
manufacturer's util will at least rule out failing hardware before going
further.

-- 
Ryan Underwood, <nemesis@icequake.net>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: reiserfs rebuild-tree time?

[Vladimir Saveliev]

Hello

On Thu, 2004-03-11 at 09:27, jlewis@lewis.org wrote:
> I've got a several year old Red Hat 7.0 server with a 70gb partition on
> [adaptec i2o, 5 18gb drive] hardware raid5.  This morning, it crashed, and
> each time someone rebooted it, it would freeze|reboot|panic seconds after
> the bootup process was complete.  It seems this reiserfs partition has
> corruption, which was not properly handled in the older kernel (it's a Red
> Hat 7.0 system).
> 
> I booted it single user and installed the latest 7.3 kernel from the
> fedorlegacy project.  That kernel didn't crash, but instead began logging
> lots of reiserfs corruption warnings, which is actually what alerted us as
> to what the problem was.
> 
> reiserfsck (from reiserfsprogs-3.6.13) found problems and said it could
> not fix them without doing a --rebuild-tree.  So I ran it again with that
> option.  That was about 13 hours ago...and its still running.  I didn't
> think to redirect output to a file, but AFAIK, it's in Pass 3 now.  Lots
> of file names are scrolling by, but only on the last line of my terminal,
> so it's difficult to tell what its up to.
> 
> At this point, I'm wondering how much longer this could possibly take to
> finish?  The partition has about 45gb of files (maildirs, so lots of
> files).  My next question was already answered by the FAQ...regardless of
> the safety of trying it, mounting it ro at this point to try to get to
> some of the files isn't possible.  Am I correct in assuming that stopping
> reiserfsck before the rebuild-tree finishes would be "bad"...i.e. we won't
> be able to access the data, and would have to start over anyway?
> 
Yes


> It seems to be taking an awful long time (hours) scrolling path names
> inside the same user's maildir...so I'm beginning to wonder if it's caught
> in some sort of loop or if this maildir is just huge and its taking time
> to get through.
> 
Yes, this looks like reiserfsck got into endless loop. If it is still
there lets break and fix reiserfsck bug.
For this we need you to do extract filesystem metadata with

debugreiserfs -p /dev/thatdevice | bzip2 -c > metadata.bz2

and allow us to download that file.

> Is there any possibility of getting to the data (most of it anyway) on
> this partition if we have to stop the rebuild before it completes?...i.e.
> if it really is caught in a loop.
> 
> The system is a dual PIII-733.  Other than the rebuild and a few admins
> ssh'd in to keep an eye on it, the system is idle.  I'm not seeing a whole
> lot of activity (1MB/s in, 1MB/5s or so out according to vmstat) and CPU
> idle 98%.  If CPU and disk i/o aren't limiting it, what's causing the
> rebuild-tree to take so long?
> 
> ----------------------------------------------------------------------
>  Jon Lewis *jlewis@lewis.org*|  I route
>  Senior Network Engineer     |  therefore you are
>  Atlantic Net                |
> _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
> 

Re: reiserfs rebuild-tree time?

[Philippe Gramoullé]

Hello,

On Thu, 11 Mar 2004 01:27:47 -0500 (EST)
jlewis@lewis.org wrote:

  | That was about 13 hours ago...and its still running.  I didn't
  | think to redirect output to a file, but AFAIK, it's in Pass 3 now.  Lots
  | of file names are scrolling by, but only on the last line of my terminal,
  | so it's difficult to tell what its up to.

Well, not using "-q" option of reiserfsck is a major slowdown, especially if
you use a serial console. I always use it with -q and -l /path/to/logfile.log
which i tail if i want to know the reiserfsck progress.

  |   Am I correct in assuming that stopping
  | reiserfsck before the rebuild-tree finishes would be "bad"...i.e. we won't
  | be able to access the data, and would have to start over anyway?

Don't ever do that :) or then prepare to go there:

http://www.namesys.com/support.html

  | 
  | Is there any possibility of getting to the data (most of it anyway) on
  | this partition if we have to stop the rebuild before it completes?...i.e.
  | if it really is caught in a loop.

Again, *don't* stop it, just wait. I know it's always too long but you need yo be patient if
you want your data back :)

  | 
  | The system is a dual PIII-733.  Other than the rebuild and a few admins
  | ssh'd in to keep an eye on it, the system is idle.  I'm not seeing a whole
  | lot of activity (1MB/s in, 1MB/5s or so out according to vmstat) and CPU
  | idle 98%.  If CPU and disk i/o aren't limiting it, what's causing the
  | rebuild-tree to take so long?

Maybe broken hardware ? just a guess.

Thanks,

Philippe

Re: reiserfs rebuild-tree time?

[jlewis]

On Thu, 11 Mar 2004, Vladimir Saveliev wrote:

> > some of the files isn't possible.  Am I correct in assuming that stopping
> > reiserfsck before the rebuild-tree finishes would be "bad"...i.e. we won't
> > be able to access the data, and would have to start over anyway?
> >
> Yes

The next question then is, if we have to stop it and start over, will
reiserfsck be able to start over with as much chance of success as we had
the first time...or are we likely to lose lots of data by doing this?

> Yes, this looks like reiserfsck got into endless loop. If it is still
> there lets break and fix reiserfsck bug.
> For this we need you to do extract filesystem metadata with
>
> debugreiserfs -p /dev/thatdevice | bzip2 -c > metadata.bz2
>
> and allow us to download that file.

How soon could we expect a response to that?  The $25 (or multiples of
that) support option, if that would speed it up, is no problem.

> > Is there any possibility of getting to the data (most of it anyway) on
> > this partition if we have to stop the rebuild before it completes?...i.e.
> > if it really is caught in a loop.

Is there any way to at least get the system back to the state it was in
before --rebuild-tree started?  i.e. make the partition mountable, and try
to extract as much data as we can from it...or are we stuck at this point
needing --rebuild-tree to finish/start over and finish before we can try
getting at the data again?

I kind of doubt broken hardware.  There have been no error messages from
the i2o driver (it likes to whine if there are disk problems) and in my
experience, if we had bad CPUs or RAM, it wouldn't run this long...it'd be
randomly crashing.

----------------------------------------------------------------------
 Jon Lewis *jlewis@lewis.org*|  I route
 Senior Network Engineer     |  therefore you are
 Atlantic Net                |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

Re: reiserfs rebuild-tree time?

[Vladimir Saveliev]

Hello

On Thu, 2004-03-11 at 16:58, jlewis@lewis.org wrote:
> On Thu, 11 Mar 2004, Vladimir Saveliev wrote:
> 
> > > some of the files isn't possible.  Am I correct in assuming that stopping
> > > reiserfsck before the rebuild-tree finishes would be "bad"...i.e. we won't
> > > be able to access the data, and would have to start over anyway?
> > >
> > Yes
> 
> The next question then is, if we have to stop it and start over, will
> reiserfsck be able to start over with as much chance of success as we had
> the first time...or are we likely to lose lots of data by doing this?
> 

reiserfsck will start from the beginning. You should not lose data
because of this.

> > Yes, this looks like reiserfsck got into endless loop. If it is still
> > there lets break and fix reiserfsck bug.
> > For this we need you to do extract filesystem metadata with
> >
> > debugreiserfs -p /dev/thatdevice | bzip2 -c > metadata.bz2
> >
> > and allow us to download that file.
> 
> How soon could we expect a response to that?  

fsck maintainer will work on it when he is on his working place. Now he
is not, though
Btw, is reiserfsck still looping in its semantic pass?

> The $25 (or multiples of
> that) support option, if that would speed it up, is no problem.
> 
fixing reiserfs bugs has nothing to do with this $25 thing.


> > > Is there any possibility of getting to the data (most of it anyway) on
> > > this partition if we have to stop the rebuild before it completes?...i.e.
> > > if it really is caught in a loop.
> 
> Is there any way to at least get the system back to the state it was in
> before --rebuild-tree started?  i.e. make the partition mountable, and try
> to extract as much data as we can from it...or are we stuck at this point
> needing --rebuild-tree to finish/start over and finish before we can try
> getting at the data again?
> 
reiserfsck --rebuild-tree must complete

> I kind of doubt broken hardware.  There have been no error messages from
> the i2o driver (it likes to whine if there are disk problems) and in my
> experience, if we had bad CPUs or RAM, it wouldn't run this long...it'd be
> randomly crashing.
> 
> ----------------------------------------------------------------------
>  Jon Lewis *jlewis@lewis.org*|  I route
>  Senior Network Engineer     |  therefore you are
>  Atlantic Net                |
> _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
> 
> 

Re: reiserfs rebuild-tree time?

[jlewis]

On Thu, 11 Mar 2004, Vladimir Saveliev wrote:

> > > Yes, this looks like reiserfsck got into endless loop. If it is still
> > > there lets break and fix reiserfsck bug.

Actually, I've just straced reiserfsck for a while and in 20mb of strace
output, I see it's written to stdout more than 60000 message file names
(all unique) for this one user it's been 'stuck' on since last night.  So
I'm not so sure it's looping, but just taking an awful long time on a dir
with some insane number of files in it.  If that's the case, I'd hate to
stop it just to start over and take as long again.

I ran this reiserfsck via a session logged in (across a 100mb ethernet)
via ssh.  Is it possible that not using -q could make it orders of
magnitude slower than it would have been if I'd used -q -l /some/logpath ?
If so, maybe it is worth stopping and restarting just to add those
options.

----------------------------------------------------------------------
 Jon Lewis *jlewis@lewis.org*|  I route
 Senior Network Engineer     |  therefore you are
 Atlantic Net                |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

Re: reiserfs rebuild-tree time?

[Jens Benecke]

jlewis@lewis.org wrote:

> I ran this reiserfsck via a session logged in (across a 100mb ethernet)
> via ssh.  Is it possible that not using -q could make it orders of
> magnitude slower than it would have been if I'd used -q -l /some/logpath ?
> If so, maybe it is worth stopping and restarting just to add those
> options.

Perhaps the following would be a good idea (no idea how easy it is to
implement):

Stop displaying every file name after $NUM seconds (default:60?), print "Not
printing any more file names. Press SPACE if you still want to see all of
them.", and periodically look if a key has been pressed, while reiserfsck
continues scanning.


-- 
Jens Benecke (jens at spamfreemail.de)
http://www.hitchhikers.de - Europaweite kostenlose Mitfahrzentrale
http://www.spamfreemail.de - 100% saubere Postfächer - garantiert!
http://www.rb-hosting.de - PHP ab 9? - SSH ab 19? - günstiger Traffic
.
Please DO NOT CC: me, I read the lists and newsgroups I post in!

Re: reiserfs rebuild-tree time?

[Vitaly Fertman]

On Thursday 11 March 2004 17:52, jlewis@lewis.org wrote:
> On Thu, 11 Mar 2004, Vladimir Saveliev wrote:
> > > > Yes, this looks like reiserfsck got into endless loop. If it is still
> > > > there lets break and fix reiserfsck bug.
>
> Actually, I've just straced reiserfsck for a while and in 20mb of strace
> output, I see it's written to stdout more than 60000 message file names
> (all unique) for this one user it's been 'stuck' on since last night.  So
> I'm not so sure it's looping, but just taking an awful long time on a dir
> with some insane number of files in it.  If that's the case, I'd hate to
> stop it just to start over and take as long again.
>
> I ran this reiserfsck via a session logged in (across a 100mb ethernet)
> via ssh.  Is it possible that not using -q could make it orders of
> magnitude slower than it would have been if I'd used -q -l /some/logpath ?

it is possible that not using -l logfile or -n make it orders of magnitude 
slower, but not -q.

> If so, maybe it is worth stopping and restarting just to add those
> options.

-- 
Thanks,
Vitaly Fertman

Re: reiserfs rebuild-tree time?

[Vitaly Fertman]

> > > Yes, this looks like reiserfsck got into endless loop. If it is still
> > > there lets break and fix reiserfsck bug.
> > > For this we need you to do extract filesystem metadata with
> > >
> > > debugreiserfs -p /dev/thatdevice | bzip2 -c > metadata.bz2
> > >
> > > and allow us to download that file.
> >
> > How soon could we expect a response to that?
>
> fsck maintainer will work on it when he is on his working place. Now he
> is not, though

wrong, I can look into it just after you make the metadata.

-- 
Thanks,
Vitaly Fertman

Re: reiserfs rebuild-tree time?

[Vitaly Fertman]

On Thursday 11 March 2004 18:48, Vitaly Fertman wrote:
> On Thursday 11 March 2004 17:52, jlewis@lewis.org wrote:
> > On Thu, 11 Mar 2004, Vladimir Saveliev wrote:
> > > > > Yes, this looks like reiserfsck got into endless loop. If it is
> > > > > still there lets break and fix reiserfsck bug.
> >
> > Actually, I've just straced reiserfsck for a while and in 20mb of strace
> > output, I see it's written to stdout more than 60000 message file names
> > (all unique) for this one user it's been 'stuck' on since last night.  So
> > I'm not so sure it's looping, but just taking an awful long time on a dir
> > with some insane number of files in it.  If that's the case, I'd hate to
> > stop it just to start over and take as long again.
> >
> > I ran this reiserfsck via a session logged in (across a 100mb ethernet)
> > via ssh.  Is it possible that not using -q could make it orders of
> > magnitude slower than it would have been if I'd used -q -l /some/logpath
> > ?
>
> it is possible that not using -l logfile or -n make it orders of magnitude
> slower, but not -q.

ah, I forgot that there is no delays in progress at semantic pass in 
v3 progs, in other words -q would speedup semantic pass for you also.

-- 
Thanks,
Vitaly Fertman

Re: reiserfs rebuild-tree time?

[jlewis]

On Thu, 11 Mar 2004, Ryan Underwood wrote:

> Maybe it's time to test the HDD?  A quick run through with the
> manufacturer's util will at least rule out failing hardware before going
> further.

I kind of doubt that's it.  The Adaptec i2o cards are somewhat finicky,
but if the drives even look back at the card funny, it'll declare them
dead, sound the alarm that annoys people in the offices nearby, and
degrade the array.  That isn't happening right now.

----------------------------------------------------------------------
 Jon Lewis *jlewis@lewis.org*|  I route
 Senior Network Engineer     |  therefore you are
 Atlantic Net                |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

Re: reiserfs rebuild-tree time?

[jlewis]

On Thu, 11 Mar 2004, Vladimir Saveliev wrote:

> Yes, this looks like reiserfsck got into endless loop. If it is still
> there lets break and fix reiserfsck bug.
> For this we need you to do extract filesystem metadata with
>
> debugreiserfs -p /dev/thatdevice | bzip2 -c > metadata.bz2

I started doing this today, but it seemed to be going slowly and making a
very large file.  So I stopped it and went back to reiserfsck with -l -q
-n.

How large relative to the size (or used space) of the fs should the
metadata file generated by the above command be (without compression)?
If it's going to approach the size of the partition, I'll have to NFS
mount some space to store it.

----------------------------------------------------------------------
 Jon Lewis *jlewis@lewis.org*|  I route
 Senior Network Engineer     |  therefore you are
 Atlantic Net                |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

Re: reiserfs rebuild-tree time?

[Vitaly Fertman]

On Friday 12 March 2004 05:05, jlewis@lewis.org wrote:
> On Thu, 11 Mar 2004, Vladimir Saveliev wrote:
> > Yes, this looks like reiserfsck got into endless loop. If it is still
> > there lets break and fix reiserfsck bug.
> > For this we need you to do extract filesystem metadata with
> >
> > debugreiserfs -p /dev/thatdevice | bzip2 -c > metadata.bz2
>
> I started doing this today, but it seemed to be going slowly and making a
> very large file.  So I stopped it and went back to reiserfsck with -l -q
> -n.

How big was it? There are many directories at the beginning of the fs
and directories need more time for packing, the speed usually increases 
when ~20% of partition is processed.

How is CPU activity now?

> How large relative to the size (or used space) of the fs should the
> metadata file generated by the above command be (without compression)?
> If it's going to approach the size of the partition, I'll have to NFS
> mount some space to store it.

the metadata size depends on your data pattern, thus then higher percent 
of small files, then larger metadata file. In the usual case I would expect 
~7-10M zipped metadata file of 70G fs, but it may exceed 70M.

-- 
Thanks,
Vitaly Fertman

Re: reiserfs rebuild-tree time?

[jlewis]

On Fri, 12 Mar 2004, Vitaly Fertman wrote:

> How big was it? There are many directories at the beginning of the fs
> and directories need more time for packing, the speed usually increases
> when ~20% of partition is processed.

It's about a 70gb fs with about 45gb in use.  The bzip'd metadata file was
about 60mb, and there were still about 10M blocks left in the countdown.
I see now that the file gets pretty big to start with, but growth slows
down.  I'm collecting this again.

Before stopping the rebuild again just now, I straced it and could see
that it was apparently working on the same maildir again.

> How is CPU activity now?

When reiserfsck gets to "Pass 3 (semantic):" CPU load is very low (>90%
idle) and disk I/O is fairly low, but it seems to get to a particular
maildir and stop making progress.

debugreiserfs 3.6.13 (2003 www.namesys.com)

Loading on-disk bitmap .. 11377568 bits set - done
super block..ok
bitmaps..(545).. ok
journal (from 18 to 8210)..ok
Super block, bitmaps, journal - 8739 blocks - done, 11368829 blocks left
0%....20%....40%                                        left 6679893, 4402 /sec

The bzip'd output file is >100mb already.  What's the best way to get you
a copy once it finishes?

----------------------------------------------------------------------
 Jon Lewis *jlewis@lewis.org*|  I route
 Senior Network Engineer     |  therefore you are
 Atlantic Net                |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

Re: reiserfs rebuild-tree time?

[jlewis]

On Thu, 11 Mar 2004, Vitaly Fertman wrote:

> > > How soon could we expect a response to that?
> >
> > fsck maintainer will work on it when he is on his working place. Now he
> > is not, though
>
> wrong, I can look into it just after you make the metadata.

Maybe there's no problem.  I've been running
reiserfsck --rebuild-tree -l /var/tmp/fsck.log2 -q -n /dev/sdb2
for about 27 hours now, and it's finally made it a bit further:

Pass 1 (will try to insert 2473256 leaves):
Looking for allocable blocks .. finished
0%....20%....40%....60%....80%....100%
Flushing..finished
        2473256 leaves read
                2462276 inserted
                        - pointers in indirect items pointing to metadata
76 (zeroed)
                10980 not inserted
        non-unique pointers in indirect items (zeroed) 142

Pass 2:
0%....20%....40%....60%....80%....100%
Flushing..finished
        Leaves inserted item by item 10980
Pass 3 (semantic):
Flushing..finished
        Files found: 5390892
        Directories found: 244401
        Symlinks found: 1494
        Names pointing to nowhere (removed): 10
Pass 3a (looking for lost dir/files):
Looking for lost directories:

"Pass 3 (semantic):" is as far as I'd seen it get previously.  Maybe it
just needs time?...like a few days?

----------------------------------------------------------------------
 Jon Lewis *jlewis@lewis.org*|  I route
 Senior Network Engineer     |  therefore you are
 Atlantic Net                |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

Re: reiserfs rebuild-tree time?

[jlewis]

This finally finished after a little more than 30 hours.  AFAICT, the
filesystem is intact...only 2 dirs and a handful of files got moved to
lost+found.  The maildir reiserfsck was spending so much time in (making
me think it was stuck) has 3742463 files (unread messages) in it.


 On Sat, 13 Mar 2004 jlewis@lewis.org wrote:

> Maybe there's no problem.  I've been running
> reiserfsck --rebuild-tree -l /var/tmp/fsck.log2 -q -n /dev/sdb2
> for about 27 hours now, and it's finally made it a bit further:
>
> Pass 1 (will try to insert 2473256 leaves):
> Looking for allocable blocks .. finished
> 0%....20%....40%....60%....80%....100%
> Flushing..finished
>         2473256 leaves read
>                 2462276 inserted
>                         - pointers in indirect items pointing to metadata
> 76 (zeroed)
>                 10980 not inserted
>         non-unique pointers in indirect items (zeroed) 142
>
> Pass 2:
> 0%....20%....40%....60%....80%....100%
> Flushing..finished
>         Leaves inserted item by item 10980
> Pass 3 (semantic):
> Flushing..finished
>         Files found: 5390892
>         Directories found: 244401
>         Symlinks found: 1494
>         Names pointing to nowhere (removed): 10
> Pass 3a (looking for lost dir/files):
> Looking for lost directories:
>
> "Pass 3 (semantic):" is as far as I'd seen it get previously.  Maybe it
> just needs time?...like a few days?
>
> ----------------------------------------------------------------------
>  Jon Lewis *jlewis@lewis.org*|  I route
>  Senior Network Engineer     |  therefore you are
>  Atlantic Net                |
> _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
>

----------------------------------------------------------------------
 Jon Lewis *jlewis@lewis.org*|  I route
 Senior Network Engineer     |  therefore you are
 Atlantic Net                |
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

Re: reiserfs rebuild-tree time?

[Vladimir Saveliev]

On Sun, 2004-03-14 at 09:14, jlewis@lewis.org wrote:
> This finally finished after a little more than 30 hours.  AFAICT, the
> filesystem is intact...only 2 dirs and a handful of files got moved to
> lost+found.  The maildir reiserfsck was spending so much time in (making
> me think it was stuck) has 3742463 files (unread messages) in it.
> 
Vitaly, you should probably profile reiserfsck when it runs on similar
file set. There should be posibility to speed it up.


> 
>  On Sat, 13 Mar 2004 jlewis@lewis.org wrote:
> 
> > Maybe there's no problem.  I've been running
> > reiserfsck --rebuild-tree -l /var/tmp/fsck.log2 -q -n /dev/sdb2
> > for about 27 hours now, and it's finally made it a bit further:
> >
> > Pass 1 (will try to insert 2473256 leaves):
> > Looking for allocable blocks .. finished
> > 0%....20%....40%....60%....80%....100%
> > Flushing..finished
> >         2473256 leaves read
> >                 2462276 inserted
> >                         - pointers in indirect items pointing to metadata
> > 76 (zeroed)
> >                 10980 not inserted
> >         non-unique pointers in indirect items (zeroed) 142
> >
> > Pass 2:
> > 0%....20%....40%....60%....80%....100%
> > Flushing..finished
> >         Leaves inserted item by item 10980
> > Pass 3 (semantic):
> > Flushing..finished
> >         Files found: 5390892
> >         Directories found: 244401
> >         Symlinks found: 1494
> >         Names pointing to nowhere (removed): 10
> > Pass 3a (looking for lost dir/files):
> > Looking for lost directories:
> >
> > "Pass 3 (semantic):" is as far as I'd seen it get previously.  Maybe it
> > just needs time?...like a few days?
> >
> > ----------------------------------------------------------------------
> >  Jon Lewis *jlewis@lewis.org*|  I route
> >  Senior Network Engineer     |  therefore you are
> >  Atlantic Net                |
> > _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
> >
> 
> ----------------------------------------------------------------------
>  Jon Lewis *jlewis@lewis.org*|  I route
>  Senior Network Engineer     |  therefore you are
>  Atlantic Net                |
> _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
>