i/o error (lost+found) with reiser4

i/o error (lost+found) with reiser4

[Henning Westerholt]

[-- Attachment #1: Type: text/plain, Size: 1212 bytes --]

Hello all,

after the regular stat data error, i try to fix the fs as usual. Today the 
fsck program complains the first time about a fatal corruption. I attached 
the output from the first fsck run.

I run fsck with option "build-fs" as recommended, and it seems to fix the 
error. But now a "ls" in the root of the reiser4 partiton complains about a 
input/ output error on the directory "lost+found". 
Output from the system log:

+++++++++
reiser4[ls(14750)]: check_open_format40 
(fs/reiser4/plugin/disk_format/disk_format40.c:551)[vpf-1360]:
WARNING: The object with the oid 3441763 greater then the max used oid 3441762 
found.
reiser4[ls(14831)]: check_open_format40 
[...the same with stat, bash...]
+++++++++

Error messages from ls, stat..:

+++++++++
# ls /var/tmp/
ls: /var/tmp/lost+found: Input/output error  
ccache  distfiles  emerge-webrsync  kdecache-party  portage  portage-pkg  sync 

# stat /var/tmp/lost+found
stat: cannot stat `/var/tmp/lost+found': Input/output error

# cd /var/tmp/lost+found
-bash: cd: /var/tmp/lost+found: Input/output error

#fsck.reiser4 /dev/hdd8 
...
No corruption found.
+++++++++

Should i rebuild the partition? Is this a know problem?


Best regards,

Henning

[-- Attachment #2: fsck.hdd8.out --]
[-- Type: text/plain, Size: 2007 bytes --]

*******************************************************************
This is an EXPERIMENTAL version of fsck.reiser4. Read REAMDE first.
*******************************************************************

Fscking the /dev/hdd8 block device.
Will fix minor corruptions of the Reiser4 SuperBblock.
Will fix minor corruptions of the Reiser4 FileSystem.
Continue?
(Yes/No): Yes
***** Openning the fs.
Reiser4 fs was detected on the /dev/hdd8.
Master super block (16):
magic:          ReIsEr4
blksize:        4096
format:         0x0 (format40)
uuid:           921a81ba-f1aa-47b8-846c-71af014be94d
label:          <none>

Format super block (17):
plugin:         format40
description:    Disk-format for reiser4, ver. 0.5.0
magic:          ReIsEr40FoRmAt
flushes:        0
mkfs id:        0x46a0ee0e
blocks:         2441872
free blocks:    1450517
root block:     510075
tail policy:    0x4 (smart)
next oid:       0x348463
file count:     214556
tree height:    4
key policy:     LARGE

***** Tree Traverse Pass: scanning the reiser4 internal tree.
        Read nodes 116088
        Nodes left in the tree 116087
                Leaves of them 113839, Twigs of them 2205
        Nodes with wrong delimiting keys 1
                Leaves of them 1, Twigs of them 0
        Time interval: Sun Feb 22 22:50:17 2004 - Sun Feb 22 22:51:53 2004
***** TwigScan Pass: checking extent pointers of all twigs.
        Read twigs 2205
        Time interval: Sun Feb 22 22:51:53 2004 - Sun Feb 22 22:51:54 2004

Error: Node (1073153): The first key [3220ba:4(FB):7374616e646172:3220d1:af] is not equal to 
the left delimiting key [347a51:4(FB):706d736f6c6964:347db5:80]. 
Error: Node (1073073), item (49), unit (0): Points to the node [1073153] with wrong delimiting
keys. The whole subtree is skipped. 

Fatal corruptions were found. Semantic pass is skipped.
***** fsck.reiser4 finished at Sun Feb 22 22:51:54 2004
Closing fs...done

1 fatal corruptions were detected in FileSystem. Run with --build-fs option
to fix them.

Re: i/o error (lost+found) with reiser4

[Alex Zarochentsev]

On Sun, Feb 22, 2004 at 11:38:23PM +0100, Henning Westerholt wrote:
> Hello all,
> 
> after the regular stat data error, i try to fix the fs as usual. Today the 
> fsck program complains the first time about a fatal corruption. I attached 
> the output from the first fsck run.
> 
> I run fsck with option "build-fs" as recommended, and it seems to fix the 
> error. But now a "ls" in the root of the reiser4 partiton complains about a 
> input/ output error on the directory "lost+found". 
> Output from the system log:
> 
> +++++++++
> reiser4[ls(14750)]: check_open_format40 
> (fs/reiser4/plugin/disk_format/disk_format40.c:551)[vpf-1360]:
> WARNING: The object with the oid 3441763 greater then the max used oid 3441762 
> found.
> reiser4[ls(14831)]: check_open_format40 
> [...the same with stat, bash...]
> +++++++++
> 
> Error messages from ls, stat..:
> 
> +++++++++
> # ls /var/tmp/
> ls: /var/tmp/lost+found: Input/output error  
> ccache  distfiles  emerge-webrsync  kdecache-party  portage  portage-pkg  sync 
> 
> # stat /var/tmp/lost+found
> stat: cannot stat `/var/tmp/lost+found': Input/output error
> 
> # cd /var/tmp/lost+found
> -bash: cd: /var/tmp/lost+found: Input/output error
> 
> #fsck.reiser4 /dev/hdd8 
> ...
> No corruption found.
> +++++++++
> 
> Should i rebuild the partition? Is this a know problem?

This is not known but I have an idea why it happened.  I will discuss it with
our fsck maintainer and then we tell you how to fix the corruption. 

Thanks for the report. 

> Best regards,
> 
> Henning

-- 
Alex.

Re: i/o error (lost+found) with reiser4

[Alex Zarochentsev]

On Mon, Feb 23, 2004 at 06:20:03PM +0300, Alex Zarochentcev wrote:
> On Sun, Feb 22, 2004 at 11:38:23PM +0100, Henning Westerholt wrote:
> > Hello all,
> > 
> > after the regular stat data error, i try to fix the fs as usual. Today the 
> > fsck program complains the first time about a fatal corruption. I attached 
> > the output from the first fsck run.

Can you tell more about workload which caused the corruption?  Also, we like
to know reiser4 verion, kernel and hardware configuration.

> > 
> > I run fsck with option "build-fs" as recommended, and it seems to fix the 
> > error. But now a "ls" in the root of the reiser4 partiton complains about a 
> > input/ output error on the directory "lost+found". 
> > Output from the system log:
> > 
> > +++++++++
> > reiser4[ls(14750)]: check_open_format40 
> > (fs/reiser4/plugin/disk_format/disk_format40.c:551)[vpf-1360]:
> > WARNING: The object with the oid 3441763 greater then the max used oid 3441762 
> > found.

If there was no fs activity (file creation/deletion) after "build-sb",
you can erase blocks ##19, 20 on the fs:

backup:
dd if=/dev/hdd8 skip=19 bs=4096 count=2 of=hdd8.19-20x4k.dat

erase:
dd if=/dev/zero of=/dev/hdd8 seek=19 bs=4096 count=2 conv=notrunc

> > reiser4[ls(14831)]: check_open_format40 
> > [...the same with stat, bash...]
> > +++++++++
> > 
> > Error messages from ls, stat..:
> > 
> > +++++++++
> > # ls /var/tmp/
> > ls: /var/tmp/lost+found: Input/output error  
> > ccache  distfiles  emerge-webrsync  kdecache-party  portage  portage-pkg  sync 
> > 
> > # stat /var/tmp/lost+found
> > stat: cannot stat `/var/tmp/lost+found': Input/output error
> > 
> > # cd /var/tmp/lost+found
> > -bash: cd: /var/tmp/lost+found: Input/output error
> > 
> > #fsck.reiser4 /dev/hdd8 
> > ...
> > No corruption found.
> > +++++++++
> > 
> > Should i rebuild the partition? Is this a know problem?
> 
> This is not known but I have an idea why it happened.  I will discuss it with
> our fsck maintainer and then we tell you how to fix the corruption. 
> 
> Thanks for the report. 
> 
> > Best regards,
> > 
> > Henning
> 
> -- 
> Alex.

-- 
Alex.

Re: i/o error (lost+found) with reiser4

[Alex Zarochentsev]

On Mon, Feb 23, 2004 at 08:00:17PM +0300, Alex Zarochentcev wrote:
> On Mon, Feb 23, 2004 at 06:20:03PM +0300, Alex Zarochentcev wrote:
> > On Sun, Feb 22, 2004 at 11:38:23PM +0100, Henning Westerholt wrote:
> > > Hello all,
> > > 
> > > after the regular stat data error, i try to fix the fs as usual. Today the 
> > > fsck program complains the first time about a fatal corruption. I attached 
> > > the output from the first fsck run.
> 
> Can you tell more about workload which caused the corruption?  Also, we like
> to know reiser4 verion, kernel and hardware configuration.
> 
> > > 
> > > I run fsck with option "build-fs" as recommended, and it seems to fix the 
> > > error. But now a "ls" in the root of the reiser4 partiton complains about a 
> > > input/ output error on the directory "lost+found". 
> > > Output from the system log:
> > > 
> > > +++++++++
> > > reiser4[ls(14750)]: check_open_format40 
> > > (fs/reiser4/plugin/disk_format/disk_format40.c:551)[vpf-1360]:
> > > WARNING: The object with the oid 3441763 greater then the max used oid 3441762 
> > > found.
> 
> If there was no fs activity (file creation/deletion) after "build-sb",
> you can erase blocks ##19, 20 on the fs:
> 
> backup:
> dd if=/dev/hdd8 skip=19 bs=4096 count=2 of=hdd8.19-20x4k.dat
> 
> erase:
> dd if=/dev/zero of=/dev/hdd8 seek=19 bs=4096 count=2 conv=notrunc


Those blocks are journal header and footer, they may contain nr_files, last
used object id and free_blocks counters which are different from the super
block values.  It is due to a journalling optimization.  The fsck --build-fs
should have invalidated the duplicated info in the journal footer but it
hasn't. 

So, the command above invalidates the data by erasing all journal control
blocks content, and completes the fsck --build-fs run.

-- 
Alex.

Re: i/o error (lost+found) with reiser4

[Henning Westerholt]

On Monday 23 February 2004 18:23, Alex Zarochentsev wrote:
> > If there was no fs activity (file creation/deletion) after "build-sb",
> > you can erase blocks ##19, 20 on the fs:
> >
> > backup:
> > dd if=/dev/hdd8 skip=19 bs=4096 count=2 of=hdd8.19-20x4k.dat
> >
> > erase:
> > dd if=/dev/zero of=/dev/hdd8 seek=19 bs=4096 count=2 conv=notrunc
>
> Those blocks are journal header and footer, they may contain nr_files, last
> used object id and free_blocks counters which are different from the super
> block values.  It is due to a journalling optimization.  The fsck
> --build-fs should have invalidated the duplicated info in the journal
> footer but it hasn't.
>
> So, the command above invalidates the data by erasing all journal control
> blocks content, and completes the fsck --build-fs run.

There was much activity on the fs after the "build-fs", but this procedure
seems to fix my problem! Thank you very much.

Henning