Strange directories after reisferfsck

Strange directories after reisferfsck

[Kristian Koehntopp]

Yesterday my / and /home crashed (Suse Linux 9.0 with reiserfs
for everything except /boot) due to problems with the Nvidia 8x
AGP driver. Reiserfs was able to recover / after a
--fix-fixable, but needed a rebuild for /home. I was able to
recover with almost no data loss and have disabled the Nvidia
driver since.

After the rebuild, I found lots of strange directories
obviously belonging to a / partition in my /home. Some theories,
which one is plausible:

- The memory corruption due to the AGP problems brought about 800
  MB worth of data down on /home during some erraneous sync (I
  have one GB of RAM).

- The /home partition holds lots of vmdk (VMware virtual disk
  files). reiserfsck looked through all blocks during the
  rebuild and interpreted contents of the vmdk files as valid
  leaf nodes, installing them into the /home proper instead.

- Part of the disk was used as a / a few years ago. (But the
  file dates I saw were not consistent with that, they were less
  than a month old).

What happened?

Kristian

Re: Strange directories after reisferfsck

[Vitaly Fertman]

On Tuesday 04 November 2003 09:59, Kristian Koehntopp wrote:
> Yesterday my / and /home crashed (Suse Linux 9.0 with reiserfs
> for everything except /boot) due to problems with the Nvidia 8x
> AGP driver. Reiserfs was able to recover / after a
> --fix-fixable, but needed a rebuild for /home. I was able to
> recover with almost no data loss and have disabled the Nvidia
> driver since.
>
> After the rebuild, I found lots of strange directories
> obviously belonging to a / partition in my /home. Some theories,
> which one is plausible:
>
> - The memory corruption due to the AGP problems brought about 800
>   MB worth of data down on /home during some erraneous sync (I
>   have one GB of RAM).
>
> - The /home partition holds lots of vmdk (VMware virtual disk
>   files). reiserfsck looked through all blocks during the
>   rebuild and interpreted contents of the vmdk files as valid
>   leaf nodes, installing them into the /home proper instead.

yes, if you have reiserfs images in files on reiserfs partition it 
can happen. you should compress such files or you may want 
to use  a crypto loopback device to avoid the problem, xor
encryption for example. 

reiser4 will be able to keep reiser4 images in files without such 
problems.

> - Part of the disk was used as a / a few years ago. (But the
>   file dates I saw were not consistent with that, they were less
>   than a month old).
>
> What happened?
>
> Kristian

-- 
Thanks,
Vitaly Fertman

Re: Strange directories after reisferfsck

[Kristian Koehntopp]

Vitaly Fertman wrote:
> > - The /home partition holds lots of vmdk (VMware virtual disk
> >   files). reiserfsck looked through all blocks during the
> >   rebuild and interpreted contents of the vmdk files as valid
> >   leaf nodes, installing them into the /home proper instead.
> 
> yes, if you have reiserfs images in files on reiserfs partition it 
> can happen. you should compress such files or you may want 
> to use  a crypto loopback device to avoid the problem, xor
> encryption for example. 

I assume this is, because a rebuild has to scan the entire disk
for reiserfs data structures, due to the dynamic nature of the
filesystem? That is, in a traditional filesystem you know where
the inode areas are supposed to be from the information in the
superblock and can determine what is metadata and what is not
from the block address. In reiserfs you cannot, because any
block address can hold either metadata or data.

How do you recognize metadata in a rebuild scan? I assume you
are using magic bytes that markup potential metadata, and if a
rebuild scan does see these magic bytes, you check the data
structure and see if it matches several consistency checks?

If so, wouldn't it be useful not only to have a magic byte
marker in each data structure, but also a filesystem generation
number, that is unique for each filesystem generated, for
example the time_t the filesystem was created. Thus metadata
structures from vmdk or loop files would have the same magic
bytes as metadata from the main filesystem, but would be
recognizeable as belonging to a different tree due to the
different generation number.

> reiser4 will be able to keep reiser4 images in files without such 
> problems.

How does it do that?

Kristian

Re: Strange directories after reisferfsck

[Carl-Daniel Hailfinger]

Kristian Koehntopp wrote:
> Vitaly Fertman wrote:
> 
>>>- The /home partition holds lots of vmdk (VMware virtual disk
>>>  files). reiserfsck looked through all blocks during the
>>>  rebuild and interpreted contents of the vmdk files as valid
>>>  leaf nodes, installing them into the /home proper instead.
>>
>>yes, if you have reiserfs images in files on reiserfs partition it 
>>can happen. you should compress such files or you may want 
>>to use  a crypto loopback device to avoid the problem, xor
>>encryption for example. 
> 
> 
> I assume this is, because a rebuild has to scan the entire disk
> for reiserfs data structures, due to the dynamic nature of the
> filesystem? That is, in a traditional filesystem you know where
> the inode areas are supposed to be from the information in the
> superblock and can determine what is metadata and what is not
> from the block address. In reiserfs you cannot, because any
> block address can hold either metadata or data.

Yes.


> How do you recognize metadata in a rebuild scan? I assume you
> are using magic bytes that markup potential metadata, and if a
> rebuild scan does see these magic bytes, you check the data
> structure and see if it matches several consistency checks?

AFAIK there was a dicussion on this list a few weeks ago and this was the
procedure specified at that time.


> If so, wouldn't it be useful not only to have a magic byte
> marker in each data structure, but also a filesystem generation
> number, that is unique for each filesystem generated, for
> example the time_t the filesystem was created. Thus metadata
> structures from vmdk or loop files would have the same magic
> bytes as metadata from the main filesystem, but would be
> recognizeable as belonging to a different tree due to the
> different generation number.

Yes, but:
- nobody wants to sponsor that feature
- reiserfs 3.6 is in feature freeze
- the on-disk format would be not backwards compatible


>>reiser4 will be able to keep reiser4 images in files without such 
>>problems.
> 
> 
> How does it do that?

By using the ideas you specified. Sorry to disappoint you: your ideas are
not new, but they are good. Good enough to be incorporated in reiser4.


Regards,
Carl-Daniel
-- 
http://www.hailfinger.org/

Re: Strange directories after reisferfsck

[Hans Reiser]

Carl-Daniel Hailfinger wrote:

>
>
>
>  
>
>>If so, wouldn't it be useful not only to have a magic byte
>>marker in each data structure, but also a filesystem generation
>>number, that is unique for each filesystem generated, for
>>example the time_t the filesystem was created. Thus metadata
>>structures from vmdk or loop files would have the same magic
>>bytes as metadata from the main filesystem, but would be
>>recognizeable as belonging to a different tree due to the
>>different generation number.
>>    
>>
>
>Yes, but:
>- nobody wants to sponsor that feature
>- reiserfs 3.6 is in feature freeze
>- the on-disk format would be not backwards compatible
>
>  
>
which is why it is in reiser4.....

-- 
Hans

Re: Strange directories after reisferfsck

[Mike Fedyk]

On Wed, Nov 05, 2003 at 03:07:04AM +0100, Carl-Daniel Hailfinger wrote:
> Kristian Koehntopp wrote:
> > If so, wouldn't it be useful not only to have a magic byte
> > marker in each data structure, but also a filesystem generation
> > number, that is unique for each filesystem generated, for
> > example the time_t the filesystem was created. Thus metadata
> > structures from vmdk or loop files would have the same magic
> > bytes as metadata from the main filesystem, but would be
> > recognizeable as belonging to a different tree due to the
> > different generation number.
> 
> Yes, but:
> - nobody wants to sponsor that feature
> - reiserfs 3.6 is in feature freeze
> - the on-disk format would be not backwards compatible
> 
> 
> >>reiser4 will be able to keep reiser4 images in files without such 
> >>problems.
> > 
> > 
> > How does it do that?
> 
> By using the ideas you specified. Sorry to disappoint you: your ideas are
> not new, but they are good. Good enough to be incorporated in reiser4.

Does that mean there is no generation number, or random hash in the 3.6
format, or anything else that changes on each fs image that could be keyed
upon?

Re: Strange directories after reisferfsck

[Hans Reiser]

Mike Fedyk wrote:

>
>
>Does that mean there is no generation number, or random hash in the 3.6
>format, or anything else that changes on each fs image that could be keyed
>upon?
>
>
>  
>
Right.  The V3 kernel code was written before we had experience with 
running fsck for real.  We learn.  We get older.  We release new 
versions.;-)

-- 
Hans

Re: Strange directories after reisferfsck

[Kristian Koehntopp]

Mike Fedyk wrote:
> > By using the ideas you specified. Sorry to disappoint you:
> > your ideas are not new, but they are good. Good enough to be
> > incorporated in reiser4.

> Does that mean there is no generation number, or random hash
> in the 3.6 format, or anything else that changes on each fs
> image that could be keyed upon?

I do not know a thing about reiserfs, but I'd bet there is a
UUID that is characteristic for the fs in the superblock. Also,
I bet there is an inode generation number in each structure that
is the equivalent of an inode in reiserfs. That number is
necessary for NFS security, and it is independent/different for
each file.
 
For a rebuild to recognize it's own, it would be necessary to
have such a fs specific generation number in each and every data
structure, and for it to be the same for all structures that
belong to a single fs, and to be different for structures that
belong to different fs'es. That's different from both a
superblock UUID and an inode generation number.


Kristian