vpf-10680, minor corruptions

vpf-10680, minor corruptions

[Christian Kujau]

Hi,

i'm running linux 2.5.70/2.5.72 and have a 2GB fs with Reiserfs on it.
from time to time reiserfsck the fs and some corrutions came up, which 
were fixed as suggested by --fix-fixable.
how comes? the data on the fs is not that important, it's a cache for 
Debian packages. I'm just curious _why_ there are corruptions anyway, 
since there is not even heavy load on the fs.


lila:~# reiserfsck /dev/sde2
<--------reiserfsck 3.6.6, 2003-------->

[...]

Do you want to run this program?[N/Yes] (note need to type Yes if you 
do):Yes
###########
reiserfsck --check started at Wed Jun 18 16:32:20 2003
###########
Replaying journal..
0 transactions replayed
Checking internal tree..finished
Comparing bitmaps..vpf-10640: The on-disk and the correct bitmaps differs.
Checking Semantic tree:
/debian/dists/woody/main/binary-powerpc/Packages.gzvpf-10680: The file 
[2256 3144] has the wrong block count in the StatData (3344), should be 
(1680)
/source/Sources.gzvpf-10680: The file [2275 2349] has the wrong block 
count in the StatData (1432), should be (720)
/contrib/binary-powerpc/Packages.gzvpf-10680: The file [2225 523] has 
the wrong block count in the StatData (80), should be (40)
/source/Sources.gzvpf-10680: The file [2246 2785] has the wrong block 
count in the StatData (48), should be (24)
/non-free/binary-powerpc/Packages.gzvpf-10680: The file [2289 2782] has 
the wrong block count in the StatData (104), should be (56)
/source/Sources.gzvpf-10680: The file [2310 2471] has the wrong block 
count in the StatData (56), should be (32)
/non-US/dists/woody/non-US/main/binary-powerpc/Packages.gzvpf-10680: The 
file [2627 2495] has the wrong block count in the StatData (88), should 
be (48)
finished 

9 found corruptions can be fixed with --fix-fixable
###########
reiserfsck finished at Wed Jun 18 16:32:35 2003
###########
lila:~#


(--fix-fixable fixed all corruptions, none left).
the machine was booted yesterday evening, and is booted once in a week 
or so. it's an Alpha/EV45, Debian testing on it, glibc 2.3.1.


Thanks,
Christian.

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Wed, Jun 18, 2003 at 04:45:15PM +0200, Christian Kujau wrote:
> i'm running linux 2.5.70/2.5.72 and have a 2GB fs with Reiserfs on it.
> from time to time reiserfsck the fs and some corrutions came up, which 
> were fixed as suggested by --fix-fixable.
> how comes? the data on the fs is not that important, it's a cache for 
> Debian packages. I'm just curious _why_ there are corruptions anyway, 
> since there is not even heavy load on the fs.

Hm, interesting. Do you had crashes/unexpected shutdowns before corruptions appears
or are they appear without any reason at all?

> /debian/dists/woody/main/binary-powerpc/Packages.gzvpf-10680: The file 
> [2256 3144] has the wrong block count in the StatData (3344), should be 
> (1680)
> /source/Sources.gzvpf-10680: The file [2275 2349] has the wrong block 
> count in the StatData (1432), should be (720)
> /contrib/binary-powerpc/Packages.gzvpf-10680: The file [2225 523] has 
> the wrong block count in the StatData (80), should be (40)
> /source/Sources.gzvpf-10680: The file [2246 2785] has the wrong block 
> count in the StatData (48), should be (24)
> /non-free/binary-powerpc/Packages.gzvpf-10680: The file [2289 2782] has 
> the wrong block count in the StatData (104), should be (56)
> /source/Sources.gzvpf-10680: The file [2310 2471] has the wrong block 
> count in the StatData (56), should be (32)
> /non-US/dists/woody/non-US/main/binary-powerpc/Packages.gzvpf-10680: The 
> file [2627 2495] has the wrong block count in the StatData (88), should 
> be (48)
> finished 

Are there more files on this fs at the time of check?

> the machine was booted yesterday evening, and is booted once in a week 
> or so. it's an Alpha/EV45, Debian testing on it, glibc 2.3.1.

Well, I guess it's time to clear the dust off our alpha and do some testing.

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Christian Kujau]

Oleg Drokin schrieb:
> Hm, interesting. Do you had crashes/unexpected shutdowns before corruptions appears
> or are they appear without any reason at all?

i had this issue once before -- did a check and noticed vpf-10680/some 
corruptions. but these must have been from an crash.
but now, i think as i rebooted the machine yesterday (because i upgraded 
to kernel 2.5.72) the journal was checked (replayed?) anyway at boot:

found reiserfs format "3.6" with standard journal
Reiserfs journal params: device sde2, size 8192, journal first block 18, 
max trans len 1024, max batch 900, max commit age 30, max trans age 30
reiserfs: checking transaction log (sde2) for (sde2)
Using r5 hash to sort names

(from dmesg, booting process)

and i thought the fs is "O.K." at least after boot, because ReiserFS 
cares about consistency for itsself. if not, the corruptions are likely 
from the unclean shutdowns. but that would mean, that i still have to 
manually reiserfsck from time to time.

btw, is there a switch like "Maximum mount counft before doing the next 
fsck while booting"?

> Well, I guess it's time to clear the dust off our alpha and do some testing.

hehe, should it be architecture related?

Thank you,
Christian.

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Wed, Jun 18, 2003 at 08:01:12PM +0200, Christian Kujau wrote:

> >Hm, interesting. Do you had crashes/unexpected shutdowns before 
> >corruptions appears
> >or are they appear without any reason at all?
> i had this issue once before -- did a check and noticed vpf-10680/some 
> corruptions. but these must have been from an crash.
> but now, i think as i rebooted the machine yesterday (because i upgraded 
> to kernel 2.5.72) the journal was checked (replayed?) anyway at boot:
> found reiserfs format "3.6" with standard journal
> Reiserfs journal params: device sde2, size 8192, journal first block 18, 
> max trans len 1024, max batch 900, max commit age 30, max trans age 30
> reiserfs: checking transaction log (sde2) for (sde2)
> Using r5 hash to sort names
> (from dmesg, booting process)

No, there is no sign of replaying journal.
If there was replay, you'd normally see "x transactions replayed in y seconds" message.

> and i thought the fs is "O.K." at least after boot, because ReiserFS 
> cares about consistency for itsself. if not, the corruptions are likely 
> from the unclean shutdowns. but that would mean, that i still have to 
> manually reiserfsck from time to time.

Well, normally reiserfs is caring about consistency.
There are two noticeable omissions, though:
1. if the unexpected shutdown was because of power loss and you have write cache enabled
   and your write reorders write requests, then it is possible invalid data gets
   written to disk, before "transaction is finished" mark is written to the drive.
   (there is a way to avoid this with some drives, by explicitly flushing
    drive cache in some cases, but this method seems to create some problems
    on itself. So this is not yet merged in any mainstream kernel).
2. there is no protection against kernel bugs.

1st usually leads to bitmap problems, but I also seen names pointing to nowhere.
Your corruption is somewhat strange by the fact the number of blocks
in statdata is ~ 2x bigger than it should be (on several files). Sounds like a pattern
to me.

> btw, is there a switch like "Maximum mount counft before doing the next 
> fsck while booting"?

No.

> >Well, I guess it's time to clear the dust off our alpha and do some 
> >testing.
> hehe, should it be architecture related?

This is also possible.

So can you say check/fix the fs, mount it write some files to it,
unmount it and run fsck again to see if everything is ok?

Thank you.

Bye,
    Oleg

Re: vpf-10680, minor corruptions -- oooh!

[Christian Kujau]

Oleg Drokin schrieb:
> Well, normally reiserfs is caring about consistency.
> There are two noticeable omissions, though:
> 1. if the unexpected shutdown was because of power loss and you have write cache enabled
>    and your write reorders write requests, then it is possible invalid data gets
>    written to disk, before "transaction is finished" mark is written to the drive.

yes, the on-disk write cache. this could be indeed a problem hard to
cover from any fs. i could disable it, yes.

> So can you say check/fix the fs, mount it write some files to it,
> unmount it and run fsck again to see if everything is ok?

oh, oh! i was about to answer this question with a plain "Yes". ok, with
--fix-fixable the corruptions got fixed, a reiserfsck went O.K. with "no
corruptions". i mounted the device yesterday, but no files were written
to it until today. now, i've just unmounted the partition, reiserfsck
went O.K. again, no corruptions. mounted again, i created a directory on
the fs and copied 329 files into it (cp -a /lib /path-to-reiser-fs/).

unmounted, reiserfsck found 131 corruptions in an instant:


lila:~# reiserfsck /dev/sde2
<--------reiserfsck 3.6.6, 2003-------->

[...]

Do you want to run this program?[N/Yes] (note need to type Yes if you
do):Yes
###########
reiserfsck --check started at Thu Jun 19 16:51:49 2003
###########
Replaying journal..
0 transactions replayed
Checking internal tree..finished
Comparing bitmaps..vpf-10640: The on-disk and the correct bitmaps differs.
Checking Semantic tree:
/temp/lib/libnss_files-2.3.1.sovpf-10680: The file [3214 3538] has the
wrong block count in the StatData (104), should be (56)

[...]

finished

131 found corruptions can be fixed with --fix-fixable
###########
reiserfsck finished at Thu Jun 19 16:52:04 2003
###########
lila:~# find /lib/ | wc -l
     329
lila:~#


the pathnames (/temp/lib/...) are the same files i just copied to the
fs. i was not aware of a reproduceable bug (?) at all on this issue.

the fs is used once in a week very often, but rarely _during_ week. this
could be the cause, that i never recognized the errors before or were
fixed by a journal replay at boot time.

fyi only: i have _one_ weird issue with this alpha: it has 128 MB RAM
inside, but only 64 MB are recognized. putting 64 MB into it gives 32
MB, but 32 MB is still 32MB. this is odd, but kernel compiling / heavy
load causes no ooopses, well i got some with 2.5.6x kernels, but this is
long ago. and: the hd is a little old, it's a ST34573N (SCSI, 2 GB). but
there are no odd kernel messages or failures in the log.  i say this,
because often "bad RAM" or other issues are often on-topic here.


Thank you,
Christian.

PS: sorry for the delay. mail probs.

Re: vpf-10680, minor corruptions -- oooh!

[Christian Kujau]

hi,

after diggin a little more into this, i think we really have some weird 
problem here. as said before, the data on my fs is not _that_ important.
and with 2.5.72 it's some kind of testbed anyway.

as described in the last mail, i get instant corruptions on this 
ReiserFS. furthermore, new created files on this fs *differ* from its 
source:

lila:~# cp -a /lib/ /var/cache/apt-proxy/temp/ | wc -l
     329
lila:~#

(with sde2 mounted on /var/cache/apt-proxy/ (reiserfs).
then i did:

lila:~# diff -r /lib/ /var/cache/apt-proxy/temp/lib/ | wc -l
      62
lila:~#

so 62 differ in an instant! there are probably symlinks in /lib, dunno 
how they get counted with "diff" or "cp". the partition with ReiserFS 
(and ext3 too) are mounted with "noatime" only, no other mount-options used.
(i hate to say that, but the same thing on a ext3 gives no diff's at 
all. after "cp" the files should really be the same)

so, what to do now? the ReiserFS was created as "format 3.6", but some 
time ago, with a 2.4 kernel i think. will it help to test this with 
different kernels, tools, archs? any hints?

for the record: there are still 329 files in /lib, and every time 
exactly 131 corruptions are found.

Thank you,
Christian

Re: vpf-10680, minor corruptions -- oooh!

[Oleg Drokin]

Hello!

On Fri, Jun 20, 2003 at 01:42:13PM +0200, Christian Kujau wrote:

> after diggin a little more into this, i think we really have some weird 
> problem here. as said before, the data on my fs is not _that_ important.
> and with 2.5.72 it's some kind of testbed anyway.
> as described in the last mail, i get instant corruptions on this 
> ReiserFS. furthermore, new created files on this fs *differ* from its 
> source:
> lila:~# cp -a /lib/ /var/cache/apt-proxy/temp/ | wc -l
>     329
> lila:~#

Was /var/cache/apt-proxy/temp/ empty before copying?

> (with sde2 mounted on /var/cache/apt-proxy/ (reiserfs).
> then i did:
> lila:~# diff -r /lib/ /var/cache/apt-proxy/temp/lib/ | wc -l
>      62
> lila:~#
> so 62 differ in an instant! there are probably symlinks in /lib, dunno 
> how they get counted with "diff" or "cp". the partition with ReiserFS 
> (and ext3 too) are mounted with "noatime" only, no other mount-options used.
> (i hate to say that, but the same thing on a ext3 gives no diff's at 
> all. after "cp" the files should really be the same)
> so, what to do now? the ReiserFS was created as "format 3.6", but some 
> time ago, with a 2.4 kernel i think. will it help to test this with 
> different kernels, tools, archs? any hints?

"diff -r" will emit "only in PATH" message for each file present only
in one compared dir, so if you have some extra files at the destination,
thisis kind of expected.

> for the record: there are still 329 files in /lib, and every time 
> exactly 131 corruptions are found.

Ok, I am looking into this.
In fact this "non-matching blocks" stuff is not dangerous to actual data.

I am interested in what will happen if you'd do a cp -a/diff -r on a
freshly formatted reiserfs, will there be any differences?
(and send me output of diff in this case.)

Bye,
    Oleg

Re: vpf-10680, minor corruptions -- oooh!

[Christian Kujau]

Oleg Drokin schrieb:
>>lila:~# cp -a /lib/ /var/cache/apt-proxy/temp/ | wc -l
>>    329
>>lila:~#
> 
> Was /var/cache/apt-proxy/temp/ empty before copying?

yes.

> "diff -r" will emit "only in PATH" message for each file present only
> in one compared dir, so if you have some extra files at the destination,
> thisis kind of expected.

yes, i know. see, the severity is really as strong as it seems. there 
are no "only in path" msg, but "file1 and file1 differ" messages.


> Ok, I am looking into this.
> In fact this "non-matching blocks" stuff is not dangerous to actual data.

are you sure? when even "diff" says that the files are different, the 
data is wrong in some way.

under /lib are modeuls, inserting them is fine. however, the copied 
files (modules) gaive only a segfault!

Thanks,
Christian.

> I am interested in what will happen if you'd do a cp -a/diff -r on a
> freshly formatted reiserfs, will there be any differences?
> (and send me output of diff in this case.)

will do later.

Re: vpf-10680 corruptions

[Christian Kujau]

Christian Kujau schrieb:
>> Ok, I am looking into this.
>> In fact this "non-matching blocks" stuff is not dangerous to actual data.
> 
> are you sure? when even "diff" says that the files are different, the 
> data is wrong in some way.
> 
> under /lib are modeuls, inserting them is fine. however, the copied 
> files (modules) gaive only a segfault!

now i have even more time to answer this: data is indeed corrupt on the 
reiserfs. when i saw "segmentation fault" on stdout, i mean a 
kernel-Ooops (but no freeze!) in the logs. so modules on the ReiserFS 
could no be loaded, diff's are shown, .gz files are not "OK".

>> I am interested in what will happen if you'd do a cp -a/diff -r on a
>> freshly formatted reiserfs, will there be any differences?
>> (and send me output of diff in this case.)

i have copied all data from the said ReiserFS to another partition and 
have now 2GB of spare media to test on. i want to explain an issue here, 
i hope i am not confusing: /var/cache/apt-proxy is the said ReiserFS. i 
noticed corruptions now and "some time ago". but there is data on far 
longer. i was able to cp all data from /var/cache/apt-proxy to another 
fs, a "diff -r" showed *no* differences. i deleted everything under 
/var/cache/apt-proxy and copied all from my apt-proxy backup to 
/var/cache/apt-proxy again. then diff showed differences.
like this:

# cp -a /var/cache/apt-proxy /backup
# diff -r /var/cache/apt-proxy /backup/apt-proxy
(showed 0 (zero) differences)
# rm -rf /var/cache/apt-proxy/*
# cp -a /backup/apt-proxy/* /var/cache/apt-proxy
# diff -r /backup/apt-proxy/ /var/cache/apt-proxy
(showed errors, a following reiserfsck brought corruptions again.)

i have now remformatted /dev/sde1 (*) with ReiserFS, but the symptoms 
behave the same. i'm still using 2.5.72 on an alpha. oh, yes, there is 
*one* patch used, i'll post it here for the record, but i don't think it 
has anything to do with the problem:

-------------
--- 2.5.72/arch/alpha/kernel/srmcons.c	Mon Jun 16 16:00:39 2003
+++ linux/arch/alpha/kernel/srmcons.c	Mon Jun 16 16:13:17 2003
@@ -291,6 +291,7 @@ srmcons_init(void)
  		driver->type = TTY_DRIVER_TYPE_SYSTEM;
  		driver->subtype = SYSTEM_TYPE_SYSCONS;
  		driver->init_termios = tty_std_termios;
+		tty_set_operations(driver, &srmcons_ops);
  		err = tty_register_driver(driver);
  		if (err) {
  			put_tty_driver(driver);
-------------

without this, 2.5.72/2.5.72 would not compile.

i tried the tests with default mount options, often "-o notail" (via 
fstab) was used, same results.

the output of "diff -r ..." was not "only in directory..." but:

Binary files /lib/modules/2.5.70/kernel/drivers/block/floppy.ko and 
/var/cache/apt-proxy/lib/modules/2.5.70/kernel/drivers/block/floppy.ko 
differ

and as i said before: loading the latter one gave me an Oops in the logfile.

i'll restest with the "newest reiserfsprogs" from namesys, since i have 
3.6.6. here (from debian testing)

Thanks,
Christian.


(*) it used to be sde2, but i have re-paritioned sde into one single 2GB 
sde1 to test on.

Re: vpf-10680 corruptions

[Christian Kujau]

Christian Kujau schrieb:
> i'll restest with the "newest reiserfsprogs" from namesys, since i have 
> 3.6.6. here (from debian testing)
> 

no change with reiserfsprogs-3.6.8.

but: on ia32 the problem does not occur. i have an AMD 900 here, 2.5.72, 
reiserfsprogs 3.6.6 (also debian/testing) and there are no corruptions 
at all with reiserfs. i guess i'll resend this to lk-ml or so.

sorry for wasting time after it turned out to be probably an arch 
problem....

Thanks,
Christian.

Re: vpf-10680 corruptions

[Oleg Drokin]

Hello!

On Sat, Jun 21, 2003 at 12:37:49PM +0200, Christian Kujau wrote:

> >i'll restest with the "newest reiserfsprogs" from namesys, since i have 
> >3.6.6. here (from debian testing)
> no change with reiserfsprogs-3.6.8.
> but: on ia32 the problem does not occur. i have an AMD 900 here, 2.5.72, 
> reiserfsprogs 3.6.6 (also debian/testing) and there are no corruptions 
> at all with reiserfs. i guess i'll resend this to lk-ml or so.
> sorry for wasting time after it turned out to be probably an arch 
> problem....

Actually it may be not.
There alpha have different page size and that might lead to some unexpected
problems in reiserfs (but it should not).
I want to test that myself, but unfortunatelly it turned out that our
alpha have old suse 6.3 install that is uncapable of building 2.5 kernel.
I guess I wil ltry to cross-compile something.
I just want to judge for myself whenever it is reiserfs problem or not.
Thank you for all of the feedback and tests.

Bye,
    Oleg

Re: vpf-10680 corruptions

[Christian Kujau]

i'm about to test with 2.4.21, but it needs *a bit time* to compile :-)

Christian.

Re: vpf-10680, minor corruptions

[Hans Reiser]

Oleg Drokin wrote:

>
>Well, normally reiserfs is caring about consistency.
>There are two noticeable omissions, though:
>1. if the unexpected shutdown was because of power loss and you have write cache enabled
>   and your write reorders write requests, then it is possible invalid data gets
>   written to disk, before "transaction is finished" mark is written to the drive.
>   (there is a way to avoid this with some drives, by explicitly flushing
>    drive cache in some cases, but this method seems to create some problems
>    on itself. So this is not yet merged in any mainstream kernel).
>  
>

Can you provide details on this not being merged?

-- 
Hans

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Mon, Jun 23, 2003 at 11:38:15AM +0400, Hans Reiser wrote:
> >Well, normally reiserfs is caring about consistency.
> >There are two noticeable omissions, though:
> >1. if the unexpected shutdown was because of power loss and you have write 
> >cache enabled
> >  and your write reorders write requests, then it is possible invalid data 
> >  gets
> >  written to disk, before "transaction is finished" mark is written to the 
> >  drive.
> >  (there is a way to avoid this with some drives, by explicitly flushing
> >   drive cache in some cases, but this method seems to create some problems
> >   on itself. So this is not yet merged in any mainstream kernel).
> Can you provide details on this not being merged?

It was never submitted to inclusion into mainstream kernel. Hence it was never merged.
I saw that Jens Axboe (I think) posted the code for public review/verification, but that's it.

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Hans Reiser]

Oleg Drokin wrote:

>Hello!
>
>On Mon, Jun 23, 2003 at 11:38:15AM +0400, Hans Reiser wrote:
>  
>
>>>Well, normally reiserfs is caring about consistency.
>>>There are two noticeable omissions, though:
>>>1. if the unexpected shutdown was because of power loss and you have write 
>>>cache enabled
>>> and your write reorders write requests, then it is possible invalid data 
>>> gets
>>> written to disk, before "transaction is finished" mark is written to the 
>>> drive.
>>> (there is a way to avoid this with some drives, by explicitly flushing
>>>  drive cache in some cases, but this method seems to create some problems
>>>  on itself. So this is not yet merged in any mainstream kernel).
>>>      
>>>
>>Can you provide details on this not being merged?
>>    
>>
>
>It was never submitted to inclusion into mainstream kernel. Hence it was never merged.
>I saw that Jens Axboe (I think) posted the code for public review/verification, but that's it.
>
>Bye,
>    Oleg
>
>
>  
>
Please pursue the matter with Jens and Chris.

-- 
Hans

Re: vpf-10680, minor corruptions

[Christian Kujau]

hi again,

i did some further testings on this issue and the results are as follows:

as stated before, the corruptions occur only on this very alpha machine, 
i386 (2.5.72) is not affected here.

i staretd the alpha then with a 2.4.21. i created an fs on /dev/sde2, 
reiserfsck passed with no errors. i mounted the partition, copied /lib 
to it, diff showed _no_ differences, which was good. then something 
unexecpected happened: i unmounted the partition right after the 
"diff..." finished and wanted to reiserfsck the patition again, but this 
time reiserfsck complained about:

[...]
The problem has occurred looks like a hardware problem.
If you have bad blocks, we advise you to get a new hard
drive, because once you get one bad block that the disk
[...]
that so many blocks have gone bad that none remain in
reserve to allocate.
 

bread: Cannot read the block (523914): (Input/output error).

hah! i was not aware that the disk might have an hw problem, not a 
single error ever showed up in my logs. this was weird. so i 
re-partitioned the disk with a 10MB sde (to circumvent the bread error) 
on the beginning and a 2 GB sde2. now reiserfsck/cp/diff are all working 
fine under 2.4.21, but 2.5.72 is still erroneous.

btw: i am still using reiserfsprogs 3.6.8 now (since debian/testing has 
3.6.6) and i have compiled these utils under a 2.5.72 kernel. is it safe 
to use them under 2.4 ?

thanks,
Christian.

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Mon, Jun 23, 2003 at 03:38:20PM +0200, Christian Kujau wrote:

> as stated before, the corruptions occur only on this very alpha machine, 

Well, I still cannot build the kernel myself and still working on it.
(having "make: *** [vmlinux] Error 139" and zero length vmlinux)

BTW, I realised that I have not looked into your kernel config for that box,
can you send it to me please?

> bread: Cannot read the block (523914): (Input/output error).

Hm, but still it means kernel returned some error for read request.

> hah! i was not aware that the disk might have an hw problem, not a 
> single error ever showed up in my logs. this was weird. so i 
> re-partitioned the disk with a 10MB sde (to circumvent the bread error) 
> on the beginning and a 2 GB sde2. now reiserfsck/cp/diff are all working 
> fine under 2.4.21, but 2.5.72 is still erroneous.

Sigh.

> 
> btw: i am still using reiserfsprogs 3.6.8 now (since debian/testing has 
> 3.6.6) and i have compiled these utils under a 2.5.72 kernel. is it safe 
> to use them under 2.4 ?

I see that you have used 2.5.70 and earlier kernels on alpha too.
Do you have any idea of when stuff broke for you?

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Christian Kujau]

Oleg Drokin schrieb:
>>>I see that you have used 2.5.70 and earlier kernels on alpha too.
>>>Do you have any idea of when stuff broke for you?
>>
>>hm, i used 2.5.6x kernels too on this machine, but i recognized the 
>>vpf-10680 the first time with 2.5.70.

of course, the best thing i can do is the el-cheapo-hacking approach: 
compiling 2.5.60...up to 2.5.72 and see *when* it breaks. hm, compiling 
a 2.5 kernel takes 180min on this machine. but anyway, i'll start with 
2.5.60 now, see what it gives.

> 
> You are certainly not the one person with alpha and 2.5, but I do not know
> if others are using reiserfs.

you gotta send ads (read: spam) to all the linux-alpha lists :-)

> BTW, have you tried to run with CONFIG_REISERFS_CHECK enabled to see if it will break
> and panic in kernel or something like that?

no, only CONFIG_REISERFS_PROC_INFO, but i'll do so now.

Thanks,
Christian.

Re: vpf-10680, minor corruptions

[Christian Kujau]

Christian Kujau schrieb:
> of course, the best thing i can do is the el-cheapo-hacking approach: 
> compiling 2.5.60...up to 2.5.72 and see *when* it breaks. hm, compiling 
> a 2.5 kernel takes 180min on this machine. but anyway, i'll start with 
> 2.5.60 now, see what it gives.

no, i started with 2.5.66 but the kernel did not compile. 2.5.65 did 
compile (don't ask how long....) and has already booted. but trying to 
mount the newly created reiserfs gives:

module reiserfs: Relocation overflow vs section 9

in the log. the reiserfs module was not loaded. "modprobe reiserfs" gives:

lila:~# modprobe reiserfs
FATAL: Error inserting reiserfs 
(/lib/modules/2.5.65/kernel/fs/reiserfs/reiserfs.ko): Invalid module format
lila:~# uname -a
Linux lila 2.5.65 #4 Wed Jun 25 00:48:46 CEST 2003 alpha GNU/Linux

i compiled the module with CONFIG_REISERFS_CHECK=y.

shall i go on with 2.5.64 or better 2.5.67 ?

good night,
Christian.

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Wed, Jun 25, 2003 at 02:42:24AM +0200, Christian Kujau wrote:
> (/lib/modules/2.5.65/kernel/fs/reiserfs/reiserfs.ko): Invalid module format
> lila:~# uname -a
> Linux lila 2.5.65 #4 Wed Jun 25 00:48:46 CEST 2003 alpha GNU/Linux
> i compiled the module with CONFIG_REISERFS_CHECK=y.
> shall i go on with 2.5.64 or better 2.5.67 ?

Try to compile with CONFIG_REISERFS_CHECK=y the kernel that known-bad for you.
(e.g. 2.5.72/2.5.73)

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Christian Kujau]

Oleg Drokin schrieb:
> Try to compile with CONFIG_REISERFS_CHECK=y the kernel that known-bad for you.
> (e.g. 2.5.72/2.5.73)
> 
yes, 2.5.72 with CONFIG_REISERFS_CHECK=y is compiling now.

over night the alpha finished compiling 2.5.65 and 2.5.69. i had to 
compile reiserfs statically, inserting modules gave these "Invalid 
module format" errors.

under both (2.5.65+2.5.69) i was able to mkreiserfs sde2. mounting the 
fs went ok, but copying data (cp -a /lib /mnt/reiserfs) brought several 
kernel-errors (see https://ephigenie.kicks-ass.net/browse/reiserfs/).

but: diff -r showed _no_ differences betweeen the directories, a 
following reiserfsck brought no vpf-10680 anymore!

so i'd say the problem occurs somewhere between 2.5.69 and 2.5.70.

thanks,
Christian.

Re: vpf-10680, minor corruptions

[Christian Kujau]

finally,

2.5.72 with CONFIG_REISERFS_CHECK is ready and was booting fine. but 
another issue occured: i always had reiserfs built as a module, now i 
got "Invalid module format" on the reiserfs module too (i got the error 
in earlier versions too, but i _had_ once a 2.7.72 with reiserfs as 
module. only now i get this error. btw: i other modules are ok, but only 
jfs/xfs/reiserfs are showing these errors). anyway, i built 2.5.72 with 
static reiserfs support but i don't get much debug info.

this is the mounting:

found reiserfs format "3.6" with standard journal
reiserfs:warning: CONFIG_REISERFS_CHECK is set ON
reiserfs:warning: - it is slow mode for debugging.
Reiserfs journal params: device sde2, size 8192, journal first block 18, 
max trans len 1024, max batch 900, max commit age 30, max trans age 30
reiserfs: checking transaction log (sde2) for (sde2)
journal-1153: found in header: first_unflushed_offset 71, 
last_flushed_trans_id 15
journal-1206: Starting replay from offset 71, trans_id 16
journal-1299: Setting newest_mount_id to 11
Using r5 hash to sort names

cp'ing files to it gives no info in stderr/logfiles, but the corruptions 
are back again:

lila:~# reiserfsck --fix-fixable /dev/sde2

<-------------reiserfsck, 2003------------->
reiserfsprogs 3.6.8

   *************************************************************
   ** If you are using the latest reiserfsprogs and  it fails **
   ** please  email bug reports to reiserfs-list@namesys.com, **
   ** providing  as  much  information  as  possible --  your **
   ** hardware,  kernel,  patches,  settings,  all reiserfsck **
   ** messages  (including version),  the reiserfsck logfile, **
   ** check  the  syslog file  for  any  related information. **
   ** If you would like advice on using this program, support **
   ** is available  for $25 at  www.namesys.com/support.html. **
   *************************************************************

Will check consistency of the filesystem on /dev/sde2
and will fix what can be fixed without --rebuild-tree
Will put log info to 'stdout'

Do you want to run this program?[N/Yes] (note need to type Yes if you 
do):Yes
###########
reiserfsck --fix-fixable started at Wed Jun 25 20:01:56 2003
###########
Replaying journal..
0 transactions replayed
Checking internal tree..finished
Comparing bitmaps..vpf-10630: The on-disk and the correct bitmaps 
differs. Will be fixed later.
Checking Semantic tree:
/lib/libnss_files-2.3.1.sovpf-10680: The file [3 415] has the wrong 
block count in the StatData (104) - corrected to (56)
[...]
/libpthread-0.10.sovpf-10680: The file [3 423] has the wrong block count 
in the StatData (232) - corrected to (120)
finished
No corruptions found
There are on the filesystem:
	Leaves 33
	Internal nodes 1
	Directories 99
	Other files 323
	Data block pointers 5477 (2616 of them are zero)
	Safe links 0
###########
reiserfsck finished at Wed Jun 25 20:02:05 2003
###########
lila:~#
---------------------------------

i really don't know why this alpha has these odd problems. remember the 
strange "bread error" i got with 2.4? today, under 2.5.72 i noticed for 
the first time an scsi-error in my logfile while "mkreiserfs /dev/sdb2" 
was about to finish. (scsi0: ERROR on channel 0, id 5, lun 0/sense key 
Medium Error). can't it be an hw issue too?
as described in an earlier mail i already re-partitioned sde into sde1 
(10MB) and sde2 (2135MB). now, after getting these scsi error i 
re-partitioned again into sde1(10MB), sde2(2125MB) and sde3(10MB), to 
exclude the beginning and the end of the disk from being touched.
sure, the only good workaround is buy a new disk :-)

so, after saying that i made another test. ok, perhaps is sde really 
dying, but sdc is not, it's my rootfs :-)
i created a 100MB image, mkreiserfs -f on it, mounted it and copied 
files to it. a following "diff" gave no errors! fine, but "reiserfsck 
test.img" (after unmounting) presented the very known errors again. i 
did not use --fix-fixable but mounted the test.img again, now "diff -r" 
showed differences where none should be :-(


Thank you for your time,
Christian.

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Wed, Jun 25, 2003 at 08:26:01PM +0200, Christian Kujau wrote:

> i really don't know why this alpha has these odd problems. remember the 
> strange "bread error" i got with 2.4? today, under 2.5.72 i noticed for 
> the first time an scsi-error in my logfile while "mkreiserfs /dev/sdb2" 
> was about to finish. (scsi0: ERROR on channel 0, id 5, lun 0/sense key 
> Medium Error). can't it be an hw issue too?

Hm.

> so, after saying that i made another test. ok, perhaps is sde really 
> dying, but sdc is not, it's my rootfs :-)
> i created a 100MB image, mkreiserfs -f on it, mounted it and copied 
> files to it. a following "diff" gave no errors! fine, but "reiserfsck 

Sure, 100M perfectly fits into your RAM, so you just verified cached copies.
They are the same.

> test.img" (after unmounting) presented the very known errors again. i 
> did not use --fix-fixable but mounted the test.img again, now "diff -r" 
> showed differences where none should be :-(

Now you unmounted the thing and flushed the cache. Next attempt to
compare stuff lead to data being fetched off disk.
Hm.
Another datapoint to the picture.
Thank you.

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Wed, Jun 25, 2003 at 08:26:01PM +0200, Christian Kujau wrote:

> so, after saying that i made another test. ok, perhaps is sde really 
> dying, but sdc is not, it's my rootfs :-)
> i created a 100MB image, mkreiserfs -f on it, mounted it and copied 
> files to it. a following "diff" gave no errors! fine, but "reiserfsck 
> test.img" (after unmounting) presented the very known errors again. i 
> did not use --fix-fixable but mounted the test.img again, now "diff -r" 
> showed differences where none should be :-(

BTW, can you send me the original file and its corrupted counterpart?

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Christian Kujau]

Oleg Drokin schrieb:
> BTW, can you send me the original file and its corrupted counterpart?
> 

i've copied several files to a loopmounted reiserfs and selected a text 
file (some random text) and a binary file 
(/lib/iptables/libip6t_icmpv6.so), for which i have the original and the 
corrupted copy on https://ephigenie.kicks-ass.net/browse/reiserfs/diffs

Thanks,
Christian.

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Thu, Jun 26, 2003 at 02:38:11PM +0200, Christian Kujau wrote:

> >BTW, can you send me the original file and its corrupted counterpart?
> i've copied several files to a loopmounted reiserfs and selected a text 
> file (some random text) and a binary file 
> (/lib/iptables/libip6t_icmpv6.so), for which i have the original and the 
> corrupted copy on https://ephigenie.kicks-ass.net/browse/reiserfs/diffs

Well, there is really a pattern.
Each 4k of data you copy us placed in its own 8k page (the rest 4k are freed).
But I really cannot see how that may happen! It simply cannot, I'd say.
Would you mind to try 2.5.72 with the following patch, and copy some file under
128k in size to some reiserfs filesystem and see if any warnings were
produced. (if they are, show me what were those warnings).
Also check that files are still corrupted.

Thank you.

Bye,
    Oleg
===== fs/reiserfs/file.c 1.20 vs edited =====
--- 1.20/fs/reiserfs/file.c	Wed Jun  4 11:50:34 2003
+++ edited/fs/reiserfs/file.c	Fri Jun 27 13:05:12 2003
@@ -578,10 +578,15 @@
     long page_fault=0; // status of copy_from_user.
     int i; // loop counter.
     int offset; // offset in page
+    int write_bytes_saved = write_bytes;
 
     for ( i = 0, offset = (pos & (PAGE_CACHE_SIZE-1)); i < num_pages ; i++,offset=0) {
 	int count = min_t(int,PAGE_CACHE_SIZE-offset,write_bytes); // How much of bytes to write to this page
 	struct page *page=prepared_pages[i]; // Current page we process.
+
+	if ( write_bytes <= 0 ) {
+		printk("Allocated %d pages that are not needed (of %d), pagesize is %ld, to_write %d!\n", num_pages - i, num_pages, PAGE_CACHE_SIZE, write_bytes_saved );
+	}
 
 	fault_in_pages_readable( buf, count);
 

Re: vpf-10680, minor corruptions

[Christian Kujau]

Oleg Drokin schrieb:
> Well, there is really a pattern.
> Each 4k of data you copy us placed in its own 8k page (the rest 4k are freed).
> But I really cannot see how that may happen! It simply cannot, I'd say.

i put some more info about this alpha on the very same web-address, 
alpha machines differ much, maybe it's just this kind of alpha (Avanti) 
showing this behaviour...

> Would you mind to try 2.5.72 with the following patch, and copy some file under
> 128k in size to some reiserfs filesystem and see if any warnings were
> produced. (if they are, show me what were those warnings).
> Also check that files are still corrupted.

yes, of course.

Thanks,
Christian.

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Fri, Jun 27, 2003 at 02:18:34PM +0200, Christian Kujau wrote:
> >Well, there is really a pattern.
> >Each 4k of data you copy us placed in its own 8k page (the rest 4k are 
> >freed).
> >But I really cannot see how that may happen! It simply cannot, I'd say.
> i put some more info about this alpha on the very same web-address, 
> alpha machines differ much, maybe it's just this kind of alpha (Avanti) 
> showing this behaviour...

Sigh, I have found what happens. And it will happen on every 64 bit platform with recent 2.5 kernel.
the internal reiserfs blocknumbers were referred to as 'unsigned long', that's where these
extra blocks of zeroes are from, this also explains all other anomalies.
Except when I make it 'unsigned int' as it ought to be, it still breaks,
only in some other way which I have not understood yet.
(yes, I seem to finally got our alpha to some working state and can do experiments locally now).

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Christian Kujau]

Oleg Drokin schrieb:
> Sigh, I have found what happens. And it will happen on every 64 bit platform with recent 2.5 kernel.
> the internal reiserfs blocknumbers were referred to as 'unsigned long', that's where these
> extra blocks of zeroes are from, this also explains all other anomalies.
> Except when I make it 'unsigned int' as it ought to be, it still breaks,
> only in some other way which I have not understood yet.
> (yes, I seem to finally got our alpha to some working state and can do experiments locally now).
> 

this sound like good news, yes? hm, but when you say on every 64bit 
platform with recent 2.5 kernels, i wonder again where all the 
sparc64/ppc64/opteron/itanium guys are to report corruptions...

compiling 2.5.72 with your patch is under way...

thank you Oleg,
Christian.

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Fri, Jun 27, 2003 at 02:32:42PM +0200, Christian Kujau wrote:
> >Sigh, I have found what happens. And it will happen on every 64 bit 
> >platform with recent 2.5 kernel.
> >the internal reiserfs blocknumbers were referred to as 'unsigned long', 
> >that's where these
> >extra blocks of zeroes are from, this also explains all other anomalies.
> >Except when I make it 'unsigned int' as it ought to be, it still breaks,
> >only in some other way which I have not understood yet.
> >(yes, I seem to finally got our alpha to some working state and can do 
> >experiments locally now).
> this sound like good news, yes? hm, but when you say on every 64bit 
> platform with recent 2.5 kernels, i wonder again where all the 
> sparc64/ppc64/opteron/itanium guys are to report corruptions...

Nobody uses reiserfs on 64 bit arches on recent 2.5 kernels?
Or may be they have not noticed corruptions yet?

> compiling 2.5.72 with your patch is under way...

Thank you,
this is not needed already.
I was looking in the wrong direction, when I produced that patch,
so it will produce zero output.
I hope to come up with ultimate fix soon enough. ;)

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Oleg Drokin]

[-- Attachment #1: Type: text/plain, Size: 13637 bytes --]

Hello!

On Fri, Jun 27, 2003 at 04:38:00PM +0400, Oleg Drokin wrote:

> I was looking in the wrong direction, when I produced that patch,
> so it will produce zero output.
> I hope to come up with ultimate fix soon enough. ;)

Well, there is a patch below that does *not* work for me ;)
But it should work.
I have traced the new problem to a cross compiler that compiles
code in a different way than native compiler for whatever reason
(demo is attached as test.c program, it should print "result is 1"
in case it is compiled correctly and stuff about unknown
uniqueness if it is miscompiled. In fact may be this is just correct compiler behaviour.)
I now think that when I compile a kernel with native compiler, it should work
with below patch. But I can verify that only tomorrow it seems.
You might try that patch as well to see if it helps you before I try it ;)
The patch is "obviously correct" one. (except that it does not work
with my cross compiler and kernel does work without patch which is really-really strange).

===== fs/reiserfs/bitmap.c 1.26 vs edited =====
--- 1.26/fs/reiserfs/bitmap.c	Sun May 18 01:09:36 2003
+++ edited/fs/reiserfs/bitmap.c	Fri Jun 27 16:58:44 2003
@@ -43,7 +43,7 @@
     test_bit(_ALLOC_ ## optname , &SB_ALLOC_OPTS(s))
 
 static inline void get_bit_address (struct super_block * s,
-				    unsigned long block, int * bmap_nr, int * offset)
+				    b_blocknr_t block, int * bmap_nr, int * offset)
 {
     /* It is in the bitmap block number equal to the block
      * number divided by the number of bits in a block. */
@@ -54,7 +54,7 @@
 }
 
 #ifdef CONFIG_REISERFS_CHECK
-int is_reusable (struct super_block * s, unsigned long block, int bit_value)
+int is_reusable (struct super_block * s, b_blocknr_t block, int bit_value)
 {
     int i, j;
 
@@ -107,7 +107,7 @@
 static inline  int is_block_in_journal (struct super_block * s, int bmap, int
 off, int *next)
 {
-    unsigned long tmp;
+    b_blocknr_t tmp;
 
     if (reiserfs_in_journal (s, bmap, off, 1, &tmp)) {
 	if (tmp) {              /* hint supplied */
@@ -235,7 +235,7 @@
 /* Tries to find contiguous zero bit window (given size) in given region of
  * bitmap and place new blocks there. Returns number of allocated blocks. */
 static int scan_bitmap (struct reiserfs_transaction_handle *th,
-			unsigned long *start, unsigned long finish,
+			b_blocknr_t *start, b_blocknr_t finish,
 			int min, int max, int unfm, unsigned long file_block)
 {
     int nr_allocated=0;
@@ -281,7 +281,7 @@
 }
 
 static void _reiserfs_free_block (struct reiserfs_transaction_handle *th,
-				  unsigned long block)
+				  b_blocknr_t block)
 {
     struct super_block * s = th->t_super;
     struct reiserfs_super_block * rs;
@@ -327,7 +327,7 @@
 }
 
 void reiserfs_free_block (struct reiserfs_transaction_handle *th, 
-                          unsigned long block)
+                          b_blocknr_t block)
 {
     struct super_block * s = th->t_super;
 
@@ -340,7 +340,7 @@
 
 /* preallocated blocks don't need to be run through journal_mark_freed */
 void reiserfs_free_prealloc_block (struct reiserfs_transaction_handle *th, 
-                          unsigned long block) {
+                          b_blocknr_t block) {
     RFALSE(!th->t_super, "vs-4060: trying to free block on nonexistent device");
     RFALSE(is_reusable (th->t_super, block, 1) == 0, "vs-4070: can not free such block");
     _reiserfs_free_block(th, block) ;
@@ -589,15 +589,15 @@
 
 static inline int old_hashed_relocation (reiserfs_blocknr_hint_t * hint)
 {
-    unsigned long border;
-    unsigned long hash_in;
+    b_blocknr_t border;
+    u32 long hash_in;
     
     if (hint->formatted_node || hint->inode == NULL) {
 	return 0;
       }
 
     hash_in = le32_to_cpu((INODE_PKEY(hint->inode))->k_dir_id);
-    border = hint->beg + (unsigned long) keyed_hash(((char *) (&hash_in)), 4) % (hint->end - hint->beg - 1);
+    border = hint->beg + (u32) keyed_hash(((char *) (&hash_in)), 4) % (hint->end - hint->beg - 1);
     if (border > hint->search_start)
 	hint->search_start = border;
 
@@ -606,7 +606,7 @@
   
 static inline int old_way (reiserfs_blocknr_hint_t * hint)
 {
-    unsigned long border;
+    b_blocknr_t border;
     
     if (hint->formatted_node || hint->inode == NULL) {
 	return 0;
@@ -622,7 +622,7 @@
 static inline void hundredth_slices (reiserfs_blocknr_hint_t * hint)
 {
     struct key * key = &hint->key;
-    unsigned long slice_start;
+    b_blocknr_t slice_start;
 
     slice_start = (keyed_hash((char*)(&key->k_dir_id),4) % 100) * (hint->end / 100);
     if ( slice_start > hint->search_start || slice_start + (hint->end / 100) <= hint->search_start) {
@@ -910,7 +910,7 @@
 int reiserfs_can_fit_pages ( struct super_block *sb /* superblock of filesystem
 						       to estimate space */ )
 {
-	unsigned long space;
+	b_blocknr_t space;
 
 	spin_lock(&REISERFS_SB(sb)->bitmap_lock);
 	space = (SB_FREE_BLOCKS(sb) - REISERFS_SB(sb)->reserved_blocks) >> ( PAGE_CACHE_SHIFT - sb->s_blocksize_bits);
===== fs/reiserfs/do_balan.c 1.17 vs edited =====
--- 1.17/fs/reiserfs/do_balan.c	Sun May 18 01:09:35 2003
+++ edited/fs/reiserfs/do_balan.c	Fri Jun 27 17:07:28 2003
@@ -1250,12 +1250,12 @@
 
 static void free_thrown(struct tree_balance *tb) {
     int i ;
-    unsigned long blocknr ;
+    b_blocknr_t blocknr ;
     for (i = 0; i < sizeof (tb->thrown)/sizeof (tb->thrown[0]); i++) {
 	if (tb->thrown[i]) {
 	    blocknr = tb->thrown[i]->b_blocknr ;
 	    if (buffer_dirty (tb->thrown[i]))
-	      printk ("free_thrown deals with dirty buffer %ld\n", blocknr);
+	      printk ("free_thrown deals with dirty buffer %d\n", blocknr);
 	    brelse(tb->thrown[i]) ; /* incremented in store_thrown */
 	    reiserfs_free_block (tb->transaction_handle, blocknr);
 	}
@@ -1339,7 +1339,7 @@
 
 #ifdef CONFIG_REISERFS_CHECK
 
-int is_reusable (struct super_block * s, unsigned long block, int bit_value);
+int is_reusable (struct super_block * s, b_blocknr_t block, int bit_value);
 static void check_internal_node (struct super_block * s, struct buffer_head * bh, char * mes)
 {
   struct disk_child * dc;
===== fs/reiserfs/fix_node.c 1.28 vs edited =====
--- 1.28/fs/reiserfs/fix_node.c	Tue Sep  3 10:28:56 2002
+++ edited/fs/reiserfs/fix_node.c	Fri Jun 27 17:07:12 2003
@@ -758,7 +758,7 @@
             ) {
   struct buffer_head  * p_s_new_bh,
     		      *	p_s_Sh = PATH_H_PBUFFER (p_s_tb->tb_path, n_h);
-  unsigned long	      *	p_n_blocknr,
+  b_blocknr_t	      *	p_n_blocknr,
     			a_n_blocknrs[MAX_AMOUNT_NEEDED] = {0, };
   int       		n_counter,
    			n_number_of_freeblk,
@@ -879,7 +879,7 @@
             ) {
   struct buffer_head  * p_s_father, * left;
   struct super_block  * p_s_sb = p_s_tb->tb_sb;
-  unsigned long         n_left_neighbor_blocknr;
+  b_blocknr_t		n_left_neighbor_blocknr;
   int                   n_left_neighbor_position;
 
   if ( ! p_s_tb->FL[n_h] ) /* Father of the left neighbor does not exist. */
@@ -2501,7 +2501,7 @@
     /* deal with list of allocated (used and unused) nodes */
     for ( i = 0; i < MAX_FEB_SIZE; i++ ) {
 	if ( tb->FEB[i] ) {
-	    unsigned long blocknr  = tb->FEB[i]->b_blocknr ;
+	    b_blocknr_t blocknr  = tb->FEB[i]->b_blocknr ;
 	    /* de-allocated block which was not used by balancing and
                bforget about buffer for it */
 	    brelse (tb->FEB[i]);
===== fs/reiserfs/inode.c 1.78 vs edited =====
--- 1.78/fs/reiserfs/inode.c	Wed Jun  4 11:50:34 2003
+++ edited/fs/reiserfs/inode.c	Fri Jun 27 15:55:43 2003
@@ -506,7 +506,7 @@
 			struct buffer_head * bh_result, int create)
 {
     int repeat, retval;
-    b_blocknr_t allocated_block_nr = 0;// b_blocknr_t is unsigned long
+    b_blocknr_t allocated_block_nr = 0;// b_blocknr_t is (unsigned) 32 bit int
     INITIALIZE_PATH(path);
     int pos_in_item;
     struct cpu_key key;
===== fs/reiserfs/journal.c 1.69 vs edited =====
--- 1.69/fs/reiserfs/journal.c	Wed Jun  4 11:50:34 2003
+++ edited/fs/reiserfs/journal.c	Fri Jun 27 17:11:29 2003
@@ -507,7 +507,7 @@
 */
 int reiserfs_in_journal(struct super_block *p_s_sb,
                         int bmap_nr, int bit_nr, int search_all, 
-			unsigned long *next_zero_bit) {
+			b_blocknr_t *next_zero_bit) {
   struct reiserfs_journal_cnode *cn ;
   struct reiserfs_list_bitmap *jb ;
   int i ;
@@ -761,7 +761,7 @@
 */
 static struct reiserfs_journal_list *find_newer_jl_for_cn(struct reiserfs_journal_cnode *cn) {
   struct super_block *sb = cn->sb;
-  unsigned long blocknr = cn->blocknr ;
+  b_blocknr_t blocknr = cn->blocknr ;
 
   cn = cn->hprev ;
   while(cn) {
@@ -791,7 +791,7 @@
   while(cn) {
     if (cn->blocknr != 0) {
       if (debug) {
-        printk("block %lu, bh is %d, state %ld\n", cn->blocknr, cn->bh ? 1: 0, 
+        printk("block %u, bh is %d, state %ld\n", cn->blocknr, cn->bh ? 1: 0, 
 	        cn->state) ;
       }
       cn->state = 0 ;
@@ -1105,7 +1105,7 @@
 {
     struct reiserfs_journal_list *pjl ; /* previous list for this cn */
     struct reiserfs_journal_cnode *cn, *walk_cn ;
-    unsigned long blocknr ;
+    b_blocknr_t blocknr ;
     int run = 0 ;
     int orig_trans_id = jl->j_trans_id ;
     struct buffer_head *saved_bh ; 
@@ -2421,7 +2421,7 @@
 **
 ** returns 1 if it cleaned and relsed the buffer. 0 otherwise
 */
-static int remove_from_transaction(struct super_block *p_s_sb, unsigned long blocknr, int already_cleaned) {
+static int remove_from_transaction(struct super_block *p_s_sb, b_blocknr_t blocknr, int already_cleaned) {
   struct buffer_head *bh ;
   struct reiserfs_journal_cnode *cn ;
   int ret = 0;
@@ -2474,7 +2474,7 @@
 */
 static int can_dirty(struct reiserfs_journal_cnode *cn) {
   struct super_block *sb = cn->sb;
-  unsigned long blocknr = cn->blocknr  ;
+  b_blocknr_t blocknr = cn->blocknr  ;
   struct reiserfs_journal_cnode *cur = cn->hprev ;
   int can_dirty = 1 ;
   
@@ -2710,7 +2710,7 @@
 **
 ** Then remove it from the current transaction, decrementing any counters and filing it on the clean list.
 */
-int journal_mark_freed(struct reiserfs_transaction_handle *th, struct super_block *p_s_sb, unsigned long blocknr) {
+int journal_mark_freed(struct reiserfs_transaction_handle *th, struct super_block *p_s_sb, b_blocknr_t blocknr) {
   struct reiserfs_journal_cnode *cn = NULL ;
   struct buffer_head *bh = NULL ;
   struct reiserfs_list_bitmap *jb = NULL ;
@@ -2719,7 +2719,7 @@
   if (reiserfs_dont_log(th->t_super)) {
     bh = sb_find_get_block(p_s_sb, blocknr) ;
     if (bh && buffer_dirty (bh)) {
-      printk ("journal_mark_freed(dont_log): dirty buffer on hash list: %lx %ld\n", bh->b_state, blocknr);
+      printk ("journal_mark_freed(dont_log): dirty buffer on hash list: %lx %d\n", bh->b_state, blocknr);
       BUG ();
     }
     brelse (bh);
===== include/linux/reiserfs_fs.h 1.49 vs edited =====
--- 1.49/include/linux/reiserfs_fs.h	Wed Jun  4 11:50:34 2003
+++ edited/include/linux/reiserfs_fs.h	Fri Jun 27 17:01:42 2003
@@ -269,7 +269,7 @@
 #define NO_BALANCING_NEEDED  (-4)
 #define NO_MORE_UNUSED_CONTIGUOUS_BLOCKS (-5)
 
-typedef unsigned long b_blocknr_t;
+typedef __u32 b_blocknr_t;
 typedef __u32 unp_t;
 
 struct unfm_nodeinfo {
@@ -1730,11 +1730,11 @@
 int journal_end(struct reiserfs_transaction_handle *, struct super_block *, unsigned long) ;
 int journal_end_sync(struct reiserfs_transaction_handle *, struct super_block *, unsigned long) ;
 int journal_mark_dirty_nolog(struct reiserfs_transaction_handle *, struct super_block *, struct buffer_head *bh) ;
-int journal_mark_freed(struct reiserfs_transaction_handle *, struct super_block *, unsigned long blocknr) ;
+int journal_mark_freed(struct reiserfs_transaction_handle *, struct super_block *, b_blocknr_t blocknr) ;
 int push_journal_writer(char *w) ;
 int pop_journal_writer(int windex) ;
 int journal_transaction_should_end(struct reiserfs_transaction_handle *, int) ;
-int reiserfs_in_journal(struct super_block *p_s_sb, int bmap_nr, int bit_nr, int searchall, unsigned long *next) ;
+int reiserfs_in_journal(struct super_block *p_s_sb, int bmap_nr, int bit_nr, int searchall, b_blocknr_t *next) ;
 int journal_begin(struct reiserfs_transaction_handle *, struct super_block *p_s_sb, unsigned long) ;
 void flush_async_commits(struct super_block *p_s_sb) ;
 
@@ -2105,8 +2105,8 @@
 typedef struct __reiserfs_blocknr_hint reiserfs_blocknr_hint_t;
 
 int reiserfs_parse_alloc_options (struct super_block *, char *);
-int is_reusable (struct super_block * s, unsigned long block, int bit_value);
-void reiserfs_free_block (struct reiserfs_transaction_handle *th, unsigned long);
+int is_reusable (struct super_block * s, b_blocknr_t block, int bit_value);
+void reiserfs_free_block (struct reiserfs_transaction_handle *th, b_blocknr_t);
 int reiserfs_allocate_blocknrs(reiserfs_blocknr_hint_t *, b_blocknr_t * , int, int);
 extern inline int reiserfs_new_form_blocknrs (struct tree_balance * tb,
 					      b_blocknr_t *new_blocknrs, int amount_needed)
===== include/linux/reiserfs_fs_sb.h 1.23 vs edited =====
--- 1.23/include/linux/reiserfs_fs_sb.h	Wed Jun  4 11:50:34 2003
+++ edited/include/linux/reiserfs_fs_sb.h	Fri Jun 27 17:10:56 2003
@@ -133,7 +133,7 @@
 struct reiserfs_journal_cnode {
   struct buffer_head *bh ;		 /* real buffer head */
   struct super_block *sb ;		 /* dev of real buffer head */
-  unsigned long blocknr ;		 /* block number of real buffer head, == 0 when buffer on disk */		 
+  __u32 blocknr ;		 /* block number of real buffer head, == 0 when buffer on disk */		 
   long state ;
   struct reiserfs_journal_list *jlist ;  /* journal list this cnode lives in */
   struct reiserfs_journal_cnode *next ;  /* next in transaction list */

Bye,
    Oleg

[-- Attachment #2: test.c --]
[-- Type: text/plain, Size: 923 bytes --]

#include <stdio.h>
#include <sys/types.h>
#define V1_DIRECT_UNIQUENESS 0xffffffff
#define V1_DIRENTRY_UNIQUENESS 500
#define V1_ANY_UNIQUENESS 555 // FIXME: comment is required
#define V1_INDIRECT_UNIQUENESS 0xfffffffe
#define V1_SD_UNIQUENESS 0

#define TYPE_STAT_DATA 0
#define TYPE_INDIRECT 1
#define TYPE_DIRECT 2
#define TYPE_DIRENTRY 3
#define TYPE_MAXTYPE 3
#define TYPE_ANY 15 // FIXME: comment is required

static inline int uniqueness2type (unsigned int uniqueness)
{
    switch (uniqueness) {
    case V1_SD_UNIQUENESS: return TYPE_STAT_DATA;
    case V1_INDIRECT_UNIQUENESS: return TYPE_INDIRECT;
    case V1_DIRECT_UNIQUENESS: return TYPE_DIRECT;
    case V1_DIRENTRY_UNIQUENESS: return TYPE_DIRENTRY;
    default:
printf("vs-500: unknown uniqueness %d\n", uniqueness);
return TYPE_ANY;
    }
}

int main(void)
{
unsigned int a=V1_INDIRECT_UNIQUENESS;
printf("result is %d\n", uniqueness2type(a));
return 0;
}

Re: vpf-10680, minor corruptions

[Chris Mason]

On Fri, 2003-06-27 at 12:13, Oleg Drokin wrote:
> Hello!
> 
> On Fri, Jun 27, 2003 at 04:38:00PM +0400, Oleg Drokin wrote:
> 
> > I was looking in the wrong direction, when I produced that patch,
> > so it will produce zero output.
> > I hope to come up with ultimate fix soon enough. ;)
> 
> Well, there is a patch below that does *not* work for me ;)
> But it should work.
> I have traced the new problem to a cross compiler that compiles
> code in a different way than native compiler for whatever reason
> (demo is attached as test.c program, it should print "result is 1"
> in case it is compiled correctly and stuff about unknown
> uniqueness if it is miscompiled. In fact may be this is just correct compiler behaviour.)
> I now think that when I compile a kernel with native compiler, it should work
> with below patch. But I can verify that only tomorrow it seems.
> You might try that patch as well to see if it helps you before I try it ;)
> The patch is "obviously correct" one. (except that it does not work
> with my cross compiler and kernel does work without patch which is really-really strange).
> 

Most of these changes are in 2.4.21, which I've been using on an AMD64
bit box for a while without any problems.  The bug should be somewhere
else, it looks to me like these spots aren't trying to send an unsigned
long to disk.

-chris

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Fri, Jun 27, 2003 at 12:23:07PM -0400, Chris Mason wrote:

> Most of these changes are in 2.4.21, which I've been using on an AMD64

Not the reiserfs_file_write() ones.

> bit box for a while without any problems.  The bug should be somewhere
> else, it looks to me like these spots aren't trying to send an unsigned
> long to disk.

the reiserfs_file_write() code
have an array of b_blocknr_t elements.
It then submits this array to reiserfs_paste_into_item/reiserfs_insert_item,
but b_blocknr_t is unsigned long (read - 64 bit on alpha - oops).
Funny thing is when I declare b_blocknr_t as u32, kernel basically falls apart
if cross compiled. E.g. key comparison does not work and
all kind of weird things start to happen.

In short - if you want to make sure the bug is there - compile 2.5.70+ code
on any 64 bit platform, write any file bigger than 2 blocks,
unmount and remount the fs and see what's in the file.

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Christian Kujau]

Oleg Drokin schrieb:
> I have traced the new problem to a cross compiler that compiles
> code in a different way than native compiler for whatever reason
> (demo is attached as test.c program, it should print "result is 1"

yes, that what it prints, no warnings were shown.

> You might try that patch as well to see if it helps you before I try it ;)

yes, compiling with _this_ patch but _not_ with the last patch you sent 
(file.c) is under way again...


Thank you,
Christian.

Re: vpf-10680, minor corruptions

[Christian Kujau]

Oleg Drokin schrieb:
> I have traced the new problem to a cross compiler that compiles
> code in a different way than native compiler for whatever reason
> (demo is attached as test.c program, it should print "result is 1"

yes, that what it prints, no warnings were shown.

> You might try that patch as well to see if it helps you before I try it ;)

yes, compiling with _this_ patch but _not_ with the last patch you sent 
(file.c) is under way again...


Thank you,
Christian.

Re: vpf-10680, minor corruptions

[Christian Kujau]

hi,

2.5.72 with your patch applied compiled and booted.
but, i'm almost afraid to say it: vpf-10680 is not gone :-(

https://ephigenie.kicks-ass.net/browse/reiserfs/vpf-10680/test.log

is the output of "reiserfsck -l test.log test.img".
test.img is a 200MB file, right after booting i created a fs on it and 
copied files to it.

again, i'll put some files under ../reiserfs/diffs2 just in case the 
corruptions are different now.

sorry,
Christian.



evil@prinz:~$ ssh alpha

BOFH excuse #439:

Hot Java has gone cold

lila:~# uname -a
Linux lila 2.5.72 #8 Fri Jun 27 19:39:58 CEST 2003 alpha GNU/Linux
lila:~#

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Fri, Jun 27, 2003 at 09:20:57PM +0200, Christian Kujau wrote:

> 2.5.72 with your patch applied compiled and booted.
> but, i'm almost afraid to say it: vpf-10680 is not gone :-(
> https://ephigenie.kicks-ass.net/browse/reiserfs/vpf-10680/test.log
> is the output of "reiserfsck -l test.log test.img".
> test.img is a 200MB file, right after booting i created a fs on it and 
> copied files to it.
> again, i'll put some files under ../reiserfs/diffs2 just in case the 
> corruptions are different now.

Well, the corruptions are different indeed.
Can you also publish debugreiserfs -d /dev/whateverdev
output (it is preferable that there would be new FS with only
one or two files written and immediately unmounted).

Thank you.

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Christian Kujau]

Oleg Drokin schrieb:
> Well, the corruptions are different indeed.
> Can you also publish debugreiserfs -d /dev/whateverdev
> output (it is preferable that there would be new FS with only
> one or two files written and immediately unmounted).

i've done so, debugreiserfs-test.stdout is in ../reiserfs on the known
address. let me repeat, that not _all_ files in the fs are corrupted.
the two textfiles i have just written into the newly created reiserfs
are _not_ different from their origins. however, when i copy many files,
a certain amount of files get corrupted.

does it make a difference to test on a file-image or is it better to
continue testing on a "real" block-device?

Thanks,
Chrsitian.

PS: may i ask you, in what timezone you are? i'm not "always online"; to
know *when* you could read my mail could minimize the delay it takes me
to reply :-)

Re: vpf-10680, minor corruptions

[Christian Kujau]

Christian Kujau schrieb:
> address. let me repeat, that not _all_ files in the fs are corrupted.
> the two textfiles i have just written into the newly created reiserfs
> are _not_ different from their origins. however, when i copy many files,
> a certain amount of files get corrupted.
> 

that said, i've done some testings (yes, i really have time-> holidays!)
to find out the amount of files triggering the corruptions. started with 
10, 20, 200 (text)files gave me _no_ corruptions. i was already about 
doubting myself, that i copied again my /lib to the fresh fs:

lila:~# mount -o loop test.img /mnt/bench/
lila:~# /bin/cp -a /lib/ /mnt/bench/
lila:~# find /lib/ | wc -l
     475
lila:~# diff -r /lib/ /mnt/bench/lib/ | wc -l
diff: /lib/modules/2.5.65/build: No such file or directory
diff: /mnt/bench/lib/modules/2.5.65/build: No such file or directory
diff: /lib/modules/2.5.69/build: No such file or directory
diff: /mnt/bench/lib/modules/2.5.69/build: No such file or directory
diff: /lib/modules/2.5.70/build: No such file or directory
diff: /mnt/bench/lib/modules/2.5.70/build: No such file or directory
     168
lila:~#

(the warnings are ok, these are correct dead links.)

so, it's 168 out of 475--> 35%. but i still don't know the exact amount 
of file needed to get a corruption. (even with 200 files i should get 70 
corruptions...)

if that would be important/helpful i could do further tests...

Christian.

Re: vpf-10680, minor corruptions

[Christian Kujau]

Christian Kujau schrieb:
> address. let me repeat, that not _all_ files in the fs are corrupted.
> the two textfiles i have just written into the newly created reiserfs
> are _not_ different from their origins. however, when i copy many files,
> a certain amount of files get corrupted.
> 

that said, i've done some testings (yes, i really have time-> holidays!)
to find out the amount of files triggering the corruptions. started with 
10, 20, 200 (text)files gave me _no_ corruptions. i was already about 
doubting myself, that i copied again my /lib to the fresh fs:

lila:~# mount -o loop test.img /mnt/bench/
lila:~# /bin/cp -a /lib/ /mnt/bench/
lila:~# find /lib/ | wc -l
     475
lila:~# diff -r /lib/ /mnt/bench/lib/ | wc -l
diff: /lib/modules/2.5.65/build: No such file or directory
diff: /mnt/bench/lib/modules/2.5.65/build: No such file or directory
diff: /lib/modules/2.5.69/build: No such file or directory
diff: /mnt/bench/lib/modules/2.5.69/build: No such file or directory
diff: /lib/modules/2.5.70/build: No such file or directory
diff: /mnt/bench/lib/modules/2.5.70/build: No such file or directory
     168
lila:~#

(the warnings are ok, these are correct dead links.)

so, it's 168 out of 475--> 35%. but i still don't know the exact amount 
of file needed to get a corruption. (even with 200 files i should get 70 
corruptions...)

if that would be important/helpful i could do further tests...

Christian.

Re: vpf-10680, minor corruptions

[Christian Kujau]

Christian Kujau schrieb:
> address. let me repeat, that not _all_ files in the fs are corrupted.
> the two textfiles i have just written into the newly created reiserfs
> are _not_ different from their origins. however, when i copy many files,
> a certain amount of files get corrupted.
> 

that said, i've done some testings (yes, i really have time-> holidays!)
to find out the amount of files triggering the corruptions. started with 
10, 20, 200 (text)files gave me _no_ corruptions. i was already about 
doubting myself, that i copied again my /lib to the fresh fs:

lila:~# mount -o loop test.img /mnt/bench/
lila:~# /bin/cp -a /lib/ /mnt/bench/
lila:~# find /lib/ | wc -l
     475
lila:~# diff -r /lib/ /mnt/bench/lib/ | wc -l
diff: /lib/modules/2.5.65/build: No such file or directory
diff: /mnt/bench/lib/modules/2.5.65/build: No such file or directory
diff: /lib/modules/2.5.69/build: No such file or directory
diff: /mnt/bench/lib/modules/2.5.69/build: No such file or directory
diff: /lib/modules/2.5.70/build: No such file or directory
diff: /mnt/bench/lib/modules/2.5.70/build: No such file or directory
     168
lila:~#

(the warnings are ok, these are correct dead links.)

so, it's 168 out of 475--> 35%. but i still don't know the exact amount 
of file needed to get a corruption. (even with 200 files i should get 70 
corruptions...)

if that would be important/helpful i could do further tests...

Christian.

Re: vpf-10680, minor corruptions (sorry)

[Christian Kujau]

sorry,

my last mail was sent 3 times, i'll blame mozilla this time.

and: i won't stop coming up with odd phenomenas regarding this issue.
while playing around with mkreiserfs/cp/diff i have to say that i 
sometimes copy files with "mc", a curses based file-manager (Midnight 
Commander, much like NortonCommander, i think it's very a common tool).

then, when copying files with this mc i do _not_ get corruptions on the 
fs. yes, it's reproduceable. no differences are shown (diff) and 
reiserfsck shows no corruptions. i can copy the whole /var/lib to this 
fs (which fills it up to 80% or so) with "mc" - no corruptions.
i copy 7MB (35 files) of textfiles to the fs (always after a fresh 
mkreiserfs) with "cp" and i get differences (*) and corruptions reported 
by reiserfsck.

this is odd, i don't know how "mc" copies files internally, it's perhaps 
not that important, but i wanted to share these results, just in case 
you/sbd. else wants to reproduce corruptions and cannot...

Thanks,
Christian.

(*) here: 2 files out of 35 differ when using "cp".

lila:~# cp --version
cp (coreutils) 5.0

lila:~# mc -V
GNU Midnight Commander 4.6.0
Virtual File System: tarfs, extfs, cpiofs, ftpfs, fish
With builtin Editor
Using included S-Lang library with terminfo database
With subshell support as default
With support for background operations
With mouse support on xterm and Linux console
With internationalization support

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Sat, Jun 28, 2003 at 12:37:35AM +0200, Christian Kujau wrote:

> >Well, the corruptions are different indeed.
> >Can you also publish debugreiserfs -d /dev/whateverdev
> >output (it is preferable that there would be new FS with only
> >one or two files written and immediately unmounted).
> i've done so, debugreiserfs-test.stdout is in ../reiserfs on the known
> address. let me repeat, that not _all_ files in the fs are corrupted.
> the two textfiles i have just written into the newly created reiserfs
> are _not_ different from their origins. however, when i copy many files,
> a certain amount of files get corrupted.

Well, looking at this metadata dump, I cann reasonably assure you that
this dump was taken from fs that was written to without first applying
the patch I sent. (same block, zero, block, zero pattern in indirect items)

In other mails you write that now you see only some of files are differs
(as opposed to every file gets corrupted). Or may be there is a pattern
like "files less than 8k in size are uncorrupted and everything bigger
is corrupted"?
So can I look at the debugreiserfs -d output for the filesystm where
that happens (and I want to know filename of a corrupted file too).

My own alpha boot attempts are doomed it seems. the kernel I build on alpha itself
jumps to address zero before starting init. Hm. Or may be it is just unhappy with gcc 3.2.2,
time to build gcc 2.95.3 there, it seems.

Thank you.

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Christian Kujau]

Oleg Drokin schrieb:
> Well, looking at this metadata dump, I cann reasonably assure you that
> this dump was taken from fs that was written to without first applying
> the patch I sent. (same block, zero, block, zero pattern in indirect items)

?
should i have missed something? this is bad; so just now i've fetched a 
vanilla 2.5.70, patched ist up to 2.5.72 and applied your patch (again).
damn, will take another 3h to compile now.

> In other mails you write that now you see only some of files are differs
> (as opposed to every file gets corrupted). Or may be there is a pattern
> like "files less than 8k in size are uncorrupted and everything bigger
> is corrupted"?

hm, the fact that not _all_ files get corrupted was written in some 
early mail of mine:

[from 20.06.2003]
 > lila:~# cp -a /lib/ /var/cache/apt-proxy/temp/ | wc -l
 >     329
 > lila:~#
 >
 > (with sde2 mounted on /var/cache/apt-proxy/ (reiserfs).
 > then i did:
 >
 > lila:~# diff -r /lib/ /var/cache/apt-proxy/temp/lib/ | wc -l
 >      62
 > lila:~#

i just wanted to repeat that, because when you asked me to copy only 2 
files to the fs, then do the debugreiserfs, the 2 files were not corrupted!

> So can I look at the debugreiserfs -d output for the filesystm where
> that happens (and I want to know filename of a corrupted file too).

ok, i'll provide this information right after the compile finished.

> My own alpha boot attempts are doomed it seems. the kernel I build on alpha itself
> jumps to address zero before starting init. Hm. Or may be it is just unhappy with gcc 3.2.2,
> time to build gcc 2.95.3 there, it seems.

do you still have my .config file ? on my Avanti the option 
"CONFIG_ALPHA_LEGACY_START_ADDRESS=y" was somehow important to get the 
alpha even started. and perhaps CONFIG_ALPHA_SRM=y too.

Thanks,
Christian.

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Sat, Jun 28, 2003 at 01:23:44PM +0200, Christian Kujau wrote:

> do you still have my .config file ? on my Avanti the option 

Yes, I do. But we have Ruffian alpha.

> "CONFIG_ALPHA_LEGACY_START_ADDRESS=y" was somehow important to get the 
> alpha even started. and perhaps CONFIG_ALPHA_SRM=y too.

I have a config. When I build a kernel using cross compiler - it boots just fine.
If I build kernel using native compiler - it dies when trying to start init.
Config is the same in both cases.

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Christian Kujau]

hi,

your patch applied with no warnings, i really seem to have _not_ applied 
the patch last time (sorry!),  because now i get a compiler error:


   LD      fs/proc/proc.o
   LD      fs/proc/built-in.o
   CC      fs/ramfs/inode.o
   LD      fs/ramfs/ramfs.o
   LD      fs/ramfs/built-in.o
   CC      fs/reiserfs/bitmap.o
fs/reiserfs/bitmap.c: In function `old_hashed_relocation':
fs/reiserfs/bitmap.c:593: error: long, short, signed or unsigned used 
invalidly for `hash_in'
make[3]: *** [fs/reiserfs/bitmap.o] Error 1
make[2]: *** [fs/reiserfs] Error 2
make[1]: *** [fs] Error 2
make: *** [vmlinux] Error 2

real	32m56.960s
user	26m37.120s
sys	1m44.777s
lila:/usr/src/linux-2.5#

this is still with gcc-3.3, and:

lila:/usr/src/linux-2.5# ld -V
GNU ld version 2.14.90.0.4 20030523 Debian GNU/Linux
   Supported emulations:
    elf64alpha
    alpha


i could use another compiler, though, if that would be helpful.

Thank you,
Christian.

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Sat, Jun 28, 2003 at 01:45:57PM +0200, Christian Kujau wrote:

> your patch applied with no warnings, i really seem to have _not_ applied 
> the patch last time (sorry!),  because now i get a compiler error:
>   LD      fs/proc/proc.o
>   LD      fs/proc/built-in.o
>   CC      fs/ramfs/inode.o
>   LD      fs/ramfs/ramfs.o
>   LD      fs/ramfs/built-in.o
>   CC      fs/reiserfs/bitmap.o
> fs/reiserfs/bitmap.c: In function `old_hashed_relocation':
> fs/reiserfs/bitmap.c:593: error: long, short, signed or unsigned used 
> invalidly for `hash_in'
> make[3]: *** [fs/reiserfs/bitmap.o] Error 1
> make[2]: *** [fs/reiserfs] Error 2
> make[1]: *** [fs] Error 2
> make: *** [vmlinux] Error 2

Aha, there is "u32 long" in line 593 in fs/reiserfs/bitmap.c file.
Remove the "long" word and it should compile.
Strange enough that it compiles with gcc 2.95 and gcc 3.2.2

Hm, interesting that I compiled gcc 2.95.3 on my alpha and still the kernel
built with this gcc dies aprox at a time when starting init too (jumps to zero again).

Thank you.

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Christian Kujau]

Oleg Drokin schrieb:
> Aha, there is "u32 long" in line 593 in fs/reiserfs/bitmap.c file.
> Remove the "long" word and it should compile.

ok, to make sure i've done the right thing, i'll put your patch with 
this change on this webserver.

> Strange enough that it compiles with gcc 2.95 and gcc 3.2.2

hehe 8-)

> Hm, interesting that I compiled gcc 2.95.3 on my alpha and still the kernel
> built with this gcc dies aprox at a time when starting init too (jumps to zero again).

oh, when init starts up, so the options i called were for booting only. 
i never had probs with init...


thanks,
Christian.

Re: vpf-10680, minor corruptions

[Christian Kujau]

Christian Kujau schrieb:
> ok, to make sure i've done the right thing, i'll put your patch with 
> this change on this webserver.

i meant: i'll put bitmap.c in

https://ephigenie.kicks-ass.net/browse/reiserfs/2.5.72-oleg/bitmap.c_removed-long

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Sat, Jun 28, 2003 at 02:15:56PM +0200, Christian Kujau wrote:
> >ok, to make sure i've done the right thing, i'll put your patch with 
> >this change on this webserver.
> i meant: i'll put bitmap.c in
> https://ephigenie.kicks-ass.net/browse/reiserfs/2.5.72-oleg/bitmap.c_removed-long

They are the same as my fixed version now:

green@angband:~/bk/linux-2.5-alpha/fs/reiserfs> diff -u bitmap.c_removed-long bitmap.c
green@angband:~/bk/linux-2.5-alpha/fs/reiserfs>

Bye,
    Oleg

Re: vpf-10680, minor corruptions (solved?)

[Christian Kujau]

Hi Oleg,

2.5.72 with your patch applied booted just fine and a freshly created 
reiserfs on test.img brought no differences any more! Yes!

reiserfsck showed no corruptions, so the issue seems to be solved :-)

i'll provide a output of "debugreiserfs -d test.img" after a fresh 
mkreiserfs on the very known address; tell me if you need any further 
details.

happy,
Christian.

Re: vpf-10680, minor corruptions (solved?)

[Oleg Drokin]

Hello!

On Sat, Jun 28, 2003 at 05:52:12PM +0200, Christian Kujau wrote:

> 2.5.72 with your patch applied booted just fine and a freshly created 
> reiserfs on test.img brought no differences any more! Yes!

Good. Can you stress it for some time just to be sure?

> reiserfsck showed no corruptions, so the issue seems to be solved :-)

Good, if only I can be able to verify that myself...
I guess I need to try on Itanium, may be I will be more lucky with it.

> i'll provide a output of "debugreiserfs -d test.img" after a fresh 
> mkreiserfs on the very known address; tell me if you need any further 
> details.

No need for that. debugreiserfs output is sometimes useful when something goes
wrong.

Thanks for all the testing.

Bye,
    Oleg

Re: vpf-10680, minor corruptions (solved!)

[Christian Kujau]

Oleg Drokin wrote:
> Good. Can you stress it for some time just to be sure?

i've done so, a kernel compile on a fresh reierfs finished with no 
errors, and in such a compile *a lot* of files are created/read, i guess 
the compile would horrible fail if there were corruptions in any of them.

and yes, if i ever stumble again over such an odd issue, i know your 
mail address :-[]

Thanks again for this bug hunting, i think many other 64bit users will 
be happy too.

Regards,
Christian.
-- 
BOFH excuse #274:

It was OK before you touched it.

Re: vpf-10680, minor corruptions (solved!)

[Oleg Drokin]

Hello!

On Mon, Jun 30, 2003 at 02:31:03AM +0200, Christian Kujau wrote:

> >Good. Can you stress it for some time just to be sure?
> i've done so, a kernel compile on a fresh reierfs finished with no 
> errors, and in such a compile *a lot* of files are created/read, i guess 
> the compile would horrible fail if there were corruptions in any of them.

Well, it seems you will only see corruptions when your workload does not fit the RAM.

> and yes, if i ever stumble again over such an odd issue, i know your 
> mail address :-[]

Sure, let us know if something goes wrong.

Bye,
    Oleg

Re: vpf-10680, minor corruptions

[Oleg Drokin]

Hello!

On Wed, Jun 25, 2003 at 03:17:42PM +0200, Christian Kujau wrote:

> under both (2.5.65+2.5.69) i was able to mkreiserfs sde2. mounting the 
> fs went ok, but copying data (cp -a /lib /mnt/reiserfs) brought several 
> kernel-errors (see https://ephigenie.kicks-ass.net/browse/reiserfs/).

This "buffer layer error" problem was fixed in 2.5.71 or something like that, I think.
I found it while merging "variable blocksize" patch. (that allowed me to have
the same situation in which you are - blocksize is less than pagesize, only on x86)

> but: diff -r showed _no_ differences betweeen the directories, a 
> following reiserfsck brought no vpf-10680 anymore!
> so i'd say the problem occurs somewhere between 2.5.69 and 2.5.70.

Hm. Ok, thank you for the testing.
I am going to read the patches submitted at that time again.

Bye,
    Oleg