mkreiserfs and big RAID-Systems

mkreiserfs and big RAID-Systems

[Andreas Abele]

Hello,

I have a RAID5-Set with 840 GB netto here.

1.) fdisk shows:

Festplatte /dev/sdf: 255 Köpfe, 63 Sektoren, 104623 Zylinder
Einheiten: Zylinder mit 16065 * 512 Bytes
    Device boot.  Start       End    Blocks   Id  Dateisystemtyp
/dev/sdf1   *         1    104623 840384216   83  Linux
                                  ^^^^^^^^^
2.) mkreiserfs /dev/sdf1
3.) mount /dev/sdf1 /users1/p1
4.) df /dev/sdf1
Dateisystem          1K-Blöcke    Benutzt Verfügbar Ben% montiert auf
/dev/sdf1            720312060     32840 720279220   1% /users1/p1
                     ^^^^^^^^^

Where are the remaining 120 GBytes?

Cross-Test with ext2

2a.) mke2fs /dev/sdf1
3a.) mount /dev/sdf1 /users1/p1
4a.) df /dev/sdf1
Dateisystem          1K-Blöcke    Benutzt Verfügbar Ben% montiert auf
/dev/sdf1            827197400        20 785178172   1% /users1/p1
                     ^^^^^^^^^

What is happening here? When i run mkreiserfs, the programs last
blocknumber seen is 180060160, which is exactly a1/4 of the mounted
kilobytes (4k-Blocks okay). But where are the remaining 120 G-Blocks?

Any ideas? According to the reiserfs-specs a filesystem can hold 17.6
TBytes.

kind regards
Andreas Abele

Re: mkreiserfs and big RAID-Systems

[Vitaly Fertman]

HI, 

which reiserfsprogs version do you use?
what does 
	debugreiserfs /dev/sdf1
say?

-- 

Thanks,
Vitaly Fertman

On Monday 15 July 2002 17:59, Andreas Abele wrote:
> Hello,
>
> I have a RAID5-Set with 840 GB netto here.
>
> 1.) fdisk shows:
>
> Festplatte /dev/sdf: 255 Köpfe, 63 Sektoren, 104623 Zylinder
> Einheiten: Zylinder mit 16065 * 512 Bytes
>     Device boot.  Start       End    Blocks   Id  Dateisystemtyp
> /dev/sdf1   *         1    104623 840384216   83  Linux
>                                   ^^^^^^^^^
> 2.) mkreiserfs /dev/sdf1
> 3.) mount /dev/sdf1 /users1/p1
> 4.) df /dev/sdf1
> Dateisystem          1K-Blöcke    Benutzt Verfügbar Ben% montiert auf
> /dev/sdf1            720312060     32840 720279220   1% /users1/p1
>                      ^^^^^^^^^
>
> Where are the remaining 120 GBytes?
>
> Cross-Test with ext2
>
> 2a.) mke2fs /dev/sdf1
> 3a.) mount /dev/sdf1 /users1/p1
> 4a.) df /dev/sdf1
> Dateisystem          1K-Blöcke    Benutzt Verfügbar Ben% montiert auf
> /dev/sdf1            827197400        20 785178172   1% /users1/p1
>                      ^^^^^^^^^
>
> What is happening here? When i run mkreiserfs, the programs last
> blocknumber seen is 180060160, which is exactly a1/4 of the mounted
> kilobytes (4k-Blocks okay). But where are the remaining 120 G-Blocks?
>
> Any ideas? According to the reiserfs-specs a filesystem can hold 17.6
> TBytes.
>
> kind regards
> Andreas Abele

Re: mkreiserfs and big RAID-Systems

[Andreas Abele]

Hi Vitaly,

do these Informations help you?

thanks 
Andreas Abele


Am Mon, 2002-07-15 um 14.39 schrieb Vitaly Fertman:
> 
> HI, 
> 
> which reiserfsprogs version do you use?
Name        : reiserfsprogs                Relocations: (not
relocateable)
Version     : 3.x.0j                            Vendor: MandrakeSoft
> what does 
> 	debugreiserfs /dev/sdf1
> say?
[root@saturn6 root]# debugreiserfs /dev/sdf1
debugreiserfs, 2001 - reiserfsprogs 3.x.0jSuper block of format 3.6
found on the 0x3 in block 16
Block count 180083512
Blocksize 4096
Free blocks 180069805
Busy blocks (skipped 16, bitmaps - 5496, journal blocks - 8193
1 super blocks, 1 data blocks
Root block 8211
Journal block (first) 18
Journal dev 0
Journal orig size 8192
Filesystem state VALID
Tree height 2
Hash function used to sort names: "r5"
Objectid map size 2, max 972
Version 2
> 
> -- 
> 
> Thanks,
> Vitaly Fertman
> 
> On Monday 15 July 2002 17:59, Andreas Abele wrote:
> > Hello,
> >
> > I have a RAID5-Set with 840 GB netto here.
> >
> > 1.) fdisk shows:
> >
> > Festplatte /dev/sdf: 255 Köpfe, 63 Sektoren, 104623 Zylinder
> > Einheiten: Zylinder mit 16065 * 512 Bytes
> >     Device boot.  Start       End    Blocks   Id  Dateisystemtyp
> > /dev/sdf1   *         1    104623 840384216   83  Linux
> >                                   ^^^^^^^^^
> > 2.) mkreiserfs /dev/sdf1
> > 3.) mount /dev/sdf1 /users1/p1
> > 4.) df /dev/sdf1
> > Dateisystem          1K-Blöcke    Benutzt Verfügbar Ben% montiert auf
> > /dev/sdf1            720312060     32840 720279220   1% /users1/p1
> >                      ^^^^^^^^^
> >
> > Where are the remaining 120 GBytes?
> >
> > Cross-Test with ext2
> >
> > 2a.) mke2fs /dev/sdf1
> > 3a.) mount /dev/sdf1 /users1/p1
> > 4a.) df /dev/sdf1
> > Dateisystem          1K-Blöcke    Benutzt Verfügbar Ben% montiert auf
> > /dev/sdf1            827197400        20 785178172   1% /users1/p1
> >                      ^^^^^^^^^
> >
> > What is happening here? When i run mkreiserfs, the programs last
> > blocknumber seen is 180060160, which is exactly a1/4 of the mounted
> > kilobytes (4k-Blocks okay). But where are the remaining 120 G-Blocks?
> >
> > Any ideas? According to the reiserfs-specs a filesystem can hold 17.6
> > TBytes.
> >
> > kind regards
> > Andreas Abele

Re: mkreiserfs and big RAID-Systems

[Vitaly Fertman]

> Hi Vitaly,
>
> do these Informations help you?
>
> thanks
> Andreas Abele
>
> Am Mon, 2002-07-15 um 14.39 schrieb Vitaly Fertman:
> > HI,
> >
> > which reiserfsprogs version do you use?
>
> Name        : reiserfsprogs                Relocations: (not
> relocateable)
> Version     : 3.x.0j                            Vendor: MandrakeSoft

First of all get the latest reiserfsprogs-3.6.2 from our ftp site please 
and try again.

-- 

Thanks,
Vitaly Fertman

Re: mkreiserfs and big RAID-Systems

[Andreas Abele]

> First of all get the latest reiserfsprogs-3.6.2 from our ftp site please 
> and try again.

Hi Vitaly,

i have built the 3.6.2-Progs and get the following behaviour:

1.) mkreiserfs -d /dev/sdf1 180000000 (less than 720 GB)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[root@saturn6 mkreiserfs]# ./mkreiserfs -d /dev/sdf1 180000000

<-------------mkreiserfs, 2002------------->
reiserfsprogs 3.6.2

mkreiserfs: Guessing about desired format.. 
mkreiserfs: Kernel 2.4.18 is running.
Block 16 (0x851) contains super block. Format 3.6 with standard journal
Count of blocks on the device: 180000000
Number of blocks consumed by mkreiserfs formatting process: 13705
Free blocks: 179986295
Blocksize: 4096
Hash function used to sort names: "r5"
Number of bitmaps: 5494
Root block: 8211
Tree height: 2
Objectid map size 2, max 972
Journal parameters:
        Device [0x0]
        Magic [0x1162e2dd]
        Size 8193 blocks (including 1 for journal header) (first block
18)
        Max transaction length 1024 blocks
        Max batch size 900 blocks
        Max commit age 30
Filesystem state 0x0
sb_version 2
inode generation number: 0
UUID: df15b6bc-8c2f-484e-876b-9930cbcc3548
ATTENTION: YOU SHOULD REBOOT AFTER FDISK!
        ALL DATA WILL BE LOST ON '/dev/sdf1'!
Continue (y/n):y
Initializing journal - 0%....20%....40%....60%....80%....100%
Syncing..ok

=>Everything seems to be okay until now.
2.) => Now the following command (bigger than 720 GB):

[root@saturn6 mkreiserfs]# ./mkreiserfs -d /dev/sdf1 190000000

<-------------mkreiserfs, 2002------------->
reiserfsprogs 3.6.2

mkreiserfs: Guessing about desired format.. 
mkreiserfs: Kernel 2.4.18 is running.
Block 16 (0x851) contains super block. Format 3.6 with standard journal
Count of blocks on the device: 190000000
Number of blocks consumed by mkreiserfs formatting process: 14010
Free blocks: 189985990
Blocksize: 4096
Hash function used to sort names: "r5"
Number of bitmaps: 5799
Root block: 8211
Tree height: 2
Objectid map size 2, max 972
Journal parameters:
        Device [0x0]
        Magic [0x7457c150]
        Size 8193 blocks (including 1 for journal header) (first block
18)
        Max transaction length 1024 blocks
        Max batch size 900 blocks
        Max commit age 30
Filesystem state 0x0
sb_version 2
inode generation number: 0
UUID: 8e4a9fe8-1e96-47f3-9d2d-e5a7e05cc01f
ATTENTION: YOU SHOULD REBOOT AFTER FDISK!
        ALL DATA WILL BE LOST ON '/dev/sdf1'!
Continue (y/n):y
Initializing journal - 0%....20%....40%....60%....80%....100%

bread: Cannot read a block # 189999999.
Abgebrochen (core dumped)

=> the same behaviour with no filesize-param at the end.

any Ideas?

thanks
Andreas Abele

Re: mkreiserfs and big RAID-Systems

[Vitaly Fertman]

> > First of all get the latest reiserfsprogs-3.6.2 from our ftp site please
> > and try again.
>
> Hi Vitaly,
>
> i have built the 3.6.2-Progs and get the following behaviour:
>
> 1.) mkreiserfs -d /dev/sdf1 180000000 (less than 720 GB)

Could you run just 
    mkreiserfs /dev/sdf1

> [root@saturn6 mkreiserfs]# ./mkreiserfs -d /dev/sdf1 180000000
> Free blocks: 179986295
> Syncing..ok
>
> =>Everything seems to be okay until now.
> 2.) => Now the following command (bigger than 720 GB):
>
> [root@saturn6 mkreiserfs]# ./mkreiserfs -d /dev/sdf1 190000000
> Free blocks: 189985990
> Blocksize: 4096
> bread: Cannot read a block # 189999999.
> Abgebrochen (core dumped)

Do you have anything related in your syslog?

-- 

Thanks,
Vitaly Fertman

Re: mkreiserfs and big RAID-Systems

[Andreas Abele]

Hi Vitaly,

> Could you run just 
>     mkreiserfs /dev/sdf1
> 


this has no effect, except the number of affected blocks.

Here is the result
[root@saturn6 mkreiserfs]# ./mkreiserfs /dev/sdf1

<-------------mkreiserfs, 2002------------->
reiserfsprogs 3.6.2

mkreiserfs: Guessing about desired format.. 
mkreiserfs: Kernel 2.4.18 is running.
Format 3.6 with standard journal
Count of blocks on the device: 210096054
Number of blocks consumed by mkreiserfs formatting process: 14623
Blocksize: 4096
Hash function used to sort names: "r5"
Journal Size 8193 blocks (first block 18)
Journal Max transaction length 1024
inode generation number: 0
UUID: e4df4ab3-dd1b-4426-80f1-dbf6d60f8b96
ATTENTION: YOU SHOULD REBOOT AFTER FDISK!
        ALL DATA WILL BE LOST ON '/dev/sdf1'!
Continue (y/n):y
Initializing journal - 0%....20%....40%....60%....80%....100%

bread: Cannot read a block # 210096053.

Abgebrochen (core dumped)

This is the syslog-entry from fdisk
===================================
Jul 16 01:23:14 saturn6 kernel: SCSI device sdf: 1680779520 512-byte hdwr sectors (-238952 MB)
Jul 16 01:23:14 saturn6 kernel:  /dev/scsi/host4/bus0/target4/lun0: p1
Jul 16 01:23:16 saturn6 kernel: SCSI device sdf: 1680779520 512-byte hdwr sectors (-238952 MB)
Jul 16 01:23:16 saturn6 kernel:  /dev/scsi/host4/bus0/target4/lun0: p1


This is the syslogentry from mkreiserfs
=======================================
Jul 16 01:26:39 saturn6 kernel:  I/O error: dev 08:51, sector 1680768424

I don't think this is a hardware problem, because i have 5 raids of that kind and they all behave 
the same way.

thanks 
Andreas Abele

Re: mkreiserfs and big RAID-Systems

[Vitaly Fertman]

> Here is the result
> [root@saturn6 mkreiserfs]# ./mkreiserfs /dev/sdf1
> Count of blocks on the device: 210096054

As you can see block number is correct - 840384216 blocks.

> bread: Cannot read a block # 210096053.
>
> Abgebrochen (core dumped)
>
> This is the syslog-entry from fdisk
> ===================================
> Jul 16 01:23:14 saturn6 kernel: SCSI device sdf: 1680779520 512-byte hdwr
> sectors (-238952 MB) Jul 16 01:23:14 saturn6 kernel: 
> /dev/scsi/host4/bus0/target4/lun0: p1 Jul 16 01:23:16 saturn6 kernel: SCSI
> device sdf: 1680779520 512-byte hdwr sectors (-238952 MB) Jul 16 01:23:16
> saturn6 kernel:  /dev/scsi/host4/bus0/target4/lun0: p1
>
>
> This is the syslogentry from mkreiserfs
> =======================================
> Jul 16 01:26:39 saturn6 kernel:  I/O error: dev 08:51, sector 1680768424

This is likely to be a bad block. Run 
    /sbin/badblock -b 4096 /dev/sdf1 210096053 210096053 
or with some range.

-- 

Thanks,
Vitaly Fertman

Re: mkreiserfs and big RAID-Systems

[Andreas Abele]

Hi Vitaly,

Am Mon, 2002-07-15 um 18.08 schrieb Vitaly Fertman:
> > Here is the result
> > [root@saturn6 mkreiserfs]# ./mkreiserfs /dev/sdf1
> > Count of blocks on the device: 210096054
> 
> As you can see block number is correct - 840384216 blocks.
> 
> > bread: Cannot read a block # 210096053.
it looks like every block on these devices that is aboth about
Block-Number 18000000 is declared as a bad block in reiserfs, in ext2fs
that doesnt happen. (Thats why i hve tried 'mkreiserfs /dev/sdf1
190000000) and i got ' bread: Cannot read a block # 189999999.'

> This is likely to be a bad block. Run 
>     /sbin/badblock -b 4096 /dev/sdf1 210096053 210096053 
> or with some range.
> 
i tried the command, but if i am right, i have to declare 120 G-Blocks
as bad, and i dont believe that in every of my 5 RAID-Systems there is
exactly 1 defect Harddisk.

thanks
Andreas Abele

Re: mkreiserfs and big RAID-Systems

[Vitaly Fertman]

> Hi Vitaly,
>
> Am Mon, 2002-07-15 um 18.08 schrieb Vitaly Fertman:
> > > Here is the result
> > > [root@saturn6 mkreiserfs]# ./mkreiserfs /dev/sdf1
> > > Count of blocks on the device: 210096054
> >
> > As you can see block number is correct - 840384216 blocks.
> >
> > > bread: Cannot read a block # 210096053.
>
> it looks like every block on these devices that is aboth about
> Block-Number 18000000 is declared as a bad block in reiserfs, in ext2fs
> that doesnt happen. (Thats why i hve tried 'mkreiserfs /dev/sdf1
> 190000000) and i got ' bread: Cannot read a block # 189999999.'

Badblocks list is still disabled in reiserfs. And mkreiserfs just want
to read that block and the kernel (not reiserfs part of it) prints I/O 
error into syslog.

> > This is likely to be a bad block. Run
> >     /sbin/badblock -b 4096 /dev/sdf1 210096053 210096053
> > or with some range.
>
> i tried the command, but if i am right, i have to declare 120 G-Blocks
> as bad, and i dont believe that in every of my 5 RAID-Systems there is
> exactly 1 defect Harddisk.

badblock tests disk blocks, not reiserfs. And as you can see you cannot 
access some blocks on your device. This is not reiserfs problem. Test with
badblock that harddisk separately, test your RAID configuration, etc.

-- 

Thanks,
Vitaly Fertman

Re: mkreiserfs and big RAID-Systems

[Kuba Ober]

> > > Jul 16 01:26:39 saturn6 kernel:  I/O error: dev 08:51, sector 1680768424

> > This is likely to be a bad block. Run
> >     /sbin/badblock -b 4096 /dev/sdf1 210096053 210096053
> > or with some range.

> i tried the command

You couldn't. It should be badblocks, not badblock (or did you catch that?)

> but if i am right, i have to declare 120 G-Blocks
> as bad

You are not right. Read 
man badblocks
first, so that you have idea of what you're doing. The numbers (210096053) 
specify a range of 4096 byte long blocks to test.

He advised you to *test* reading from the supposedly bad block first, and 
that's why you should call badblocks on that block.

The thing is that the kernel has problems reading a block from your raid 
array:
> > > Jul 16 01:26:39 saturn6 kernel:  I/O error: dev 08:51, sector 1680768424
Fix that first. This has nothing to do with reiserfs -- reiserfs won't help if 
kernel cannot read a block from the device.

Cheers, Kuba Ober

Re: mkreiserfs and big RAID-Systems

[Kuba Ober]

> > > [root@saturn6 mkreiserfs]# ./mkreiserfs /dev/sdf1
> > > Count of blocks on the device: 210096054
> >
> > As you can see block number is correct - 840384216 blocks.
> >
> > > bread: Cannot read a block # 210096053.
>
> it looks like every block on these devices that is aboth about
> Block-Number 18000000 is declared as a bad block in reiserfs,

It's not declared as a bad block. It's just that reiserfs cannot read that, 
because your kernel cannot read it as well, and that's because the raid 
driver reports an error.

So *please* first make sure that the whole raid array is readable. Try 
something like 
dd if=/dev/sdf1 of=/dev/null
and see if it succeeds (i.e. it doesn't report an error before end of the raid 
device).

> in ext2fs
> that doesnt happen.
Try mke2fs -c /dev/sdf1
I'm quite sure it'll complain loudly (-c enables checking the device for read 
errors).

So it didn't happen in mke2fs probably because it just didn't try 
writing/reading from the bad blocks. 

AFAICT, you're definitely having some device access problem -- either your 
kernel is bad, or the raid driver in the kernel, or your raid controller is 
failing, or your motherboard, or your raid array is dying. Whatever it is, as 
long as kernel cannot read blocks from the device, you cannot use it, and it 
doesn't depend on reiserfs at all.

This thread is off-topic. You're having bad hardware. This has nothing to do 
with reiserfs.

Cheers, Kuba Ober

Re: mkreiserfs and big RAID-Systems

[Alexander Saers]

Are you sure that you dont have looked at the wrong valus. I mean, when
running raid-5 you loose some storage space and gain some redundance. You do
that by adding checksum on one or several disks. Very often you also have
stand in disks "spare - disks" that actually not are used at all. But when
one disk fails they are connected in.

So my advice are to check your raid config and se what the actual raid size
is. Also check what harddrive that are in the raid array and start counting
and se if it all matches your numbers

/Alexander

----- Original Message -----
From: "Andreas Abele" <aba@IZS.FhG.de>
To: <reiserfs-list@namesys.com>
Sent: Monday, July 15, 2002 3:59 PM
Subject: [reiserfs-list] mkreiserfs and big RAID-Systems


Hello,

I have a RAID5-Set with 840 GB netto here.

1.) fdisk shows:

Festplatte /dev/sdf: 255 Köpfe, 63 Sektoren, 104623 Zylinder
Einheiten: Zylinder mit 16065 * 512 Bytes
    Device boot.  Start       End    Blocks   Id  Dateisystemtyp
/dev/sdf1   *         1    104623 840384216   83  Linux
                                  ^^^^^^^^^
2.) mkreiserfs /dev/sdf1
3.) mount /dev/sdf1 /users1/p1
4.) df /dev/sdf1
Dateisystem          1K-Blöcke    Benutzt Verfügbar Ben% montiert auf
/dev/sdf1            720312060     32840 720279220   1% /users1/p1
                     ^^^^^^^^^

Where are the remaining 120 GBytes?

Cross-Test with ext2

2a.) mke2fs /dev/sdf1
3a.) mount /dev/sdf1 /users1/p1
4a.) df /dev/sdf1
Dateisystem          1K-Blöcke    Benutzt Verfügbar Ben% montiert auf
/dev/sdf1            827197400        20 785178172   1% /users1/p1
                     ^^^^^^^^^

What is happening here? When i run mkreiserfs, the programs last
blocknumber seen is 180060160, which is exactly a1/4 of the mounted
kilobytes (4k-Blocks okay). But where are the remaining 120 G-Blocks?

Any ideas? According to the reiserfs-specs a filesystem can hold 17.6
TBytes.

kind regards
Andreas Abele