problems formatting raid5

problems formatting raid5

[sk]

Hi There,

I am trying to create file system on a software raid5 device but after while it hangs on:

Writing superblocks and filesystem accounting information:


some details:
OS: Scientific Linux release 6.3 (Carbon), 2.6.32-279.el6.x86_64; supermicro 4047R-7JRFT

the command i've used:
mkfs.ext3 -v -m .1 -b 4096 -E stride=128,stripe-width=2304 /dev/md0

mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Wed Feb 13 15:06:39 2013
Raid Level : raid5
Array Size : 9001838592 (8584.82 GiB 9217.88 GB)
Used Dev Size : 500102144 (476.93 GiB 512.10 GB)
Raid Devices : 19
Total Devices : 20
Persistence : Superblock is persistent

Update Time : Fri Feb 15 15:38:08 2013
State : clean
Active Devices : 19
Working Devices : 20
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 512K

Name : sandy2:0 (local to host sandy2)
UUID : 9e313caa:c33e5d9c:e56f7848:b0c67189
Events : 52

Number Major Minor RaidDevice State
0 8 33 0 active sync /dev/sdc1
1 8 49 1 active sync /dev/sdd1
2 8 65 2 active sync /dev/sde1
3 8 81 3 active sync /dev/sdf1
4 8 97 4 active sync /dev/sdg1
5 8 113 5 active sync /dev/sdh1
6 8 129 6 active sync /dev/sdi1
7 8 145 7 active sync /dev/sdj1
8 8 161 8 active sync /dev/sdk1
9 8 177 9 active sync /dev/sdl1
10 8 193 10 active sync /dev/sdm1
11 8 209 11 active sync /dev/sdn1
12 8 225 12 active sync /dev/sdo1
13 8 241 13 active sync /dev/sdp1
14 65 1 14 active sync /dev/sdq1
15 65 17 15 active sync /dev/sdr1
16 65 33 16 active sync /dev/sds1
17 65 49 17 active sync /dev/sdt1
20 65 65 18 active sync /dev/sdu1

19 8 17 - spare /dev/sdb1

i have tried with mkfs.ext4, and mke2fs as well but the result was the same.
any ideas what could be wrong ?
Thx!
SK

Re: problems formatting raid5

[Mikael Abrahamsson]

On Sun, 17 Feb 2013, sk wrote:

> Hi There,
>
> I am trying to create file system on a software raid5 device but after while it hangs on:
>
> Writing superblocks and filesystem accounting information:
>
> i have tried with mkfs.ext4, and mke2fs as well but the result was the same.
> any ideas what could be wrong ?

"iostat -x 5", check if there is activity going on. Start it before you do 
the mkfs to make sure nothing is going on (for instance a resync) before 
you start creating the filesystem.

A large filesystem with ext3/ext4 takes quite a while to create. How long 
did you wait until you decided it was "hung" ?

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: problems formatting raid5

[sk]

i wait about 10 hours
and start after the resync process completed. 
it was not mount so i don't think there were any activity. 
sk




----- Original Message -----
From: Mikael Abrahamsson <swmike@swm.pp.se>
To: sk <sdk0084@yahoo.com>
Cc: "linux-raid@vger.kernel.org" <linux-raid@vger.kernel.org>
Sent: Monday, February 18, 2013 11:17 AM
Subject: Re: problems formatting raid5

On Sun, 17 Feb 2013, sk wrote:

> Hi There,
> 
> I am trying to create file system on a software raid5 device but after while it hangs on:
> 
> Writing superblocks and filesystem accounting information:
> 
> i have tried with mkfs.ext4, and mke2fs as well but the result was the same.
> any ideas what could be wrong ?

"iostat -x 5", check if there is activity going on. Start it before you do the mkfs to make sure nothing is going on (for instance a resync) before you start creating the filesystem.

A large filesystem with ext3/ext4 takes quite a while to create. How long did you wait until you decided it was "hung" ?

-- Mikael Abrahamsson    email: swmike@swm.pp.se
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: problems formatting raid5

[Mikael Abrahamsson]

On Mon, 18 Feb 2013, sk wrote:

> i wait about 10 hours
> and start after the resync process completed. 
> it was not mount so i don't think there were any activity.

Well, then output from iostat -x 5 during the creation would be of 
interest. Also, try other filesystems such as xfs and see if that behaves 
differently.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: problems formatting raid5

[sk]

I have tried xfs as well and it was ok. However, later it hangs several times while transferring files and the only way to stop the transfer was rebooting.


 Original Message -----

From: Mikael Abrahamsson <swmike@swm.pp.se>
To: sk <sdk0084@yahoo.com>
Cc: "linux-raid@vger.kernel.org" <linux-raid@vger.kernel.org>
Sent: Monday, February 18, 2013 12:46 PM
Subject: Re: problems formatting raid5

On Mon, 18 Feb 2013, sk wrote:

> i wait about 10 hours
> and start after the resync process completed. it was not mount so i don't think there were any activity.

Well, then output from iostat -x 5 during the creation would be of interest. Also, try other filesystems such as xfs and see if that behaves differently.

-- Mikael Abrahamsson    email: swmike@swm.pp.se

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: problems formatting raid5

[Mikael Abrahamsson]

On Mon, 18 Feb 2013, sk wrote:

> I have tried xfs as well and it was ok. However, later it hangs several 
> times while transferring files and the only way to stop the transfer was 
> rebooting.

Then you need to start looking at your logs. Seems you have some other 
problem causing IO to stall.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: problems formatting raid5

[Roy Sigurd Karlsbakk]

> Raid Level : raid5
> Raid Devices : 19

that's like BASE jumping with a large umbrella, as earlier noted. Don't do that, it'll just hurt you. At least use RAID-5 and preferably use more RAID sets and RAID-0 or LVM on top.

Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
roy@karlsbakk.net
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: problems formatting raid5

[Roy Sigurd Karlsbakk]

> that's like BASE jumping with a large umbrella, as earlier noted.
> Don't do that, it'll just hurt you. At least use RAID-5 and preferably
> use more RAID sets and RAID-0 or LVM on top.

Make that RAID-6

Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
roy@karlsbakk.net
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: problems formatting raid5

[Steven Haigh]

[-- Attachment #1: Type: text/plain, Size: 993 bytes --]

On 19/02/2013 9:15 AM, Roy Sigurd Karlsbakk wrote:
>> that's like BASE jumping with a large umbrella, as earlier noted.
>> Don't do that, it'll just hurt you. At least use RAID-5 and preferably
>> use more RAID sets and RAID-0 or LVM on top.
>
> Make that RAID-6

I agree. I think a more sane outlook on this would be 19 drives made up of:
	* RAID Level: 6
	* Hot Spares: 2

Using RAID5 then its not a matter of if you are going to lose your 
array, its a matter of WHEN considering the number of spindles here.

If you have the drives in separate enclosures or different controllers, 
have one spare per controller - ie don't but both spares on the same 
controller.

*Maybe* if you are pushed for space, only use 1 hot spare - but be sure 
to have a drive sat in the anti-static bag on a shelf within walking 
distance of the server.

--
Steven Haigh

Email: netwiz@crc.id.au
Web: https://www.crc.id.au
Phone: (03) 9001 6090 - 0412 935 897
Fax: (03) 8338 0299


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4240 bytes --]

Re: problems formatting raid5

[Emilio Hodge]

sk wrote:
> Hi There,
>
>
Do you have the possibility to use another controller?
And 19 spindles with raid 5 is a little mad. But let first find the 
problem could you find anything in the logs?

Emilio Hodge

Re: problems formatting raid5

[Emilio Hodge]

sk wrote:
> Hi There,
>
>
Do you have the possibility to use another controller?
And 19 spindles with raid 5 is a little mad. But let first find the 
problem could you find anything in the logs?

Emilio Hodge