RAID question

RAID question

[Peter Waltenberg]

We have a RAID 5 system thats had 2 of 6 disks in the RAID go into thermal
shutdown. (Air-con failure).

The disks are functional, but the RAID won't restart because the superblock
timestamps on those two disks are now out of step with the rest of the array and
there aren't enough "good" disks to reconstruct the array.

We know there was very little activity when this happened.

Does anyone out there know of a way to hack the superblocks on the "bad" disks
to force them to appear to be O.K. so that the RAID will restart. 

Thanks
Peter 

Re: RAID question

[Jakob Østergaard]

On Tue, May 08, 2001 at 12:48:25PM +1000, Peter Waltenberg wrote:
> We have a RAID 5 system thats had 2 of 6 disks in the RAID go into thermal
> shutdown. (Air-con failure).
> 
> The disks are functional, but the RAID won't restart because the superblock
> timestamps on those two disks are now out of step with the rest of the array and
> there aren't enough "good" disks to reconstruct the array.
> 
> We know there was very little activity when this happened.
> 
> Does anyone out there know of a way to hack the superblocks on the "bad" disks
> to force them to appear to be O.K. so that the RAID will restart. 

As documented in the HOWTO (http://unthought.net/Software-RAID.HOWTO), you should
re-run mkraid after making dead sure that your raidtab still corrosponds to the
RAID on your disks (it usually does unless someone screwed it up).

Run fsck on the RAID after mkraid.

-- 
................................................................
:   jakob@unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............:

RAID question

[Roy Sigurd Karlsbakk]

hi

I booted up 2.4.13-ac5 today, and got this message

raid5: measuring checksumming speed
   8regs     :  1480.800 MB/sec
   32regs    :   711.200 MB/sec
   pIII_sse  :  1570.400 MB/sec
   pII_mmx   :  1787.200 MB/sec
   p5_mmx    :  1904.000 MB/sec
raid5: using function: pIII_sse (1570.400 MB/sec)

Why is raid5 using function pIII_sse when p5_MMX is way faster?

roy

--
Roy Sigurd Karlsbakk, MCSE, MCNE, CLS, LCA

Computers are like air conditioners.
They stop working when you open Windows.

RE: RAID question

[Bene, Martin]

Hi Roy,

> raid5: measuring checksumming speed
>    8regs     :  1480.800 MB/sec
>    32regs    :   711.200 MB/sec
>    pIII_sse  :  1570.400 MB/sec
>    pII_mmx   :  1787.200 MB/sec
>    p5_mmx    :  1904.000 MB/sec
> raid5: using function: pIII_sse (1570.400 MB/sec)
> 
> Why is raid5 using function pIII_sse when p5_MMX is way faster?

The sse version is prefered over the others and gets used regardless of
speed if it's available:

/* We force the use of the SSE xor block because it can write around L2.
   We may also be able to load into the L1 only depending on how the cpu
   deals with a load to a line that is being prefetched.  */

Bye, Martin

Re: RAID question

[Lionel Bouton]

Bene, Martin wrote:

>Hi Roy,
>
>>raid5: measuring checksumming speed
>>   8regs     :  1480.800 MB/sec
>>   32regs    :   711.200 MB/sec
>>   pIII_sse  :  1570.400 MB/sec
>>   pII_mmx   :  1787.200 MB/sec
>>   p5_mmx    :  1904.000 MB/sec
>>raid5: using function: pIII_sse (1570.400 MB/sec)
>>
>>Why is raid5 using function pIII_sse when p5_MMX is way faster?
>>
>
>The sse version is prefered over the others and gets used regardless of
>speed if it's available:
>
>/* We force the use of the SSE xor block because it can write around L2.
>   We may also be able to load into the L1 only depending on how the cpu
>   deals with a load to a line that is being prefetched.  */
>
As your cpu(s) work on the data before sending it back to the devices in 
the case of RAID5 and as in the general case memory writes invalidate an 
amount of L2 cache corresponding to the amount of data to write, you'll 
push out of L2 data of your running applications, hence slowing them down.

Imagine you have a block you want to write to/read from your 4k chunk 
size, 3 drive RAID 5 array.
For an atomic write, you'll read 8k from memory and write 12k back. It's 
20k of L2 cache involved. The 8k read may already be in cache (you write 
to disk what you worked on earlier) but the 12k are generated so they 
surely won't.
On x86 12-20k of L2 cache is a substansial amount of cache. And it's the 
lowest atomic read/write operation you can imagine, with 64kb chunck 
size and 5 drives it would be 64 x 5 = 320 to 64 x 9 = 576k -> whole L2 
invalidated on most modern x86 cpu.

What does cache give you ? Fast access to recently used memory and 
write-back capability.
Cache use in this case is *bad* because you won't reuse the cached data. 
Write-back won't give much (you stream whole chunks down the memory bus 
anyway).
You *may* slow down disk writes (if you have something near 2GB/s of 
disk bandwidth on your RAID...) but you'll *surely* slow down your 
applications a lot by taking away their data from your L2 cache(s).

So the raid code doesn't use the optimal disk throughput method, it uses 
the optimal system performance method...

raid question

[jshankar]

Hello,

I have couple of questions regarding raid routine.

1> When i do mkraid, is there any specific routines in md.c that is 
responsible for adding and tracking the device. I have maintain the entries in 
/etc/raidtab. When i copy a file into the raid device, is there any specific 
routine that track  the  device to write the next block( raid 0  case).

2> Also the routine that  keeps track the amount
of buffer size that can be written to each particular device and the case when
the writing to one disk fails.



3> Also i was  wondering whether the source code of raid tools are availaible 
in
linux directory.

It would be great if somebody  can throw some light.

Thanks in advance.


-Jay



>===== Original Message From Neil Brown <neilb@cse.unsw.edu.au> =====
>On Wednesday July 30, jshankar@CS.ColoState.EDU wrote:
>> Hello ,
>>
>> I am Jayshankar, Masters student in Colorado State UNiversity.
>>
>> A small question on raid.
>>
>> Suppose i have got raid0 connected to disk1, disk2 , disk3 , disk4.
>>
>> The operating system writes a block of data to disk1 then to disk2, disk3,
>> disk4. Do the I/O needs to be successful on disk1, for writing the next
block
>> of data on disk2, disk3, disk4. Or it checks the I/O is successful on
disk1,
>> after the block of data has been written to disk2,disk3,disk4.
>>
>>
>> Looking forward for your reply.
>>
>> THanks in advance.
>>
>> -Jay
>
>Please send future questions to
>  linux-raid@vger.kernel.org
>rather than to me personally.
>
>All disk IO in Linux is asynchronous.
>i.e. submitting a block schedules it for IO.  If you submit lots of
>blocks they all get scheduled for IO.  This is true whether your are
>accessing a single device or a raid array.
>
>So there is no check for success of one block before writing the
>next.  quite often the blocks writen to the different drive will all
>be written at one.
>
>NeilBrown

Re: raid question

[Neil Brown]

On Sunday August 17, jshankar@CS.ColoState.EDU wrote:
> Hello,
> 
> I have couple of questions regarding raid routine.
> 
> 1> When i do mkraid, is there any specific routines in md.c that is 
> responsible for adding and tracking the device. I have maintain the entries in 
> /etc/raidtab. When i copy a file into the raid device, is there any specific 
> routine that track  the  device to write the next block( raid 0
> case).

There is no concept of a "next block"
raid0 gets given a number of blocks to write (or read).  Each block
has an address.
raid0 does some simple arithmetic to decide which device, and which
block on that device, to use.
This code is in
   raid0.c(raid0_make_request).

> 
> 2> Also the routine that  keeps track the amount
> of buffer size that can be written to each particular device and the case when
> the writing to one disk fails.
> 

This questions doesn't really make sense to me.

> 
> 
> 3> Also i was  wondering whether the source code of raid tools are availaible 
> in
> linux directory.

google can probably find it for you.
I think there is a link somewhere like
  http://www.kernel.org/pub/linux/utils/raid

NeilBrown

> 
> It would be great if somebody  can throw some light.
> 
> Thanks in advance.
> 
> 
> -Jay
> 
> 
> 
> >===== Original Message From Neil Brown <neilb@cse.unsw.edu.au> =====
> >On Wednesday July 30, jshankar@CS.ColoState.EDU wrote:
> >> Hello ,
> >>
> >> I am Jayshankar, Masters student in Colorado State UNiversity.
> >>
> >> A small question on raid.
> >>
> >> Suppose i have got raid0 connected to disk1, disk2 , disk3 , disk4.
> >>
> >> The operating system writes a block of data to disk1 then to disk2, disk3,
> >> disk4. Do the I/O needs to be successful on disk1, for writing the next
> block
> >> of data on disk2, disk3, disk4. Or it checks the I/O is successful on
> disk1,
> >> after the block of data has been written to disk2,disk3,disk4.
> >>
> >>
> >> Looking forward for your reply.
> >>
> >> THanks in advance.
> >>
> >> -Jay
> >
> >Please send future questions to
> >  linux-raid@vger.kernel.org
> >rather than to me personally.
> >
> >All disk IO in Linux is asynchronous.
> >i.e. submitting a block schedules it for IO.  If you submit lots of
> >blocks they all get scheduled for IO.  This is true whether your are
> >accessing a single device or a raid array.
> >
> >So there is no check for success of one block before writing the
> >next.  quite often the blocks writen to the different drive will all
> >be written at one.
> >
> >NeilBrown
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html