which disk the the one that data is on?

which disk the the one that data is on?

[Shai]

Hi,

I rebooted my server today to find out that one of the arrays is being
re-synced (see output below)
.
1. What does the (S) to the right of hdh1[5](S) mean?
2. How do I know, from this output, which disk is the one holding the
most current data and from which all the other drives are syncing
from? Or are they all containing the data and this sync process is
something else? Maybe I'm just not understanding what is being done
exactly?

Thanks in advance for the help.

Shai
---

md0 : active raid5 hdd1[1] hdc1[0] hdh1[5](S) hdg1[4] hdf1[3] hde1[2]
      781433344 blocks level 5, 64k chunk, algorithm 2 [5/5] [UUUUU]
      [=>...................]  resync =  5.1% (9965184/195358336)
finish=180.5min speed=17110K/sec

-------------------
# mdadm -Q --detail /dev/md0
/dev/md0:
        Version : 00.90.02
  Creation Time : Mon Mar 20 20:40:24 2006
     Raid Level : raid5
     Array Size : 781433344 (745.23 GiB 800.19 GB)
    Device Size : 195358336 (186.31 GiB 200.05 GB)
   Raid Devices : 5
  Total Devices : 6
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Tue Jul 18 10:28:39 2006
          State : active, resyncing
 Active Devices : 5
Working Devices : 6
 Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 64K

 Rebuild Status : 8% complete

           UUID : 4bc2c7f7:b7ec1184:9c2c382d:2aaf7a98
         Events : 0.386738

    Number   Major   Minor   RaidDevice State
       0      22        1        0      active sync   /dev/.static/dev/hdc1
       1      22       65        1      active sync   /dev/.static/dev/hdd1
       2      33        1        2      active sync   /dev/.static/dev/hde1
       3      33       65        3      active sync   /dev/.static/dev/hdf1
       4      34        1        4      active sync   /dev/.static/dev/hdg1

       5      34       65        -      spare   /dev/.static/dev/hdh1

Re: which disk the the one that data is on?

[Neil Brown]

On Tuesday July 18, shaibn@gmail.com wrote:
> Hi,
> 
> I rebooted my server today to find out that one of the arrays is being
> re-synced (see output below)
> .
> 1. What does the (S) to the right of hdh1[5](S) mean?

Spare.

> 2. How do I know, from this output, which disk is the one holding the
> most current data and from which all the other drives are syncing
> from? Or are they all containing the data and this sync process is
> something else? Maybe I'm just not understanding what is being done
> exactly?

This is raid5.  The data is distributed over all of the drives.
There are also 'parity' blocks used for coping with missing devices.
This 'resync' process is checking that the parity blocks are all
correct and will correct any that are wrong.

NeilBrown


> 
> Thanks in advance for the help.
> 
> Shai
> ---
> 
> md0 : active raid5 hdd1[1] hdc1[0] hdh1[5](S) hdg1[4] hdf1[3] hde1[2]
>       781433344 blocks level 5, 64k chunk, algorithm 2 [5/5] [UUUUU]
>       [=>...................]  resync =  5.1% (9965184/195358336)
> finish=180.5min speed=17110K/sec
> 

Re: which disk the the one that data is on?

[Shai]

Hi,

So if I were to want to stop the resync process on a very large array
(1.4T), since it is in the middle of the day and makes work slower...
How can I tell which drive is the one that is being used to check all
the rest of the data? Or in other words, how can I stop the resync
process and let it cont. later?

Shai

On 7/18/06, Neil Brown <neilb@suse.de> wrote:
> On Tuesday July 18, shaibn@gmail.com wrote:
> > Hi,
> >
> > I rebooted my server today to find out that one of the arrays is being
> > re-synced (see output below)
> > .
> > 1. What does the (S) to the right of hdh1[5](S) mean?
>
> Spare.
>
> > 2. How do I know, from this output, which disk is the one holding the
> > most current data and from which all the other drives are syncing
> > from? Or are they all containing the data and this sync process is
> > something else? Maybe I'm just not understanding what is being done
> > exactly?
>
> This is raid5.  The data is distributed over all of the drives.
> There are also 'parity' blocks used for coping with missing devices.
> This 'resync' process is checking that the parity blocks are all
> correct and will correct any that are wrong.
>
> NeilBrown
>
>
> >
> > Thanks in advance for the help.
> >
> > Shai
> > ---
> >
> > md0 : active raid5 hdd1[1] hdc1[0] hdh1[5](S) hdg1[4] hdf1[3] hde1[2]
> >       781433344 blocks level 5, 64k chunk, algorithm 2 [5/5] [UUUUU]
> >       [=>...................]  resync =  5.1% (9965184/195358336)
> > finish=180.5min speed=17110K/sec
> >
>

Re: which disk the the one that data is on?

[Neil Brown]

On Tuesday July 18, shaibn@gmail.com wrote:
> Hi,
> 
> So if I were to want to stop the resync process on a very large array
> (1.4T), since it is in the middle of the day and makes work slower...
> How can I tell which drive is the one that is being used to check all
> the rest of the data? Or in other words, how can I stop the resync
> process and let it cont. later?

There is no "one that is being used to check all the rest".  They are
all in this together.

But if you want to stop the resync because it is slowing things down
then write a small number to /proc/sys/dev/raid/speed_limit_min
e.g.

  echo 10 > /proc/sys/dev/raid/speed_limit_min

This won't actually stop it, but it will make it go very slowly so as
not to interfere with anything else.

NeilBrown

Re: which disk the the one that data is on?

[Shai]

Hi,

Another question on this matter please:
If there is a raid5 with 4 disks and 1 missing, and we add that disk,
while its doing the resync of that disk, how do we know which disk it
is (if we forgot what one we added)?

Thanks,
Shai

On 7/18/06, Neil Brown <neilb@suse.de> wrote:
> On Tuesday July 18, shaibn@gmail.com wrote:
> > Hi,
> >
> > So if I were to want to stop the resync process on a very large array
> > (1.4T), since it is in the middle of the day and makes work slower...
> > How can I tell which drive is the one that is being used to check all
> > the rest of the data? Or in other words, how can I stop the resync
> > process and let it cont. later?
>
> There is no "one that is being used to check all the rest".  They are
> all in this together.
>
> But if you want to stop the resync because it is slowing things down
> then write a small number to /proc/sys/dev/raid/speed_limit_min
> e.g.
>
>   echo 10 > /proc/sys/dev/raid/speed_limit_min
>
> This won't actually stop it, but it will make it go very slowly so as
> not to interfere with anything else.
>
> NeilBrown
>

Re: which disk the the one that data is on?

[Janos Farkas]

On 2006-07-19 at 08:33:21, Shai wrote:
> Another question on this matter please:
> If there is a raid5 with 4 disks and 1 missing, and we add that disk,
> while its doing the resync of that disk, how do we know which disk it
> is (if we forgot what one we added)?

Everything is already in the /proc/mdstat (and mdadm -Q -D /dev/md0)
output you posted:

md0 : active raid5 hdd1[1] hdc1[0] hdh1[5](S) hdg1[4] hdf1[3] hde1[2]
     781433344 blocks level 5, 64k chunk, algorithm 2 [5/5] [UUUUU]
     [=>...................]  resync =  5.1% (9965184/195358336) finish=180.5min speed=17110K/sec

There's a number after each component (hdc1[0]), and there's an U in the
next line for each component.  If some of the component drives are not
there, or resyncing, something else will be there.  You can also see
that hdh1 is a (S)pare right now.  [_UUUU] would mean that the component
0 is not (U)p to date.  Make yourself familiar with this output so you
are not confused should there be any problem.

Similarly, mdadm output is containing largely the same information, but
in a different layout, maybe that's more clear in some cases.

Janos

Re: which disk the the one that data is on?

[Neil Brown]

On Wednesday July 19, shaibn@gmail.com wrote:
> Hi,
> 
> Another question on this matter please:
> If there is a raid5 with 4 disks and 1 missing, and we add that disk,
> while its doing the resync of that disk, how do we know which disk it
> is (if we forgot what one we added)?

mdadm --detail will list all the devices.  The one that is being
recovered won't be 'sync' - the others will.

NeilBrown

Re: which disk the the one that data is on?

[Shai]

Thanks thus far :)

Shai

On 7/20/06, Neil Brown <neilb@suse.de> wrote:
> On Wednesday July 19, shaibn@gmail.com wrote:
> > Hi,
> >
> > Another question on this matter please:
> > If there is a raid5 with 4 disks and 1 missing, and we add that disk,
> > while its doing the resync of that disk, how do we know which disk it
> > is (if we forgot what one we added)?
>
> mdadm --detail will list all the devices.  The one that is being
> recovered won't be 'sync' - the others will.
>
> NeilBrown
>

Re: which disk the the one that data is on?

[Bill Davidsen]

Shai wrote:

> Hi,
>
> I rebooted my server today to find out that one of the arrays is being
> re-synced (see output below)
> .
> 1. What does the (S) to the right of hdh1[5](S) mean?
> 2. How do I know, from this output, which disk is the one holding the
> most current data and from which all the other drives are syncing
> from? Or are they all containing the data and this sync process is
> something else? Maybe I'm just not understanding what is being done
> exactly? 

In addition to what you have already been told, if you "find out" that 
the array is in rebuild I would be a lot more worried to find out why. 
If it was from unclean shutdown you really should look into a bitmap if 
you don't have one.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979