RE: now on to tuning....

RE: now on to tuning....

[peter.greis]

Good point about maxing out the pci bus... - I already use the nForce for mirrored boot drives, so that's not an option. The IDE controllers are empty at the moment (save for a DVD drive); I will give this a thought.

Thanks for the feedback,

-P

-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org]On Behalf Of Nicola Fankhauser
Sent: Wednesday, March 09, 2005 10:48 AM
To: linux-raid@vger.kernel.org
Subject: Re: now on to tuning....


hi peter

peter.greis@ubs.com wrote:
> I have been lurking for a while.... I recently put together a raid 5
> system (Asus K8NE SIL 3114/2.6.8.1 kernel) with 4 300GB SATA Seagate
> drives (a lot smaller than the bulk of what seems to be on this
> list!). Currently this is used for video and mp3 storage, being
> Reiser on LVM2.

beware that LVM2 _can_ affect your performance. I too believed that the 
concept of dynamic drives is good, but I experienced a performance hit 
of about 50% (especially in sequential reads).

see my blog entry describing how I built my 2TB file-server at 
http://variant.ch/phpwiki/WikiBlog/2005-02-27 for some numbers and more 
explanation.

the K8NE has the same SiI 3114 controller as the board I used has; it is 
connected by a 33mhz 32bit PCI bus and maxes out at 133MiB/s, so for 
maxmimum performance you might want to connect only two drives to this 
controller, the other two to the nforce3 chipset SATA ports.

> Bonnie++ to test, but with which parameters ? 

normally it's enough to specify a test-file larger (e.g. twice) the 
memory capacity of the machine you are testing. for a machine with 1GiB RAM:

# bonnie++ -s 2gb {other options}

you might as well want to specify the "fast" option which skips per-char 
operations (which are quite useless to test IMHO):

# bonnie++ -f {other options}

HTH
nicola
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: now on to tuning....

[Derek Piper]

Hmm..  for me:

> smartctl -A -d ata /dev/sda

On my work machine with Debian Sarge:
----------------------------------------------------------------------------
smartctl version 5.32 Copyright (C) 2002-4 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

Smartctl: Device Read Identity Failed (not an ATA/ATAPI device)

A mandatory SMART command failed: exiting. To continue, add one or
more '-T permissive' options.
----------------------------------------------------------------------------

Did you apply the libata patch? I saw that here:

http://smartmontools.sourceforge.net/#testinghelp

I'm on kernel 2.6.10 and haven't applied any patches.. maybe it's
included on 2.6.11 now or a difference between smartctl 5.32 and 5.33?

The drives I have are on an Intel ICH5 SATA controller. I am doing a
few RAIDed partitions between a couple of 120GB drives since I
reinstalled my work machine a few weeks ago. I think I'm using libata
(the option marked as 'conflicting' with it isn't enabled in my
kernel). Any thoughts?

Derek

On Wed, 9 Mar 2005 11:53:11 +0100, peter.greis@ubs.com
<peter.greis@ubs.com> wrote:
> Good point about maxing out the pci bus... - I already use the nForce for mirrored boot drives, so that's not an option. The IDE controllers are empty at the moment (save for a DVD drive); I will give this a thought.
> 
> Thanks for the feedback,
> 
> -P
> 
> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org
> [mailto:linux-raid-owner@vger.kernel.org]On Behalf Of Nicola Fankhauser
> Sent: Wednesday, March 09, 2005 10:48 AM
> To: linux-raid@vger.kernel.org
> Subject: Re: now on to tuning....
> 
> hi peter
> 
> peter.greis@ubs.com wrote:
> > I have been lurking for a while.... I recently put together a raid 5
> > system (Asus K8NE SIL 3114/2.6.8.1 kernel) with 4 300GB SATA Seagate
> > drives (a lot smaller than the bulk of what seems to be on this
> > list!). Currently this is used for video and mp3 storage, being
> > Reiser on LVM2.
> 
> beware that LVM2 _can_ affect your performance. I too believed that the
> concept of dynamic drives is good, but I experienced a performance hit
> of about 50% (especially in sequential reads).
> 
> see my blog entry describing how I built my 2TB file-server at
> http://variant.ch/phpwiki/WikiBlog/2005-02-27 for some numbers and more
> explanation.
> 
> the K8NE has the same SiI 3114 controller as the board I used has; it is
> connected by a 33mhz 32bit PCI bus and maxes out at 133MiB/s, so for
> maxmimum performance you might want to connect only two drives to this
> controller, the other two to the nforce3 chipset SATA ports.
> 
> > Bonnie++ to test, but with which parameters ?
> 
> normally it's enough to specify a test-file larger (e.g. twice) the
> memory capacity of the machine you are testing. for a machine with 1GiB RAM:
> 
> # bonnie++ -s 2gb {other options}
> 
> you might as well want to specify the "fast" option which skips per-char
> operations (which are quite useless to test IMHO):
> 
> # bonnie++ -f {other options}
> 
> HTH
> nicola
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 


-- 
Derek Piper - derek.piper@gmail.com
http://doofer.org/

now on to tuning....

[peter.greis]

Greetings All,

I have been lurking for a while.... I recently put together a raid 5 system (Asus K8NE SIL 3114/2.6.8.1 kernel) with 4 300GB SATA Seagate drives (a lot smaller than the bulk of what seems to be on this list!). Currently this is used for video and mp3 storage, being Reiser on LVM2.

So a couple of questions:

Bonnie++ to test, but with which parameters ? Also, I have seen the mount option "nolargeio=1" for reiser, but not a lot of information on the impact in raid systems.

Any thoughts ?

regards,

-Peter

Re: now on to tuning....

[Gordon Henderson]

On Wed, 9 Mar 2005 peter.greis@ubs.com wrote:

> Greetings All,
>
> I have been lurking for a while.... I recently put together a raid 5
> system (Asus K8NE SIL 3114/2.6.8.1 kernel) with 4 300GB SATA Seagate
> drives (a lot smaller than the bulk of what seems to be on this list!).

Size isn't important :)

> Currently this is used for video and mp3 storage, being Reiser on LVM2.
>
> So a couple of questions:
>
> Bonnie++ to test, but with which parameters ? Also, I have seen the
> mount option "nolargeio=1" for reiser, but not a lot of information on
> the impact in raid systems.
>
> Any thoughts ?

Why LVM2? Are you taking snapshots, or expecting to re-size the array?

I just could never work out why I needed yet another layer of software
between the application and the disk platter. There may well by a good
reason for it - I did look into LVM some 18 months ago when I was looking
at snapshot solutions though, but it was hideously slow (or appeared to
be) after I'd taken a snapshot. (My intention was to tape dump from the
snapshots, and provide a 'yesterday' & 'the day before' type things. I now
use rsync for that - takes time to make the snapshot, but accesses to it
are no slower than accessing the live aray.

I know nothing about reiser either, so can't help there, I'm afraid,
however, XFS has some real-time facilities which might be useful for
streaming data, but again, it's not something I've looked into.

Anyway - if you are streaming big files, you want nothing more than time
dd to test the speed of the thing. Bonnie++ will limit the size of a
single file to 2GB (and write multiple 2GB files if you specify a bigger
size) and you'll get an indication of how busy the CPU is when it's
writing & reading. Your drive will have between 55 and 65MB/sec of head
bandwidth, so anything better than that is a bonus, athough the 3114 will
have all 4 ports on the same PCI bus, so thats going to be your
bottleneck.

If you want more speed, you might want to try to reconfigure it in a
RAID-10 way, but then you'll only get 2 disks worth of storage rather than
the 3 in a RAID-5 setup.

Also test streaming over LAN if thats what you are doing - if it's just a
100Mb LAN, then all your disk system needs to be able to do is 12MB/ses
and anything more is a bonus.

And do check your disks regularly, although I don't think current version
of smartmontools fully supports sata under the scsi subsystem yet...

Good luck,

Gordon

Re: now on to tuning....

[Brad Campbell]

Gordon Henderson wrote:

> And do check your disks regularly, although I don't think current version
> of smartmontools fully supports sata under the scsi subsystem yet...
> 

Actually, if you are using a UP machine, the libata-dev tree has patches that make this work. I 
believe there may be races on SMP machines however.

All 29 drives get a short test every morning and a long test every Sunday morning. Odd results are 
immediately E-mailed to me by smartd.

storage1:/home/brad# smartctl -A -d ata /dev/sda
smartctl version 5.32 Copyright (C) 2002-4 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
   3 Spin_Up_Time            0x0027   252   252   063    Pre-fail  Always       -       5622
   4 Start_Stop_Count        0x0032   253   253   000    Old_age   Always       -       20
   5 Reallocated_Sector_Ct   0x0033   253   253   063    Pre-fail  Always       -       0
   6 Read_Channel_Margin     0x0001   253   253   100    Pre-fail  Offline      -       0
   7 Seek_Error_Rate         0x000a   253   252   000    Old_age   Always       -       0
   8 Seek_Time_Performance   0x0027   250   248   187    Pre-fail  Always       -       35232
   9 Power_On_Minutes        0x0032   252   252   000    Old_age   Always       -       457h+24m
  10 Spin_Retry_Count        0x002b   252   252   157    Pre-fail  Always       -       0
  11 Calibration_Retry_Count 0x002b   253   252   223    Pre-fail  Always       -       0
  12 Power_Cycle_Count       0x0032   253   253   000    Old_age   Always       -       34
192 Power-Off_Retract_Count 0x0032   253   253   000    Old_age   Always       -       0
193 Load_Cycle_Count        0x0032   253   253   000    Old_age   Always       -       0
194 Temperature_Celsius     0x0032   253   253   000    Old_age   Always       -       35
195 Hardware_ECC_Recovered  0x000a   253   252   000    Old_age   Always       -       1411
196 Reallocated_Event_Count 0x0008   253   253   000    Old_age   Offline      -       0
197 Current_Pending_Sector  0x0008   253   253   000    Old_age   Offline      -       0
198 Offline_Uncorrectable   0x0008   253   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0008   199   199   000    Old_age   Offline      -       0
200 Multi_Zone_Error_Rate   0x000a   253   252   000    Old_age   Always       -       0
201 Soft_Read_Error_Rate    0x000a   253   252   000    Old_age   Always       -       3
202 TA_Increase_Count       0x000a   253   252   000    Old_age   Always       -       0
203 Run_Out_Cancel          0x000b   253   252   180    Pre-fail  Always       -       0
204 Shock_Count_Write_Opern 0x000a   253   252   000    Old_age   Always       -       0
205 Shock_Rate_Write_Opern  0x000a   253   252   000    Old_age   Always       -       0
207 Spin_High_Current       0x002a   252   252   000    Old_age   Always       -       0
208 Spin_Buzz               0x002a   252   252   000    Old_age   Always       -       0
209 Offline_Seek_Performnce 0x0024   194   194   000    Old_age   Offline      -       0
  99 Unknown_Attribute       0x0004   253   253   000    Old_age   Offline      -       0
100 Unknown_Attribute       0x0004   253   253   000    Old_age   Offline      -       0
101 Unknown_Attribute       0x0004   253   253   000    Old_age   Offline      -       0

Regards,
Brad
-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams

Re: now on to tuning....

[Gordon Henderson]

On Wed, 9 Mar 2005, Brad Campbell wrote:

> Gordon Henderson wrote:
>
> > And do check your disks regularly, although I don't think current version
> > of smartmontools fully supports sata under the scsi subsystem yet...
> >
>
> Actually, if you are using a UP machine, the libata-dev tree has patches
> that make this work. I believe there may be races on SMP machines
> however.
>
> All 29 drives get a short test every morning and a long test every
> Sunday morning. Odd results are immediately E-mailed to me by smartd.
>
> storage1:/home/brad# smartctl -A -d ata /dev/sda

Ahhh... I've been waiting for the magical "-d libata" to make it work.. I
never through to just try the -d ata!

And hurrah:

Old:

  skylla:/home/gordonh# smartctl -a /dev/sda
  smartctl version 5.33 [i686-pc-linux-gnu] Copyright (C) 2002-4 Bruce Allen
  Home page is http://smartmontools.sourceforge.net/

  Device: ATA      Maxtor 6Y080M0   Version: YAR5

  SATA disks accessed via libata are not currently supported by
  smartmontools. When libata is given an ATA pass-thru ioctl() then an
  additional '-d libata' device type will be added to smartmontools.

New:

  smartctl version 5.33 [i686-pc-linux-gnu] Copyright (C) 2002-4 Bruce Allen
  Home page is http://smartmontools.sourceforge.net/

  === START OF INFORMATION SECTION ===
  Device Model:     Maxtor 6Y080M0
  Serial Number:    Y2ASRT5E
  Firmware Version: YAR51HW0
  User Capacity:    80,000,000,000 bytes
  Device is:        In smartctl database [for details use: -P show]
  ATA Version is:   7
  ATA Standard is:  ATA/ATAPI-7 T13 1532D revision 0
  Local Time is:    Wed Mar  9 11:09:12 2005 GMT
  SMART support is: Available - device has SMART capability.
  SMART support is: Enabled

Cheers,

Gordon
Ps. This is a Dell 1U poweredge FWIW - Maxtor disks )-:

Re: now on to tuning....

[Nicola Fankhauser]

hi peter

peter.greis@ubs.com wrote:
> I have been lurking for a while.... I recently put together a raid 5
> system (Asus K8NE SIL 3114/2.6.8.1 kernel) with 4 300GB SATA Seagate
> drives (a lot smaller than the bulk of what seems to be on this
> list!). Currently this is used for video and mp3 storage, being
> Reiser on LVM2.

beware that LVM2 _can_ affect your performance. I too believed that the 
concept of dynamic drives is good, but I experienced a performance hit 
of about 50% (especially in sequential reads).

see my blog entry describing how I built my 2TB file-server at 
http://variant.ch/phpwiki/WikiBlog/2005-02-27 for some numbers and more 
explanation.

the K8NE has the same SiI 3114 controller as the board I used has; it is 
connected by a 33mhz 32bit PCI bus and maxes out at 133MiB/s, so for 
maxmimum performance you might want to connect only two drives to this 
controller, the other two to the nforce3 chipset SATA ports.

> Bonnie++ to test, but with which parameters ? 

normally it's enough to specify a test-file larger (e.g. twice) the 
memory capacity of the machine you are testing. for a machine with 1GiB RAM:

# bonnie++ -s 2gb {other options}

you might as well want to specify the "fast" option which skips per-char 
operations (which are quite useless to test IMHO):

# bonnie++ -f {other options}

HTH
nicola