Re: Optimize RAID0 for max IOPS?

[Roberto Spadim <roberto@spadim.com.br>]

a good idea....
why not start a opensource raid controller?
what we need? a cpu, memory, power supply with battery or capacitor,
sas/sata (disk interfaces), pci-express or another (computer
interface)
it don´t need a operational system, since it will only run one program
with some threads (ok a small operational system to implement threads
easly)

we could use arm, fpga, intel core2duo, atlhon, xeon, or another system...
instead using a computer with ethernet interface (nbd nfs samba or
another file/device sharing iscsi ethernet sata), we need a computer
with pci-express interface and native operational system module


2011/1/19 Roberto Spadim <roberto@spadim.com.br>:
> the problem....
> if you use iostat, or iotop
> with software raid:
>   you just see disk i/o
>   you don´t see memory (cache) i/o
> when using hardware raid:
>   you just see raid i/o (it can be a cache read or a real disk read)
>
>
> if you check memory+disk i/o, you will get similar values, if not, you
> will see high cpu usage
> for example you are using raidx with 10disks on a hardware raid
> change hardware raid to use only disks (10 disks for linux)
> make the same raidx with 10disks
> you will get a slower i/o since it have a controler between disk and cpu
> try it without hardware raid cpu, just a (sas/sata) optimized
> controller, or 10 (sata/sas) one port
> you still with a slow i/o then hardware controller (that´s right!)
>
> now let´s remove the sata/sas channel, let´s use a pci-express
> revodrive or pci-express texas ssd drive
> you will get better values then a hardware raid, but... why? you
> changed the hardware (ok, i know) but you make cpu more close to disk
> if you use disks with cache, you will get more speed (a memory ssd
> harddisk is faster than a harddisk only disk)
>
> why hardware are more faster than linux? i don´t think they are...
> they can make smaller latencies with good memory cache
> but if you computer use ddr3 and your hardware raid controller use i2c
> memory, your ddr3 cache is faster...
>
> how to benchmark? check disk i/o+memory cache i/o
> if linux is faster ok, you use more cpu and memory of your computer
> if linux is slower ok, you use less cpu and memory, but will have it
> on hardware raid...
> if you upgrade you memory and cpu, it can be faster than you hardware
> raid controller, what´s better for you?
>
> want a better read/write solution for software raid? make a new
> read/write code, you can do it, linux is easier than hardware raid to
> code!
> want a better read/write solution for hardware raid? call your
> hardware seller and talk, please i need a better firmware, could you
> send me?
>
> got?
>
>
> 2011/1/19 Stefan /*St0fF*/ Hübner <stefan.huebner@stud.tu-ilmenau.de>:
>> @Roberto: I guess you're right. BUT: i have not seen 900MB/s coming from
>> (i.e. read access) a software raid, but I've seen it from a 9750 on a
>> LSI SASx28 backplane, running RAID6 over 16disks (HDS722020ALA330).  So
>> one might not be wrong assuming on current raid-controllers
>> hardware/software matching and timing is way more optimized than what
>> mdraid might get at all.
>>
>> The 9650 and 9690 are considerably slower, but I've seen 550MB/s thruput
>> from those, also (I don't recall the setup anymore, tho).
>>
>> Max reading I saw from a software raid was around 350MB/s - so hence my
>> answers.  And if people had problems with controllers which are 5 years
>> or older by now, the numbers are not really comparable...
>>
>> Now again there's the point where there are also parameters on the
>> controller that can be tweaked, and a simple way to recreate the testing
>> scenario.  We may discuss and throw in further numbers and experience,
>> but not being able to recreate your specific scenario makes us talk past
>> each other...
>>
>> stefan
>>
>> Am 19.01.2011 20:50, schrieb Roberto Spadim:
>>> So can anybody help answering these questions:
>>>
>>> - are there any special options when creating the RAID0 to make it
>>> perform faster for such a use case?
>>> - are there other tunables, any special MD / LVM / file system / read
>>> ahead / buffer cache / ... parameters to look for?
>>>
>>> lets see:
>>> what´s your disk (ssd or sas or sata) best block size to write/read?
>>> write this at ->(A)
>>> what´s your work load? 50% write 50% read ?
>>>
>>> raid0 block size should be multiple of (A)
>>> *****filesystem size should be multiple of (A) of all disks
>>> *****read ahead should be a multiple of (A)
>>> for example
>>> /dev/sda 1kb
>>> /dev/sdb 4kb
>>>
>>> you should use 6kb... you should use 4kb, 8kb, 16kb (multiple of 1kb and 4kb)
>>>
>>> check i/o sheduller per disk too (ssd should use noop, disks should
>>> use cfq, deadline or another...)
>>> async and sync option at mount /etc/fstab, noatime reduce a lot of i/o
>>> too, you should optimize your application too
>>> hdparm each disk to use dma and fastest i/o options
>>>
>>> are you using only filesystem? are you using somethink more? samba?
>>> mysql? apache? lvm?
>>> each of this programs have some tunning, check their benchmarks
>>>
>>>
>>> getting back....
>>> what´s a raid controller?
>>> cpu + memory + disk controller + disks
>>> but... it only run raid software (it can run linux....)
>>>
>>> if you computer is slower than raid cpu+memory+disk controller, you
>>> will have a slower software raid, than hardware raid
>>> it´s like load balance on cpu/memory utilization of disk i/o (use
>>> dedicated hardware, or use your hardware?)
>>> got it?
>>> using a super fast xeon with ddr3 and optical fiber running software
>>> raid, is faster than a hardware raid using a arm (or fpga) ddrX memory
>>> and sas(fiber optical) connection to disks
>>>
>>> two solutions for the same problem
>>> what´s fast? benchmark it
>>> i think that if your xeon run a database and a very workloaded apache,
>>> a dedicated hardware raid can run faster, but a light xeon can run
>>> faster than a dedicated hardware raid
>>>
>>>
>>>
>>> 2011/1/19 Wolfgang Denk <wd@denx.de>:
>>>> Dear =?ISO-8859-15?Q?Stefan_/*St0fF*/_H=FCbner?=,
>>>>
>>>> In message <4D361F26.3060507@stud.tu-ilmenau.de> you wrote:
>>>>>
>>>>> [in German:] Schätzelein, Dein Problem sind die Platten, nicht der
>>>>> Controller.
>>>>>
>>>>> [in English:] Dude, the disks are your bottleneck.
>>>> ...
>>>>
>>>> Maybe we can stop speculations about what might be the cause of the
>>>> problems in some setup I do NOT intend to use, and rather discuss the
>>>> questions I asked.
>>>>
>>>>>> I will have 4 x 1 TB disks for this setup.
>>>>>>
>>>>>> The plan is to build a RAID0 from the 4 devices, create a physical
>>>>>> volume and a volume group on the resulting /dev/md?, then create 2 or
>>>>>> 3 logical volumes that will be used as XFS file systems.
>>>>
>>>> Clarrification: I'll run /dev/md* on the raw disks, without any
>>>> partitions on them.
>>>>
>>>>>> My goal is to optimize for maximum number of I/O operations per
>>>>>> second. ...
>>>>>>
>>>>>> Is this a reasonable approach for such a task?
>>>>>>
>>>>>> Should I do anything different to acchive maximum performance?
>>>>>>
>>>>>> What are the tunables in this setup?  [It seems the usual recipies are
>>>>>> more oriented in maximizing the data troughput for large, mostly
>>>>>> sequential accesses - I figure that things like increasing read-ahead
>>>>>> etc. will not help me much here?]
>>>>
>>>> So can anybody help answering these questions:
>>>>
>>>> - are there any special options when creating the RAID0 to make it
>>>>  perform faster for such a use case?
>>>> - are there other tunables, any special MD / LVM / file system /
>>>>  read ahead / buffer cache / ... parameters to look for?
>>>>
>>>> Thanks.
>>>>
>>>> Wolfgang Denk
>>>>
>>>> --
>>>> DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
>>>> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
>>>> Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
>>>> Boykottiert Microsoft - Kauft Eure Fenster bei OBI!
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>
>>>
>>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
>
>
> --
> Roberto Spadim
> Spadim Technology / SPAEmpresarial
>



-- 
Roberto Spadim
Spadim Technology / SPAEmpresarial
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html