raid1 - ssd, doubts

raid1 - ssd, doubts

[Roberto Spadim]

hi guys!
i will use a ssd raid1, i want know if raid1 trim is supported at mdadm
i will use a 840 evo (or evo pro not selected the right one yet) 500gb
each, raid1, today database size is 100gb, i think it will grow
10gb/year, i had many space...

the point are: madm raid1 trim is supported? or should i use lvm?
should i partition it with 400gb and leave 100gb untouched? or should
i use a hdd+ssd and dmcache?


:) thanks guys, that's a small enterprise solution, they can't buy
raid cards and sas harddisk are same price of ssd :)

any idea/experience and information is wellcome


-- 
Roberto Spadim

Re: raid1 - ssd, doubts

[Andrei Banu]

Hi,

I am in no way of the same technical caliber as many of the people on this
list so please take my advice with a grain of salt. However I do have some
experience with Samsung SSDs in RAID 1 software (md-raid) and I would like
to warn you against it.

My setup is with 840 PROs of 512GB. I did leave free space (not partitioned)
for OP. The SSDs are connected on SATA3.

I'll give you the result of 2 tests and if you are happy, go ahead:

READ
root [~]# hdparm -t /dev/sda
/dev/sda:
  Timing buffered disk reads: 292 MB in  3.01 seconds =  96.92 MB/sec

WRITE
root [~]# dd if=/dev/zero of=testn bs=4k count=256k conv=fdatasync
262144+0 records in
262144+0 records out
1073741824 bytes (1.1 GB) copied, 13.4047 s, 80.1 MB/s

When the setup is fresh you'll get significantly more out of it but 
after a while
(this setup is roughly 1 year old) this is what you get with Samsung SSDs in
RAID1 SW. At least this is my experience and I did try to improve it. As 
a matter
of fact, before making a secure erase of one of the SSDs the write speed 
was
under 10MB/s. What you see above is a lot better than what it started out
(but it's true that after the secure erase the read/write speeds were a lot
better but it dropped in a very short time frame).

A few points:
1. CAUTION: 840 EVO are proved to have a read speed degradation for old
data written to the drive (this is unrelated to RAID but it probably 
affects
performance in RAID).
2. I believe that any SSDs, regardless of the brand, in a software 
RAID-1 setup
might be a bad idea.
3. I believe that the same SSDs in a hardware RAID setup might lead to a
different story.

Again: I am not a technical wiz (like many of the people on this list) 
but I did
have some experience with this so I thought I should let you know.

Kind regards!



On 30.09.2014 17:50, Roberto Spadim wrote:
> hi guys!
> i will use a ssd raid1, i want know if raid1 trim is supported at mdadm
> i will use a 840 evo (or evo pro not selected the right one yet) 500gb
> each, raid1, today database size is 100gb, i think it will grow
> 10gb/year, i had many space...
>
> the point are: madm raid1 trim is supported? or should i use lvm?
> should i partition it with 400gb and leave 100gb untouched? or should
> i use a hdd+ssd and dmcache?
>
>
> :) thanks guys, that's a small enterprise solution, they can't buy
> raid cards and sas harddisk are same price of ssd :)
>
> any idea/experience and information is wellcome
>
>

Re: raid1 - ssd, doubts

[Roberto Spadim]

nice :) well today it use hdd with sata, 500GB with raid 1 (2 disks)
the only problem is the disk being old, i must replace it
i'm considering ssd only by the price and myth, hdd works well today
i have some doubts about performace without the raid card as you told,
the last ssd i used was enterprise level not desktop/non enterprise
level, and with a raid card

well, others experiences and comments are wellcome :)
comments about cache (bcache,flashcache,dm cache) are wellcome too


2014-09-30 12:46 GMT-03:00 Andrei Banu <andrei.banu@redhost.ro>:
> Hi,
>
> I am in no way of the same technical caliber as many of the people on this
> list so please take my advice with a grain of salt. However I do have some
> experience with Samsung SSDs in RAID 1 software (md-raid) and I would like
> to warn you against it.
>
> My setup is with 840 PROs of 512GB. I did leave free space (not partitioned)
> for OP. The SSDs are connected on SATA3.
>
> I'll give you the result of 2 tests and if you are happy, go ahead:
>
> READ
> root [~]# hdparm -t /dev/sda
> /dev/sda:
>  Timing buffered disk reads: 292 MB in  3.01 seconds =  96.92 MB/sec
>
> WRITE
> root [~]# dd if=/dev/zero of=testn bs=4k count=256k conv=fdatasync
> 262144+0 records in
> 262144+0 records out
> 1073741824 bytes (1.1 GB) copied, 13.4047 s, 80.1 MB/s
>
> When the setup is fresh you'll get significantly more out of it but after a
> while
> (this setup is roughly 1 year old) this is what you get with Samsung SSDs in
> RAID1 SW. At least this is my experience and I did try to improve it. As a
> matter
> of fact, before making a secure erase of one of the SSDs the write speed was
> under 10MB/s. What you see above is a lot better than what it started out
> (but it's true that after the secure erase the read/write speeds were a lot
> better but it dropped in a very short time frame).
>
> A few points:
> 1. CAUTION: 840 EVO are proved to have a read speed degradation for old
> data written to the drive (this is unrelated to RAID but it probably affects
> performance in RAID).
> 2. I believe that any SSDs, regardless of the brand, in a software RAID-1
> setup
> might be a bad idea.
> 3. I believe that the same SSDs in a hardware RAID setup might lead to a
> different story.
>
> Again: I am not a technical wiz (like many of the people on this list) but I
> did
> have some experience with this so I thought I should let you know.
>
> Kind regards!
>
>
>
>
> On 30.09.2014 17:50, Roberto Spadim wrote:
>>
>> hi guys!
>> i will use a ssd raid1, i want know if raid1 trim is supported at mdadm
>> i will use a 840 evo (or evo pro not selected the right one yet) 500gb
>> each, raid1, today database size is 100gb, i think it will grow
>> 10gb/year, i had many space...
>>
>> the point are: madm raid1 trim is supported? or should i use lvm?
>> should i partition it with 400gb and leave 100gb untouched? or should
>> i use a hdd+ssd and dmcache?
>>
>>
>> :) thanks guys, that's a small enterprise solution, they can't buy
>> raid cards and sas harddisk are same price of ssd :)
>>
>> any idea/experience and information is wellcome
>>
>>
>



-- 
Roberto Spadim
SPAEmpresarial
Eng. Automação e Controle
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: raid1 - ssd, doubts

[Robert L Mathews]

On 9/30/14 9:20 AM, Roberto Spadim wrote:

> well, others experiences and comments are wellcome :)
> comments about cache (bcache,flashcache,dm cache) are wellcome too

Keep in mind that sequential read and write speeds are not everything.
For our use pattern, for example (busy Web and mail servers with
millions of files that aren't necessarily grouped physically on the
disk, even when in the same directory [think maildirs where each file
represents one message]), latency is more important -- particularly read
latency.

Our servers use three-disk RAID 1 arrays. When we replaced one of the
spinning hard drives in each array with an SSD (some of which were
Samsung 840 Pros), then marked the remaining two spinning disks as
"write-mostly", the average read latency on the array dropped from
around 12 ms to less than 2 ms.

More importantly, the average amount of time any process on a server is
waiting in the "D" state (read or write) dropped from 8% to 3%.

Note that improving the read performance this way also improves the
write performance of the entire array, because when a write occurs, it
will never be queued behind a spinning disk read: the spinning disk is
more likely to be idle.

So our experience confirms that adding a single SSD to a RAID 1 array,
then marking the others write-mostly, is a good stopgap measure on the
road to replacing all spinning disks with SSDs. It effectively doubled
the average performance of our storage. No additional caching layers
required.

-- 
Robert L Mathews, Tiger Technologies, http://www.tigertech.net/

Re: raid1 - ssd, doubts

[Roberto Spadim]

wow, nice :)
i think that's something near what i have today, the latency is the
main problem that i need to reduce and control, checking every time
with users, they don't care if system is slow to send a big file, they
care if system stop for some seconds and get back, and stop again and
get back, a smothly operation is prefered instead of batchs of high
speed

+1 to this experience, here i have some "D" state process too,
interesting common point

let'me ask others questions....

today you have 3 disk and/or ssd, before you had 3 disks, any time you
got 2 disks setup and tried to change only one disk to ssd? that will
be my scenario if i stay with raid and ssd solution without new cache
layers, about trim commands and this setup life time, there's any
scheduler cleanup method? how many time running this setup? here 1
year is a nice time to consider as a sucessfull project, i replace
disks near to 2 years






2014-09-30 16:58 GMT-03:00 Robert L Mathews <lists@tigertech.com>:
> On 9/30/14 9:20 AM, Roberto Spadim wrote:
>
>> well, others experiences and comments are wellcome :)
>> comments about cache (bcache,flashcache,dm cache) are wellcome too
>
> Keep in mind that sequential read and write speeds are not everything.
> For our use pattern, for example (busy Web and mail servers with
> millions of files that aren't necessarily grouped physically on the
> disk, even when in the same directory [think maildirs where each file
> represents one message]), latency is more important -- particularly read
> latency.
>
> Our servers use three-disk RAID 1 arrays. When we replaced one of the
> spinning hard drives in each array with an SSD (some of which were
> Samsung 840 Pros), then marked the remaining two spinning disks as
> "write-mostly", the average read latency on the array dropped from
> around 12 ms to less than 2 ms.
>
> More importantly, the average amount of time any process on a server is
> waiting in the "D" state (read or write) dropped from 8% to 3%.
>
> Note that improving the read performance this way also improves the
> write performance of the entire array, because when a write occurs, it
> will never be queued behind a spinning disk read: the spinning disk is
> more likely to be idle.
>
> So our experience confirms that adding a single SSD to a RAID 1 array,
> then marking the others write-mostly, is a good stopgap measure on the
> road to replacing all spinning disks with SSDs. It effectively doubled
> the average performance of our storage. No additional caching layers
> required.
>
> --
> Robert L Mathews, Tiger Technologies, http://www.tigertech.net/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Roberto Spadim

Re: raid1 - ssd, doubts

[Brassow Jonathan]

On Sep 30, 2014, at 9:50 AM, Roberto Spadim wrote:
> the point are: madm raid1 trim is supported? or should i use lvm?
> should i partition it with 400gb and leave 100gb untouched? or should
> i use a hdd+ssd and dmcache?

LVM does not yet support trim on RAID.  (Funny you should ask!  We've just been talking about enabling this in LVM recently and it should go upstream very soon - following into the various distros a little later.)

 brassow

Re: raid1 - ssd, doubts

[Roberto Spadim]

humm interesting , just mdraid1 work with trim :)

other doubt now.. when using ssd+hdd, should trim be enabled at filesystem?

2014-09-30 21:36 GMT-03:00 Brassow Jonathan <jbrassow@redhat.com>:
>
> On Sep 30, 2014, at 9:50 AM, Roberto Spadim wrote:
>> the point are: madm raid1 trim is supported? or should i use lvm?
>> should i partition it with 400gb and leave 100gb untouched? or should
>> i use a hdd+ssd and dmcache?
>
> LVM does not yet support trim on RAID.  (Funny you should ask!  We've just been talking about enabling this in LVM recently and it should go upstream very soon - following into the various distros a little later.)
>
>  brassow



-- 
Roberto Spadim
SPAEmpresarial
Eng. Automação e Controle
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: raid1 - ssd, doubts

[David Brown]

On 30/09/14 16:50, Roberto Spadim wrote:
> hi guys!
> i will use a ssd raid1, i want know if raid1 trim is supported at mdadm
> i will use a 840 evo (or evo pro not selected the right one yet) 500gb
> each, raid1, today database size is 100gb, i think it will grow
> 10gb/year, i had many space...
> 
> the point are: madm raid1 trim is supported? or should i use lvm?
> should i partition it with 400gb and leave 100gb untouched? or should
> i use a hdd+ssd and dmcache?
> 
> 
> :) thanks guys, that's a small enterprise solution, they can't buy
> raid cards and sas harddisk are same price of ssd :)
> 
> any idea/experience and information is wellcome
> 

With reasonably modern SSD's, don't worry about trim.  Make sure you've
got enough over-provisioning (look at the specs for the SSD - and if
necessary, leave a 5-10% of the space on the disk as unpartitioned free
space).

The main point of trim is to return blocks to the list of blocks
available for recycling - if you have plenty of overprovisioning, this
list (along with the list of blocks already erased) is always going to
have plenty in it.  Thus trim doesn't add anything useful here.

The second effect of trim is to reduce the number of blocks copied
during garbage collection.  As it is unlikely that many extra blocks are
copied, and it is done at off-peak times by the SSD, this makes very
little difference - certainly nothing you would notice in most use-cases.

In other words, you should not notice whether trim is used or not.

Having said that, I believe trim works on raid1 with current kernels.
But make sure you only do off-line trim (fstrim), not the "discard"
mount option in ext4 (or equivalent on other filesystems), as discard
mount will give you noticeably slower performance on many operations.

Re: raid1 - ssd, doubts

[Mikael Abrahamsson]

On Tue, 30 Sep 2014, Roberto Spadim wrote:

> any idea/experience and information is wellcome

Please read the treads in the archives regarding this issue. It's been 
discussed multiple times.

http://board.issociate.de/thread/508953/Re-Software-RAID-and-TRIM.html is 
one of them.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se