swap on raid

swap on raid

[Peter Rabbitson]

Hi,

I need to use a raid volume for swap, utilizing partitions from 4 
physical drives I have available. From my experience I have three 
options - raid5, raid10 with 2 offset chunks, and two raid 1 volumes 
that are swapon-ed with equal priority. However I have a hard time 
figuring out what to use as I am not really sure how can I detect the 
usage patterns of swap, left alone benchmark it. Has anyone done 
anything like this, or is there information on what kind of reads/writes 
the kernel performs when paging in and out?

Before you answer my question - yes, I am painfully aware of the 
paradigm "swap on raid is bad", and I know there are other ways to solve 
it, but my situation requires me to have swap. Several weeks ago a drive 
failed and took a full partition away bringing the system to its knees 
and causied massive data corruption. I am also aware that I can use a 
file that will reside alongside my other data, but fragmentation makes 
this approach inefficient. So I am looking into placing the swap 
directly on a raid voulme.


Thanks 

Peter

Re: swap on raid

[Justin Piszcz]

Swap on a redundant RAID is a smart idea, not sure who would tell you 
otherwise.  Depends which disks are faster?  I have a RAID5 which is 2x as 
fast as a RAID1 (both SW raid)-- whichever is faster, do some 
benchmarking.  I would assume RAID10, but I have not benchmarked to 
confirm this.

Justin.

On Thu, 1 Mar 2007, Peter Rabbitson wrote:

> Hi,
>
> I need to use a raid volume for swap, utilizing partitions from 4
> physical drives I have available. From my experience I have three
> options - raid5, raid10 with 2 offset chunks, and two raid 1 volumes
> that are swapon-ed with equal priority. However I have a hard time
> figuring out what to use as I am not really sure how can I detect the
> usage patterns of swap, left alone benchmark it. Has anyone done
> anything like this, or is there information on what kind of reads/writes
> the kernel performs when paging in and out?
>
> Before you answer my question - yes, I am painfully aware of the
> paradigm "swap on raid is bad", and I know there are other ways to solve
> it, but my situation requires me to have swap. Several weeks ago a drive
> failed and took a full partition away bringing the system to its knees
> and causied massive data corruption. I am also aware that I can use a
> file that will reside alongside my other data, but fragmentation makes
> this approach inefficient. So I am looking into placing the swap
> directly on a raid voulme.
>
>
> Thanks
>
> Peter
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: swap on raid

[Richard Scobie]

Peter Rabbitson wrote:
> Hi,
> 
> I need to use a raid volume for swap, utilizing partitions from 4 
> physical drives I have available. From my experience I have three 
> options - raid5, raid10 with 2 offset chunks, and two raid 1 volumes 
> that are swapon-ed with equal priority. However I have a hard time 

Another option here would be a 4 disk RAID1, on the basis that it's 
giving you a simple, highly redundant setup.

If the performance requirement is such that you need the speed of RAID5, 
10 etc., then you probably need to be looking at adding RAM or otherwise 
avoiding swapping in the first place.

The fact that you mention you are using partitions on disks that 
possibly have other partions doing other things, means raw performance 
will be compromised anyway.

Regards,

Richard

Re: swap on raid

[Bill Davidsen]

Peter Rabbitson wrote:
> Hi,
>
> I need to use a raid volume for swap, utilizing partitions from 4 
> physical drives I have available. From my experience I have three 
> options - raid5, raid10 with 2 offset chunks, and two raid 1 volumes 
> that are swapon-ed with equal priority. However I have a hard time 
> figuring out what to use as I am not really sure how can I detect the 
> usage patterns of swap, left alone benchmark it. Has anyone done 
> anything like this, or is there information on what kind of reads/writes 
> the kernel performs when paging in and out?
>
> Before you answer my question - yes, I am painfully aware of the 
> paradigm "swap on raid is bad", and I know there are other ways to solve 
> it, but my situation requires me to have swap. Several weeks ago a drive 
> failed and took a full partition away bringing the system to its knees 
> and causied massive data corruption. I am also aware that I can use a 
> file that will reside alongside my other data, but fragmentation makes 
> this approach inefficient. So I am looking into placing the swap 
> directly on a raid voulme.
I did some benchmarking, using mem= to force swap. It seems that RAID10 
is faster, has nice even disk utilization, etc, etc. The downside is 
that many recovery CDs do not know about RAID10 and can't handle running 
swap there. Test before you decide. Two fast and two slow can be done 
with RAID0 over RAID1 pairs, RAID1 one fast and one slow and mark the 
slow one "write-mostly" to reduce use. RAID1 on two active with the 
other two as hot spares is viable as well.

In other words, you can do it, tell us the size and speed of the 
partitions and you will get a bunch of ideas matched to your hardware.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: swap on raid

[Peter Rabbitson]

> The fact that you mention you are using partitions on disks that 
> possibly have other partions doing other things, means raw performance 
> will be compromised anyway.
> 
> Regards,
> 
> Richard

You know I never thought about it, but you are absolutely right. The 
times at which my memory usage peaks coincide with high disk activity 
(mostly reads). In this light it actually might be better to keep the 
swap in a file on my raid10 (-p n3) which occupies most of these 4 
drives, and hope that the md code will be able to distribute the io 
across idle drives. Does this sound about right?

Re: swap on raid

[Mark Hahn]

>> The fact that you mention you are using partitions on disks that
>> possibly have other partions doing other things, means raw performance
>> will be compromised anyway.

with normal unraided swap (partition or file), swapouts are not a performance
problem, since they're lazy, relatively cheap, cpu-wise, and not likely to be
voluminous.  if you ever have more than a trivial number of swapins, you'll
be crawling; this is merely a crutch until you can fix your memory problem.

swap over raid5 is somewhat opposed to this: the swapouts now start
eating more CPU.  but non-degraded raid5 swapins run like an N-1 disk raid0.

> (mostly reads). In this light it actually might be better to keep the
> swap in a file on my raid10 (-p n3) which occupies most of these 4
> drives, and hope that the md code will be able to distribute the io
> across idle drives. Does this sound about right?

yes.  especially better than the absurd way some installers insist on 
putting swap partitions at the slow/distant tail of the disk...

but if you're planning to actually _use_ swap, you should probably 
start over, planning more memory instead.