Re: [linux-lvm] Allocation Policy for Cloud Computing needed

[Lars Ellenberg <lars.ellenberg@linbit.com>]

On Thu, Feb 16, 2012 at 02:50:07PM +0100, Sebastian Riemer wrote:
> Hi LVM list,
> 
> I'm experimenting with storage for many QEMU/KVM virtual machines in
> cloud computing. I've got many concurrent IO processes and 24 hard
> drives. I've tested the scalability with a single IO reader process per
> hard drive. Single drives scale best and have the best performance of
> cause, but we need mirroring and volume management. So I've created MD
> RAID-1 arrays and created on each a VG and two LVs. This gives me good
> overall performance (up to 2 GB/s, HBA limit: 2.2 GB/s).
> 
> Then, I've tested to put all my RAID-1 arrays into a single VG, because
> LV size should be adjustable over all hard drives. I've tried all
> allocation policies but none does what I want to achieve here. Yeah,
> that this isn't implemented fully is in the man page, ... .
> 
> I want to have an allocation which distributes the LVs equally over the
> PVs as long as space is left and LVs aren't resized. The goal is to
> minimize the number of concurrent IO processes per hard drive (striping
> is total crap in this situation).
> 
> I've tested LVM2 2.02.66 and kernel 3.0.15. Is something like that
> implemented in newer releases or is something like that intended to be
> implemented in near future?

I don't know.  Does not look like it, though.

> Or does someone want to implement this together with me?

I would certainly be here for discussions.

Though, as you always will be more flexible with scripts than with
pre-implemented fixed algorithms, I probably would first check if I can
solve it with some scripting.
[completely untested, but you get the idea]

#!/bin/bash
export LANG=C LC_ALL=C
name=$1 vg=$2 size_in_MiB=$3
PVS=$(vgs --nohead --unit m -o pv_name,pv_free -O -pv_free,pv_name $vg |
	awk -v need=$size_in_MiB '{ print $1; sum += $2;
	if (sum >= need) exit; }')
lvcreate -n $name -L ${size_in_MiB}m $vg $PVS

(similar for lvextend)

Which basically implements this allocation policy:
use the pvs with most free space available,
and no more than necessary.

If I understand you correctly, that would almost do what you asked for.

You can get pretty complex in similar scripts, if you really want to...
consider using
  pvs -o vg_name,lv_name,pv_name,pvseg_start,pvseg_size,seg_pe_ranges
and explicitly listing not only the PVS, but even the PE ranges to your
lvcreate commands...

	Lars

-- 
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com