lvm2 performance data with linux-2.6

lvm2 performance data with linux-2.6

[markw]

I've started collecting various data (including oprofile) using our
DBT-2 (OLTP) workload with lvm2 on linux 2.6.2 and 2.6.3 on ia32 and
ia64 platforms:
	http://developer.osdl.org/markw/lvm2/

So far I've only varied the stripe width with lvm, from 8 KB to 512 KB,
for PostgreSQL that is using 8 KB sized blocks with ext2.  It appears
that a stripe width of 16 KB through 128KB on the ia64 system gives the
best throughput for the DBT-2 workload on a volume that should be doing
mostly sequential writes.

I'm going to run through more tests varying the block size that
PostgreSQL uses, but I wanted to share what I had so far in case there
were other suggestions or recommendations.

-- 
Mark Wong - - markw@osdl.org
Open Source Development Lab Inc - A non-profit corporation
12725 SW Millikan Way - Suite 400 - Beaverton, OR 97005
(503) 626-2455 x 32 (office)
(503) 626-2436      (fax)
http://developer.osdl.org/markw/

Re: lvm2 performance data with linux-2.6

[Miquel van Smoorenburg]

In article <200403081916.i28JGgE25794@mail.osdl.org>,
 <markw@osdl.org> wrote:
>I've started collecting various data (including oprofile) using our
>DBT-2 (OLTP) workload with lvm2 on linux 2.6.2 and 2.6.3 on ia32 and
>ia64 platforms:
>	http://developer.osdl.org/markw/lvm2/

For write-heavy loads, you might want to see if 2.6.4-rc2-mm1 makes
any difference. It fixes a case where both pdflush and a writing
process itself would queue writes to an LVM device, resulting in
less than optimal I/O ordering for some cases. The fix will probably
be in 2.6.4 proper as well.

Mike.

Re: lvm2 performance data with linux-2.6

[Matthias Andree]

On Mon, 08 Mar 2004, markw@osdl.org wrote:

> I've started collecting various data (including oprofile) using our
> DBT-2 (OLTP) workload with lvm2 on linux 2.6.2 and 2.6.3 on ia32 and
> ia64 platforms:
> 	http://developer.osdl.org/markw/lvm2/
> 
> So far I've only varied the stripe width with lvm, from 8 KB to 512 KB,
> for PostgreSQL that is using 8 KB sized blocks with ext2.  It appears

Does ext2 write 8 KByte sized blocks atomically on IA32?

Or is this no requirement for PostgreSQL consistency?

-- 
Matthias Andree

Encrypt your mail: my GnuPG key ID is 0x052E7D95

Re: lvm2 performance data with linux-2.6

[Bill Davidsen]

markw@osdl.org wrote:
> I've started collecting various data (including oprofile) using our
> DBT-2 (OLTP) workload with lvm2 on linux 2.6.2 and 2.6.3 on ia32 and
> ia64 platforms:
> 	http://developer.osdl.org/markw/lvm2/
> 
> So far I've only varied the stripe width with lvm, from 8 KB to 512 KB,
> for PostgreSQL that is using 8 KB sized blocks with ext2.  It appears
> that a stripe width of 16 KB through 128KB on the ia64 system gives the
> best throughput for the DBT-2 workload on a volume that should be doing
> mostly sequential writes.
> 
> I'm going to run through more tests varying the block size that
> PostgreSQL uses, but I wanted to share what I had so far in case there
> were other suggestions or recommendations.
> 
Here's one thought: look at the i/o rates on individual drives using 
each stripe size. You *might* see that one size does far fewer seeks 
than others, which is a secondary thing to optimize after throughput IMHO.

If you don't have a tool for this I can send you the latest diorate 
which does stuff like this, io rate perdrive or per partition, something 
I occasionally find revealing.

		-bill

Re: lvm2 performance data with linux-2.6

[Mark Wong]

On Thu, Mar 11, 2004 at 05:12:35PM -0500, Bill Davidsen wrote:
> markw@osdl.org wrote:
> > I've started collecting various data (including oprofile) using our
> > DBT-2 (OLTP) workload with lvm2 on linux 2.6.2 and 2.6.3 on ia32 and
> > ia64 platforms:
> > 	http://developer.osdl.org/markw/lvm2/
> > 
> > So far I've only varied the stripe width with lvm, from 8 KB to 512 KB,
> > for PostgreSQL that is using 8 KB sized blocks with ext2.  It appears
> > that a stripe width of 16 KB through 128KB on the ia64 system gives the
> > best throughput for the DBT-2 workload on a volume that should be doing
> > mostly sequential writes.
> > 
> > I'm going to run through more tests varying the block size that
> > PostgreSQL uses, but I wanted to share what I had so far in case there
> > were other suggestions or recommendations.
> > 
> Here's one thought: look at the i/o rates on individual drives using 
> each stripe size. You *might* see that one size does far fewer seeks 
> than others, which is a secondary thing to optimize after throughput IMHO.
> 
> If you don't have a tool for this I can send you the latest diorate 
> which does stuff like this, io rate perdrive or per partition, something 
> I occasionally find revealing.

Yeah, please do send me a copy.  I'd be interested to see what that might 
turn up.  I've just been using iostat -x so far.

Thanks,
Mark

Re: [linux-lvm] Re: lvm2 performance data with linux-2.6

[Chris Croswhite]

Is this generally available to all?  Where can it be had?

TIA

On Thu, 2004-03-11 at 14:25, Mark Wong wrote:
> On Thu, Mar 11, 2004 at 05:12:35PM -0500, Bill Davidsen wrote:
> > markw@osdl.org wrote:
> > > I've started collecting various data (including oprofile) using our
> > > DBT-2 (OLTP) workload with lvm2 on linux 2.6.2 and 2.6.3 on ia32 and
> > > ia64 platforms:
> > > 	http://developer.osdl.org/markw/lvm2/
> > > 
> > > So far I've only varied the stripe width with lvm, from 8 KB to 512 KB,
> > > for PostgreSQL that is using 8 KB sized blocks with ext2.  It appears
> > > that a stripe width of 16 KB through 128KB on the ia64 system gives the
> > > best throughput for the DBT-2 workload on a volume that should be doing
> > > mostly sequential writes.
> > > 
> > > I'm going to run through more tests varying the block size that
> > > PostgreSQL uses, but I wanted to share what I had so far in case there
> > > were other suggestions or recommendations.
> > > 
> > Here's one thought: look at the i/o rates on individual drives using 
> > each stripe size. You *might* see that one size does far fewer seeks 
> > than others, which is a secondary thing to optimize after throughput IMHO.
> > 
> > If you don't have a tool for this I can send you the latest diorate 
> > which does stuff like this, io rate perdrive or per partition, something 
> > I occasionally find revealing.
> 
> Yeah, please do send me a copy.  I'd be interested to see what that might 
> turn up.  I've just been using iostat -x so far.
> 
> Thanks,
> Mark
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] Re: lvm2 performance data with linux-2.6

[Bill Davidsen]

Chris Croswhite wrote:
> Is this generally available to all?  Where can it be had?

I stuck it on the page I use to avoid beating my fractional T1 to death.
   http://pages.prodigy.net/davidsen/diorate.pl

Enjoy, any misfeatures please feed back to me.

-- 
bill davidsen <davidsen@tmr.com>
   CTO TMR Associates, Inc
   Doing interesting things with small computers since 1979

Re: lvm2 performance data with linux-2.6

[bill davidsen]

In article <20040311142515.A27177@osdlab.pdx.osdl.net>,
Mark Wong  <markw@osdl.org> wrote:
| On Thu, Mar 11, 2004 at 05:12:35PM -0500, Bill Davidsen wrote:

| > Here's one thought: look at the i/o rates on individual drives using 
| > each stripe size. You *might* see that one size does far fewer seeks 
| > than others, which is a secondary thing to optimize after throughput IMHO.
| > 
| > If you don't have a tool for this I can send you the latest diorate 
| > which does stuff like this, io rate perdrive or per partition, something 
| > I occasionally find revealing.
| 
| Yeah, please do send me a copy.  I'd be interested to see what that might 
| turn up.  I've just been using iostat -x so far.

Okay, I posted the pointer a few days ago to LKML, did you get a chance
to try it? And if so, did it tell you anything?
-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: lvm2 performance data with linux-2.6

[markw]

On 16 Mar, bill davidsen wrote:
> In article <20040311142515.A27177@osdlab.pdx.osdl.net>,
> Mark Wong  <markw@osdl.org> wrote:
> | On Thu, Mar 11, 2004 at 05:12:35PM -0500, Bill Davidsen wrote:
> 
> | > Here's one thought: look at the i/o rates on individual drives using 
> | > each stripe size. You *might* see that one size does far fewer seeks 
> | > than others, which is a secondary thing to optimize after throughput IMHO.
> | > 
> | > If you don't have a tool for this I can send you the latest diorate 
> | > which does stuff like this, io rate perdrive or per partition, something 
> | > I occasionally find revealing.
> | 
> | Yeah, please do send me a copy.  I'd be interested to see what that might 
> | turn up.  I've just been using iostat -x so far.
> 
> Okay, I posted the pointer a few days ago to LKML, did you get a chance
> to try it? And if so, did it tell you anything?

I've grabbed it but I haven't had a chance to try it yet.  Hopefully
later today or tomorrow.

Mark