[linux-lvm] LVM performance

[linux-lvm] LVM performance

[Bradley M Alexander]

Hi all,

All right. I've got LVM installed and running on 2 30GB IDE drives, a VG on
each, and the partitions divided just about evenly between them. This is on
a 1GHz Athlon with 640MB of RAM. I am running Debian/sid with lvm 1.0.1rc4
and kernel 2.4.13.

A few things I have noticed in the week or so that I have been using LVM.
First, the drive activity light stays lit all the time. I'm not terribly
worried about it, since things seem to be behaving normally, but I figured
I would ask.

The second, and far more disturbing thing is that any heavy accesses to the
filesystem (copying large files, scp-ing files mastering CDs, anything with
a lot of IO brings my machine to its knees. Case in point. When I started
this message, I was mastering a CD using xcdroast, which had called
mkisofs. Not only did the mastering take (I'm guessing here, since I didn't
note the time) about 10 minutes, the load average jumped, at one point, to
8.23, and generally hovered around the 6-to-7 range. Copying large files
from filesystem to filesystem on the machine yields similar results.
Originally, I had the following configuration:

hda 30GB HD
hdb 30GB HD
hdc DVD-ROM
hdd CD-RW

A friend suggested moving the HD on hdb to the other channel, so now the
drives are on hda and hdc, the dvd is on hdb and the cdrw is on hdd.

Can anyone give me any ideas as to why the machine gets beaten about so
much during IO operations and more importantly how can I minimize the
impact. 

Regards,
-- 
--Brad
============================================================================
Bradley M. Alexander, CISSP              |   Co-Chairman,
Beowulf System Admin/Security Specialist |    NoVALUG/DCLUG Security SIG
Debian/GNU Linux Developer		 |   storm@debian.org
                                         |   storm@tux.org
============================================================================
Never sign a contract which contains the phrases "sort of," "kind of," or
"and stuff."

RE: [linux-lvm] LVM performance

[Steve Wray]

[big snip]
> Originally, I had the following configuration:
>
> hda 30GB HD
> hdb 30GB HD
> hdc DVD-ROM
> hdd CD-RW
>
> A friend suggested moving the HD on hdb to the other channel, so now the
> drives are on hda and hdc, the dvd is on hdb and the cdrw is on hdd.

thats dead right; ide can't simultaneously write to the master and slave
on the same controller. Also, the pair of drives uses the controller
circuitry
of the master, so it always pays to make the master the most modern one.


> Can anyone give me any ideas as to why the machine gets beaten about so
> much during IO operations and more importantly how can I minimize the
> impact.

Dunno, if it was because the fs is striped across those drives then
splitting them across controllers would have made it go away.

I've seen no performance problems at all and really thrashed an LVM-root
machine for test purposes while working on a movie. It took it well,
performance-wise. (reliability is another issue; never go LVM-root...
but thats just my 2 cents, YMMV).

Re: [linux-lvm] LVM performance

[Bradley M Alexander]

On Fri, Nov 23, 2001 at 05:32:07PM +1300, Steve Wray wrote:
> 
> thats dead right; ide can't simultaneously write to the master and slave
> on the same controller. Also, the pair of drives uses the controller
> circuitry
> of the master, so it always pays to make the master the most modern one.

Since both controllers have a drive master and a cd slave, this shouldn't
matter, per se.

> 
> > Can anyone give me any ideas as to why the machine gets beaten about so
> > much during IO operations and more importantly how can I minimize the
> > impact.
> 
> Dunno, if it was because the fs is striped across those drives then
> splitting them across controllers would have made it go away.

Nothing should be striped across the drives. I have two PVs and made two
VGs, one on each drive. The way it works out, the data I am mastering is on
vg01 (hdc), and the ISO is being created there as well. the CD write actually
happens from hdc -> hdd.

> I've seen no performance problems at all and really thrashed an LVM-root
> machine for test purposes while working on a movie. It took it well,
> performance-wise. (reliability is another issue; never go LVM-root...
> but thats just my 2 cents, YMMV).

Yeah, I wasn't brave enough to go all out...And with the problems I was
having, I'm glad I made that decision. :)

> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

-- 
--Brad
============================================================================
Bradley M. Alexander, CISSP              |   Co-Chairman,
Beowulf System Admin/Security Specialist |    NoVALUG/DCLUG Security SIG
Debian/GNU Linux Developer		 |   storm@debian.org
                                         |   storm@tux.org
============================================================================
You start with a bag full of luck and an empty bag of
experience. The trick is to fill the bag of experience before you
empty the bag of luck.
					--Rules of the Air, #16

RE: [linux-lvm] LVM performance

[Steve Wray]

> From: linux-lvm-admin@sistina.com [mailto:linux-lvm-admin@sistina.com]On
> Behalf Of Bradley M Alexander
> On Fri, Nov 23, 2001 at 05:32:07PM +1300, Steve Wray wrote:
> >
> > thats dead right; ide can't simultaneously write to the master and slave
> > on the same controller. Also, the pair of drives uses the controller
> > circuitry
> > of the master, so it always pays to make the master the most modern one.
>
> Since both controllers have a drive master and a cd slave, this shouldn't
> matter, per se.

yeah but I thought I'd mention it, for completeness
:)


> > > Can anyone give me any ideas as to why the machine gets
> beaten about so
> > > much during IO operations and more importantly how can I minimize the
> > > impact.
> >
> > Dunno, if it was because the fs is striped across those drives then
> > splitting them across controllers would have made it go away.
>
> Nothing should be striped across the drives. I have two PVs and made two
> VGs, one on each drive. The way it works out, the data I am
> mastering is on

yup you have to tell it to stripe, as I recall. Never used that feature
tho.

> > I've seen no performance problems at all and really thrashed an LVM-root
> > machine for test purposes while working on a movie. It took it well,
> > performance-wise. (reliability is another issue; never go LVM-root...
> > but thats just my 2 cents, YMMV).
>
> Yeah, I wasn't brave enough to go all out...And with the problems I was
> having, I'm glad I made that decision. :)

I think the worst part is the lack of backward compatibility
at some stages of LVM. With root on LVM it can be a bit dangerous
to upgrade! Or at least it was...

Re: [linux-lvm] LVM performance

[Benjamin Scott]

On Fri, 23 Nov 2001, Bradley M Alexander wrote:
> the CD write actually happens from hdc -> hdd.

  This might be *a* problem.  As others have said, IDE handles bus
contention either very poorly or not at all.  Since you have all your I/O on
one bus, you may be seeing a problem there.

  For IDE, I generally favor putting HDs on one bus, and CDs/tapes on the
other.  Actually, I favor putting a single device on each bus.  Actually, I
favor SCSI.  :-)

-- 
Ben Scott <bscott@ntisys.com>
| The opinions expressed in this message are those of the author and do not |
| necessarily represent the views or policy of any other person, entity or  |
| organization.  All information is provided without warranty of any kind.  |

Re: [linux-lvm] LVM performance

[Benjamin Scott]

On Fri, 23 Nov 2001, Volker Jahns wrote:
> Also, why don't you use a "cheap" raid controller which will do the
> homework for you in hardware _not_ software?

  Most "cheap" IDE "RAID controllers" implement RAID in the device driver.
The card itself is just a regular IDE controller with a special BIOS.  The
CPU still does all the work, and you still have to deal with IDE bus
contention problems.

  For IDE hardware RAID, we have had good luck with 3Ware's Escalade series.
We have never tried anything beyond RAID-1 mirroring, though.

-- 
Ben Scott <bscott@ntisys.com>
| The opinions expressed in this message are those of the author and do not |
| necessarily represent the views or policy of any other person, entity or  |
| organization.  All information is provided without warranty of any kind.  |

RE: [linux-lvm] LVM performance

[Steven Lembark]

-- Steve Wray <steve.wray@the.net.nz>

>> From: linux-lvm-admin@sistina.com [mailto:linux-lvm-admin@sistina.com]On
>> Behalf Of Bradley M Alexander
>> On Fri, Nov 23, 2001 at 05:32:07PM +1300, Steve Wray wrote:
>> >
>> > thats dead right; ide can't simultaneously write to the master and
>> > slave on the same controller. Also, the pair of drives uses the
>> > controller circuitry
>> > of the master, so it always pays to make the master the most modern
>> > one.
>>
>> Since both controllers have a drive master and a cd slave, this shouldn't
>> matter, per se.
>
> yeah but I thought I'd mention it, for completeness
> :)

Dependong on the bios, for example, there still could be
contention on the SB chipset for access to the controllers.
Point is that there can be multiple bottlenecks.

>> > > Can anyone give me any ideas as to why the machine gets
>> beaten about so
>> > > much during IO operations and more importantly how can I minimize the
>> > > impact.
>> >
>> > Dunno, if it was because the fs is striped across those drives then
>> > splitting them across controllers would have made it go away.
>>
>> Nothing should be striped across the drives. I have two PVs and made two
>> VGs, one on each drive. The way it works out, the data I am
>> mastering is on
>
> yup you have to tell it to stripe, as I recall. Never used that feature
> tho.

I'm using LVM with several striped volumes. Hardware is Tekram
160MB scsi + pair of 80MB drives. The striping works rather
nicely: sustained I/O is high for large copies and latency is
low for reads. The file system uses a 4KB page w/ the inodes
reduced to a reasonable value (e.g., mkfs.ext2 -b4096 -i10240).

If you are using the default of 1block / page then it can
hammer a system due to 2ndary access and higher write rate.

>> > I've seen no performance problems at all and really thrashed an
>> > LVM-root machine for test purposes while working on a movie. It took
>> > it well, performance-wise. (reliability is another issue; never go
>> > LVM-root... but thats just my 2 cents, YMMV).
>>
>> Yeah, I wasn't brave enough to go all out...And with the problems I was
>> having, I'm glad I made that decision. :)
>
> I think the worst part is the lack of backward compatibility
> at some stages of LVM. With root on LVM it can be a bit dangerous
> to upgrade! Or at least it was...

<broken-record>
This can also be a real pain to fix if something goes wrong. If
the lvm is --prefix-ed to, say, /lvm and the root volume is
on a paritition then you can fix most things w/o having to
play with rescue file systems.
</broken-record>

--
Steven Lembark                               2930 W. Palmer
Workhorse Computing                       Chicago, IL 60647
                                            +1 800 762 1582

Re: [linux-lvm] LVM performance

[Steven Lembark]

-- Benjamin Scott <bscott@ntisys.com>

>  Actually,
> I favor SCSI.  :-)

Me too, but more scsi-ish behavior can be arrived at by
tinkering with hdparm. Mainly the delayed write, buffer
size can help enormously.

--
Steven Lembark                               2930 W. Palmer
Workhorse Computing                       Chicago, IL 60647
                                            +1 800 762 1582

Re: [linux-lvm] LVM performance

[Duncan Young]

I hope this isn't stating the obvious, but I asume you have used hdparm to 
adjust the default setting on your drives appropriatly (i.e. dma).  I have 4 
disks striped and get average sequential throuput of at least 40mb/s and 
35mb/s (dd to /dev/null) on only 2 disks.

cpu is approx 60% system (666mhz system).

If I have the disks untuned I get %100 cpu, machine "stuttering" and throuput 
10 times less.

Duncan

On Friday 23 Nov 2001 4:17 am, you wrote:
> Hi all,
>
> All right. I've got LVM installed and running on 2 30GB IDE drives, a VG on
> each, and the partitions divided just about evenly between them. This is on
> a 1GHz Athlon with 640MB of RAM. I am running Debian/sid with lvm 1.0.1rc4
> and kernel 2.4.13.
>
> A few things I have noticed in the week or so that I have been using LVM.
> First, the drive activity light stays lit all the time. I'm not terribly
> worried about it, since things seem to be behaving normally, but I figured
> I would ask.
>
> The second, and far more disturbing thing is that any heavy accesses to the
> filesystem (copying large files, scp-ing files mastering CDs, anything with
> a lot of IO brings my machine to its knees. Case in point. When I started
> this message, I was mastering a CD using xcdroast, which had called
> mkisofs. Not only did the mastering take (I'm guessing here, since I didn't
> note the time) about 10 minutes, the load average jumped, at one point, to
> 8.23, and generally hovered around the 6-to-7 range. Copying large files
> from filesystem to filesystem on the machine yields similar results.
> Originally, I had the following configuration:
>
> hda 30GB HD
> hdb 30GB HD
> hdc DVD-ROM
> hdd CD-RW
>
> A friend suggested moving the HD on hdb to the other channel, so now the
> drives are on hda and hdc, the dvd is on hdb and the cdrw is on hdd.
>
> Can anyone give me any ideas as to why the machine gets beaten about so
> much during IO operations and more importantly how can I minimize the
> impact.
>
> Regards,

[linux-lvm] LVM performance

[Alan Willis]

  Is there any performance data comparing an md software raid 0 to a basic
lvm
volume?

  I'd like to use lvm instead of md, but I'd have to justify it with
something solid.  We don't need alot of the extra lvm features.  But I'd
like to be able to upgrade this to use device-mapper eventually, and if
new installations are made md, that probably won't happen.

-alan

Re: [linux-lvm] LVM performance

[Joe Thornber]

On Fri, Nov 08, 2002 at 06:28:35PM -0500, Alan Willis wrote:
> 
>   Is there any performance data comparing an md software raid 0 to a basic
> lvm
> volume?
> 
>   I'd like to use lvm instead of md, but I'd have to justify it with
> something solid.  We don't need alot of the extra lvm features.  But I'd
> like to be able to upgrade this to use device-mapper eventually, and if
> new installations are made md, that probably won't happen.

Performance is similar as far as I know.

- Joe