[linux-lvm] Parallel IO on striped logic volume?

[linux-lvm] Parallel IO on striped logic volume?

[Xiaoxiang Liu]

I have a performance problem about parallel IO on striped logic volume.

I build a striped LV on 4 scsi disks. If I do sequential IO on one disk
with buffer size 4k, the bandwidth is 9MB/s. Then I do sequential IO on
LV with buffer size 16k. Theoretically I can get almost 4*9=36MB/s because
LVM stripe 4k IO on every disk but I only get 18MB/s. I don't know where
I lost so much performance or it is the overhead of LVM? I do believe LVM 
do parallel IO because if I increase buffer size to 1024k, I can get 
51MB/s which is close to 4*14 MB/s and 14MB/s is the maximum
bandwidth of a single disk.

My question is why I can't see nearly linear scaling of the bandwidth 
when the buffer size is small? Does striping LVM do real parallel IO
similar to software RAID0?
 

--xiaoxiang

Re: [linux-lvm] Parallel IO on striped logic volume?

[Eric M. Hopper]

[-- Attachment #1: Type: text/plain, Size: 1775 bytes --]

On Wed, Sep 05, 2001 at 12:29:50PM -0500, Xiaoxiang Liu wrote:
> I have a performance problem about parallel IO on striped logic volume.
> 
> I build a striped LV on 4 scsi disks. If I do sequential IO on one disk
> with buffer size 4k, the bandwidth is 9MB/s. Then I do sequential IO on
> LV with buffer size 16k. Theoretically I can get almost 4*9=36MB/s because
> LVM stripe 4k IO on every disk but I only get 18MB/s. I don't know where
> I lost so much performance or it is the overhead of LVM? I do believe LVM 
> do parallel IO because if I increase buffer size to 1024k, I can get 
> 51MB/s which is close to 4*14 MB/s and 14MB/s is the maximum
> bandwidth of a single disk.
> 
> My question is why I can't see nearly linear scaling of the bandwidth 
> when the buffer size is small? Does striping LVM do real parallel IO
> similar to software RAID0?

	Here's a random guess...

	You have to schedule the reads on each of the 4 disks, then wait
for the reads to complete on each of the 4 disks.  Due to details of
disk positioning, the latency to actually get to the data you're asking
for is different on each of the 4 disks.  Each time you do this read,
you end up with the maximum latency any of the disks will give you.

	The bigger your buffer size, the greater the ratio of data read
to latency.  That ratio will be magnified by the maximum latency
characteristics of reading from multiple devices simultaneously.

Just a guess,
-- 
"It does me no injury for my neighbor to say there are twenty gods or no God.
It neither picks my pocket nor breaks my leg."  --- Thomas Jefferson
"Go to Heaven for the climate, Hell for the company."  -- Mark Twain
-- Eric Hopper (hopper@omnifarious.org  http://www.omnifarious.org/~hopper) --

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]