From mboxrd@z Thu Jan 1 00:00:00 1970 From: Steven Haigh Subject: Re: IO speed limited by size of IO request (for RBD driver) Date: Tue, 23 Apr 2013 23:41:57 +1000 Message-ID: <51768FA5.6090609@crc.id.au> References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org On 23/04/2013 11:33 PM, Sylvain Munaut wrote: > Hi, > > > I was observing a pretty severe performance impact when using Xen VM > with RBD (Ceph) backed storage, especially when doing large sequential > access. > > And I think I finally found a major cause for it: even large user > space requests seem to be split into small requests of 11 * 4096 > bytes. ( 44k ) > > This is caused by : > > blk_queue_max_segments(rq, BLKIF_MAX_SEGMENTS_PER_REQUEST); > blk_queue_max_segment_size(rq, PAGE_SIZE); > > What's the impact of modifying those ? > I've seen some justification for the BLKIF_MAX_SEGMENTS_PER_REQUEST > limit, but why limit segment size to page_size ? I'm seeing the same as you - see the thread in the archives over the last few weeks - subject "RE: Xen disk write slowness in kernel 3.8.x" I get ~50MB/sec max write speeds due to probably the same problem. -- Steven Haigh Email: netwiz@crc.id.au Web: https://www.crc.id.au Phone: (03) 9001 6090 - 0412 935 897 Fax: (03) 8338 0299