From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark Kirkwood Subject: Re: fio rbd completions (Was: fio rbd hang for block sizes > 1M) Date: Fri, 31 Oct 2014 21:54:32 +1300 Message-ID: <54534E48.8080505@catalyst.net.nz> References: <5449BBB3.7090109@catalyst.net.nz> <544E6691.106@kernel.dk> <544E6A8D.1040608@kernel.dk> <544EC05D.7040807@catalyst.net.nz> <544EC7F1.6010900@kernel.dk> <544ED37D.6060800@catalyst.net.nz> <79432849-2F58-4738-82E1-E4D8AB10A6AE@kernel.dk> <544FB2F2.2060703@kernel.dk> <544FCDBD.7070106@kernel.dk> <5450FA47.2030203@kernel.dk> <5451EC63.9070600@catalyst.net.nz> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Return-path: Received: from bertrand.catalyst.net.nz ([202.78.240.40]:38997 "EHLO mail.catalyst.net.nz" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752825AbaJaIyi (ORCPT ); Fri, 31 Oct 2014 04:54:38 -0400 In-Reply-To: Sender: ceph-devel-owner@vger.kernel.org List-ID: To: Ketor D Cc: Jens Axboe , Mark Nelson , Mark Nelson , "fio@vger.kernel.org" , "xan.peng" , "ceph-devel@vger.kernel.org" On 30/10/14 21:04, Ketor D wrote: > Hi Mark, > Could you do a fio test in your env with the busy_poll=1 ? > I am very interested in the busy_poll result. Thanks! > Sure: blocksize k | head iops | head iops (busy_pool=1) ------------+---------------+-------------------------- 4 | 11114 | 12608 128 | 4551 | 6422 1024 | 1195 | 1175 4096 | 320 | 316 So looks like the busy_pool=1 improves performance for small and mid range blocksizes but is a little slower at the larger end. However there are a lot of variables here - I'm using iodepth=32 for instance, and altering that may change the pattern I'm seeing, also a system with more osd's may bring out different behaviours as it runs the fio client out of available cpu power in the smaller block sizes. Regards Mark From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Message-ID: <54534E48.8080505@catalyst.net.nz> Date: Fri, 31 Oct 2014 21:54:32 +1300 From: Mark Kirkwood MIME-Version: 1.0 Subject: Re: fio rbd completions (Was: fio rbd hang for block sizes > 1M) References: <5449BBB3.7090109@catalyst.net.nz> <544E6691.106@kernel.dk> <544E6A8D.1040608@kernel.dk> <544EC05D.7040807@catalyst.net.nz> <544EC7F1.6010900@kernel.dk> <544ED37D.6060800@catalyst.net.nz> <79432849-2F58-4738-82E1-E4D8AB10A6AE@kernel.dk> <544FB2F2.2060703@kernel.dk> <544FCDBD.7070106@kernel.dk> <5450FA47.2030203@kernel.dk> <5451EC63.9070600@catalyst.net.nz> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit To: Ketor D Cc: Jens Axboe , Mark Nelson , Mark Nelson , "fio@vger.kernel.org" , "xan.peng" , "ceph-devel@vger.kernel.org" List-ID: On 30/10/14 21:04, Ketor D wrote: > Hi Mark, > Could you do a fio test in your env with the busy_poll=1 ? > I am very interested in the busy_poll result. Thanks! > Sure: blocksize k | head iops | head iops (busy_pool=1) ------------+---------------+-------------------------- 4 | 11114 | 12608 128 | 4551 | 6422 1024 | 1195 | 1175 4096 | 320 | 316 So looks like the busy_pool=1 improves performance for small and mid range blocksizes but is a little slower at the larger end. However there are a lot of variables here - I'm using iodepth=32 for instance, and altering that may change the pattern I'm seeing, also a system with more osd's may bring out different behaviours as it runs the fio client out of available cpu power in the smaller block sizes. Regards Mark