From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark Kirkwood Subject: Re: fio rbd completions (Was: fio rbd hang for block sizes > 1M) Date: Tue, 28 Oct 2014 12:21:33 +1300 Message-ID: <544ED37D.6060800@catalyst.net.nz> References: <5449BBB3.7090109@catalyst.net.nz> <5449E50E.7000808@kernel.dk> <5449EEF1.1060407@catalyst.net.nz> <544A51C7.40803@gmail.com> <544A5DA6.2010709@gmail.com> <544AD67D.4030603@catalyst.net.nz> <544AEAE7.6080603@redhat.com> <544AF0D2.1050405@catalyst.net.nz> <544B0C7F.4080109@catalyst.net.nz> <544B1D50.4010101@kernel.dk> <544B2C19.7070009@catalyst.net.nz> <544BF808.2090800@kernel.dk> <544C2371.1020403@catalyst.net.nz> <544E547C.30009@kernel.dk> <544E6330.1000202@kernel.dk> <544E63EA.1010204@kernel.dk> <544E6691.106@kernel.dk> <544E6A8D.1040608@kernel.dk> <544EC05D.7040807@catalyst.net.nz> <544EC7F1.6010900@kernel.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <544EC7F1.6010900-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org> Sender: fio-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org To: Jens Axboe , Ketor D Cc: Mark Nelson , Mark Nelson , fio-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, "xan.peng" , "ceph-devel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" List-Id: ceph-devel.vger.kernel.org On 28/10/14 11:32, Jens Axboe wrote: > On 10/27/2014 03:59 PM, Mark Kirkwood wrote: >> On 28/10/14 05:20, Ketor D wrote: >>> V8 patch runs good. >>> >>> The iops is 33032. If I just comment the usleep(100) in the master, I >>> can get iops 35245. >>> The CPU usage about the two test is same 120%. >>> So maybe this patch could be better! >>> >> >> Yeah, v8 is working for me. >> >> I'm seeing it a bit slower for some blocksizes, but faster (or perhaps >> about the same within repeat measurement error) for others: >> >> blocksize k | patched iops | orig iops >> ------------+---------------+----------- >> 4 | 12265 | 11930 >> 128 | 5800 | 7100 >> 1024 | 1193 | 1196 > > As for most things, the difference should be in IOPS, not bandwidth. So > I would assume that the above are within normal variance, since 4k > should show the biggest difference, then drop off after that and match > at 128/1024k. > Yeah, I suspect the 4K numbers are the same as we are bottlenecked by Ceph's small blocksize performance, not fio itself. If Ketor has a setup that can get higher IOPS @4K it would be interesting to see his numbers for patched vs orig! Cheers Mark From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Message-ID: <544ED37D.6060800@catalyst.net.nz> Date: Tue, 28 Oct 2014 12:21:33 +1300 From: Mark Kirkwood MIME-Version: 1.0 Subject: Re: fio rbd completions (Was: fio rbd hang for block sizes > 1M) References: <5449BBB3.7090109@catalyst.net.nz> <5449E50E.7000808@kernel.dk> <5449EEF1.1060407@catalyst.net.nz> <544A51C7.40803@gmail.com> <544A5DA6.2010709@gmail.com> <544AD67D.4030603@catalyst.net.nz> <544AEAE7.6080603@redhat.com> <544AF0D2.1050405@catalyst.net.nz> <544B0C7F.4080109@catalyst.net.nz> <544B1D50.4010101@kernel.dk> <544B2C19.7070009@catalyst.net.nz> <544BF808.2090800@kernel.dk> <544C2371.1020403@catalyst.net.nz> <544E547C.30009@kernel.dk> <544E6330.1000202@kernel.dk> <544E63EA.1010204@kernel.dk> <544E6691.106@kernel.dk> <544E6A8D.1040608@kernel.dk> <544EC05D.7040807@catalyst.net.nz> <544EC7F1.6010900@kernel.dk> In-Reply-To: <544EC7F1.6010900@kernel.dk> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit To: Jens Axboe , Ketor D Cc: Mark Nelson , Mark Nelson , fio@vger.kernel.org, "xan.peng" , "ceph-devel@vger.kernel.org" List-ID: On 28/10/14 11:32, Jens Axboe wrote: > On 10/27/2014 03:59 PM, Mark Kirkwood wrote: >> On 28/10/14 05:20, Ketor D wrote: >>> V8 patch runs good. >>> >>> The iops is 33032. If I just comment the usleep(100) in the master, I >>> can get iops 35245. >>> The CPU usage about the two test is same 120%. >>> So maybe this patch could be better! >>> >> >> Yeah, v8 is working for me. >> >> I'm seeing it a bit slower for some blocksizes, but faster (or perhaps >> about the same within repeat measurement error) for others: >> >> blocksize k | patched iops | orig iops >> ------------+---------------+----------- >> 4 | 12265 | 11930 >> 128 | 5800 | 7100 >> 1024 | 1193 | 1196 > > As for most things, the difference should be in IOPS, not bandwidth. So > I would assume that the above are within normal variance, since 4k > should show the biggest difference, then drop off after that and match > at 128/1024k. > Yeah, I suspect the 4K numbers are the same as we are bottlenecked by Ceph's small blocksize performance, not fio itself. If Ketor has a setup that can get higher IOPS @4K it would be interesting to see his numbers for patched vs orig! Cheers Mark