From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754995AbbK0QOp (ORCPT ); Fri, 27 Nov 2015 11:14:45 -0500 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:55887 "EHLO mx0b-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754185AbbK0QOn (ORCPT ); Fri, 27 Nov 2015 11:14:43 -0500 Subject: Re: QUEUE_FLAG_NO_SG_MERGE and non-block-mq To: Hannes Reinecke , Ming Lei References: <5656BF25.3000407@suse.de> <565868E7.2010807@suse.de> CC: Christoph Hellwig , "Martin K. Petersen" , Johannes Thumshirn , Linux Kernel , , SCSI Mailing List From: Jens Axboe Message-ID: <56588150.1080900@fb.com> Date: Fri, 27 Nov 2015 09:14:08 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <565868E7.2010807@suse.de> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [192.168.54.13] X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2015-11-27_03:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 11/27/2015 07:29 AM, Hannes Reinecke wrote: > On 11/26/2015 10:21 AM, Ming Lei wrote: >> On Thu, Nov 26, 2015 at 4:13 PM, Hannes Reinecke wrote: >>> Hi all, >>> >>> while investigating the crash in scsi_lib.c I found a rather curious >>> behaviour for QUEUE_FLAG_NO_SG_MERGE. >>> >>> While the flag is evaluated in blk_recalc_rq_segments and >>> blk_recount_segments (resulting in nr_phys_segments being >>> computed based on that flag) it is completely ignored >>> during blk_rq_map_sg() or the actual merging itself. >> >> Yes, I guess Jens introduced the flag for decreasing CPU >> consumption when comuputing segments, but it is still >> ignored by blk_rq_map_sg(), but it may not be used >> by some drivers. >> >> After bio splitting is introduced, the flag is also ignored >> when computing segments. >> >>> >>> This typically shouldn't be an issue, seeing that with >>> QUEUE_FLAG_NO_SG_MERGE nr_phys_segments will always be >>> larger than the actual segment count. >>> >>> However, it still makes me wonder: >>> What is the point of having a QUEUE_FLAG_NO_SG_MERGE >>> which doesn't work as advertised? >>> Or, to be precise, which only works for blk-mq? >>> Should we make it work for non-block-mq, too? >> >> Thanks bio splitting, this flag has little effect on performance now, >> so I think it can be removed if Jens has no objection. >> > As per your suggestion we've made some performance measurements, > and 4k fio showed little if no impact: > > NO_SG_MERGE: > IOPS R/W: 148097.7+-125.7 / 148124.1+-123.1 > BW R/W: 592392.4+-502.7 / 592498.3+-492.3 > SG_MERGE: > IOPS R/W: 148054.4+-123.3 / 148082.6+-120.0 > BW R/W: 592219.2+-493.5 / 592332.3+-479.7 > > So the performance benefit lies squarely within the > error margin, making me wonder if it's worth bothering > with having the NO_SG_MERGE flag at all. > > Thanks to Johannes for doing the measurements :-) 150K iops is on the slow side, though. It's pointless to iterate the sg list if we don't have to. I can try and run a few tests next week. -- Jens Axboe