From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31D3AC5519F for ; Thu, 12 Nov 2020 16:17:58 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 742E5206D5 for ; Thu, 12 Nov 2020 16:17:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RFJYOhVX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 742E5206D5 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=tempfail smtp.mailfrom=dm-devel-bounces@redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1605197876; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=USmh7h5QhHJdtwDDLLOiAvMd4MbLQeLOn+VSUSs6IG0=; b=RFJYOhVXyAkboHrNOlrCMFG2GmiqwqcTtv8fCndsOtguiAz3aAswM9CznLHDtwfHU6sTxg wJSiZ+M364EB7xwCi8vf8pbTqeVJ4cg9Cv/q7ARKjj+4t66V+PzmmCs28xkyuY2J4uxoZH e7198gDD+Ej4e92w9j8UfPwiKp4QX3k= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-23-RScHAs-DPT-rHHrq7DmdbA-1; Thu, 12 Nov 2020 11:17:53 -0500 X-MC-Unique: RScHAs-DPT-rHHrq7DmdbA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 035931868405; Thu, 12 Nov 2020 16:17:38 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 25C9C6EF48; Thu, 12 Nov 2020 16:17:37 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 650FE58103; Thu, 12 Nov 2020 16:17:34 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 0ACGBUcI031117 for ; Thu, 12 Nov 2020 11:11:30 -0500 Received: by smtp.corp.redhat.com (Postfix) id 1F8A71001901; Thu, 12 Nov 2020 16:11:30 +0000 (UTC) Received: from localhost (unknown [10.18.25.174]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A5FF310013D0; Thu, 12 Nov 2020 16:11:26 +0000 (UTC) Date: Thu, 12 Nov 2020 11:11:26 -0500 From: Mike Snitzer To: JeffleXu Message-ID: <20201112161125.GA29249@redhat.com> References: <20201110065558.22694-1-jefflexu@linux.alibaba.com> <20201111153824.GB22834@redhat.com> <533a3b6b-146b-afe6-2e3e-d1bc2180a8c8@linux.alibaba.com> MIME-Version: 1.0 In-Reply-To: <533a3b6b-146b-afe6-2e3e-d1bc2180a8c8@linux.alibaba.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-loop: dm-devel@redhat.com Cc: axboe@kernel.dk, linux-block@vger.kernel.org, joseph.qi@linux.alibaba.com, dm-devel@redhat.com, io-uring@vger.kernel.org, koct9i@gmail.com Subject: Re: [dm-devel] dm: add support for DM_TARGET_NOWAIT for various targets X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=dm-devel-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 T24gVGh1LCBOb3YgMTIgMjAyMCBhdCAgMTowNWFtIC0wNTAwLApKZWZmbGVYdSA8amVmZmxleHVA bGludXguYWxpYmFiYS5jb20+IHdyb3RlOgoKPiBIaSBKZW5zIGFuZCBndXlzIGluIGJsb2NrL2lv X3VyaW5nIG1haWxpbmcgbGlzdCwgdGhpcyBtYWlsIGNvbnRhaW5zCj4gc29tZSBkaXNjdXNzaW9u IGFib3VudAo+IAo+IFJXRl9OT1dBSVQsIHBsZWFzZSBzZWUgdGhlIGZvbGxvd2luZyBjb250ZW50 cy4KPiAKPiAKPiAKPiBPbiAxMS8xMS8yMCAxMTozOCBQTSwgTWlrZSBTbml0emVyIHdyb3RlOgo+ ID5PbiBUdWUsIE5vdiAxMCAyMDIwIGF0ICAxOjU1YW0gLTA1MDAsCj4gPkplZmZsZSBYdSA8amVm ZmxleHVAbGludXguYWxpYmFiYS5jb20+IHdyb3RlOgo+ID4KPiA+PlRoaXMgaXMgb25lIHByZXAg cGF0Y2ggZm9yIHN1cHBvcnRpbmcgaW9wb2xsIGZvciBkbSBkZXZpY2UuCj4gPj4KPiA+PlRoZSBk aXJlY3QgSU8gcm91dGluZSB3aWxsIHNldCBSRVFfTk9XQUlUIGZsYWcgZm9yIFJFUV9ISVBSSSBJ TyAodGhhdAo+ID4+aXMsIElPIHdpbGwgZG8gaW9wb2xsKSBpbiBiaW9fc2V0X3BvbGxlZCgpLiBU aGVuIGluIHRoZSBJTyBzdWJtaXNzaW9uCj4gPj5yb3V0aW5lLCB0aGUgYWJpbGl0eSBvZiBoYW5k bGluZyBSRVFfTk9XQUlUIG9mIHRoZSBibG9jayBkZXZpY2Ugd2lsbAo+ID4+YmUgY2hlY2tlZCBm b3IgUkVRX0hJUFJJIElPIGluIHN1Ym1pdF9iaW9fY2hlY2tzKCkuIC1FT1BOT1RTVVBQIHdpbGwK PiA+PmJlIHJldHVybmVkIGlmIHRoZSBibG9jayBkZXZpY2UgZG9lc24ndCBzdXBwb3J0IFJFUV9O T1dBSVQuCj4gPnN1Ym1pdF9iaW9fY2hlY2tzKCkgdmVyaWZpZXMgdGhlIHJlcXVlc3RfcXVldWUg aGFzIFFVRVVFX0ZMQUdfTk9XQUlUIHNldAo+ID5pZiB0aGUgYmlvIGhhcyBSRVFfTk9XQUlULgo+ IFllcyB0aGF0J3MgdGhlIGNhc2UuCj4gPgo+ID4+RE0gbGFja3Mgc3VwcG9ydCBmb3IgUkVRX05P V0FJVCB1bnRpbCBjb21taXQgNmFiYzQ5NDY4ZWVhICgiZG06IGFkZAo+ID4+c3VwcG9ydCBmb3Ig UkVRX05PV0FJVCBhbmQgZW5hYmxlIGl0IGZvciBsaW5lYXIgdGFyZ2V0IikuIFNpbmNlIHRoZW4s Cj4gPj5kbSB0YXJnZXRzIHRoYXQgc3VwcG9ydCBSRVFfTk9XQUlUIHNob3VsZCBhZHZlcnRpc2Ug RE1fVEFSR0VUX05PV0FJVAo+ID4+ZmVhdHVyZS4KPiA+SSdtIG5vdCBzZWVpbmcgd2h5IERNX1RB UkdFVF9OT1dBSVQgaXMgbmVlZGVkIChzaW5jZSB5b3UgZGlkbid0IGFkZCBhbnkKPiA+Y29kZSB0 aGF0IGNvbnN1bWVzIHRoZSBmbGFnKS4KPiAKPiBBcyBJIHNhaWQsIGl0J3MgbmVlZGVkIGlmIHdl IHN1cHBvcnQgaW9wb2xsIGZvciBkbSBkZXZpY2UuwqAgT25seSBpZgo+IGEgYmxvY2sgZGV2aWNl IGlzIGNhcGFibGUgb2YKPiAKPiBoYW5kbGluZyBOT1dBSVQsIHRoZW4gaXQgY2FuIHN1cHBvcnQg aW9wb2xsLgo+IAo+IAo+IElPIHN1Ym1pdHRlZCBmb3IgaW9wb2xsIChtYXJrZWQgd2l0aCBJT0NC X0hJUFJJKSBpcyB1c3VhbGx5IGFsc28KPiBtYXJrZWQgd2l0aCBSRVFfTk9XQUlULgo+IAo+IFRo ZXJlIGFyZSB0d28gc2NlbmFyaW8gd2hlbiBpdCBjb3VsZCBoYXBwZW4uCj4gCj4gCj4gMS4gaW9f dXJpbmcgd2lsbCBzZXQgUkVRX05PV0FJVAo+IAo+IFRoZSBJTyBzdWJtaXNzaW9uIG9mIGlvX3Vy aW5nIGNhbiBiZSBkaXZpZGVkIGludG8gdHdvIHBoYXNlLiBGaXJzdCwKPiBJTyB3aWxsIGJlIHN1 Ym1pdHRlZAo+IAo+IHN5bmNocm9ub3VzbHkgaW4gdXNlciBwcm9jZXNzIGNvbnRleHQgKHdoZW4g c3F0aHJlYWQgZmVhdHVyZQo+IGRpc2FibGVkKSwgb3Igc3F0aHJlYWQKPiAKPiBjb250ZXh0ICh3 aGVuIHNxdGhyZWFkIGZlYXR1cmUgZW5hYmxlZCkuCj4gCj4gCj4gYGBgc2gKPiAtIGN1cnJlbnQg cHJvY2VzcyBjb250ZXh0IHdoZW4gc3F0aHJlYWQgZGlzYWJsZWQsIG9yIHNxdGhyZWFkIHdoZW4K PiBpdCdzIGVuYWJsZWQKPiDCoMKgwqAgaW9fdXJpbmdfZW50ZXIKPiDCoMKgwqDCoMKgwqDCoCBp b19zdWJtaXRfc3Flcwo+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgaW9fc3VibWl0X3NxZQo+IMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBpb19xdWV1ZV9zcWUKPiDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBfX2lvX3F1ZXVlX3NxZQo+IMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgaW9faXNzdWVfc3FlIC8vIHdpdGggQGZv cmNlX25vbmJsb2NrIGlzIHRydWUKPiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgaW9fcmVhZC9pb193cml0ZQo+IGBgYAo+IAo+IEluIHRoaXMg Y2FzZSwgSU8gc2hvdWxkIGJlIGhhbmRsZWQgaW4gYSBOT1dBSVQgd2F5LCBzaW5jZSB0aGUgdXNl cgo+IHByb2Nlc3Mgb3Igc3F0aHJlYWQKPiAKPiBjYW4gbm90IGJlIGJsb2NrZWQgZm9yIHBlcmZv cm1hbmNlLgo+IAo+IGBgYAo+IAo+IGlvX3JlYWQvaW9fd3JpdGUKPiAKPiDCoMKgwqAgLyogRW5z dXJlIHdlIGNsZWFyIHByZXZpb3VzbHkgc2V0IG5vbi1ibG9jayBmbGFnICovCj4gwqDCoMKgIGlm ICghZm9yY2Vfbm9uYmxvY2spCj4gwqDCoMKgIMKgwqDCoCBraW9jYi0+a2lfZmxhZ3MgJj0gfklP Q0JfTk9XQUlUOwo+IMKgwqDCoCBlbHNlCj4gwqDCoMKgIMKgwqDCoCBraW9jYi0+a2lfZmxhZ3Mg fD0gSU9DQl9OT1dBSVQ7Cj4gCj4gYGBgCj4gCj4gCj4gMi4gVGhlIGRpcmVjdCBJTyByb3V0aW5l IHdpbGwgc2V0IFJFUV9OT1dBSVQgZm9yIHBvbGxpbmcgSU8KPiAKPiBCb3RoIGZzL2Jsb2NrX2Rl di5jOiBfX2Jsa2Rldl9kaXJlY3RfSU8gYW5kIGZzL2lvbWFwL2RpcmVjdC1pby5jOgo+IGlvbWFw X2Rpb19zdWJtaXRfYmlvIHdpbGwKPiAKPiBjYWxsIGJpb19zZXRfcG9sbGVkKCksIGluIHdoaWNo IHdpbGwgc2V0IFJFUV9OT1dBSVQgZm9yIHBvbGxpbmcgSU8uCj4gCj4gCj4gYGBgc2gKPiBfX2Js a2Rldl9kaXJlY3RfSU8gLyBpb21hcF9kaW9fc3VibWl0X2JpbzoKPiDCoMKgwqAgaWYgKGRpby0+ aW9jYi0+a2lfZmxhZ3MgJiBJT0NCX0hJUFJJKQo+IMKgwqDCoCDCoMKgwqAgYmlvX3NldF9wb2xs ZWQKPiDCoMKgwqAgwqDCoMKgIMKgIGJpby0+Ymlfb3BmIHw9IFJFUV9OT1dBSVQKPiBgYGAKPiAK PiAKPiBUaHVzIHRvIHN1cHBvcnQgaW9wb2xsIGZvciBkbSBkZXZpY2UsIHRoZSBkbSB0YXJnZXQg c2hvdWxkIGJlCj4gY2FwYWJsZSBvZiBoYW5kbGluZyBOT1dBSVQsCj4gCj4gb3Igc3VibWl0X2Jp b19jaGVja3MoKSB3aWxsIGZhaWwgd2l0aCAtRU9QTk9UU1VQUCB3aGVuIHN1Ym1pdHRpbmcKPiBi aW8gdG8gZG0gZGV2aWNlLgo+IAo+IAo+ID4KPiA+ZG0tdGFibGUuYzpkbV90YWJsZV9zZXRfcmVz dHJpY3Rpb25zKCkgaGFzOgo+ID4KPiA+ICAgICAgICAgaWYgKGRtX3RhYmxlX3N1cHBvcnRzX25v d2FpdCh0KSkKPiA+ICAgICAgICAgICAgICAgICBibGtfcXVldWVfZmxhZ19zZXQoUVVFVUVfRkxB R19OT1dBSVQsIHEpOwo+ID4gICAgICAgICBlbHNlCj4gPiAgICAgICAgICAgICAgICAgYmxrX3F1 ZXVlX2ZsYWdfY2xlYXIoUVVFVUVfRkxBR19OT1dBSVQsIHEpOwo+ID4KPiA+PlRoaXMgcGF0Y2gg YWRkcyBzdXBwb3J0IGZvciBETV9UQVJHRVRfTk9XQUlUIGZvciB0aG9zZSBkbSB0YXJnZXRzLCB0 aGUKPiA+Pi5tYXAoKSBhbGdvcml0aG0gb2Ygd2hpY2gganVzdCBpbnZvbHZlcyBzZWN0b3IgcmVj YWxjdWxhdGlvbi4KPiA+U28geW91J3JlIGxvb2tpbmcgdG8gY29uc3RyYWluIHdoaWNoIHRhcmdl dHMgd2lsbCBwcm9wZXJseSBzdXBwb3J0Cj4gPlJFUV9OT1dBSVQsIGJhc2VkIG9uIHdoZXRoZXIg dGhleSBkbyBhIHNpbXBsZSByZW1hcHBpbmc/Cj4gCj4gVG8gYmUgaG9uZXN0LCBJJ20gYSBsaXR0 bGUgY29uZnVzZWQgYWJvdXQgdGhlIHNlbWFudGljcyBvZgo+IFJFUV9OT1dBSVQuIEplbnMgbWF5 IGhhZCBldmVyCj4gCj4gZXhwbGFpbmVkIGl0IGluIGJsb2NrIG9yIGlvX3VyaW5nIG1haWxpbmcg bGlzdCwgYnV0IEkgY2FuJ3QgZmluZCB0aGUKPiBzcGVjaWZpYyBtYWlsLgo+IAo+IAo+IFRoZSBt YW4gcGFnZSBleHBsYWlucyBGTU9ERV9OT1dBSVQgYXMgJ0ZpbGUgaXMgY2FwYWJsZSBvZiByZXR1 cm5pbmcKPiAtRUFHQUlOIGlmIEkvTyB3aWxsCj4gCj4gYmxvY2snLgo+IAo+IAo+IEFuZCBSV0Zf Tk9XQUlUIGFzCj4gCj4gYGBgCj4gCj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgUldGX05P V0FJVCAoc2luY2UgTGludXggNC4xNCkKPiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIERvbid0IHdhaXQgaWYgdGhlIEkvTyB3aWxsIGJsb2NrIGZvciBvcGVyYXRpb25z Cj4gc3VjaCBhcwo+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgZmls ZSBibG9jayBhbGxvY2F0aW9ucywgZGlydHkgcGFnZSBmbHVzaCwgbXV0ZXggbG9ja3MsCj4gwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBvciBhIGNvbmdlc3RlZCBibG9j ayBkZXZpY2UgaW5zaWRlIHRoZSBrZXJuZWwuwqAgSWYgYW55Cj4gwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBvZiB0aGVzZSBjb25kaXRpb25zIGFyZSBtZXQsIHRoZSBj b250cm9sIGJsb2NrIGlzIHJl4oCQCj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCB0dXJuZWQgaW1tZWRpYXRlbHkgd2l0aCBhIHJldHVybiB2YWx1ZSBvZiAtRUFHQUlO IGluCj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB0aGUgcmVzIGZp ZWxkIG9mIHRoZSBpb19ldmVudCBzdHJ1Y3R1cmUgKHNlZQo+IMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgaW9fZ2V0ZXZlbnRzKDIpKS4KPiAKPiBgYGAKPiAKPiAKPiBj b21taXQgNmFiYzQ5NDY4ZWVhICgiZG06IGFkZCBzdXBwb3J0IGZvciBSRVFfTk9XQUlUIGFuZCBl bmFibGUgaXQKPiBmb3IgbGluZWFyCj4gCj4gdGFyZ2V0IikgaGFuZGxlcyBOT1dBSVQgZm9yIERN IGNvcmUgYXMKPiAKPiAKPiBgYGAKPiAKPiBAQCAtMTgwMiw3ICsxODAyLDkgQEAgc3RhdGljIGJs a19xY190IGRtX3N1Ym1pdF9iaW8oc3RydWN0IGJpbyAqYmlvKQo+IMKgwqDCoMKgwqDCoMKgIGlm ICh1bmxpa2VseSh0ZXN0X2JpdChETUZfQkxPQ0tfSU9fRk9SX1NVU1BFTkQsICZtZC0+ZmxhZ3Mp KSkgewo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGlmIChiaW8tPmJpX29wZiAmIFJF UV9OT1dBSVQpCj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg IGJpb193b3VsZGJsb2NrX2Vycm9yKGJpbyk7Cj4gCj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqAgZWxzZSBpZiAoIShiaW8tPmJpX29wZiAmIFJFUV9SQUhFQUQpKQo+IMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcXVldWVfaW8obWQsIGJpbyk7Cj4g Cj4gYGBgCj4gCj4gCj4gVGhlb3JldGljYWxseSB0aGUgYmxvY2sgZGV2aWNlIGNvdWxkIGFkdmVy dGlzZSBRVUVVRV9GTEFHX05PV0FJVCBhcwo+IGxvbmcgYXMgaXQgY291bGQKPiAKPiAncmV0dXJu IC1FQUdBSU4gaWYgSS9PIHdpbGwgYmxvY2snIGFzIHRoZSBtYW4gcGFnZSBzYWlkLiBIb3dldmVy LAo+IGNvbnNpZGVyaW5nIHdoZW4gdGhlCj4gCj4gZG0gZGV2aWNlIGRldGVjdGVkIGFzIHN1c3Bl bmRpbmcsIHRoZSBzdWJtaXR0ZWQgYmlvcyBhcmUgZGVmZXJyZWQgdG8KPiB3b3JrcXVldWUgaW4K PiAKPiBkcml2ZXJzL2RtL2RtLmM6IGRtX3N1Ym1pdF9iaW8uIEluIHRoaXMgY2FzZSwgSU8gZ2V0 cyAqKmRlZmVycmVkKioKPiB3aGlsZSB0aGUgdXNlciBwcm9jZXNzCj4gCj4gd2lsbCBub3QgYmUg KipibG9ja2VkKiouIENhbiB3ZSBzYXkgSU8gZ2V0cyAqKmJsb2NrZWQqKiBpbiB0aGlzIGNhc2U/ Cj4gCj4gCj4gQWN0dWFsbHkgc2V2ZXJhbCBkbSB0YXJnZXRzIGhhbmRsZSBzdWJtaXR0ZWQgYmlv IGluIHRoaXMgZGVmZXJyZWQKPiB3YXksIHN1Y2ggYXMgZG0tY3J5cHQvCj4gCj4gZG0tZGVsYXkv ZG0tZXJhL2RtLWVicy4gQ2FuIHdlIHNheSB0aGVzZSB0YXJnZXRzIGFyZSBub3QgY2FwYWJsZSBv Zgo+IGhhbmRsaW5nIE5PV0FJVD8KPiAKPiAKPiBBbHNvIHdoZW4gc3lzdGVtIGlzIHNob3J0IG9m IG1lbW9yeSwgYmlvIGFsbG9jYXRpb24gaW4KPiBiaW9fYWxsb2NfYmlvc2V0KCkgbWF5IHRyaWdn ZXIgbWVtb3J5Cj4gCj4gZGlyZWN0IHJlY2xhaW0sIGFzIHRoZSBnZnBfbWFzayBpcyB1c3VhbGx5 IEdGUF9OT0lPLiBXaGlsZSBpbiBtZW1vcnkKPiBkaXJlY3QgcmVjbGFpbSwgdGhlCj4gCj4gcHJv Y2VzcyBtYXkgYmUgc2NoZWR1bGVkIG91dCwgYnV0IEkgaGF2ZSBuZXZlciBzZWVuIHRoZSBwcm9w ZXIKPiBoYW5kbGluZyBmb3IgTk9XQUlUIGluIHRoaXMKPiAKPiBzaXR1YXRpb24uIE1heWJlIHRo ZSBibG9jayBvciBpb191cmluZyBndXlzIGhhdmUgbW9yZSBpbnNpZ2h0cz8KPiAKPiAKPiBTbyB0 aGVyZSdzIGp1c3QgdG9vIG1hbnkgcG9zc2liaWxpdGllcyB0aGF0IG1heSBnZXQgYmxvY2tlZCwg bm90IHRvCj4gc2F5IG11dGV4IGxvY2tzLgo+IAo+IAo+ID4KPiA+Cj4gPj5TaWduZWQtb2ZmLWJ5 OiBKZWZmbGUgWHUgPGplZmZsZXh1QGxpbnV4LmFsaWJhYmEuY29tPgo+ID4+LS0tCj4gPj5IaSBN aWtlLAo+ID4+Cj4gPj5JIGNvdWxkIHNwbGl0IHRoZXNlIGJvaWxlcnBsYXRlIGNvZGUgdGhhdCBl YWNoIGRtIHRhcmdldCBoYXZlIG9uZQo+ID4+c2VwZXJhdGUgcGF0Y2ggaWYgeW91IHRoaW5rIHRo YXQgd291bGQgYmUgYmV0dGVyLgo+ID5PbmUgcGF0Y2ggZm9yIGFsbCB0aGVzZSBpcyBmaW5lLiAg QnV0IGl0IHNob3VsZCBpbmNsdWRlIHRoZSBjb2RlIHRoYXQgSQo+ID5hc3N1bWUgeW91J2xsIGJl IGFkZGluZyB0byBkbV90YWJsZV9zdXBwb3J0c19ub3dhaXQoKSB0byBmdXJ0aGVyIHZlcmlmeQo+ ID50aGF0IHRoZSB0YXJnZXRzIGluIHRoZSB0YWJsZSBhcmUgYWxsIERNX1RBUkdFVF9OT1dBSVQu Cj4gPgo+ID5BbmQgd2h5IGlzbid0IGRtLWxpbmVhciBzZXR0aW5nIERNX1RBUkdFVF9OT1dBSVQ/ Cj4gVGhlc2UgYXJlIGFsbCBkb25lIGluIGNvbW1pdCA2YWJjNDk0NjhlZWEgKCJkbTogYWRkIHN1 cHBvcnQgZm9yCj4gUkVRX05PV0FJVCBhbmQgZW5hYmxlIGl0IGZvcgo+IGxpbmVhciB0YXJnZXQi KS4KCkhhLCBvb3BzLiAgWW91J2QgdGhpbmsgSSdkIGhhdmUgcmVtZW1iZXJlZCBhZGRpbmcgRE1f VEFSR0VUX05PV0FJVCB0bwpkbS1saW5lYXIgYW5kIGRtX3RhcmdldF9zdXBwb3J0c19ub3dhaXQs IGV0Yy4KClRoYW5rcyBmb3IgY2xhcmlmeWluZy4gIFNvIHlvdXIgcGF0Y2ggd2FzIGp1c3QgYWJv dXQgZXh0ZW5kaW5nCnRoZSBjYXBhYmlsaXR5IHRvIG90aGVyIHRhcmdldHMgd2hlcmUgeW91IHRo aW5rIGl0IGFwcGxpY2FibGUuCgpXZSdsbCBiZSBhYmxlIHRvIG1ha2UgdGhhdCBjbGVhcmVyIGJ5 IHR3ZWFraW5nIHRoZSBoZWFkZXIgYSBiaXQuCgpNaWtlCgotLQpkbS1kZXZlbCBtYWlsaW5nIGxp c3QKZG0tZGV2ZWxAcmVkaGF0LmNvbQpodHRwczovL3d3dy5yZWRoYXQuY29tL21haWxtYW4vbGlz dGluZm8vZG0tZGV2ZWw= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C7A4C61DD8 for ; Thu, 12 Nov 2020 16:11:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B1E8322256 for ; Thu, 12 Nov 2020 16:11:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bGDjHQWN" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728821AbgKLQLg (ORCPT ); Thu, 12 Nov 2020 11:11:36 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:47737 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728414AbgKLQLg (ORCPT ); Thu, 12 Nov 2020 11:11:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1605197493; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h7UdIT8ZId8iYTHJvW2BvD8NkTKm/RjlnhZH0DdqO6k=; b=bGDjHQWNUDq2kJySjlgSPLzzeWAiS6Dvv4x2aXMDse3H4TSF/rdVaqmQYo0rB6B7eqM7cj 6Rbntr2CsYGbVFv0iNFHvY3zGeg+2YHo19P5bLLGHDgQG37jAXrMQX0d/BIO9fXBL/fzTc HUeJh8I0AM9UjUtNLbOK6UhsGcbjB6Y= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-351-xGN0Qhq8PFm1LictJq_YPQ-1; Thu, 12 Nov 2020 11:11:31 -0500 X-MC-Unique: xGN0Qhq8PFm1LictJq_YPQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2124D1006CA7; Thu, 12 Nov 2020 16:11:30 +0000 (UTC) Received: from localhost (unknown [10.18.25.174]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A5FF310013D0; Thu, 12 Nov 2020 16:11:26 +0000 (UTC) Date: Thu, 12 Nov 2020 11:11:26 -0500 From: Mike Snitzer To: JeffleXu Cc: joseph.qi@linux.alibaba.com, dm-devel@redhat.com, koct9i@gmail.com, axboe@kernel.dk, io-uring@vger.kernel.org, linux-block@vger.kernel.org Subject: Re: dm: add support for DM_TARGET_NOWAIT for various targets Message-ID: <20201112161125.GA29249@redhat.com> References: <20201110065558.22694-1-jefflexu@linux.alibaba.com> <20201111153824.GB22834@redhat.com> <533a3b6b-146b-afe6-2e3e-d1bc2180a8c8@linux.alibaba.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <533a3b6b-146b-afe6-2e3e-d1bc2180a8c8@linux.alibaba.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org On Thu, Nov 12 2020 at 1:05am -0500, JeffleXu wrote: > Hi Jens and guys in block/io_uring mailing list, this mail contains > some discussion abount > > RWF_NOWAIT, please see the following contents. > > > > On 11/11/20 11:38 PM, Mike Snitzer wrote: > >On Tue, Nov 10 2020 at 1:55am -0500, > >Jeffle Xu wrote: > > > >>This is one prep patch for supporting iopoll for dm device. > >> > >>The direct IO routine will set REQ_NOWAIT flag for REQ_HIPRI IO (that > >>is, IO will do iopoll) in bio_set_polled(). Then in the IO submission > >>routine, the ability of handling REQ_NOWAIT of the block device will > >>be checked for REQ_HIPRI IO in submit_bio_checks(). -EOPNOTSUPP will > >>be returned if the block device doesn't support REQ_NOWAIT. > >submit_bio_checks() verifies the request_queue has QUEUE_FLAG_NOWAIT set > >if the bio has REQ_NOWAIT. > Yes that's the case. > > > >>DM lacks support for REQ_NOWAIT until commit 6abc49468eea ("dm: add > >>support for REQ_NOWAIT and enable it for linear target"). Since then, > >>dm targets that support REQ_NOWAIT should advertise DM_TARGET_NOWAIT > >>feature. > >I'm not seeing why DM_TARGET_NOWAIT is needed (since you didn't add any > >code that consumes the flag). > > As I said, it's needed if we support iopoll for dm device.  Only if > a block device is capable of > > handling NOWAIT, then it can support iopoll. > > > IO submitted for iopoll (marked with IOCB_HIPRI) is usually also > marked with REQ_NOWAIT. > > There are two scenario when it could happen. > > > 1. io_uring will set REQ_NOWAIT > > The IO submission of io_uring can be divided into two phase. First, > IO will be submitted > > synchronously in user process context (when sqthread feature > disabled), or sqthread > > context (when sqthread feature enabled). > > > ```sh > - current process context when sqthread disabled, or sqthread when > it's enabled >     io_uring_enter >         io_submit_sqes >             io_submit_sqe >                 io_queue_sqe >                     __io_queue_sqe >                         io_issue_sqe // with @force_nonblock is true >                             io_read/io_write > ``` > > In this case, IO should be handled in a NOWAIT way, since the user > process or sqthread > > can not be blocked for performance. > > ``` > > io_read/io_write > >     /* Ensure we clear previously set non-block flag */ >     if (!force_nonblock) >         kiocb->ki_flags &= ~IOCB_NOWAIT; >     else >         kiocb->ki_flags |= IOCB_NOWAIT; > > ``` > > > 2. The direct IO routine will set REQ_NOWAIT for polling IO > > Both fs/block_dev.c: __blkdev_direct_IO and fs/iomap/direct-io.c: > iomap_dio_submit_bio will > > call bio_set_polled(), in which will set REQ_NOWAIT for polling IO. > > > ```sh > __blkdev_direct_IO / iomap_dio_submit_bio: >     if (dio->iocb->ki_flags & IOCB_HIPRI) >         bio_set_polled >           bio->bi_opf |= REQ_NOWAIT > ``` > > > Thus to support iopoll for dm device, the dm target should be > capable of handling NOWAIT, > > or submit_bio_checks() will fail with -EOPNOTSUPP when submitting > bio to dm device. > > > > > >dm-table.c:dm_table_set_restrictions() has: > > > > if (dm_table_supports_nowait(t)) > > blk_queue_flag_set(QUEUE_FLAG_NOWAIT, q); > > else > > blk_queue_flag_clear(QUEUE_FLAG_NOWAIT, q); > > > >>This patch adds support for DM_TARGET_NOWAIT for those dm targets, the > >>.map() algorithm of which just involves sector recalculation. > >So you're looking to constrain which targets will properly support > >REQ_NOWAIT, based on whether they do a simple remapping? > > To be honest, I'm a little confused about the semantics of > REQ_NOWAIT. Jens may had ever > > explained it in block or io_uring mailing list, but I can't find the > specific mail. > > > The man page explains FMODE_NOWAIT as 'File is capable of returning > -EAGAIN if I/O will > > block'. > > > And RWF_NOWAIT as > > ``` > >               RWF_NOWAIT (since Linux 4.14) >                      Don't wait if the I/O will block for operations > such as >                      file block allocations, dirty page flush, mutex locks, >                      or a congested block device inside the kernel.  If any >                      of these conditions are met, the control block is re‐ >                      turned immediately with a return value of -EAGAIN in >                      the res field of the io_event structure (see >                      io_getevents(2)). > > ``` > > > commit 6abc49468eea ("dm: add support for REQ_NOWAIT and enable it > for linear > > target") handles NOWAIT for DM core as > > > ``` > > @@ -1802,7 +1802,9 @@ static blk_qc_t dm_submit_bio(struct bio *bio) >         if (unlikely(test_bit(DMF_BLOCK_IO_FOR_SUSPEND, &md->flags))) { > +               if (bio->bi_opf & REQ_NOWAIT) > +                       bio_wouldblock_error(bio); > > +               else if (!(bio->bi_opf & REQ_RAHEAD)) >                         queue_io(md, bio); > > ``` > > > Theoretically the block device could advertise QUEUE_FLAG_NOWAIT as > long as it could > > 'return -EAGAIN if I/O will block' as the man page said. However, > considering when the > > dm device detected as suspending, the submitted bios are deferred to > workqueue in > > drivers/dm/dm.c: dm_submit_bio. In this case, IO gets **deferred** > while the user process > > will not be **blocked**. Can we say IO gets **blocked** in this case? > > > Actually several dm targets handle submitted bio in this deferred > way, such as dm-crypt/ > > dm-delay/dm-era/dm-ebs. Can we say these targets are not capable of > handling NOWAIT? > > > Also when system is short of memory, bio allocation in > bio_alloc_bioset() may trigger memory > > direct reclaim, as the gfp_mask is usually GFP_NOIO. While in memory > direct reclaim, the > > process may be scheduled out, but I have never seen the proper > handling for NOWAIT in this > > situation. Maybe the block or io_uring guys have more insights? > > > So there's just too many possibilities that may get blocked, not to > say mutex locks. > > > > > > > >>Signed-off-by: Jeffle Xu > >>--- > >>Hi Mike, > >> > >>I could split these boilerplate code that each dm target have one > >>seperate patch if you think that would be better. > >One patch for all these is fine. But it should include the code that I > >assume you'll be adding to dm_table_supports_nowait() to further verify > >that the targets in the table are all DM_TARGET_NOWAIT. > > > >And why isn't dm-linear setting DM_TARGET_NOWAIT? > These are all done in commit 6abc49468eea ("dm: add support for > REQ_NOWAIT and enable it for > linear target"). Ha, oops. You'd think I'd have remembered adding DM_TARGET_NOWAIT to dm-linear and dm_target_supports_nowait, etc. Thanks for clarifying. So your patch was just about extending the capability to other targets where you think it applicable. We'll be able to make that clearer by tweaking the header a bit. Mike