From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D1B43C636CC for ; Tue, 31 Jan 2023 20:47:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1675198054; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=1YaRmla/5K193LMos0fHFBZqWDm6s+jxealYMXzyyU8=; b=Oot/raLD2jidFtcef4X2gRwzmkrY7FOhHIJnAdo0hyEJUyekxQlKd2gq6EOQ+1o+hJxagc +JzG0BTUMEx/SpJJ3Vao3R49LKujpggvI6cEUmK+Tu0PodOALjSCA7k/gQl0Dk7yjPHrO5 /P1oKMSpR6Ey7a/bwlfpcZ7aIHoe8Y4= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-616-SmBAr1SmNTCCt5usPE_7_g-1; Tue, 31 Jan 2023 15:47:30 -0500 X-MC-Unique: SmBAr1SmNTCCt5usPE_7_g-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 55F04281DE6E; Tue, 31 Jan 2023 20:47:27 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id E0228492C3E; Tue, 31 Jan 2023 20:47:25 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id C0B3F1946589; Tue, 31 Jan 2023 20:47:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 40A221946587 for ; Tue, 31 Jan 2023 20:47:24 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 2CF64492B06; Tue, 31 Jan 2023 20:47:24 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast03.extmail.prod.ext.rdu2.redhat.com [10.11.55.19]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 24E2C492B05 for ; Tue, 31 Jan 2023 20:47:24 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F00E387A38A for ; Tue, 31 Jan 2023 20:47:23 +0000 (UTC) Received: from mail-qt1-f172.google.com (mail-qt1-f172.google.com [209.85.160.172]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-602-aO_p0cNIMBO57k0simU-yw-1; Tue, 31 Jan 2023 15:47:22 -0500 X-MC-Unique: aO_p0cNIMBO57k0simU-yw-1 Received: by mail-qt1-f172.google.com with SMTP id cr22so2580806qtb.10 for ; Tue, 31 Jan 2023 12:47:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GNvDVbG7CFVraYi/TOfzOl0f+P6XVrld8Cgwy6F2tQc=; b=6nc9R8cyodtEDsacEO3XF0AeJHMqbnVchurpCARBOL1m5pQvBKyAfDMuy0nCIB96hH npHKkDcbRMy3QeLZ3vjAXu5ai6N2BUs2r3Z8ftDelVTLbKV75Ik0AMt7PAaseZUYQ6YG l16PrDTa/bW1SMBR4zEKLWOJbYg6LmFhluzod7enfuyCqPNhG/2uI6Y+uh5QApgPW2y+ fT4D7NV3rLHKi8dsXeC73Dezden2AeRFIue1wG5efCuDoaNm8xmLpSi/EnIJu2QWZoPm 22pZihQEJPstqPvluwCZTpqIg8XJRjZN9GTaTVPxzF2HKUFNGNGD8RoAAVKpm9WuanSw VWqg== X-Gm-Message-State: AO0yUKWmcd88rPbc3s3Jq6gNfWyvDXGxZxP8wy5x6+EYzsjR3xowYU2q wk1NPvh6mL6tA9o5s6AMd6XKyqc= X-Google-Smtp-Source: AK7set+PDMu0N8Uwh8+1p7Os6/xZFQVW0hOeiq3lGbhmrIKFFcHBzzLNY8AXEKM3pGHUO+UzcSRu3Q== X-Received: by 2002:ac8:5f89:0:b0:3b6:2fba:3d46 with SMTP id j9-20020ac85f89000000b003b62fba3d46mr125295qta.42.1675198041743; Tue, 31 Jan 2023 12:47:21 -0800 (PST) Received: from localhost (pool-68-160-166-30.bstnma.fios.verizon.net. [68.160.166.30]) by smtp.gmail.com with ESMTPSA id eb10-20020a05620a480a00b007112aa42c4fsm10494479qkb.135.2023.01.31.12.47.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 31 Jan 2023 12:47:21 -0800 (PST) Date: Tue, 31 Jan 2023 15:47:20 -0500 From: Mike Snitzer To: Sergei Shtepa Message-ID: References: <20221209142331.26395-1-sergei.shtepa@veeam.com> <15ffd4bb-cb87-4bc9-53fc-4e0b941db0b7@veeam.com> MIME-Version: 1.0 In-Reply-To: <15ffd4bb-cb87-4bc9-53fc-4e0b941db0b7@veeam.com> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 Subject: Re: [dm-devel] [PATCH v2 00/21] blksnap - block devices snapshots module X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "axboe@kernel.dk" , "linux-doc@vger.kernel.org" , "corbet@lwn.net" , "linux-kernel@vger.kernel.org" , "linux-block@vger.kernel.org" , dm-devel@redhat.com Errors-To: dm-devel-bounces@redhat.com Sender: "dm-devel" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 T24gVHVlLCBKYW4gMjQgMjAyMyBhdCAgNjozNFAgLTA1MDAsClNlcmdlaSBTaHRlcGEgPHNlcmdl aS5zaHRlcGFAdmVlYW0uY29tPiB3cm90ZToKCj4gRGVhck1pa2UKPiAKPiBJIGFtIHZlcnkgZ2xh ZCB0aGF0IHlvdSBub3RpY2VkIG15IHBhdGNoIHZlcnNpb24gMi4KPiBJJ20gc3VyZSB5b3VyIGV4 cGVyaWVuY2Ugd2lsbCBoZWxwLgo+IAo+IE9uIDEvMTcvMjMgMjI6MDQsIE1pa2UgU25pdHplciB3 cm90ZToKPiA+IFN1YmplY3Q6Cj4gPiBSZTogW1BBVENIIHYyIDAwLzIxXSBibGtzbmFwIC0gYmxv Y2sgZGV2aWNlcyBzbmFwc2hvdHMgbW9kdWxlCj4gPiBGcm9tOgo+ID4gTWlrZSBTbml0emVyIDxz bml0emVyQHJlZGhhdC5jb20+Cj4gPiBEYXRlOgo+ID4gMS8xNy8yMywgMjI6MDQKPiA+IAo+ID4g VG86Cj4gPiBTZXJnZWkgU2h0ZXBhIDxzZXJnZWkuc2h0ZXBhQHZlZWFtLmNvbT4KPiA+IENDOgo+ ID4gImF4Ym9lQGtlcm5lbC5kayIgPGF4Ym9lQGtlcm5lbC5kaz4sICJjb3JiZXRAbHduLm5ldCIg PGNvcmJldEBsd24ubmV0PiwgImxpbnV4LWJsb2NrQHZnZXIua2VybmVsLm9yZyIgPGxpbnV4LWJs b2NrQHZnZXIua2VybmVsLm9yZz4sICJsaW51eC1kb2NAdmdlci5rZXJuZWwub3JnIiA8bGludXgt ZG9jQHZnZXIua2VybmVsLm9yZz4sICJsaW51eC1rZXJuZWxAdmdlci5rZXJuZWwub3JnIiA8bGlu dXgta2VybmVsQHZnZXIua2VybmVsLm9yZz4KPiA+IAo+ID4gCj4gPiBPbiBGcmksIERlYyAwOSAy MDIyIGF0IDk6MjNQIC0wNTAwLAo+ID4gU2VyZ2VpIFNodGVwYSB3cm90ZToKPiA+IAo+ID4gID4g SGkgSmVucy4gSGkgSm9uYXRoYW4uIEhpIGFsbC4KPiA+ICA+Cj4gPiAgPiBJIGFtIGhhcHB5IHRv IG9mZmVyIGEgbW9kaWZpZWQgdmVyc2lvbiBvZiB0aGUgQmxvY2sgRGV2aWNlcyBTbmFwc2hvdHMK PiA+ICA+IE1vZHVsZS4gSXQgYWxsb3dzIHRvIGNyZWF0ZSBub24tcGVyc2lzdGVudCBzbmFwc2hv dHMgb2YgYW55IGJsb2NrIGRldmljZXMuCj4gPiAgPiBUaGUgbWFpbiBwdXJwb3NlIG9mIHN1Y2gg c25hcHNob3RzIGlzIHRvIHByb3ZpZGUgYmFja3VwcyBvZiBibG9jayBkZXZpY2VzLgo+ID4gID4g U2VlIG1vcmUgaW4gRG9jdW1lbnRhdGlvbi9ibG9jay9ibGtzbmFwLnJzdC4KPiA+ICA+Cj4gPiAg PiBUaGUgQmxvY2sgRGV2aWNlIEZpbHRlcmluZyBNZWNoYW5pc20gaXMgYWRkZWQgdG8gdGhlIGJs b2NrIGxheWVyLiBUaGlzCj4gPiAgPiBhbGxvd3MgdG8gYXR0YWNoIGFuZCBkZXRhY2ggYmxvY2sg ZGV2aWNlIGZpbHRlcnMgdG8gdGhlIGJsb2NrIGxheWVyLgo+ID4gID4gRmlsdGVycyBhbGxvdyB0 byBleHRlbmQgdGhlIGZ1bmN0aW9uYWxpdHkgb2YgdGhlIGJsb2NrIGxheWVyLgo+ID4gID4gU2Vl IG1vcmUgaW4gRG9jdW1lbnRhdGlvbi9ibG9jay9ibGtmaWx0ZXIucnN0Lgo+ID4gID4KPiA+ICA+ IEEgdG9vbCwgYSBsaWJyYXJ5IGZvciB3b3JraW5nIHdpdGggYmxrc25hcCBhbmQgdGVzdHMgY2Fu IGJlIGZvdW5kIGF0Cj4gPiAgPiBodHRwczovL2dpdGh1Yi5jb20vdmVlYW0vYmxrc25hcC4KPiA+ ICA+Cj4gPiAgPiBUaGUgZmlyc3QgdmVyc2lvbiB3YXMgc3VnZ2VzdGVkIGF0IDEzIEp1bmUgMjAy Mi4gTWFueSB0aGFua3MgdG8KPiA+ICA+IENocmlzdG9waCBIZWxsd2lnIGFuZCBSYW5keSBEdW5s YXAgZm9yIHRoZSByZXZpZXcgb2YgdGhhdCB2ZXJzaW9uLgo+ID4gID4KPiA+ICA+IENoYW5nZXM6 Cj4gPiAgPiAtIEZvcmdvdHRlbiAic3RhdGljIiBkZWNsYXJhdGlvbnMgaGF2ZSBiZWVuIGFkZGVk Lgo+ID4gID4gLSBUaGUgdGV4dCBvZiB0aGUgY29tbWVudHMgaGFzIGJlZW4gY29ycmVjdGVkLgo+ ID4gID4gLSBJdCBpcyBwb3NzaWJsZSB0byBjb25uZWN0IG9ubHkgb25lIGZpbHRlciwgc2luY2Ug dGhlcmUgYXJlIG5vIG90aGVycyBpbgo+ID4gID4gdXBzdHJlYW0uCj4gPiAgPiAtIERvIG5vdCBo YXZlIGFkZGl0aW9uYWwgbG9ja3MgZm9yIGF0dGFjaC9kZXRhY2ggZmlsdGVyLgo+ID4gID4gLSBi bGtzbmFwLmggbW92ZWQgdG8gaW5jbHVkZS91YXBpLy4KPiA+ICA+IC0gI3ByYWdtYSBvbmNlIGFu ZCBjb21tZW50ZWQgY29kZSByZW1vdmVkLgo+ID4gID4gLSB1dWlkX3QgcmVtb3ZlZCBmcm9tIHVz ZXIgQVBJLgo+ID4gID4gLSBSZW1vdmVkIGRlZmF1bHQgdmFsdWVzIGZvciBtb2R1bGUgcGFyYW1l dGVycyBmcm9tIHRoZSBjb25maWd1cmF0aW9uIGZpbGUuCj4gPiAgPiAtIFRoZSBkZWJ1Z2dpbmcg Y29kZSBmb3IgdHJhY2tpbmcgbWVtb3J5IGxlYWtzIGhhcyBiZWVuIHJlbW92ZWQuCj4gPiAgPiAt IFNpbXBsaWZpZWQgTWFrZWZpbGUuCj4gPiAgPiAtIE9wdGltaXplZCB3b3JrIHdpdGggbGFyZ2Ug bWVtb3J5IGJ1ZmZlcnMsIENCVCB0YWJsZXMgYXJlIG5vdyBpbiB2aXJ0dWFsCj4gPiAgPiBtZW1v cnkuCj4gPiAgPiAtIFRoZSBhbGxvY2F0aW9uIGNvZGUgb2YgbWlub3IgbnVtYmVycyBoYXMgYmVl biBvcHRpbWl6ZWQuCj4gPiAgPiAtIFRoZSBpbXBsZW1lbnRhdGlvbiBvZiB0aGUgc25hcHNob3Qg aW1hZ2UgYmxvY2sgZGV2aWNlIGhhcyBiZWVuCj4gPiAgPiBzaW1wbGlmaWVkLCBub3cgaXQgaXMg YSBiaW8tYmFzZWQgYmxvY2sgZGV2aWNlLgo+ID4gID4gLSBSZW1vdmVkIGluaXRpYWxpemF0aW9u IG9mIGdsb2JhbCB2YXJpYWJsZXMgd2l0aCBudWxsIHZhbHVlcy4KPiA+ICA+IC0gT25seSBvbmUg YmlvIGlzIHVzZWQgdG8gY29weSBvbmUgY2h1bmsuCj4gPiAgPiAtIENoZWNrZWQgb24gcHBjNjRs ZS4KPiA+ICA+Cj4gPiAgPiBUaGUgdjEgdmVyc2lvbiB3YXMgc3VnZ2VzdGVkIGF0IDIgTm92ZW1i ZXIgMjAyMi4gTWFueSB0aGFua3MgdG8gRmFiaW8KPiA+ICA+IEZhbnRvbmkgZm9yIGhpcyBmb3Ig aGlzIHBhcnRpY2lwYXRpb24gaW4gdGhlICJibGtzbmFwIiBwcm9qZWN0IG9uIGdpdGh1Ygo+ID4g ID4gYW5kIEpvbmF0aGFuIENvcmJldCBmb3IgaGlzIGFydGljbGUgaHR0cHM6Ly9sd24ubmV0L0Fy dGljbGVzLzkxNDAzMS8uCj4gPiAgPiBUaGFua3MgdG8gdGhlIGltcGFydGlhbCBrZXJuZWwgdGVz dCByb2JvdC4KPiA+ICA+Cj4gPiAgPiBDaGFuZ2VzOgo+ID4gID4gLSBBZGRlZCBkb2N1bWVudGF0 aW9uIGZvciBCbG9jayBEZXZpY2UgRmlsdGVyaW5nIE1lY2hhbmlzbS4KPiA+ICA+IC0gQWRkZWQg ZG9jdW1lbnRhdGlvbiBmb3IgQmxvY2sgRGV2aWNlcyBTbmFwc2hvdHMgTW9kdWxlIChibGtzbmFw KS4KPiA+ICA+IC0gVGhlIE1BSU5UQUlORVJTIGZpbGUgaGFzIGJlZW4gdXBkYXRlZC4KPiA+ICA+ IC0gT3B0aW1pemVkIHF1ZXVlIGNvZGUgZm9yIHNuYXBzaG90IGltYWdlcy4KPiA+ICA+IC0gRml4 ZWQgY29tbWVudHMsIGxvZyBtZXNzYWdlcyBhbmQgY29kZSBmb3IgYmV0dGVyIHJlYWRhYmlsaXR5 Lgo+ID4gCj4gPiBbdGhpcyByZXBseSBnb3QgbG9uZy4uLl0KPiA+IAo+ID4gSSB0aGluayBpdCBp cyBpbXBvcnRhbnQgdG8gcmV2aXNpdCBob3cgd2UndmUgZ290dGVuIHRvIHRoaXMgcG9pbnQuCj4g PiBUaGUgY2F0YWx5c3QgZm9yIGJsa2ZpbHRlciBhbmQgbm93IGJsa3NuYXAgd2FzIHRoYXQgSSBy ZW1vdmVkIHRoZQo+ID4gZXhwb3J0IGZvciBibGtfbXFfc3VibWl0X2JpbyAtLSB3aGljaCB2ZWVh bSB3YXMgdXNpbmcgZm9yIGVuYWJsZW1lbnQKPiA+IG9mIHRoZWlyIGNvbW1lcmNpYWwgYmFja3Vw IHNvZnR3YXJlIHByb2R1Y3QsIHRoZSBvZmZlbmRpbmcgY29tbWl0IHdhcwo+ID4gNjgxY2M1ZTg2 NjdlICgiZG06IGZpeCByZXF1ZXN0LWJhc2VkIERNIHRvIG5vdCBib3VuY2UgdGhyb3VnaCBpbmRp cmVjdCAKPiA+IGRtX3N1Ym1pdF9iaW8iKQo+ID4gCj4gPiBUaGUgd2F5IHZlZWFtIHN0YXJ0ZWQg dG8gYWRkcmVzcyB0aGlzIGNoYW5nZSB3YXMgdG8gcmVxdWVzdCBSZWQgSGF0Cj4gPiBtb2RpZnkg UkhFTCAoYnkgcmV2ZXJ0aW5nIG15IGNoYW5nZSBpbiBSSEVMOCwgd2hlcmVieSByZXN0b3Jpbmcg dGhlCj4gPiBleHBvcnQpIHRvIGdpdmUgdGhlbSBhIHN0b3BnYXAgd2hpbGUgdGhleSB3b3JrZWQg dG8gaWRlbnRpZnkgYSBtb3JlCj4gPiBsYXN0aW5nIHNvbHV0aW9uIHRvIHRoZW0gaGF2aW5nIGRl cGVuZGVkIG9uIHN1Y2ggYSBmcmFnaWxlIGludGVyZmFjZQo+ID4gd2l0aCB3aGljaCB0byBoaWph Y2sgSU8gZm9yIGFsbCBMaW51eCBibG9jayBkZXZpY2VzLgo+ID4gCj4gCj4gVGhhbmsgeW91IHNv IG11Y2ggZm9yIHlvdXIgaGVscCB3aXRoIHRoZSBmdW5jdGlvbiBibGtfbXFfc3VibWl0X2Jpbygp Lgo+IFRoaXMgZml4IGhlbHBlZCB0aGUgdmVlYW1zbmFwIG1vZHVsZSB0byBjb250aW51ZSB3b3Jr aW5nIG9uIFJIRUwgOAo+IHdpdGggYSBtaW5pbWFsIHNldCBvZiBjcnV0Y2hlcy4gQnV0IGJlc2lk ZXMgUkhFTCA4LCB0aGVyZSBhcmUgb3RoZXIKPiBkaXN0cmlidXRpb25zIHRoYXQgcmVxdWlyZWQg c3VwcG9ydC4KPiAKPiBUaGUgY2F0YWx5c3Qgd2FzIHRoZSBvcHRpbWl6YXRpb24gb2YgQ2hyaXN0 b3BoIGFuZCB0aGUgcmVtb3ZhbCBvZgo+IHRoZSBtYWtlX3JlcXVlc3RfZm4oKSBjYWxsYmFjayBm dW5jdGlvbiBpbiB2NS45LiBTZWU6Cj4gaHR0cHM6Ly9lbGl4aXIuYm9vdGxpbi5jb20vbGludXgv djUuOC4xOC9zb3VyY2UvaW5jbHVkZS9saW51eC9ibGtkZXYuaCNMNDA2Cj4gT3ZlcnJpZGluZyB0 aGlzIGZ1bmN0aW9uIGFsbG93ZWQgaGFuZGxpbmcgSS9PIHVuaXRzLgo+IAo+ID4gVGhleSB0aGVu IGNhbWUgdXAgd2l0aCBibGstaW50ZXJwb3Nlci4gSSB0cmllZCB0byBiZSBoZWxwZnVsIGFuZAo+ ID4gcmVwbGllZCBxdWl0ZSByZWd1bGFybHkgdG8gYmxrLWludGVycG9zZXIgcGF0Y2hzZXRzLCBl LmcuOgo+ID4gaHR0cHM6Ly9saXN0bWFuLnJlZGhhdC5jb20vYXJjaGl2ZXMvZG0tZGV2ZWwvMjAy MS1NYXJjaC8wNDU5MDAuaHRtbAo+ID4gaHR0cHM6Ly9saXN0bWFuLnJlZGhhdC5jb20vYXJjaGl2 ZXMvZG0tZGV2ZWwvMjAyMS1NYXJjaC8wNDU4MzguaHRtbAo+ID4gKEkgd29uJ3QgZGlnIG91dCBt b3JlIHBvaW50ZXJzLCBidXQgY2FuIGlmIHlvdSBkb3VidCB0aGlzIGFzc2VydGlvbikuCj4gPiBU aGUgbGFzdCByZXBseSBJIGdvdCBvbiB0aGlzIHRvcGljIHdhcyB2ZXJ5IGRlbnNlIGFuZCBzbyBJ Cj4gPiB0YWJsZWQgaXQgd2l0aCB0aGUgaWRlYSBvZiByZXZpc2l0aW5nIGl0LiBCdXQgSSBkcm9w cGVkIHRoZSBiYWxsIGFuZAo+ID4gbmV2ZXIgZGlkIHJlcGx5IHRvIHRoaXM6Cj4gPiBodHRwczov L2xpc3RtYW4ucmVkaGF0LmNvbS9hcmNoaXZlcy9kbS1kZXZlbC8yMDIxLUFwcmlsLzA0NjE4NC5o dG1sCj4gPiAKPiA+IFNvcnJ5LiBCdXQgdGhhdCB3YXNuJ3Qgb3V0IG9mIG1hbGljZS4gSSB3YXMg anVzdCBidXN5IHdpdGggb3RoZXIKPiA+IHRoaW5ncyBhbmQgYmxrLWludGVycG9zZXIgdG9vayB0 aGUgYmFjayBzZWF0LiBJIG5ldmVyIGltYWdpbmVkIHRoYXQgbXkKPiA+IGluYWN0aW9uIHdvdWxk IGZvc3RlciBjb21wbGV0ZWx5IGFiYW5kb25pbmcgdGhlIGFwcHJvYWNoLgo+ID4gCj4gCj4gQW5k IHllcywgSeKAmXZlIHNwZW50IHNldmVyYWwgbW9udGhzIHRyeWluZyB0byByZWZpbmUgdGhlIERN IGluIG9yZGVyIHRvCj4gaW1wbGVtZW50IHRoZSBmZWF0dXJlIG9mIGF0dGFjaGluZyBETSBkZXZp Y2VzIG9uIHRoZSBmbHksIHRoYXQgeW91IHNhaWQgeW91Cj4gd291bGQgbGlrZSB0byBoYXZlIGlu IERNLiBBbGFzLCBJIGhhdmUgbm90IGFjaGlldmVkIHN1Y2Nlc3MuIEkgZmlndXJlZCB0aGF0Cj4g aXQgd291bGQgdGFrZSBhIGxvdCBvZiBjaGFuZ2VzIGluIHRoZSBETSBjb2RlIGJlZm9yZSBpdHMg bm9uLXBlcnNpc3RlbnQKPiBzbmFwc2hvdHMgY2FuIGJlIHVzZWQgZm9yIGJhY2t1cCwgYW5kIHNv bWUgYXJjaGl0ZWN0dXJhbCBjaGFuZ2VzIHdpbGwgYmUKPiByZXF1aXJlZC4gQWx0aG91Z2ggeW91 IGhhdmUgYmVlbiB2ZXJ5IGhlbHBmdWwgZm9yIGEgd2hpbGUsIGF0IHNvbWUgcG9pbnQKPiB5b3Ug c3RvcHBlZCBwcm92aWRpbmcgYW55IGZlZWRiYWNrIGFuZCB0aHVzIEkgc3RhcnRlZCB0byBleHBs b3JlIG90aGVyCj4gcG9zc2liaWxpdGllcyB0byBzb2x2ZSB0aGUgcHJvYmxlbS4KPiAKPiBTZWVp bmcgeW91ciBhbm5veWFuY2Ugbm93LCBJIHdvbmRlciB3aGF0IHdvdWxkIGJlIHRoZSBiZXR0ZXIg d2F5IHRvIGNvbnRpbnVlCj4gbXkgd29yayBpbiB0aGUgZ2l2ZW4gY2lyY3Vtc3RhbmNlcz8gV2hh dCBraW5kIG9mIGFjdGlvbiB3b3VsZCB5b3UgZXhwZWN0IGZyb20KPiBtZSBpbiBzdWNoIHNpdHVh dGlvbj8gQmVpbmcgbGVmdCB3aXRob3V0IGFueSBmZWVkYmFjayBhbmQgZ3VpZGFuY2Ugb24gRE0s Cj4gSSBkZWNpZGVkIHRvIHdvcmsgZGlyZWN0bHkgd2l0aCB0aGUgY29tbXVuaXR5IHRvIGZpbmQg YmV0dGVyIHNvbHV0aW9uLgo+IAo+IFRoYXQgaXMsIEkgdG9vayBhbm90aGVyIHJvdXRlIChibGtz bmFwKSBub3QgYmVjYXVzZSBJIHRob3VnaHQgdGhlIGluaXRpYWwgCj4gRE0tYmFzZWQgYXBwcm9h Y2ggcHJvcG9zZWQgYnkgeW91IHdhcyBpbmZlcmlvciBvciBiYWQsIGJ1dCBiZWNhdXNlIG9ubHkg dGhpcwo+IGRpcmVjdGlvbiByZW1haW5lZCBhdmFpbGFibGUgdG8gbWUuCgpJJ20gbGF0ZSB0byBy ZXNwb25kIHRvIHlvdXIgZW1haWwgYWdhaW4gYmVjYXVzZSBSZWQgSGF0IG5vdyBpbXBvc2VzCmdt YWlsIGFuZCBpdHMgbWFpbCBmaWx0ZXJpbmcgaXMgYWJ5c21hbC4gSSBzaG91bGQganVzdCB1c2UK c25pdHplckBrZXJuZWwub3JnIHNpbmNlIGl0IGlzIGFibGUgdG8gZW5zdXJlIEkgYW0gYXdhcmUg b2YKcmVwbGllcy4uIGFueXdheSwgdGhhdCdzIG15IHByb2JsZW0gdG8gc29ydCBvdXQhIDspCgpZ b3UncmUgZmluZSB0byBhcHByb2FjaCB0aGUgaW1wbGVtZW50YXRpb24gaG93ZXZlciB5b3UgYW5k IG90aGVycwp0aGluayBiZXN0LiAgSSBoYXZlIGEgY291cGxlIGNvcmUgcG9pbnRzIEknbGwgcmFp c2UgaW4gdGhlIGJsb2NrIGNvcmUKcGF0Y2guCgpCdXQgeWVzLCBJIHdhcyBob3Bpbmcgd2UgY291 bGQgZWZmZWN0aXZlbHkgbWFrZSBpdCBzbyB0aGF0IGFueSBibG9jawpkZXZpY2UgY291bGQgYmUg YXVnbWVudGVkIHRvIHJlbWFwIGJpb3MgKGVpdGhlciBpbiB0ZXJtcyBvZiBleGlzdGluZwpETSB0 YXJnZXRzIG9yIGEgbW9yZSBtaW5pbWFsaXN0IGFwcHJvYWNoIHRoYXQgbWVldHMgeW91ciBzbmFw c2hvdApuZWVkcykuCgpGZWVscyBzb21ld2hhdCBsaWtlIGZhaWx1cmUgdG8gbm90IGJlIGFibGUg dG8gbWFrZSB0aGF0IGFkdmFuY2VtZW50LApidXQgbm90aGluZyBpcyAiZG9uZSIgeWV0Li4uIGxl dCdzIHNlZSB3aGF0IHdlIGNhbiBkby4KCj4gPiBCdXQgbXkgdGhhbmtzIGlzIEknbSBub3cgbWFk ZSB0byBkZWZlbmQgbXlzZWxmIG9uIExXTjoKPiA+IGh0dHBzOi8vbHduLm5ldC9BcnRpY2xlcy85 MjAyNDUvCj4gPiBodHRwczovL2x3bi5uZXQvQXJ0aWNsZXMvOTIwMjQ5Lwo+ID4gCj4gPiBJIGhh cHBlbmVkIHRvIHRyaXAgb3ZlciB0aGF0IExXTiB0aHJlYWQgYmVjYXVzZSBJIHNhdyBIYW5uZXMg cmVmZXJlbmNlCj4gPiAiYmxrc25hcCIgYXMgc29tZXRoaW5nIHRoYXQgc29tZWhvdyBpcyBhIHRv bGVyYXRlZCBhZHZhbmNlIGJ1dCBvdGhlcgo+ID4gZWZmb3J0cyBhcmUgbm90Ogo+ID4gaHR0cHM6 Ly9sb3JlLmtlcm5lbC5vcmcvbGludXgtYmxvY2svMDZlNGQwM2MtM2VjZi03ZTkxLWI4MGUtNjYw MGIzNjE4Yjk4QHN1c2UuZGUvCj4gPiAKPiA+IGJsa3NuYXAgcmVhbGx5IG5lZWRzIHRvIHN0YW5k IG9uIGl0cyBvd24gbWVyaXRzIGFuZCBpdCBjb3VsZCBiZSB0aGF0Cj4gPiBpbiBjb25qdW5jdGlv biB3aXRoIGJsa2ZpbHRlciBpdCBkb2VzLiBCdXQgdGhlIHdheSBpdCBoYXMgZXZvbHZlZCBoYXMK PiA+IGJlZW4gYW50aXRoZXRpY2FsIHRvIGhvdyB0byBwcm9wZXJseSBlbmdhZ2UgdGhlIExpbnV4 IGNvbW11bml0eSBhbmQKPiA+IHN1YnN5c3RlbSBtYWluYXRpbmVycyBsaWtlIG15c2VsZi4gVGhl IFBSIGNhbXBhaWduIHRvIHJhaXNlIGF3YXJlbmVzcwo+ID4gd2l0aCBMV04gYmVjYW1lIG1vcmUg aW1wb3J0YW50IHRoYW4gY2MnaW5nIG1lLiBUaGF0IHNheXMgaXQgYWxsCj4gPiByZWFsbHkuCj4g PiAKPiAKPiBBcyBmb3IgdGhlIGFydGljbGUgb24gTFdOOgo+IAo+IEkgYW0gdmVyeSBncmF0ZWZ1 bCB0byBKb25hdGhhbiBmb3IgaGlzIGFydGljbGUuIEl0IGF0dHJhY3Qgc29tZSBhdHRlbnRpb24K PiB0byBteSB3b3JrLiBIb3dldmVyLCBpdOKAmXMgd2FzbuKAmXQgYSBkZWxpYmVyYXRlIFBSIGNv bXBhbnkgZnJvbSBWZWVhbS4KPiBJbiBmYWN0LCBWZWVhbSBoYXMgbm90aGluZyB0byBkbyB3aXRo IHRoZSBhcnRpY2xlLgo+IAo+IFllcywgSSB3b3JrIGZvciB0aGUgY29tcGFueSwgYnV0IHRoZSBj b21wYW55IGhhcyBuZWl0aGVyIHJlcXVlc3RlZCB0aGUKPiBhcnRpY2xlLCBub3IgaGFzIHN0YXJ0 ZWQgYW55IG90aGVyIFBSIHJlZ2FyZGluZyB0aGUgbWF0dGVyLgo+IAo+IElmIEkgd2VyZSBvcmdh bml6aW5nIGEgUFIgY2FtcGFpZ24sIHRoZSBhcnRpY2xlIHdvdWxkIGJlIHNpbWlsYXIgdG8gdGhp czoKPiBodHRwczovL2dpdGh1Yi5jb20vdmVlYW0vYmxrc25hcC9ibG9iL21hc3Rlci9kb2MvU29t ZXRoaW5nLXdyb25nLXdpdGgtc25hcHNob3RzLWZvci1MaW51eC5tZAo+IEJ1dCBiZSBjYXJlZnVs ISBJbiB0aGUgYXJ0aWNsZSBJIHRyeSB0byBjaGFuZ2UgdGhlIHJlYWRlcidzIG9waW5pb24gYWJv dXQKPiBzbmFwc2hvdHMuIEZlZWRiYWNrLCBhcyB1c3VhbCwgaXMgd2VsY29tZS4KCldlIGNhbiBq dXN0IG1vdmUgcGFzdCBhbGwgdGhhdC4gTm8gbGFzdGluZyBpc3N1ZS4gSSB0b29rIHRoZSBjb21t ZW50cwphYm91dCBETSBpbiB0aGF0IGx3biBhcnRpY2xlIHRvbyBwZXJzb25hbC4KCj4gPiBCdXQg aG9wZWZ1bGx5IHlvdSBjYW4gdGFrZSBteSB3b3JkcyBhcyBteSB0cnV0aDogSSB0aGluayB3aGF0 IHlvdSdyZQo+ID4gd2FudGluZyB0byBkbyBpcyB1c2VmdWwuIEkgbmV2ZXIgaW50ZW5kZWQgdG8g YWN0IGFzIHNvbWUgZ2F0ZWtlZXBlci4gSQo+ID4gZG9uJ3QgaGF2ZSBhIHByb2JsZW0gd2l0aCB3 aGF0IHlvdXIgZ29hbHMgYXJlLCBJIGp1c3QgYXNrIHRoYXQgX2hvd18KPiA+IHlvdSBhY2hpZXZl IHlvdXIgZ29hbHMgYmUgZG9uZSB3aXRoIGNhcmUgYW5kIGNvbnNpZGVyYXRpb24gKHRoZQo+ID4g YXR0ZW1wdHMgSSByZXZpZXdlZCBwcmlvciB0byB5b3VyIG1vc3QgcmVjZW50IHdvcmsgd2VyZSBs YWNraW5nKS4KPiA+IAo+IAo+IE1pa2UsIEkgYXNzdXJlIHlvdSwgSSBoYXZlIG9uZSBnb2FsLiBT bmFwc2hvdHMgb2YgYmxvY2sgZGV2aWNlcyBpbiB0aGUKPiBMaW51eCBrZXJuZWwgZm9yIGJhY2t1 cC4KPgo+IEkgdGhpbmsgd2UgZ290IG9mZiB0byBhIGdvb2Qgc3RhcnQgaW5pdGlhbGx5LCBidXQg dGhlbiBzb21ldGhpbmcgd2VudCB3cm9uZyAKPiAobWlzY29tbXVuaWNhdGlvbnMsIG1pc3VuZGVy c3RhbmRpbmcsIGFuZCBhIGxhY2sgb2YgdGltZSkgc28gaGVyZSB3ZSBhcmUuCj4gSWYgeW91IGNv dWxkIGNsYXJpZnkgd2hhdCBjb3VsZCBoYXZlIGJlZW4gdGhlIHByb3BlciB3YXkgb2YgZGVhbGlu ZyB3aXRoCj4gdGhhdCBzaXR1YXRpb24gZnJvbSBteSBzaWRlLCBJIHdvdWxkIHJlYWxseSBhcHBy ZWNpYXRlIHRoYXQuCgpJIHdpbGwgaGF2ZSB0byBsb29rIGNsb3NlciBhdCBob3cgeW91J3JlIGFi bGUgdG8gZW5zdXJlIGNvbnNpc3RlbmN5CmJldHdlZW4gdXBwZXIgbGF5ZXIgKEZTKSBhbmQgdGhl IGJsb2NrIGxheWVyLiAgVGhlIGZzZnJlZXplIGhvb2tzIHdlcmUKYWRkZWQgbG9uZyBhZ28gd2l0 aCBGUy10by1ibG9jayBjb25zaXN0ZW5jeSAoZW5zdXJpbmcgSU8gZmx1c2hlZCBhbmQKaGFsdGVk LCBldGMpLgogCj4gPiBCdXQgbXkgb25lIGFuZCBvbmx5IHJlcXVlc3QgZm9yIHRoaXMgbGluZSBv ZiBkZXZlbG9wbWVudCB3b3VsZCBiZTogSQo+ID4gX3JlYWxseV8gd2FudCBETSBjb2RlIHRvIGJl IGFibGUgdG8gdXNlZCBhcyBhbiBlbmRwb2ludCBmb3IgSU8KPiA+IHJlbWFwcGluZyBhc3NvY2lh dGVkIHdpdGggYW55IG5ldyBibG9jayBjb3JlIGhvb2sgYWRkZWQgdG8gYWNjb21wbGlzaAo+ID4g ZHluYW1pYyByZW1hcHBpbmcgb2YgSU8uIElmIEkgbmVlZCB0byB0YWtlIGFuIGFjdGl2ZSByb2xl IGluIHRoZQo+ID4gZGV2ZWxvcG1lbnQgb2YgdGhhdCBjYXBhYmlsaXR5LCBzbyBiZSBpdC4KPiA+ IAo+IAo+IEkgYW0gc3VyZSB0aGF0IHRoaXMgaXMgcXVpdGUgYWNoaWV2YWJsZS4gSWYgdGhlIGRt LWRldiB0ZWFtIGhhcyBzcGVjaWFsCj4gbmVlZHMgd2hlbiBhdHRhY2hpbmcgRE0gZGV2aWNlcyB2 aWEgYmxrZmlsdGVyLCB3ZSB3aWxsIGJlIGFibGUgdG8gdXBncmFkZSBpdC4KPiBBdCB0aGUgbW9t ZW50IEkgZG9uJ3Qgc2VlIGFueSBwcm9ibGVtcyB3aXRoIHRoaXMuCj4gSSBjYW4gcHJvbWlzZSB5 b3UgdG8gYWRkIHlvdSB0byDQodChIHdoZW4gc2VuZGluZyBuZXh0IHBhdGNoZXMuCj4gCj4gU291 bmRzIGdvb2Q/CgpTdXJlLgoKPiA+IEkndmUgeWV0IHRvIGxvb2sgY2xvc2VseSBhdCBhbGwgdGhp cyBjb2RlIChidXQgd293IHRoZXJlIGlzIHF1aXRlIGEKPiA+IGxvdCB1bmRlciBkcml2ZXJzL2Js b2NrL2Jsa3NuYXApLiBJJ2xsIGhhdmUgYSBsb29rIGF0IHRoZSBibG9jay1jb3JlCj4gPiBjaGFu Z2VzIGFuZCB0aGVuIHRyeSB0byBtYWtlIHNlbnNlIG9mIHRoaW5ncyBmcm9tIHRoZXJlLgo+ID4K PiAKPiBZZXMsIHRoYXQncyByaWdodC4gVGhlcmUgYXJlIHF1aXRlIGEgZmV3IGNoYW5nZXMgaW4g YmxvY2stY29yZS4KPiBGcm9tIHRoZSBibGtzbmFwLCBpdCBpcyBlbm91Z2ggdG8gdmlldyB0cmFj a2VyLmMuIEkvTyBoYW5kbGluZyBpcwo+IGltcGxlbWVudGVkIHRoZXJlLgo+IEJ1dCBpdCdzIHBy b2JhYmx5IGJldHRlciB0byB3YWl0IGZvciB0aGUgbmV4dCB2ZXJzaW9uIG9mIHBhdGNoIHRoYXQg dGFrZXMKPiBpbnRvIGFjY291bnQgQ2hyaXN0b3BoJ3MgY29tbWVudHMuIFRoZXJlIGFyZSBhIGxv dCBvZiBjaGFuZ2VzLCBhbmQgZmlyc3QKPiBvZiBhbGwgaW4gdGhlIGludGVyZmFjZS4KCk9LLCBJ J2xsIHN0aWxsIHJlcGx5IHRvIHRoZSBibG9jayBjb3JlIHBhdGNoIG5vdy1pc2guCgo+ID4gQnV0 IHlvdSd2ZSBhbHJlYWR5IGJ5cGFzc2VkIG1lLCBteSBob3BlIGlzIHRoYXQgSmVucyBhbmQgQ2hy aXN0b3BoCj4gPiBhZ3JlZSB0aGF0IHdlIG5lZWQgdGhpcyBsaW5lIG9mIGRldmVsb3BtZW50IHRv IGJlIGluIHNlcnZpY2UgdG8gb3RoZXIKPiA+IGFyZWFzIG9mIHRoZSBMaW51eCBibG9jayBzdWJz eXN0ZW0gYW5kIGl0cyBkcml2ZXJzIHRoYXQgd2VyZQo+ID4gZXN0YWJsaXNoZWQgZm9yIHRoZSBw dXJwb3NlcyBvZiByZW1hcHBpbmcgSU8uIEl0IGNhbm5vdCBqdXN0IGJlCj4gPiB0aGUgc3Vic2V0 IG5lZWRlZCB0byBjZW1lbnQgdmVlYW0ncyBhYmlsaXR5IHRvIHVzZSBMaW51eCBmb3IgaXRzCj4g PiBwdXJwb3NlcyAoYnV0IEkgY29tcGxldGVseSB1bmRlcnN0YW5kIHRoYXQgaXMgdGhlIHBvaW50 IG9mIHZlZWFtJ3MKPiA+IGV4ZXJjaXNlKS4KPiAKPiBJdOKAmXMgbm90IGFib3V0IFZlZWFtIGF0 IGFsbC4gSSBhbSBzdXJlIHRoYXQgbXkgd29yayB3aWxsIGhlbHAgbWFueSBiYWNrdXAgCj4gdmVu ZG9ycyBhbmQgYXZlcmFnZSB1c2VycyB0byBidWlsZCBtb3JlIHJvYnVzdCBhbmQgZWZmaWNpZW50 IGJhY2t1cCB0b29scy4KPiBTbywgdGhlIGFyZ3VtZW50IHRoYXQgSSBkbyBpdCBqdXN0IGJlY2F1 c2UgVmVlYW0gbmVlZHMgaXQgZG9lcyBub3QgaG9sZAo+IGFueSB3YXRlciDigJMgSSBrbm93IHRo YXQgbWFueSBwZW9wbGUgbmVlZCB0aGUgZmVhdHVyZSwgbm90IGp1c3QgVmVlYW0uCgpObyBvdGhl ciBzbmFwc2hvdCBjb25zdW1lcnMgaGF2ZSBzaG93biB0aGVtc2VsdmVzLiBVc2luZyB0aGVtIGFz IHNvbWUKc29ydCBvZiBpbXBsaWVkIGNvbnNlbnN1cyBvbiB3aGF0IGlzIG5lZWRlZCBmb3IgZ2Vu ZXJpYyBMaW51eCBzbmFwc2hvdAppcyBhIGJpdCBvZiBhIGxlYXAuIEFsbCB5b3UgcmVhbGx5IGhh dmUgYXJlIHlvdXIgcmVxdWlyZW1lbnRzLiBEb2Vzbid0CnJlYWxseSBoZWxwIHRvIHNheSB5b3Ug cmVwcmVzZW50IHRoZSBpbnRlcmVzdHMgb2YgYWxsIGludGVyZXN0ZWQKcGFydGllcyBpZiB0aGV5 IHJlbWFpbiBuYW1lbGVzcyBhbmQgaW4gdGhlIGJhY2tncm91bmQuCgpNaWtlCgotLQpkbS1kZXZl bCBtYWlsaW5nIGxpc3QKZG0tZGV2ZWxAcmVkaGF0LmNvbQpodHRwczovL2xpc3RtYW4ucmVkaGF0 LmNvbS9tYWlsbWFuL2xpc3RpbmZvL2RtLWRldmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87736C38142 for ; Tue, 31 Jan 2023 20:48:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231282AbjAaUsn (ORCPT ); Tue, 31 Jan 2023 15:48:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231268AbjAaUsd (ORCPT ); Tue, 31 Jan 2023 15:48:33 -0500 Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C042A5A812 for ; Tue, 31 Jan 2023 12:47:22 -0800 (PST) Received: by mail-qt1-f170.google.com with SMTP id m26so14732285qtp.9 for ; Tue, 31 Jan 2023 12:47:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GNvDVbG7CFVraYi/TOfzOl0f+P6XVrld8Cgwy6F2tQc=; b=qI4E09D35qoH80qwxIftw7BN7b6RxS6EnrwLUfj8OD+A65h40eCl8Wl3J5yeHXgMKn o17K4bJ44DfOBJATnWGY3HYRuOtsGKkskOLSeleEjejpPvz08Y37iEUI1hRUalXQXDPj ZRf47XH/tUmpkKKjk/fToVJswXLz9Vi3Rn8q5pPbc89DVptjsmPug/ezPzReUcATQXe1 fANFIrl39JlL4U6Fopz3Dq8h+yMf1SFlmPs3FltCWQEbHfNKkZQvK/oT6tSdgFKlUmJV Xm8/KXOCEApSjCxvwTj5KtDOyar6/7lOSVSIVxftZKlNAKH7mBY78fSn4B0zhEYuibL4 rziA== X-Gm-Message-State: AO0yUKV0a9Y2/Z8ijxjdfMzA95EEqXRMoLaeMB2zQ1EYSIH9tLlwFcRm 764on6M0Jlog/19RWtKArEbq X-Google-Smtp-Source: AK7set+PDMu0N8Uwh8+1p7Os6/xZFQVW0hOeiq3lGbhmrIKFFcHBzzLNY8AXEKM3pGHUO+UzcSRu3Q== X-Received: by 2002:ac8:5f89:0:b0:3b6:2fba:3d46 with SMTP id j9-20020ac85f89000000b003b62fba3d46mr125295qta.42.1675198041743; Tue, 31 Jan 2023 12:47:21 -0800 (PST) Received: from localhost (pool-68-160-166-30.bstnma.fios.verizon.net. [68.160.166.30]) by smtp.gmail.com with ESMTPSA id eb10-20020a05620a480a00b007112aa42c4fsm10494479qkb.135.2023.01.31.12.47.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 31 Jan 2023 12:47:21 -0800 (PST) Date: Tue, 31 Jan 2023 15:47:20 -0500 From: Mike Snitzer To: Sergei Shtepa Cc: "axboe@kernel.dk" , "corbet@lwn.net" , "linux-block@vger.kernel.org" , "linux-doc@vger.kernel.org" , "linux-kernel@vger.kernel.org" , dm-devel@redhat.com Subject: Re: [PATCH v2 00/21] blksnap - block devices snapshots module Message-ID: References: <20221209142331.26395-1-sergei.shtepa@veeam.com> <15ffd4bb-cb87-4bc9-53fc-4e0b941db0b7@veeam.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <15ffd4bb-cb87-4bc9-53fc-4e0b941db0b7@veeam.com> Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On Tue, Jan 24 2023 at 6:34P -0500, Sergei Shtepa wrote: > DearMike > > I am very glad that you noticed my patch version 2. > I'm sure your experience will help. > > On 1/17/23 22:04, Mike Snitzer wrote: > > Subject: > > Re: [PATCH v2 00/21] blksnap - block devices snapshots module > > From: > > Mike Snitzer > > Date: > > 1/17/23, 22:04 > > > > To: > > Sergei Shtepa > > CC: > > "axboe@kernel.dk" , "corbet@lwn.net" , "linux-block@vger.kernel.org" , "linux-doc@vger.kernel.org" , "linux-kernel@vger.kernel.org" > > > > > > On Fri, Dec 09 2022 at 9:23P -0500, > > Sergei Shtepa wrote: > > > > > Hi Jens. Hi Jonathan. Hi all. > > > > > > I am happy to offer a modified version of the Block Devices Snapshots > > > Module. It allows to create non-persistent snapshots of any block devices. > > > The main purpose of such snapshots is to provide backups of block devices. > > > See more in Documentation/block/blksnap.rst. > > > > > > The Block Device Filtering Mechanism is added to the block layer. This > > > allows to attach and detach block device filters to the block layer. > > > Filters allow to extend the functionality of the block layer. > > > See more in Documentation/block/blkfilter.rst. > > > > > > A tool, a library for working with blksnap and tests can be found at > > > https://github.com/veeam/blksnap. > > > > > > The first version was suggested at 13 June 2022. Many thanks to > > > Christoph Hellwig and Randy Dunlap for the review of that version. > > > > > > Changes: > > > - Forgotten "static" declarations have been added. > > > - The text of the comments has been corrected. > > > - It is possible to connect only one filter, since there are no others in > > > upstream. > > > - Do not have additional locks for attach/detach filter. > > > - blksnap.h moved to include/uapi/. > > > - #pragma once and commented code removed. > > > - uuid_t removed from user API. > > > - Removed default values for module parameters from the configuration file. > > > - The debugging code for tracking memory leaks has been removed. > > > - Simplified Makefile. > > > - Optimized work with large memory buffers, CBT tables are now in virtual > > > memory. > > > - The allocation code of minor numbers has been optimized. > > > - The implementation of the snapshot image block device has been > > > simplified, now it is a bio-based block device. > > > - Removed initialization of global variables with null values. > > > - Only one bio is used to copy one chunk. > > > - Checked on ppc64le. > > > > > > The v1 version was suggested at 2 November 2022. Many thanks to Fabio > > > Fantoni for his for his participation in the "blksnap" project on github > > > and Jonathan Corbet for his article https://lwn.net/Articles/914031/. > > > Thanks to the impartial kernel test robot. > > > > > > Changes: > > > - Added documentation for Block Device Filtering Mechanism. > > > - Added documentation for Block Devices Snapshots Module (blksnap). > > > - The MAINTAINERS file has been updated. > > > - Optimized queue code for snapshot images. > > > - Fixed comments, log messages and code for better readability. > > > > [this reply got long...] > > > > I think it is important to revisit how we've gotten to this point. > > The catalyst for blkfilter and now blksnap was that I removed the > > export for blk_mq_submit_bio -- which veeam was using for enablement > > of their commercial backup software product, the offending commit was > > 681cc5e8667e ("dm: fix request-based DM to not bounce through indirect > > dm_submit_bio") > > > > The way veeam started to address this change was to request Red Hat > > modify RHEL (by reverting my change in RHEL8, whereby restoring the > > export) to give them a stopgap while they worked to identify a more > > lasting solution to them having depended on such a fragile interface > > with which to hijack IO for all Linux block devices. > > > > Thank you so much for your help with the function blk_mq_submit_bio(). > This fix helped the veeamsnap module to continue working on RHEL 8 > with a minimal set of crutches. But besides RHEL 8, there are other > distributions that required support. > > The catalyst was the optimization of Christoph and the removal of > the make_request_fn() callback function in v5.9. See: > https://elixir.bootlin.com/linux/v5.8.18/source/include/linux/blkdev.h#L406 > Overriding this function allowed handling I/O units. > > > They then came up with blk-interposer. I tried to be helpful and > > replied quite regularly to blk-interposer patchsets, e.g.: > > https://listman.redhat.com/archives/dm-devel/2021-March/045900.html > > https://listman.redhat.com/archives/dm-devel/2021-March/045838.html > > (I won't dig out more pointers, but can if you doubt this assertion). > > The last reply I got on this topic was very dense and so I > > tabled it with the idea of revisiting it. But I dropped the ball and > > never did reply to this: > > https://listman.redhat.com/archives/dm-devel/2021-April/046184.html > > > > Sorry. But that wasn't out of malice. I was just busy with other > > things and blk-interposer took the back seat. I never imagined that my > > inaction would foster completely abandoning the approach. > > > > And yes, I’ve spent several months trying to refine the DM in order to > implement the feature of attaching DM devices on the fly, that you said you > would like to have in DM. Alas, I have not achieved success. I figured that > it would take a lot of changes in the DM code before its non-persistent > snapshots can be used for backup, and some architectural changes will be > required. Although you have been very helpful for a while, at some point > you stopped providing any feedback and thus I started to explore other > possibilities to solve the problem. > > Seeing your annoyance now, I wonder what would be the better way to continue > my work in the given circumstances? What kind of action would you expect from > me in such situation? Being left without any feedback and guidance on DM, > I decided to work directly with the community to find better solution. > > That is, I took another route (blksnap) not because I thought the initial > DM-based approach proposed by you was inferior or bad, but because only this > direction remained available to me. I'm late to respond to your email again because Red Hat now imposes gmail and its mail filtering is abysmal. I should just use snitzer@kernel.org since it is able to ensure I am aware of replies.. anyway, that's my problem to sort out! ;) You're fine to approach the implementation however you and others think best. I have a couple core points I'll raise in the block core patch. But yes, I was hoping we could effectively make it so that any block device could be augmented to remap bios (either in terms of existing DM targets or a more minimalist approach that meets your snapshot needs). Feels somewhat like failure to not be able to make that advancement, but nothing is "done" yet... let's see what we can do. > > But my thanks is I'm now made to defend myself on LWN: > > https://lwn.net/Articles/920245/ > > https://lwn.net/Articles/920249/ > > > > I happened to trip over that LWN thread because I saw Hannes reference > > "blksnap" as something that somehow is a tolerated advance but other > > efforts are not: > > https://lore.kernel.org/linux-block/06e4d03c-3ecf-7e91-b80e-6600b3618b98@suse.de/ > > > > blksnap really needs to stand on its own merits and it could be that > > in conjunction with blkfilter it does. But the way it has evolved has > > been antithetical to how to properly engage the Linux community and > > subsystem mainatiners like myself. The PR campaign to raise awareness > > with LWN became more important than cc'ing me. That says it all > > really. > > > > As for the article on LWN: > > I am very grateful to Jonathan for his article. It attract some attention > to my work. However, it’s wasn’t a deliberate PR company from Veeam. > In fact, Veeam has nothing to do with the article. > > Yes, I work for the company, but the company has neither requested the > article, nor has started any other PR regarding the matter. > > If I were organizing a PR campaign, the article would be similar to this: > https://github.com/veeam/blksnap/blob/master/doc/Something-wrong-with-snapshots-for-Linux.md > But be careful! In the article I try to change the reader's opinion about > snapshots. Feedback, as usual, is welcome. We can just move past all that. No lasting issue. I took the comments about DM in that lwn article too personal. > > But hopefully you can take my words as my truth: I think what you're > > wanting to do is useful. I never intended to act as some gatekeeper. I > > don't have a problem with what your goals are, I just ask that _how_ > > you achieve your goals be done with care and consideration (the > > attempts I reviewed prior to your most recent work were lacking). > > > > Mike, I assure you, I have one goal. Snapshots of block devices in the > Linux kernel for backup. > > I think we got off to a good start initially, but then something went wrong > (miscommunications, misunderstanding, and a lack of time) so here we are. > If you could clarify what could have been the proper way of dealing with > that situation from my side, I would really appreciate that. I will have to look closer at how you're able to ensure consistency between upper layer (FS) and the block layer. The fsfreeze hooks were added long ago with FS-to-block consistency (ensuring IO flushed and halted, etc). > > But my one and only request for this line of development would be: I > > _really_ want DM code to be able to used as an endpoint for IO > > remapping associated with any new block core hook added to accomplish > > dynamic remapping of IO. If I need to take an active role in the > > development of that capability, so be it. > > > > I am sure that this is quite achievable. If the dm-dev team has special > needs when attaching DM devices via blkfilter, we will be able to upgrade it. > At the moment I don't see any problems with this. > I can promise you to add you to СС when sending next patches. > > Sounds good? Sure. > > I've yet to look closely at all this code (but wow there is quite a > > lot under drivers/block/blksnap). I'll have a look at the block-core > > changes and then try to make sense of things from there. > > > > Yes, that's right. There are quite a few changes in block-core. > From the blksnap, it is enough to view tracker.c. I/O handling is > implemented there. > But it's probably better to wait for the next version of patch that takes > into account Christoph's comments. There are a lot of changes, and first > of all in the interface. OK, I'll still reply to the block core patch now-ish. > > But you've already bypassed me, my hope is that Jens and Christoph > > agree that we need this line of development to be in service to other > > areas of the Linux block subsystem and its drivers that were > > established for the purposes of remapping IO. It cannot just be > > the subset needed to cement veeam's ability to use Linux for its > > purposes (but I completely understand that is the point of veeam's > > exercise). > > It’s not about Veeam at all. I am sure that my work will help many backup > vendors and average users to build more robust and efficient backup tools. > So, the argument that I do it just because Veeam needs it does not hold > any water – I know that many people need the feature, not just Veeam. No other snapshot consumers have shown themselves. Using them as some sort of implied consensus on what is needed for generic Linux snapshot is a bit of a leap. All you really have are your requirements. Doesn't really help to say you represent the interests of all interested parties if they remain nameless and in the background. Mike