From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 11650C7EE31 for ; Thu, 25 May 2023 07:12:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684998755; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=WPCmS20U88f6m2E4Eju0Vn2X0L8Wee6sxUTAHaRJMIU=; b=h5vvqWfm+8DzAiKBkxkAaxzSCRWNCG6lWNX/d3p+JFN+YcYTCbGNTbR+ESvdz6XOvAs/JK bZeJbhWDSGEldqDSnxRg3BfvtyFKqb1PeyRRo3cVCV+WzMBb8bRRePrL25TuPIHgbmn3vi sGs4jv6H9CPe1vLPVAzcPsC4RO/pLbY= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-562-QQ3CjbvyPxay6LS01o8Kag-1; Thu, 25 May 2023 03:12:31 -0400 X-MC-Unique: QQ3CjbvyPxay6LS01o8Kag-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 56577185A7AC; Thu, 25 May 2023 07:12:29 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3F55B401026; Thu, 25 May 2023 07:12:29 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 5EA441946A46; Thu, 25 May 2023 07:12:28 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 9BD9C19465A0 for ; Wed, 24 May 2023 08:11:15 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 8AE4F20296C8; Wed, 24 May 2023 08:11:15 +0000 (UTC) Received: from localhost (unknown [10.39.192.9]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0182620296C6; Wed, 24 May 2023 08:11:14 +0000 (UTC) From: Giuseppe Scrivano To: Gao Xiang References: <9505927dabc3b6695d62dfe1be371b12f5bdebf7.1684491648.git.durui@linux.alibaba.com> Date: Wed, 24 May 2023 10:11:13 +0200 In-Reply-To: (Gao Xiang's message of "Wed, 24 May 2023 15:13:49 +0800") Message-ID: <87r0r6ywri.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Mailman-Approved-At: Thu, 25 May 2023 07:12:27 +0000 Subject: Re: [dm-devel] dm overlaybd: targets mapping OverlayBD image X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Du Rui , Mike Snitzer , linux-kernel@vger.kernel.org, dm-devel@redhat.com, Alexander Larsson , Alasdair Kergon Errors-To: dm-devel-bounces@redhat.com Sender: "dm-devel" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 R2FvIFhpYW5nIDxoc2lhbmdrYW9AbGludXguYWxpYmFiYS5jb20+IHdyaXRlczoKCj4gT24gMjAy My81LzI0IDIzOjQzLCBBbGV4YW5kZXIgTGFyc3NvbiB3cm90ZToKPj4gT24gVHVlLCBNYXkgMjMs IDIwMjMgYXQgNzoyOeKAr1BNIE1pa2UgU25pdHplciA8c25pdHplckBrZXJuZWwub3JnPiB3cm90 ZToKPj4+Cj4+PiBPbiBGcmksIE1heSAxOSAyMDIzIGF0ICA2OjI3UCAtMDQwMCwKPj4+IER1IFJ1 aSA8ZHVydWlAbGludXguYWxpYmFiYS5jb20+IHdyb3RlOgo+Pj4KPj4+PiBPdmVybGF5QkQgaXMg YSBub3ZlbCBsYXllcmluZyBibG9jay1sZXZlbCBpbWFnZSBmb3JtYXQsIHdoaWNoIGlzIGRlc2ln bgo+Pj4+IGZvciBjb250YWluZXIsIHNlY3VyZSBjb250YWluZXIgYW5kIGFwcGxpY2FibGUgdG8g dmlydHVhbCBtYWNoaW5lLAo+Pj4+IHB1Ymxpc2hlZCBpbiBVU0VOSVggQVRDICcyMAo+Pj4+IGh0 dHBzOi8vd3d3LnVzZW5peC5vcmcvc3lzdGVtL2ZpbGVzL2F0YzIwLWxpLWh1aWJhLnBkZgo+Pj4+ Cj4+Pj4gT3ZlcmxheUJEIGFscmVhZHkgaGFzIGEgQ29udGFpbmVyRCBub24tY29yZSBzdWItcHJv amVjdCBpbXBsZW1lbnRhdGlvbgo+Pj4+IGluIHVzZXJzcGFjZSwgYXMgYW4gYWNjZWxlcmF0ZWQg Y29udGFpbmVyIGltYWdlIHNlcnZpY2UKPj4+PiBodHRwczovL2dpdGh1Yi5jb20vY29udGFpbmVy ZC9hY2NlbGVyYXRlZC1jb250YWluZXItaW1hZ2UKPj4+Pgo+Pj4+IEl0IGNvdWxkIGJlIG11Y2gg bW9yZSBlZmZpY2llbnQgd2hlbiBkbyBkZWNvbXByZXNzaW5nIGFuZCBtYXBwaW5nIHdvcmtzCj4+ Pj4gaW4gdGhlIGtlcm5lbCB3aXRoIHRoZSBmcmFtZXdvcmsgb2YgZGV2aWNlLW1hcHBlciwgaW4g bWFueSBjaXJjdW1zdGFuY2VzLAo+Pj4+IHN1Y2ggYXMgc2VjdXJlIGNvbnRhaW5lciBydW50aW1l LCBtb2JpbGUtZGV2aWNlcywgZXRjLgo+Pj4+Cj4+Pj4gVGhpcyBwYXRjaCBjb250YWlucyBhIG1v ZHVsZSwgZG0tb3ZlcmxheWJkLCBwcm92aWRlcyB0d28ga2luZHMgb2YgdGFyZ2V0cwo+Pj4+IGRt LXpmaWxlIGFuZCBkbS1sc210LCB0byBleHBvc2UgYSBncm91cCBvZiBibG9jay1kZXZpY2VzIGNv bnRhaW5zCj4+Pj4gT3ZlcmxheUJEIGltYWdlIGFzIGEgb3ZlcmxhaWQgcmVhZC1vbmx5IGJsb2Nr LWRldmljZS4KPj4+Pgo+Pj4+IFNpZ25lZC1vZmYtYnk6IER1IFJ1aSA8ZHVydWlAbGludXguYWxp YmFiYS5jb20+Cj4+Pgo+Pj4gPHNuaXAsIG9yaWdpbmFsIHBhdGNoIGhlcmU6IFsxXSA+Cj4+IEEg bG9uZyBsb25nIHRpbWUgYWdvIEkgd3JvdGUgYSBkb2NrZXIgY29udGFpbmVyIGltYWdlIGJhc2Vk IG9uCj4+IGRtLXNuYXBzaG90IHRoYXQgaXMgdmFndWVseSBzaW1pbGFyIHRvIHRoaXMgb25lLiBJ dCBpcyBzdGlsbAo+PiBhdmFpbGFibGUsIGJ1dCBub2JvZHkgcmVhbGx5IHVzZXMgaXQuIEl0IGhh cyBzZXZlcmFsIHdlYWtuZXNzZXMuIEZpcnN0Cj4+IG9mIGFsbCB0aGUgY29udGFpbmVyIGltYWdl IGlzIGFuIGFjdHVhbCBmaWxlc3lzdGVtLCBzbyB5b3UgbmVlZCB0bwo+PiBwcmUtYWxsb2NhdGUg YSBmaXhlZCBtYXggc2l6ZSBmb3IgaW1hZ2VzIGF0IGNvbnN0cnVjdGlvbiB0aW1lLgo+PiBTZWNv bmRseSwgYWxsIHRoZSBsdm0gdm9sdW1lIGNoYW5nZXMgYW5kIG1vdW50cyBkdXJpbmcgcnVudGlt ZSBjYXVzZWQKPj4gd2VpcmQgYmVoYXZpb3VyIChlc3BlY2lhbGx5IGF0IHNjYWxlKSB0aGF0IHdh cyBwYWluZnVsIHRvIG1hbmFnZSAoanVzdAo+PiBzZWFyY2ggdGhlIGRvY2tlciBpc3N1ZSB0cmFj a2VyIGZvciBkZXZtYXBwZXIgYmFja2VuZCkuIEluIHRoZSBlbmQKPj4gZXZlcnlvbmUgbW92ZWQg dG8gYSBmaWxlc3lzdGVtIGJhc2VkIGltcGxlbWVudGF0aW9uIChvdmVybGF5ZnMgYmFzZWQpLgo+ Cj4gWWVhaCwgYW5kIEkgdGhpbmsgcmVwcm9kdWNpYmlsaXR5IGlzc3VlIGlzIGFub3RoZXIgcHJv YmxlbSwgd2hpY2ggbWVhbnMKPiBpdCdzIHF1aXRlIGhhcmQgdG8gc2VsZWN0IGEgcmFuZG9tIGZz IHdpdGhvdXQgc29tZSBjaGFuZ2UgdG8gZ2V0IHRoZQo+IGJlc3QgcmVzdWx0LiAgSSBkbyBmaW5k IHRoZXNlIGd1eXMgd29yayBvbiBlMmZzcHJvZ3MgYWdhaW4gYW5kIGFnYWluLgo+Cj4gSSd2ZSBh bHJlYWR5IHRvbGQgdGhlbSBpbnRlcm5hbGx5IGFnYWluIGFuZCBhZ2FpbiwgYnV0Li4gVGhleSBv bmx5IGZvY3VzCj4gb24gc29tZSBtaW5vciBwb2ludHMgc3VjaCBhcyBob3cgdG8gZG8gSS9PIGFu ZCBDUFUgcHJlZmV0Y2ggdG8gZ2V0Cj4gKHNvbWV3aGF0KSBiZXR0ZXIgcGVyZm9ybWFuY2UgYW5k IGJlYXQgRVJPRlMuICBJIGRvbid0IGtub3csIEkgaGF2ZSBubwo+IGVub3VnaCB0aW1lIHRvIGV2 ZW4gbG9vayBpbnRvIHRoYXQgd2hldGhlciB0aGlzIG5ldyBrZXJuZWwgc3R1ZmZzIGlzCj4gZmlu ZTogYmVjYXVzZSBvZiBhIHZlcnkgc2ltcGxpc3QgaWRlYToKPgo+ICBzdGFja2VkIHN0b3JhZ2Ug b3ZlcmhlYWQgZ2VuZXJhbGx5IHRha2VzIGRvdWJsZSBydW50aW1lL21lbW9yeQo+IGZvb3Rwcmlu dHM6Cj4gICAgZmlsZXN5c3RlbSArIGJsb2NrIGRyaXZlcnMKPgo+PiAKPj4+IEkgYXBwcmVjaWF0 ZSB0aGF0IHRoaXMgd29yayBpcyBiZWluZyBkb25lIHdpdGggYW4gZXllIHRvd2FyZAo+Pj4gY29u dGFpbmVyZCAiY29tbXVuaXR5IiBhbmQgc3RhbmRhcmRpemF0aW9uIGJ1dCBiYXNlZCBvbiBteSBs aW1pdGVkCj4+PiByZXNlYXJjaCBpdCBhcHBlYXJzIHRoYXQgdGhpcyBmb3JtYXQgb2YgT0NJIGlt YWdlIHN0b3JhZ2UvdXNlIGlzIG9ubHkKPj4+IHVzZWQgYnkgQWxpYmFiYT8gKGJ1dCBJIGNvdWxk IGJlIHdyb25nLi4uKQo+Pj4KPj4+IEJ1dCB5b3UnZCBkbyB3ZWxsIHRvIGV4cGxhaW4gd2h5IHRo ZSB1c2Vyc3BhY2Ugc29sdXRpb24gaXNuJ3QKPj4+IGFjY2VwdGFibGUuIEFyZSB0aGVyZSBzZWN1 cml0eSBpc3N1ZXMgdGhhdCBtb3ZpbmcgdGhlIGltcGxlbWVudGF0aW9uCj4+PiB0byBrZXJuZWwg YWRkcmVzc2VzPwo+Pj4KPj4+IEkgYWxzbyBoYXZlIGRvdWJ0cyB0aGF0IHRoaXMgc29sdXRpb24g aXMgX2FjdHVhbGx5XyBtb3JlIHBlcmZvcm1hbnQKPj4+IHRoYW4gYSBwcm9wZXIgZmlsZXN5c3Rl bSBiYXNlZCBzb2x1dGlvbiB0aGF0IGFsbG93cyBwYWdlIGNhY2hlIHNoYXJpbmcKPj4+IG9mIGNv bnRhaW5lciBpbWFnZSBkYXRhIGFjcm9zcyBtdWx0aXBsZSBjb250YWluZXJzLgo+PiBUaGlzIHNv bHV0aW9uIGRvZXNuJ3QgZXZlbiBhbGxvdyBwYWdlIGNhY2hlIHNoYXJpbmcgYmV0d2VlbiBzaGFy ZWQKPj4gbGF5ZXJzIChsaWtlIGN1cnJlbnQgY29udGFpbmVycyBkbyksIG11Y2ggbGVzcyBiZXR3 ZWVuIGluZGVwZW5kZW50Cj4+IGxheWVycy4KPj4gCj4+PiBUaGVyZSBpcyBhbiBhY3RpdmUgZGlz Y3Vzc2lvbiBhYm91dCwgYW5kIGFjdGl2ZSBkZXZlbG9wbWVudCBlZmZvcnQKPj4+IGZvciwgdXNp bmcgb3ZlcmxheWZzICsgZXJvZnMgZm9yIGNvbnRhaW5lciBpbWFnZXMuICBJJ20gcmVsdWN0YW50 IHRvCj4+PiBtZXJnZSB0aGlzIERNIGJhc2VkIGNvbnRhaW5lciBpbWFnZSBhcHByb2FjaCB3aXRo b3V0IHdpZGVyIGNvbnNlbnN1cwo+Pj4gZnJvbSBvdGhlciBjb250YWluZXIgc3Rha2Vob2xkZXJz Lgo+Pj4KPj4+IEJ1dCBzaG9ydCBvZiByZWFjaGluZyB3aWRlciBjb25zZW5zdXMgb24gdGhlIG5l ZWQgZm9yIHRoZXNlIERNCj4+PiB0YXJnZXRzOiB0aGVyZSBpcyBub3RoaW5nIHByZXZlbnRpbmcg eW91IGZyb20gY2FycnlpbmcgdGhlc2UgY2hhbmdlcwo+Pj4gaW4geW91ciBhbGliYWJhIGtlcm5l bC4KPj4gRXJvZnMgYWxyZWFkeSBoYXMgc29tZSBibG9jay1sZXZlbCBzdXBwb3J0IGZvciBjb250 YWluZXIgaW1hZ2VzCj4+ICh3aXRoCj4+IG55ZHVzKSwgYW5kIGNvbXBvc2VmcyB3b3JrcyB3aXRo IGN1cnJlbnQgaW4ta2VybmVsIEVST0ZTK292ZXJsYXlmcy4KPj4gQW5kIHRoaXMgbmV3IGFwcHJv YWNoIGRvZXNuJ3QgaGVscCBmb3IgdGhlIElNSE8gY3VycmVudCB3ZWFrIHNwb3Qgd2UKPj4gaGF2 ZSwgd2hpY2ggaXMgdW5wcml2aWxlZ2VkIGNvbnRhaW5lciBpbWFnZXMuCj4+IEFsc28sIHdoaWxl IE9DSSBhcnRpZmFjdHMgY2FuIGJlIHVzZWQgdG8gc3RvcmUgYW55IGtpbmQgb2YgaW1hZ2UKPj4g Zm9ybWF0cyAob3IgYW55IG90aGVyIGtpbmQgb2YgZmlsZSkgSSB0aGluayBmb3IgYW4gYWN0dWFs IHN0YW5kYXJkaXplZAo+PiBuZXcgaW1hZ2UgZm9ybWF0IGl0IHdvdWxkIGJlIGJldHRlciB0byB3 b3JrIHdpdGggdGhlIE9DSSBvcmcgdG8gY29tZQo+PiB1cCB3aXRoIGEgT0NJIHYyIHN0YW5kYXJk IGltYWdlIGZvcm1hdC4KPiBBZ3JlZWQsIEkgaG9wZSB5b3UgZ3V5cyBjb3VsZCBhY3R1YWxseSBz aXQgZG93biBhbmQgZXZhbHVhdGUgYSBwcm9wZXIKPiBzb2x1dGlvbiBvbiB0aGUgbmV4dCBPQ0kg djIsIGN1cnJlbnRseSBJIGtub3cgdGhlcmUgYXJlOgo+Cj4gIC0gQ29tcG9zZWZzCj4gIC0gKGUp c3Rhcmd6ICAgaHR0cHM6Ly9naXRodWIuY29tL2NvbnRhaW5lcmQvc3Rhcmd6LXNuYXBzaG90dGVy Cj4gIC0gTnlkdXMgICAgICAgaHR0cHM6Ly9naXRodWIuY29tL2NvbnRhaW5lcmQvbnlkdXMtc25h cHNob3R0ZXIKPiAgLSBPdmVybGF5QkQgICBodHRwczovL2dpdGh1Yi5jb20vY29udGFpbmVyZC9h Y2NlbGVyYXRlZC1jb250YWluZXItaW1hZ2UKPiAgLSBTT0NJICAgICAgICBodHRwczovL2dpdGh1 Yi5jb20vYXdzbGFicy9zb2NpLXNuYXBzaG90dGVyCj4gIC0gVGFyZnMKPiAgLSAobWF5YmUgZXZl biBtb3JlLi4pCj4KPiBIb25lc3RseSwgSSBkbyB0aGluayBPU1RyZWUvQ29tcG9zZWZzIGlzIHRo ZSBiZXN0IGFwcHJvYWNoIGZvciBub3cgZm9yCj4gZGVkdXBsaWNhdGlvbiBhbmQgcGFnZSBjYWNo ZSBzaGFyaW5nIChkdWUgdG8ga2VybmVsIGxpbWl0YXRpb24gb2YgcGFnZQo+IGNhY2hlIHNoYXJp bmcgYW5kIG92ZXJsYXlmcyBjb3B5dXAgbGltaXRhdGlvbikuICBJJ20gdG9vIHRpcmVkIG9mCj4g Y29udGFpbmVyIGltYWdlIHN0dWZmcyBob25lc3RseS4gIFRvbyBtdWNoIHVubmVjZXNzYXJ5IG1h bnBvd2VyIHdhc3RlLgoKZm9yIGEgZmlsZS1iYXNlZCBzdG9yYWdlIG1vZGVsLCBJIGFtIG5vdCBz dXJlIGEgbmV3IGZvcm1hdCB3b3VsZCByZWFsbHkKYnV5IHVzIG11Y2ggb3IgaXQgY2FuIGJlIHNp Z25pZmljYW50bHkgZGlmZmVyZW50LgoKV2l0aG91dCBhIHByb3BlciBzdXBwb3J0IGZyb20gdGhl IGtlcm5lbCwgYSBuZXcgZm9ybWF0IHdvdWxkIHN0aWxsIG5lZWQKdG8gY3JlYXRlIHRoZSBsYXlv dXQgb3ZlcmxheSBleHBlY3RzLCBzbyBpdCB3b24ndCBiZSBtdWNoIGRpZmZlcmVudCB0aGFuCndo YXQgd2UgaGF2ZSBub3cuCgpUaGUgY3VycmVudCBPQ0kgZm9ybWF0LCB3aXRoIHNvbWUgdHdlYWtz IGxpa2UgKGUpc3Rhcmd6IG9yIHpzdGQ6Y2h1bmtlZCwKYWxyZWFkeSBtYWtlIGl0cyBjb250ZW50 IGFkZHJlc3NhYmxlIGFuZCBhIGNsaWVudCBjYW4gcmV0cmlldmUgb25seSB0aGUKc3Vic2V0IG9m IHRoZSBmaWxlcyB0aGF0IGFyZSBuZWVkZWQuICBBdCB0aGUgc2FtZSB0aW1lIHdlIG1haW50YWlu IHRoZQpzaW1wbGljaXR5IG9mIGEgdGFyYmFsbCBhbmQgaXQgd29uJ3QgYnJlYWsgZXhpc3Rpbmcg Y2xpZW50cy4KCklNTywgdGhlIG1vc3QgaW50ZXJlc3RpbmcgcHJvYmxlbSBpcyBob3cgdG8gc3Rv cmUgdGhlc2UgaW1hZ2VzIGxvY2FsbHkKYW5kIGhvdyB0aGUga2VybmVsIGNhbiBoZWxwIHdpdGgg dGhhdC4KClRoZSBpZGVhIGJlaGluZCBjb21wb3NlZnMgaXMgdG8gcmVwbGFjZSB0aGUgZXhpc3Rp bmcgc3RvcmFnZSBtb2RlbCB1c2VkCmZvciBvdmVybGF5LCB3aGVyZSBlYWNoIGxheWVyIGhhcyBp dHMgb3duIGRpcmVjdG9yeSwgd2l0aCBhIHNpbmdsZQpkaXJlY3Rvcnkgd2hlcmUgYWxsIHRoZSBm aWxlcyBhcmUgc3RvcmVkIGJ5IHRoZWlyIGNoZWNrc3VtLiAgVGhlCmV4cGVjdGVkIGxheW91dCB0 aGVuIGlzIHJlY3JlYXRlZCBhdCBydW50aW1lLgotLQpkbS1kZXZlbCBtYWlsaW5nIGxpc3QKZG0t ZGV2ZWxAcmVkaGF0LmNvbQpodHRwczovL2xpc3RtYW4ucmVkaGF0LmNvbS9tYWlsbWFuL2xpc3Rp bmZvL2RtLWRldmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5223DC77B7A for ; Wed, 24 May 2023 08:11:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239599AbjEXIL5 (ORCPT ); Wed, 24 May 2023 04:11:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60084 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229881AbjEXILz (ORCPT ); Wed, 24 May 2023 04:11:55 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C9339E for ; Wed, 24 May 2023 01:11:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1684915877; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1LD0RBfHkt8RyyoR9/xsXmW+SWj3IJjCO9a4bxL4YoU=; b=JYz7DEvKeyBpE2Lkhf1PjmzkPnyRhdFdPRIUcgmqK9N+oaWDJJnF4f3k0FjHjUjCgOUVz0 Ojb6fYQFwm8ILVzR389To30cBRQhSLKOTc92BDfbkuKrOF4Q/UQ/PS8L9mEeYGspO3rZ4M QZx6AeZ+kLFW+pQdR9dO0vrB6OKuXag= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-544-Nm3rW3t5MSeQAD4O1iF8hA-1; Wed, 24 May 2023 04:11:16 -0400 X-MC-Unique: Nm3rW3t5MSeQAD4O1iF8hA-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 870EC281424F; Wed, 24 May 2023 08:11:15 +0000 (UTC) Received: from localhost (unknown [10.39.192.9]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 0182620296C6; Wed, 24 May 2023 08:11:14 +0000 (UTC) From: Giuseppe Scrivano To: Gao Xiang Cc: Alexander Larsson , Mike Snitzer , Du Rui , dm-devel@redhat.com, linux-kernel@vger.kernel.org, Alasdair Kergon Subject: Re: dm overlaybd: targets mapping OverlayBD image References: <9505927dabc3b6695d62dfe1be371b12f5bdebf7.1684491648.git.durui@linux.alibaba.com> Date: Wed, 24 May 2023 10:11:13 +0200 In-Reply-To: (Gao Xiang's message of "Wed, 24 May 2023 15:13:49 +0800") Message-ID: <87r0r6ywri.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Gao Xiang writes: > On 2023/5/24 23:43, Alexander Larsson wrote: >> On Tue, May 23, 2023 at 7:29=E2=80=AFPM Mike Snitzer wrote: >>> >>> On Fri, May 19 2023 at 6:27P -0400, >>> Du Rui wrote: >>> >>>> OverlayBD is a novel layering block-level image format, which is design >>>> for container, secure container and applicable to virtual machine, >>>> published in USENIX ATC '20 >>>> https://www.usenix.org/system/files/atc20-li-huiba.pdf >>>> >>>> OverlayBD already has a ContainerD non-core sub-project implementation >>>> in userspace, as an accelerated container image service >>>> https://github.com/containerd/accelerated-container-image >>>> >>>> It could be much more efficient when do decompressing and mapping works >>>> in the kernel with the framework of device-mapper, in many circumstanc= es, >>>> such as secure container runtime, mobile-devices, etc. >>>> >>>> This patch contains a module, dm-overlaybd, provides two kinds of targ= ets >>>> dm-zfile and dm-lsmt, to expose a group of block-devices contains >>>> OverlayBD image as a overlaid read-only block-device. >>>> >>>> Signed-off-by: Du Rui >>> >>> >> A long long time ago I wrote a docker container image based on >> dm-snapshot that is vaguely similar to this one. It is still >> available, but nobody really uses it. It has several weaknesses. First >> of all the container image is an actual filesystem, so you need to >> pre-allocate a fixed max size for images at construction time. >> Secondly, all the lvm volume changes and mounts during runtime caused >> weird behaviour (especially at scale) that was painful to manage (just >> search the docker issue tracker for devmapper backend). In the end >> everyone moved to a filesystem based implementation (overlayfs based). > > Yeah, and I think reproducibility issue is another problem, which means > it's quite hard to select a random fs without some change to get the > best result. I do find these guys work on e2fsprogs again and again. > > I've already told them internally again and again, but.. They only focus > on some minor points such as how to do I/O and CPU prefetch to get > (somewhat) better performance and beat EROFS. I don't know, I have no > enough time to even look into that whether this new kernel stuffs is > fine: because of a very simplist idea: > > stacked storage overhead generally takes double runtime/memory > footprints: > filesystem + block drivers > >>=20 >>> I appreciate that this work is being done with an eye toward >>> containerd "community" and standardization but based on my limited >>> research it appears that this format of OCI image storage/use is only >>> used by Alibaba? (but I could be wrong...) >>> >>> But you'd do well to explain why the userspace solution isn't >>> acceptable. Are there security issues that moving the implementation >>> to kernel addresses? >>> >>> I also have doubts that this solution is _actually_ more performant >>> than a proper filesystem based solution that allows page cache sharing >>> of container image data across multiple containers. >> This solution doesn't even allow page cache sharing between shared >> layers (like current containers do), much less between independent >> layers. >>=20 >>> There is an active discussion about, and active development effort >>> for, using overlayfs + erofs for container images. I'm reluctant to >>> merge this DM based container image approach without wider consensus >>> from other container stakeholders. >>> >>> But short of reaching wider consensus on the need for these DM >>> targets: there is nothing preventing you from carrying these changes >>> in your alibaba kernel. >> Erofs already has some block-level support for container images >> (with >> nydus), and composefs works with current in-kernel EROFS+overlayfs. >> And this new approach doesn't help for the IMHO current weak spot we >> have, which is unprivileged container images. >> Also, while OCI artifacts can be used to store any kind of image >> formats (or any other kind of file) I think for an actual standardized >> new image format it would be better to work with the OCI org to come >> up with a OCI v2 standard image format. > Agreed, I hope you guys could actually sit down and evaluate a proper > solution on the next OCI v2, currently I know there are: > > - Composefs > - (e)stargz https://github.com/containerd/stargz-snapshotter > - Nydus https://github.com/containerd/nydus-snapshotter > - OverlayBD https://github.com/containerd/accelerated-container-image > - SOCI https://github.com/awslabs/soci-snapshotter > - Tarfs > - (maybe even more..) > > Honestly, I do think OSTree/Composefs is the best approach for now for > deduplication and page cache sharing (due to kernel limitation of page > cache sharing and overlayfs copyup limitation). I'm too tired of > container image stuffs honestly. Too much unnecessary manpower waste. for a file-based storage model, I am not sure a new format would really buy us much or it can be significantly different. Without a proper support from the kernel, a new format would still need to create the layout overlay expects, so it won't be much different than what we have now. The current OCI format, with some tweaks like (e)stargz or zstd:chunked, already make its content addressable and a client can retrieve only the subset of the files that are needed. At the same time we maintain the simplicity of a tarball and it won't break existing clients. IMO, the most interesting problem is how to store these images locally and how the kernel can help with that. The idea behind composefs is to replace the existing storage model used for overlay, where each layer has its own directory, with a single directory where all the files are stored by their checksum. The expected layout then is recreated at runtime.