From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 34892C001B0 for ; Mon, 24 Jul 2023 18:04:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690221840; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=JztED9T5Ry1dC38Q6iwpU6ex9Y2xDdMvcWYZS+MJgsU=; b=hglVC7ivcqJ0Xu2uV75kCxGLAU8sKlsY18WzNuWlCttYzLHuX6AXHWGclrnBdglkfvh8kJ dJZctgn2IjeMqA2doQeuoZx8pTsW9WLp/TkRsWyk+7HQqTrraAel9ZFvGxrHa4oThNyGJh nwUGlScjgYQVKiwcuYrMLlSlqDUQL5Q= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-606-nZ7KAsrqNO2pLw7CXngAtA-1; Mon, 24 Jul 2023 14:03:58 -0400 X-MC-Unique: nZ7KAsrqNO2pLw7CXngAtA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 201D5857FF8; Mon, 24 Jul 2023 18:03:55 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (unknown [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6FC691454142; Mon, 24 Jul 2023 18:03:53 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id F05A619465B1; Mon, 24 Jul 2023 18:03:52 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id CB7EF1946589 for ; Mon, 24 Jul 2023 18:03:51 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 5893F4087C7B; Mon, 24 Jul 2023 18:03:51 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast10.extmail.prod.ext.rdu2.redhat.com [10.11.55.26]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 50A8640C6CCC for ; Mon, 24 Jul 2023 18:03:51 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-inbound-delivery-1.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 25F371C07544 for ; Mon, 24 Jul 2023 18:03:51 +0000 (UTC) Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-148-dWL7coI7Pk23k-HbcWNUaA-1; Mon, 24 Jul 2023 14:03:49 -0400 X-MC-Unique: dWL7coI7Pk23k-HbcWNUaA-1 Received: by mail-qv1-f72.google.com with SMTP id 6a1803df08f44-63d062fc6f8so9650666d6.3 for ; Mon, 24 Jul 2023 11:03:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690221828; x=1690826628; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=h1iA/74VQWfwLwNThFZsLp7nijUvl29CDijMQU67cx0=; b=brqZ1bVx3E5pYKPvCUONulnOAUtzcC/dx59z6/ofYpQE+5N/rPQ4YHqcQxD5Rj11aP Nx/46+pnl9Akt7OJsefm/DYIn3hjCVds6QAUIyF8WYec635sLsu/AnDa2Dzzv3Jwu1If PyxDCl76Ovwwn1EUzLRzcuZ3tddGzH2TEIXV7xd2QH4A9GyQlCAWuj0Eh6WjmoKZSF5z OqLx62DMkAUA3+x+X/ferzKo50hmakMvM1162zk8dtVdsZJiu4zNXE5CLnOGo76Dd8U2 nZj4kQ79aNPB84JbofZtioxUzEasVtDXTO6yo4G4TGC6mAcglhSlEjM/YZr1TsNCzXYl L9nw== X-Gm-Message-State: ABy/qLb8O+bwSuJSRad6Hy3peOZSz2NkcyaaGO4zKU90l2far+hgXjzz m0dSWv0sAcCLWRyv79u1IRmOYgNY0M2ZUwz4NCzWZRZKD/Pc5xh2u+nRRzL64SUoatcnuP07OtY yWhVxpCMiiGtJrnUQU8T2bQLLpA== X-Received: by 2002:a0c:c541:0:b0:63c:7459:b24c with SMTP id y1-20020a0cc541000000b0063c7459b24cmr486061qvi.1.1690221827705; Mon, 24 Jul 2023 11:03:47 -0700 (PDT) X-Google-Smtp-Source: APBJJlGDoVlS3FFXlpjdmnCPWfX8LPgwXcskuVzu4G58Z3n/2etCM1/Y+5M6HfC8yhSVC2CdnaBipg== X-Received: by 2002:a0c:c541:0:b0:63c:7459:b24c with SMTP id y1-20020a0cc541000000b0063c7459b24cmr486039qvi.1.1690221827407; Mon, 24 Jul 2023 11:03:47 -0700 (PDT) Received: from crash (c-24-218-80-208.hsd1.ma.comcast.net. [24.218.80.208]) by smtp.gmail.com with ESMTPSA id f8-20020ac84708000000b00403ff38d855sm3500178qtp.4.2023.07.24.11.03.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jul 2023 11:03:46 -0700 (PDT) From: Ken Raeburn To: Mike Snitzer References: <20230523214539.226387-1-corwin@redhat.com> Date: Mon, 24 Jul 2023 14:03:45 -0400 In-Reply-To: (Mike Snitzer's message of "Tue, 18 Jul 2023 11:51:15 -0400") Message-ID: <87mszl9ofy.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Subject: Re: [dm-devel] [vdo-devel] [PATCH v2 00/39] Add the dm-vdo deduplication and compression device mapper target. X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-block@vger.kernel.org, vdo-devel@redhat.com, dm-devel@redhat.com, ebiggers@kernel.org, tj@kernel.org Errors-To: dm-devel-bounces@redhat.com Sender: "dm-devel" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 CihBcG9sb2dpZXMgZm9yIHRoZSByZS1zZW5kIC4uLiBJIG5lZ2xlY3RlZCB0byB0dXJuIG9mIEhU TUwgYW5kIHNvCmxpbnV4LWJsb2NrIGJvdW5jZWQgdGhlIGVtYWlsIGFzIHNwYW0uKQoKT24gVHVl LCBKdWwgMTgsIDIwMjMgYXQgMTE6NTHigK9BTSBNaWtlIFNuaXR6ZXIgPHNuaXR6ZXJAa2VybmVs Lm9yZz4gd3JvdGU6CgogQnV0IHRoZSBsb25nLXN0YW5kaW5nIGRlcGVuZGVuY3kgb24gVkRPJ3Mg d29yay1xdWV1ZSBkYXRhCiBzdHJ1Y3QgaXMgc3RpbGwgbGluZ2VyaW5nIChkcml2ZXJzL21kL2Rt LXZkby93b3JrLXF1ZXVlLmMpLiBBdCBhCiBtaW5pbXVtIHdlIG5lZWQgdG8gd29yayB0b3dhcmQg cGlubmluZyBkb3duIF9leGFjdGx5XyB3aHkgdGhhdCBpcywgYW5kCiBJIHRoaW5rIHRoZSBiZXN0 IHdheSB0byBhbnN3ZXIgdGhhdCBpcyBieSBzaW1wbHkgY29udmVydGluZyB0aGUgVkRPCiBjb2Rl IG92ZXIgdG8gdXNpbmcgTGludXgncyB3b3JrcXVldWVzLiAgSWYgZG9pbmcgc28gY2F1c2VzIHNl cmlvdXMKIGluaGVyZW50IHBlcmZvcm1hbmNlIChvciBmdW5jdGlvbmFsaXR5KSBsb3NzIHRoZW4g d2UgbmVlZCB0bwogdW5kZXJzdGFuZCB3aHkgLS0gYW5kIGZpeCBMaW51eCdzIHdvcmtxdWV1ZSBj b2RlIGFjY29yZGluZ2x5LiAoSSd2ZQogY2MnZCBUZWp1biBzbyBoZSBpcyBhd2FyZSkuCgpXZSB0 cmllZCB0aGlzIGV4cGVyaW1lbnQgYW5kIGRpZCBpbmRlZWQgc2VlIHNvbWUgc2lnbmlmaWNhbnQK cGVyZm9ybWFuY2UgZGlmZmVyZW5jZXMuIE5lYXJseSBhIDd4IHNsb3dkb3duIGluIHNvbWUgY2Fz ZXMuCgpWRE8gY2FuIGJlIHByZXR0eSBDUFUtaW50ZW5zaXZlLiBJbiBhZGRpdGlvbiB0byBoYXNo aW5nIGFuZApjb21wcmVzc2lvbiwgaXQgc2NhbnMgc29tZSBiaWcgaW4tbWVtb3J5IGRhdGEgc3Ry dWN0dXJlcyBhcyBwYXJ0IG9mCnRoZSBkZWR1cGxpY2F0aW9uIHByb2Nlc3MuIFNvbWUgZGF0YSBz dHJ1Y3R1cmVzIGFyZSBzcGxpdCBhY3Jvc3Mgb25lCm9yIG1vcmUgInpvbmVzIiB0byBlbmFibGUg Y29uY3VycmVuY3kgKHVzdWFsbHkgc3BsaXQgYmFzZWQgb24gYml0cyBvZgphbiBhZGRyZXNzIG9y IHNvbWV0aGluZyBsaWtlIHRoYXQpLCBidXQgc29tZSBhcmUgbm90LCBhbmQgYSBjb3VwbGUgb2YK dGhvc2UgdGhyZWFkcyBjYW4gc29tZXRpbWVzIGV4Y2VlZCA1MCUgQ1BVIHV0aWxpemF0aW9uLCBl dmVuIDkwJQpkZXBlbmRpbmcgb24gdGhlIHN5c3RlbSBhbmQgdGVzdCBkYXRhIGNvbmZpZ3VyYXRp b24uIChVc3VhbGx5IHRoaXMgaXMKd2hpbGUgcHVzaGluZyBvdmVyIDFHQi9zIHRocm91Z2ggdGhl IGRlZHVwbGljYXRpb24gYW5kIGNvbXByZXNzaW9uCnByb2Nlc3Npbmcgb24gYSBzeXN0ZW0gd2l0 aCBmYXN0IHN0b3JhZ2UuIE9uIGEgc2xvdyBWTSB3aXRoIHNwaW5uaW5nCnN0b3JhZ2UsIHRoZSBD UFUgbG9hZCBpcyBtdWNoIHNtYWxsZXIuKQoKV2UgdXNlIGEgc29ydCBvZiBtZXNzYWdlLXBhc3Np bmcgYXJyYW5nZW1lbnQgd2hlcmUgYSB3b3JrZXIgdGhyZWFkIGlzCnJlc3BvbnNpYmxlIGZvciB1 cGRhdGluZyBjZXJ0YWluIGRhdGEgc3RydWN0dXJlcyBhcyBuZWVkZWQgZm9yIHRoZQpJL09zIGlu IHByb2dyZXNzLCByYXRoZXIgdGhhbiBoYXZpbmcgdGhlIHByb2Nlc3Npbmcgb2YgZWFjaCBJL08K Y29udGVuZCBmb3IgbG9ja3Mgb24gdGhlIGRhdGEgc3RydWN0dXJlcy4gSXQgZ2l2ZXMgdXMgc29t ZSBnb29kCnRocm91Z2hwdXQgdW5kZXIgbG9hZCBidXQgaXQgZG9lcyBtZWFuIHVwd2FyZHMgb2Yg YSBkb3plbiBoYW5kb2ZmcyBwZXIKNGtCIHdyaXRlLCBkZXBlbmRpbmcgb24gY29tcHJlc3NpYmls aXR5LCB3aGV0aGVyIHRoZSBibG9jayBpcyBhCmR1cGxpY2F0ZSwgYW5kIHZhcmlvdXMgb3RoZXIg ZmFjdG9ycy4gU28gcHJvY2Vzc2luZyAxIEdCL3MgbWVhbnMKaGFuZGxpbmcgb3ZlciAzTSBtZXNz YWdlcyBwZXIgc2Vjb25kLCB0aG91Z2ggZWFjaCBzdGVwIG9mIHByb2Nlc3NpbmcKaXMgZ2VuZXJh bGx5IGxpZ2h0d2VpZ2h0LiBGb3Igb3VyIGRlZGljYXRlZCB3b3JrZXIgdGhyZWFkcywgaXQncyBu b3QKdW51c3VhbCBmb3IgYSB0aHJlYWQgdG8gd2FrZSB1cCBhbmQgcHJvY2VzcyBhIGZldyB0ZW5z IG9yIGV2ZW4KaHVuZHJlZHMgb2YgdXBkYXRlcyB0byBpdHMgZGF0YSBzdHJ1Y3R1cmVzIChsaWtl bHkgYmVuZWZpdGluZyBmcm9tIENQVQpjYWNoaW5nIG9mIHRoZSBkYXRhIHN0cnVjdHVyZXMpIGJl Zm9yZSBydW5uaW5nIG91dCBvZiBhdmFpbGFibGUgd29yawphbmQgZ29pbmcgYmFjayB0byBzbGVl cC4KClRoZSBleHBlcmltZW50IEkgcmFuIHdhcyB0byBjcmVhdGUgYW4gb3JkZXJlZCB3b3JrcXVl dWUgaW5zdGVhZCBvZgplYWNoIGRlZGljYXRlZCB0aHJlYWQgd2hlcmUgd2UgbmVlZCBzZXJpYWxp emF0aW9uLCBhbmQgdW5vcmRlcmVkCndvcmtxdWV1ZXMgd2hlbiBjb25jdXJyZW5jeSBpcyBhbGxv d2VkLiBPbiBvdXIgc2xvd2VyIHRlc3Qgc3lzdGVtcyAoPgoxMHkgb2xkIFN1cGVybWljcm8gWGVv biBFNS0xNjUwIHYyLCBSQUlELTAgc3RvcmFnZSB1c2luZyBTU0RzIG9yCkhERHMpLCB0aGUgc2xv d2Rvd24gd2FzIGxlc3Mgc2lnbmlmaWNhbnQgKHVuZGVyIDJ4KSwgYnV0IG9uIG91ciBmYXN0ZXIK c3lzdGVtICg0LTU/IHllYXIgb2xkIFN1cGVybWljcm8gMTAyOVAtV1RSLCAyeCBYZW9uIEdvbGQg NjEyOCA9IDEyCmNvcmVzLCBOVk1lIHN0b3JhZ2UpIHdlIGdvdCBuZWFybHkgYSA3eCBzbG93ZG93 biBvdmVyYWxsLiBJIGhhdmVuJ3QKeWV0IGR1ZyBkZWVwbHkgaW50byBfd2h5XyB0aGUga2VybmVs IHdvcmsgcXVldWVzIGFyZSBzbG93ZXIgaW4gdGhpcwpzb3J0IG9mIHNldHVwLiBJIGRpZCBydW4g InBlcmYgdG9wIiBicmllZmx5IGR1cmluZyBvbmUgdGVzdCB3aXRoCmtlcm5lbCB3b3JrIHF1ZXVl cywgYW5kIHRoZSBsYXJnZXN0IHNpbmdsZSB1c2Ugb2YgQ1BVIGN5Y2xlcyB3YXMgaW4Kc3BpbiBs b2NrIGFjcXVpc2l0aW9uLCBidXQgSSBkaWRuJ3QgZ2V0IGNhbGwgZ3JhcGhzLgoKKFRoaXMgd2Fz IHdpdGggRmVkb3JhIDM3IDYuMi4xMi0yMDAgYW5kIDYuMi4xNS0yMDAga2VybmVscywgd2l0aG91 dAp0aGUgbGF0ZXN0IHN1Ym1pc3Npb25zIGZyb20gVGVqdW4sIHdoaWNoIGxvb2sgaW50ZXJlc3Rp bmcuIFRob3VnaCBJCnN1c3BlY3Qgd2UgY2FyZSBtb3JlIGFib3V0IGNhY2hlIGxvY2FsaXR5IGZv ciBzb21lIG9mIG91cgp0aHJlYWQtc3BlY2lmaWMgZGF0YSBzdHJ1Y3R1cmVzIHRoYW4gZm9yIGFj Y2Vzc2luZyB0aGUgSS9PCnN0cnVjdHVyZXMuKQoKS2VuCgotLQpkbS1kZXZlbCBtYWlsaW5nIGxp c3QKZG0tZGV2ZWxAcmVkaGF0LmNvbQpodHRwczovL2xpc3RtYW4ucmVkaGF0LmNvbS9tYWlsbWFu L2xpc3RpbmZvL2RtLWRldmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3DE8C001B0 for ; Mon, 24 Jul 2023 18:04:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230156AbjGXSEh (ORCPT ); Mon, 24 Jul 2023 14:04:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33544 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229495AbjGXSEh (ORCPT ); Mon, 24 Jul 2023 14:04:37 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3DF7293 for ; Mon, 24 Jul 2023 11:03:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690221830; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h1iA/74VQWfwLwNThFZsLp7nijUvl29CDijMQU67cx0=; b=JtgySdGtvJzvAl/8XlQ0oRwxq6e2Vs5MnTa5SA4LeSDkZYiZA6R5mIUyE2htQhaXypf8mn pX4jbf5Vh+UIrGjOX5oNqVdrgJySfTEvnp/+4as9d5AVVjL+Bh7VZ8W1vEhbWHJsPSetNh T/s1N49EB055xieYo+Uixaz5l4lQaXs= Received: from mail-qv1-f71.google.com (mail-qv1-f71.google.com [209.85.219.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-617-d2a4geNEPx-PxnfBeXESNg-1; Mon, 24 Jul 2023 14:03:48 -0400 X-MC-Unique: d2a4geNEPx-PxnfBeXESNg-1 Received: by mail-qv1-f71.google.com with SMTP id 6a1803df08f44-63cfe46bbb6so17162226d6.2 for ; Mon, 24 Jul 2023 11:03:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690221828; x=1690826628; h=content-transfer-encoding:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=h1iA/74VQWfwLwNThFZsLp7nijUvl29CDijMQU67cx0=; b=hAnvBke8BBImWF+wB2HMqHV+9lwfVJibM3Hu+Bgi75oCP+aC+gxttn30YevBTf25/P K3rwyu/SL+gV0YwgCx8ShOUGmi1D9FjNAb3/+tYH3kyz5QzO03spqE3ijjMjJaFv0XyO /idh0vsZNGxAqneAQA+d3Mt3NvTqVksuGnOZqmvxtX4oRtALqYArJV1gH7mLPkp3baqw iKLjGInvn0nEtp03lDaWBfr2Gn4/dKZ45DKxMoagqmn0bnejk9LJVG56UjKD3JGqtQAO ZKFBh1nwBh+VySnzdqE2Xm47bNq1PgnIi8J40BW6nbC3VUL99KUj0qUEOmM5xp2xlxs1 Jz+A== X-Gm-Message-State: ABy/qLb9nzsDNIwLJDANDjrHE5ioljZPQVppsb5kwtiZ9Gh4lR8E0lRH FLztViXb2oFqPZ6rnkNsSQSa7O647f+ndVCo8InFhBzOKKHg+FhaiheVaZ/iT2nI1uZrobjzULl 0YiBtLu/oUIu4N3nH9V7IIEk= X-Received: by 2002:a0c:c541:0:b0:63c:7459:b24c with SMTP id y1-20020a0cc541000000b0063c7459b24cmr486057qvi.1.1690221827703; Mon, 24 Jul 2023 11:03:47 -0700 (PDT) X-Google-Smtp-Source: APBJJlGDoVlS3FFXlpjdmnCPWfX8LPgwXcskuVzu4G58Z3n/2etCM1/Y+5M6HfC8yhSVC2CdnaBipg== X-Received: by 2002:a0c:c541:0:b0:63c:7459:b24c with SMTP id y1-20020a0cc541000000b0063c7459b24cmr486039qvi.1.1690221827407; Mon, 24 Jul 2023 11:03:47 -0700 (PDT) Received: from crash (c-24-218-80-208.hsd1.ma.comcast.net. [24.218.80.208]) by smtp.gmail.com with ESMTPSA id f8-20020ac84708000000b00403ff38d855sm3500178qtp.4.2023.07.24.11.03.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jul 2023 11:03:46 -0700 (PDT) From: Ken Raeburn To: Mike Snitzer Cc: dm-devel@redhat.com, linux-block@vger.kernel.org, vdo-devel@redhat.com, tj@kernel.org, ebiggers@kernel.org Subject: Re: [vdo-devel] [PATCH v2 00/39] Add the dm-vdo deduplication and compression device mapper target. References: <20230523214539.226387-1-corwin@redhat.com> Date: Mon, 24 Jul 2023 14:03:45 -0400 In-Reply-To: (Mike Snitzer's message of "Tue, 18 Jul 2023 11:51:15 -0400") Message-ID: <87mszl9ofy.fsf@redhat.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org (Apologies for the re-send ... I neglected to turn of HTML and so linux-block bounced the email as spam.) On Tue, Jul 18, 2023 at 11:51=E2=80=AFAM Mike Snitzer = wrote: But the long-standing dependency on VDO's work-queue data struct is still lingering (drivers/md/dm-vdo/work-queue.c). At a minimum we need to work toward pinning down _exactly_ why that is, and I think the best way to answer that is by simply converting the VDO code over to using Linux's workqueues. If doing so causes serious inherent performance (or functionality) loss then we need to understand why -- and fix Linux's workqueue code accordingly. (I've cc'd Tejun so he is aware). We tried this experiment and did indeed see some significant performance differences. Nearly a 7x slowdown in some cases. VDO can be pretty CPU-intensive. In addition to hashing and compression, it scans some big in-memory data structures as part of the deduplication process. Some data structures are split across one or more "zones" to enable concurrency (usually split based on bits of an address or something like that), but some are not, and a couple of those threads can sometimes exceed 50% CPU utilization, even 90% depending on the system and test data configuration. (Usually this is while pushing over 1GB/s through the deduplication and compression processing on a system with fast storage. On a slow VM with spinning storage, the CPU load is much smaller.) We use a sort of message-passing arrangement where a worker thread is responsible for updating certain data structures as needed for the I/Os in progress, rather than having the processing of each I/O contend for locks on the data structures. It gives us some good throughput under load but it does mean upwards of a dozen handoffs per 4kB write, depending on compressibility, whether the block is a duplicate, and various other factors. So processing 1 GB/s means handling over 3M messages per second, though each step of processing is generally lightweight. For our dedicated worker threads, it's not unusual for a thread to wake up and process a few tens or even hundreds of updates to its data structures (likely benefiting from CPU caching of the data structures) before running out of available work and going back to sleep. The experiment I ran was to create an ordered workqueue instead of each dedicated thread where we need serialization, and unordered workqueues when concurrency is allowed. On our slower test systems (> 10y old Supermicro Xeon E5-1650 v2, RAID-0 storage using SSDs or HDDs), the slowdown was less significant (under 2x), but on our faster system (4-5? year old Supermicro 1029P-WTR, 2x Xeon Gold 6128 =3D 12 cores, NVMe storage) we got nearly a 7x slowdown overall. I haven't yet dug deeply into _why_ the kernel work queues are slower in this sort of setup. I did run "perf top" briefly during one test with kernel work queues, and the largest single use of CPU cycles was in spin lock acquisition, but I didn't get call graphs. (This was with Fedora 37 6.2.12-200 and 6.2.15-200 kernels, without the latest submissions from Tejun, which look interesting. Though I suspect we care more about cache locality for some of our thread-specific data structures than for accessing the I/O structures.) Ken