From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8646DC28D13 for ; Mon, 22 Aug 2022 21:42:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id E0BEC4C8CC; Mon, 22 Aug 2022 17:42:24 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@kernel.org Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id u1TNpkBPvyhS; Mon, 22 Aug 2022 17:42:23 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 5EABB4C89E; Mon, 22 Aug 2022 17:42:23 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 6698B4C89E for ; Mon, 22 Aug 2022 17:42:22 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xUpmHDafi-uf for ; Mon, 22 Aug 2022 17:42:20 -0400 (EDT) Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id A23604C893 for ; Mon, 22 Aug 2022 17:42:20 -0400 (EDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 72BDE6131B; Mon, 22 Aug 2022 21:42:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A6F27C433C1; Mon, 22 Aug 2022 21:42:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1661204538; bh=m3afk3mljkvownoqx6MC00ioq6Fmulwr4fb2KxjYoHM=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=PYqyst0c/Rsc+ShBpyzFzAilHnFn3hycljOEIChGLUbJhQIsLJgPvnhh+7FgjHZRU nslyP/dSlL0Q4A+e0GgacIWJ65CO0qqnm0LTOvPiQ8TkudfxcnVN7Ua24PBegg6/08 HszYQM7XTYA8iW6ZSOlny9jOOPFo5Wwp/zXizabqwYB3BwNxV+lZwITw1jKtcYBT4t K+UqcBl3HhRozYVIOllwNBqIsXsg+n3ShVXuHmB+gre0VjUlYRbPe8vv6etglQy9wT ClUPKAQRNS3QYE4GHr4zqqZjEVzasMPBJR2JRO9XOZjlRZkmFm+XkGzo7tzSNDzzzZ FIUdpa3iZS+Pg== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oQFBY-004xE0-EA; Mon, 22 Aug 2022 22:42:16 +0100 Date: Mon, 22 Aug 2022 22:42:15 +0100 Message-ID: <87fshovtu0.wl-maz@kernel.org> From: Marc Zyngier To: Gavin Shan Subject: Re: [PATCH v1 1/5] KVM: arm64: Enable ring-based dirty memory tracking In-Reply-To: <41fb5a1f-29a9-e6bb-9fab-4c83a2a8fce5@redhat.com> References: <20220819005601.198436-1-gshan@redhat.com> <20220819005601.198436-2-gshan@redhat.com> <87lerkwtm5.wl-maz@kernel.org> <41fb5a1f-29a9-e6bb-9fab-4c83a2a8fce5@redhat.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: gshan@redhat.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, peterx@redhat.com, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, catalin.marinas@arm.com, will@kernel.org, shuah@kernel.org, seanjc@google.com, drjones@redhat.com, dmatlack@google.com, bgardon@google.com, ricarkol@google.com, zhenyzha@redhat.com, shan.gavin@gmail.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Cc: kvm@vger.kernel.org, linux-doc@vger.kernel.org, catalin.marinas@arm.com, linux-kselftest@vger.kernel.org, bgardon@google.com, shuah@kernel.org, kvmarm@lists.cs.columbia.edu, corbet@lwn.net, will@kernel.org, shan.gavin@gmail.com, drjones@redhat.com, zhenyzha@redhat.com, dmatlack@google.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu SGkgR2F2aW4sCgpPbiBNb24sIDIyIEF1ZyAyMDIyIDAyOjU4OjIwICswMTAwLApHYXZpbiBTaGFu IDxnc2hhbkByZWRoYXQuY29tPiB3cm90ZToKPiAKPiBIaSBNYXJjLAo+IAo+IE9uIDgvMTkvMjIg NjowMCBQTSwgTWFyYyBaeW5naWVyIHdyb3RlOgo+ID4gT24gRnJpLCAxOSBBdWcgMjAyMiAwMTo1 NTo1NyArMDEwMCwKPiA+IEdhdmluIFNoYW4gPGdzaGFuQHJlZGhhdC5jb20+IHdyb3RlOgo+ID4+ IAo+ID4+IFRoZSByaW5nLWJhc2VkIGRpcnR5IG1lbW9yeSB0cmFja2luZyBoYXMgYmVlbiBhdmFp bGFibGUgYW5kIGVuYWJsZWQKPiA+PiBvbiB4ODYgZm9yIGEgd2hpbGUuIFRoZSBmZWF0dXJlIGlz IGJlbmVmaWNpYWwgd2hlbiB0aGUgbnVtYmVyIG9mCj4gPj4gZGlydHkgcGFnZXMgaXMgc21hbGwg aW4gYSBjaGVja3BvaW50aW5nIHN5c3RlbSBvciBsaXZlIG1pZ3JhdGlvbgo+ID4+IHNjZW5hcmlv LiBNb3JlIGRldGFpbHMgY2FuIGJlIGZvdW5kIGZyb20gZmIwNGExZWRkYjFhICgiS1ZNOiBYODY6 Cj4gPj4gSW1wbGVtZW50IHJpbmctYmFzZWQgZGlydHkgbWVtb3J5IHRyYWNraW5nIikuCj4gPj4g Cj4gPj4gVGhpcyBlbmFibGVzIHRoZSByaW5nLWJhc2VkIGRpcnR5IG1lbW9yeSB0cmFja2luZyBv biBBUk02NC4gSXQncwo+ID4+IG5vdGFibGUgdGhhdCBubyBleHRyYSByZXNlcnZlZCByaW5nIGVu dHJpZXMgYXJlIG5lZWRlZCBvbiBBUk02NAo+ID4+IGJlY2F1c2UgdGhlIGh1Z2UgcGFnZXMgYXJl IGFsd2F5cyBzcGxpdCBpbnRvIGJhc2UgcGFnZXMgd2hlbiBwYWdlCj4gPj4gZGlydHkgdHJhY2tp bmcgaXMgZW5hYmxlZC4KPiA+IAo+ID4gQ2FuIHlvdSBwbGVhc2UgZWxhYm9yYXRlIG9uIHRoaXM/ IEFkZGluZyBhIHBlci1DUFUgcmluZyBvZiBjb3Vyc2UKPiA+IHJlc3VsdHMgaW4gZXh0cmEgbWVt b3J5IGFsbG9jYXRpb24sIHNvIHRoZXJlIG11c3QgYmUgYSBzdWJ0bGUKPiA+IHg4Ni1zcGVjaWZp YyBkZXRhaWwgdGhhdCBJJ20gbm90IGF3YXJlIG9mLi4uCj4gPiAKPiAKPiBTdXJlLiBJIGd1ZXNz IGl0J3MgaGVscGZ1bCB0byBleHBsYWluIGhvdyBpdCB3b3JrcyBpbiBuZXh0IHJldmlzaW9uLgo+ IFNvbWV0aGluZyBsaWtlIGJlbG93Ogo+IAo+IFRoaXMgZW5hYmxlcyB0aGUgcmluZy1iYXNlZCBk aXJ0eSBtZW1vcnkgdHJhY2tpbmcgb24gQVJNNjQuIFRoZSBmZWF0dXJlCj4gaXMgZW5hYmxlZCBi eSBDT05GSUdfSEFWRV9LVk1fRElSVFlfUklORywgZGV0ZWN0ZWQgYW5kIGVuYWJsZWQgYnkKPiBD T05GSUdfSEFWRV9LVk1fRElSVFlfUklORy4gQSByaW5nIGJ1ZmZlciBpcyBjcmVhdGVkIG9uIGV2 ZXJ5IHZjcHUgYW5kCj4gZWFjaCBlbnRyeSBpcyBkZXNjcmliZWQgYnkgJ3N0cnVjdCBrdm1fZGly dHlfZ2ZuJy4gVGhlIHJpbmcgYnVmZmVyIGlzCj4gcHVzaGVkIGJ5IGhvc3Qgd2hlbiBwYWdlIGJl Y29tZXMgZGlydHkgYW5kIHB1bGxlZCBieSB1c2Vyc3BhY2UuIEEgdmNwdQo+IGV4aXQgaXMgZm9y Y2VkIHdoZW4gdGhlIHJpbmcgYnVmZmVyIGJlY29tZXMgZnVsbC4gVGhlIHJpbmcgYnVmZmVycyBv bgo+IGFsbCB2Y3B1cyBjYW4gYmUgcmVzZXQgYnkgaW9jdGwgY29tbWFuZCBLVk1fUkVTRVRfRElS VFlfUklOR1MuCj4gCj4gWWVzLCBJIHRoaW5rIHNvLiBBZGRpbmcgYSBwZXItQ1BVIHJpbmcgcmVz dWx0cyBpbiBleHRyYSBtZW1vcnkgYWxsb2NhdGlvbi4KPiBIb3dldmVyLCBpdCdzIGF2b2lkaW5n IHN5bmNocm9uaXphdGlvbiBhbW9uZyBtdWx0aXBsZSB2Y3B1cyB3aGVuIGRpcnR5Cj4gcGFnZXMg aGFwcGVuIG9uIG11bHRpcGxlIHZjcHVzLiBNb3JlIGRpc2N1c3Npb24gY2FuIGJlIGZvdW5kIGZy b20gWzFdCgpPaCwgSSB0b3RhbGx5IGJ1eSB0aGUgcmVsYXhhdGlvbiBvZiB0aGUgc3luY2hyb25p c2F0aW9uICh0aG91Z2ggSQpkb3VidCB0aGlzIHdpbGwgaGF2ZSBhbnkgdmlzaWJsZSBlZmZlY3Qg dW50aWwgd2UgaGF2ZSBzb21ldGhpbmcgbGlrZQpPbGl2ZXIncyBwYXRjaGVzIHRvIGFsbG93IHBh cmFsbGVsIGZhdWx0aW5nKS4KCkJ1dCBpdCBpcyB0aGUgIm5vIGV4dHJhIHJlc2VydmVkIHJpbmcg ZW50cmllcyBhcmUgbmVlZGVkIG9uIEFSTTY0Igphcmd1bWVudCB0aGF0IEkgZG9uJ3QgZ2V0IHll dC4KCgo+Cj4gWzFdIGh0dHBzOi8vcGF0Y2h3b3JrLmtlcm5lbC5vcmcvcHJvamVjdC9rdm0vcGF0 Y2gvQkwyUFIwOE1CNDgxMkY5MjlBMjc2MEJDNDBFQTc1N0NGMDYzMEBCTDJQUjA4TUI0ODEubmFt cHJkMDgucHJvZC5vdXRsb29rLmNvbS8KPiAoY29tbWVudCM4IGZyb20gUmFkaW0gS3LEjW3DocWZ IG9uIE1heSAzLCAyMDE2LCAyOjExIHAubS4gVVRDKQo+IAo+IAo+ID4+IAo+ID4+IFNpZ25lZC1v ZmYtYnk6IEdhdmluIFNoYW4gPGdzaGFuQHJlZGhhdC5jb20+Cj4gPj4gLS0tCj4gPj4gICBEb2N1 bWVudGF0aW9uL3ZpcnQva3ZtL2FwaS5yc3QgICAgfCAyICstCj4gPj4gICBhcmNoL2FybTY0L2lu Y2x1ZGUvdWFwaS9hc20va3ZtLmggfCAxICsKPiA+PiAgIGFyY2gvYXJtNjQva3ZtL0tjb25maWcg ICAgICAgICAgICB8IDEgKwo+ID4+ICAgYXJjaC9hcm02NC9rdm0vYXJtLmMgICAgICAgICAgICAg IHwgOCArKysrKysrKwo+ID4+ICAgNCBmaWxlcyBjaGFuZ2VkLCAxMSBpbnNlcnRpb25zKCspLCAx IGRlbGV0aW9uKC0pCj4gPj4gCj4gPj4gZGlmZiAtLWdpdCBhL0RvY3VtZW50YXRpb24vdmlydC9r dm0vYXBpLnJzdCBiL0RvY3VtZW50YXRpb24vdmlydC9rdm0vYXBpLnJzdAo+ID4+IGluZGV4IGFi ZDdjMzIxMjZjZS4uMTlmYTFhYzAxN2VkIDEwMDY0NAo+ID4+IC0tLSBhL0RvY3VtZW50YXRpb24v dmlydC9rdm0vYXBpLnJzdAo+ID4+ICsrKyBiL0RvY3VtZW50YXRpb24vdmlydC9rdm0vYXBpLnJz dAo+ID4+IEBAIC04MDIyLDcgKzgwMjIsNyBAQCByZWdhcmRsZXNzIG9mIHdoYXQgaGFzIGFjdHVh bGx5IGJlZW4gZXhwb3NlZCB0aHJvdWdoIHRoZSBDUFVJRCBsZWFmLgo+ID4+ICAgOC4yOSBLVk1f Q0FQX0RJUlRZX0xPR19SSU5HCj4gPj4gICAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KPiA+ PiAgIC06QXJjaGl0ZWN0dXJlczogeDg2Cj4gPj4gKzpBcmNoaXRlY3R1cmVzOiB4ODYsIGFybTY0 Cj4gPj4gICA6UGFyYW1ldGVyczogYXJnc1swXSAtIHNpemUgb2YgdGhlIGRpcnR5IGxvZyByaW5n Cj4gPj4gICAgIEtWTSBpcyBjYXBhYmxlIG9mIHRyYWNraW5nIGRpcnR5IG1lbW9yeSB1c2luZyBy aW5nIGJ1ZmZlcnMgdGhhdAo+ID4+IGFyZQo+ID4+IGRpZmYgLS1naXQgYS9hcmNoL2FybTY0L2lu Y2x1ZGUvdWFwaS9hc20va3ZtLmggYi9hcmNoL2FybTY0L2luY2x1ZGUvdWFwaS9hc20va3ZtLmgK PiA+PiBpbmRleCAzYmIxMzQzNTU4NzQuLjdlMDRiMGI4ZDJiMiAxMDA2NDQKPiA+PiAtLS0gYS9h cmNoL2FybTY0L2luY2x1ZGUvdWFwaS9hc20va3ZtLmgKPiA+PiArKysgYi9hcmNoL2FybTY0L2lu Y2x1ZGUvdWFwaS9hc20va3ZtLmgKPiA+PiBAQCAtNDMsNiArNDMsNyBAQAo+ID4+ICAgI2RlZmlu ZSBfX0tWTV9IQVZFX1ZDUFVfRVZFTlRTCj4gPj4gICAgICNkZWZpbmUgS1ZNX0NPQUxFU0NFRF9N TUlPX1BBR0VfT0ZGU0VUIDEKPiA+PiArI2RlZmluZSBLVk1fRElSVFlfTE9HX1BBR0VfT0ZGU0VU IDY0Cj4gPiAKPiA+IEZvciBjb250ZXh0LCB0aGUgZG9jdW1lbnRhdGlvbiBzYXlzOgo+ID4gCj4g PiA8cXVvdGU+Cj4gPiAtIGlmIEtWTV9DQVBfRElSVFlfTE9HX1JJTkcgaXMgYXZhaWxhYmxlLCBh IG51bWJlciBvZiBwYWdlcyBhdAo+ID4gICAgS1ZNX0RJUlRZX0xPR19QQUdFX09GRlNFVCAqIFBB R0VfU0laRS4gWy4uLl0KPiA+IDwvcXVvdGU+Cj4gPiAKPiA+IFdoYXQgaXMgdGhlIHJlYXNvbiBm b3IgcGlja2luZyB0aGlzIHBhcnRpY3VsYXIgdmFsdWU/Cj4gPiAKPiAKPiBJdCdzIGluaGVyaXRl ZCBmcm9tIHg4Ni4gSSBkb24ndCB0aGluayBpdCBoYXMgdG8gYmUgdGhpcyBwYXJ0aWN1bGFyCj4g dmFsdWUuICBUaGUgdmFsdWUgaXMgdXNlZCB0byBkaXN0aW5ndWlzaCB0aGUgcmVnaW9uJ3Mgb3du ZXJzIGxpa2UKPiBrdm1fcnVuLCBLVk1fUElPX1BBR0VfT0ZGU0VULCBLVk1fQ09BTEVTQ0VEX01N SU9fUEFHRV9PRkZTRVQsIGFuZAo+IEtWTV9ESVJUWV9MT0dfUEFHRV9PRkZTRVQuCj4gCj4gSG93 IGFib3V0IHRvIGhhdmUgMiBmb3IgS1ZNX0RJUlRZX0xPR19QQUdFX09GRlNFVCBpbiBuZXh0IHJl dmlzaW9uPwo+IFRoZSB2aXJ0dWFsIGFyZWEgaXMgY2hlYXAsIEkgZ3Vlc3MgaXQncyBhbHNvIG5p Y2UgdG8gdXNlIHg4NidzCj4gcGF0dGVybiB0byBoYXZlIDY0IGZvciBLVk1fRElSVFlfTE9HX1BB R0VfT0ZGU0VULgo+IAo+ICAgICAjZGVmaW5lIEtWTV9DT0FMRVNDRURfTU1JT19QQUdFX09GRlNF VCAgIDEKPiAgICAgI2RlZmluZSBLVk1fRElSVFlfTE9HX1BBR0VfT0ZGU0VUICAgICAgICAyCgpH aXZlbiB0aGF0IHRoaXMgaXMganVzdCBhbiBvZmZzZXQgaW4gdGhlIHZjcHUgImZpbGUiLCBJIGRv bid0IHRoaW5rIGl0Cm1hdHRlcnMgdGhhdCBtdWNoLiA2NCBkZWZpbml0ZWx5IGFsbG93cyBmb3Ig c29tZSBzdHJ1Y3QgdmNwdSBncm93dGgsCmFuZCBpdCBkb2Vzbid0IGh1cnQgdG8gYmUgY29tcGF0 aWJsZSB3aXRoIHg4NiAoZm9yIG9uY2UuLi4pLgoKPiAKPiA+PiAgICAgI2RlZmluZSBLVk1fUkVH X1NJWkUoaWQpCj4gPj4gXAo+ID4+ICAgCSgxVSA8PCAoKChpZCkgJiBLVk1fUkVHX1NJWkVfTUFT SykgPj4gS1ZNX1JFR19TSVpFX1NISUZUKSkKPiA+PiBkaWZmIC0tZ2l0IGEvYXJjaC9hcm02NC9r dm0vS2NvbmZpZyBiL2FyY2gvYXJtNjQva3ZtL0tjb25maWcKPiA+PiBpbmRleCA4MTVjYzExOGM2 NzUuLjAzMDliMmQwZjJkYSAxMDA2NDQKPiA+PiAtLS0gYS9hcmNoL2FybTY0L2t2bS9LY29uZmln Cj4gPj4gKysrIGIvYXJjaC9hcm02NC9rdm0vS2NvbmZpZwo+ID4+IEBAIC0zMiw2ICszMiw3IEBA IG1lbnVjb25maWcgS1ZNCj4gPj4gICAJc2VsZWN0IEtWTV9WRklPCj4gPj4gICAJc2VsZWN0IEhB VkVfS1ZNX0VWRU5URkQKPiA+PiAgIAlzZWxlY3QgSEFWRV9LVk1fSVJRRkQKPiA+PiArCXNlbGVj dCBIQVZFX0tWTV9ESVJUWV9SSU5HCj4gPj4gICAJc2VsZWN0IEhBVkVfS1ZNX01TSQo+ID4+ICAg CXNlbGVjdCBIQVZFX0tWTV9JUlFDSElQCj4gPj4gICAJc2VsZWN0IEhBVkVfS1ZNX0lSUV9ST1VU SU5HCj4gPj4gZGlmZiAtLWdpdCBhL2FyY2gvYXJtNjQva3ZtL2FybS5jIGIvYXJjaC9hcm02NC9r dm0vYXJtLmMKPiA+PiBpbmRleCA5ODZjZWU2ZmJjN2YuLjNkZTZiOWIzOWRiNyAxMDA2NDQKPiA+ PiAtLS0gYS9hcmNoL2FybTY0L2t2bS9hcm0uYwo+ID4+ICsrKyBiL2FyY2gvYXJtNjQva3ZtL2Fy bS5jCj4gPj4gQEAgLTg2Niw2ICs4NjYsMTQgQEAgaW50IGt2bV9hcmNoX3ZjcHVfaW9jdGxfcnVu KHN0cnVjdCBrdm1fdmNwdSAqdmNwdSkKPiA+PiAgIAkJaWYgKCFyZXQpCj4gPj4gICAJCQlyZXQg PSAxOwo+ID4+ICAgKwkJLyogRm9yY2UgdmNwdSBleGl0IGlmIGl0cyBkaXJ0eSByaW5nIGlzIHNv ZnQtZnVsbCAqLwo+ID4+ICsJCWlmICh1bmxpa2VseSh2Y3B1LT5rdm0tPmRpcnR5X3Jpbmdfc2l6 ZSAmJgo+ID4+ICsJCQkgICAgIGt2bV9kaXJ0eV9yaW5nX3NvZnRfZnVsbCgmdmNwdS0+ZGlydHlf cmluZykpKSB7Cj4gPj4gKwkJCXZjcHUtPnJ1bi0+ZXhpdF9yZWFzb24gPSBLVk1fRVhJVF9ESVJU WV9SSU5HX0ZVTEw7Cj4gPj4gKwkJCXRyYWNlX2t2bV9kaXJ0eV9yaW5nX2V4aXQodmNwdSk7Cj4g Pj4gKwkJCXJldCA9IDA7Cj4gPj4gKwkJfQo+ID4+ICsKPiA+IAo+ID4gV2h5IGNhbid0IHRoaXMg YmUgbW92ZWQgdG8ga3ZtX3ZjcHVfZXhpdF9yZXF1ZXN0KCkgaW5zdGVhZD8gSSB3b3VsZAo+ID4g YWxzbyB2ZXJ5IG11Y2ggbGlrZSB0aGUgY2hlY2sgdG8gYmUgbWFkZSBhIGNvbW1vbiBoZWxwZXIg d2l0aCB4ODYuCj4gPiAKPiA+IEEgc2VlbWluZ2x5IGFwcHJvYWNoIHdvdWxkIGJlIHRvIG1ha2Ug dGhpcyBhIHJlcXVlc3Qgb24gZGlydHkgbG9nCj4gPiBpbnNlcnRpb24sIGFuZCBhdm9pZCB0aGUg d2hvbGUgImNoZWNrIHRoZSBsb2cgc2l6ZSIgb24gZXZlcnkgcnVuLAo+ID4gd2hpY2ggYWRkcyBw b2ludGxlc3Mgb3ZlcmhlYWQgdG8gdW5zdXNwZWN0aW5nIHVzZXJzIChha2EgZXZlcnlvbmUpLgo+ ID4gCj4gCj4gSSB0aG91Z2ggb2YgaGF2aW5nIHRoZSBjaGVjayBpbiBrdm1fdmNwdV9leGl0X3Jl cXVlc3QoKS4gVGhlIHZhcmlvdXMKPiBleGl0IHJlYXNvbnMgYXJlIHByaW9yaXRpemVkLiB4ODYg Z2l2ZXMgS1ZNX0VYSVRfRElSVFlfUklOR19GVUxMIHRoZQo+IGhpZ2hlc3QgcHJpb3JpdHkgYW5k IEFSTTY0IGlzIGp1c3QgdG8gZm9sbG93LiBJIGRvbid0IHRoaW5rIGl0IHJlYWxseQo+IG1hdHRl cnMuIEkgd2lsbCBpbXByb3ZlIGl0IGFjY29yZGluZ2x5IGluIG5leHQgcmV2aXNpb246Cj4gCj4g LSBDaGFuZ2Uga3ZtX2RpcnR5X3Jpbmdfc29mdF9mdWxsKCkgdG8gc29tZXRoaW5nIGFzIGJlbG93 IGluIGRpcnR5X3JpbmcuYwo+IAo+ICAgYm9vbCBrdm1fZGlydHlfcmluZ19zb2Z0X2Z1bGwoc3Ry dWN0IGt2bV92Y3B1ICp2Y3B1KQo+ICAgewo+ICAgICAgICBzdHJ1Y3Qga3ZtICprdm0gPSB2Y3B1 LT52Y3B1Owo+ICAgICAgICBzdHJ1Y3Qga3ZtX2RpcnR5X3JpbmcgKnJpbmcgPSAmdmNwdS0+ZGly dHlfcmluZzsKPiAKPiAgICAgICAgaWYgKHVubGlrZWx5KGt2bS0+ZGlydHlfcmluZ19zaXplICYm Cj4gICAgICAgICAgICAgICAgICAgICBrdm1fZGlydHlfcmluZ191c2VkKHJpbmcpID49IHJpbmct PnNvZnRfbGltaXQpKSB7Cj4gICAgICAgICAgICB2Y3B1LT5ydW4tPmV4aXRfcmVhc29uID0gS1ZN X0VYSVRfRElSVFlfUklOR19GVUxMOwo+ICAgICAgICAgICAgdHJhY2Vfa3ZtX2RpcnR5X3Jpbmdf ZXhpdCh2Y3B1KTsKPiAgICAgICAgICAgIHJldHVybiB0cnVlOwo+ICAgICAgICB9Cj4gCj4gICAg ICAgIHJldHVybiBmYWxzZTsKPiAgIH0KPiAKPiAtIFVzZSB0aGUgbW9kaWZpZWQga3ZtX2RpcnR5 X3Jpbmdfc29mdF9mdWxsKCkgaW4ga3ZtX3ZjcHVfZXhpdF9yZXF1ZXN0KCkuCj4gCj4gVXNlcnNw YWNlIG5lZWRzIEtWTV9FWElUX0RJUlRZX1JJTkdfRlVMTCB0byBjb2xsZWN0IHRoZSBkaXJ0eSBs b2cgaW4gdGltZS4KPiBPdGhlcndpc2UsIHRoZSBkaXJ0eSBsb2cgaW4gdGhlIHJpbmcgYnVmZmVy IHdpbGwgYmUgb3ZlcndyaXR0ZW4uIEknbSBub3QKPiBzdXJlIGlmIGFueXRoaW5nIGVsc2UgSSBt aXNzZWQ/CgpJJ20gZmluZSB3aXRoIHRoZSBhYm92ZSwgYnV0IHdoYXQgSSByZWFsbHkgd2FudGVk IGlzIGEgcmVxdWVzdCBmcm9tCnRoZSBkaXJ0eS1yaW5nIGluc2VydGlvbiwgaW5zdGVhZCBvZiBh IGNoZWNrIGluIGt2bV92cGN1X2V4aXRfcmVxdWVzdC4KU29tZXRoaW5nIGxpa2UgdGhpcyAod2hp Y2ggb2J2aW91c2x5IGRvZXNuJ3QgY29tcGlsZSwgYnV0IHlvdSdsbCBnZXQKdGhlIGlkZWEpOgoK ZGlmZiAtLWdpdCBhL2FyY2gvYXJtNjQva3ZtL2FybS5jIGIvYXJjaC9hcm02NC9rdm0vYXJtLmMK aW5kZXggOTg2Y2VlNmZiYzdmLi4wYjQxZmViNmZiN2QgMTAwNjQ0Ci0tLSBhL2FyY2gvYXJtNjQv a3ZtL2FybS5jCisrKyBiL2FyY2gvYXJtNjQva3ZtL2FybS5jCkBAIC03NDcsNiArNzQ3LDEyIEBA IHN0YXRpYyBpbnQgY2hlY2tfdmNwdV9yZXF1ZXN0cyhzdHJ1Y3Qga3ZtX3ZjcHUgKnZjcHUpCiAK IAkJaWYgKGt2bV9jaGVja19yZXF1ZXN0KEtWTV9SRVFfU1VTUEVORCwgdmNwdSkpCiAJCQlyZXR1 cm4ga3ZtX3ZjcHVfc3VzcGVuZCh2Y3B1KTsKKworCQlpZiAoa3ZtX2NoZWNrX3JlcXVlc3QoS1ZN X1JFUV9SSU5HX1NPRlRfRlVMTCwgdmNwdSkpIHsKKwkJCXZjcHUtPnJ1bi0+ZXhpdF9yZWFzb24g PSBLVk1fRVhJVF9ESVJUWV9SSU5HX0ZVTEw7CisJCQl0cmFjZV9rdm1fZGlydHlfcmluZ19leGl0 KHZjcHUpOworCQkJcmV0dXJuIDA7CisJCX0KIAl9CiAKIAlyZXR1cm4gMTsKZGlmZiAtLWdpdCBh L3ZpcnQva3ZtL2RpcnR5X3JpbmcuYyBiL3ZpcnQva3ZtL2RpcnR5X3JpbmcuYwppbmRleCBmNGMy YTZlYjE2NjYuLjA4YjJmMDExNjRmYSAxMDA2NDQKLS0tIGEvdmlydC9rdm0vZGlydHlfcmluZy5j CisrKyBiL3ZpcnQva3ZtL2RpcnR5X3JpbmcuYwpAQCAtMTQ5LDYgKzE0OSw3IEBAIGludCBrdm1f ZGlydHlfcmluZ19yZXNldChzdHJ1Y3Qga3ZtICprdm0sIHN0cnVjdCBrdm1fZGlydHlfcmluZyAq cmluZykKIAogdm9pZCBrdm1fZGlydHlfcmluZ19wdXNoKHN0cnVjdCBrdm1fZGlydHlfcmluZyAq cmluZywgdTMyIHNsb3QsIHU2NCBvZmZzZXQpCiB7CisJc3RydWN0IGt2bV92Y3B1ICp2Y3B1ID0g Y29udGFpbmVyX29mKHJpbmcsIHN0cnVjdCBrdm1fdmNwdSwgZGlydHlfcmluZyk7CiAJc3RydWN0 IGt2bV9kaXJ0eV9nZm4gKmVudHJ5OwogCiAJLyogSXQgc2hvdWxkIG5ldmVyIGdldCBmdWxsICov CkBAIC0xNjYsNiArMTY3LDkgQEAgdm9pZCBrdm1fZGlydHlfcmluZ19wdXNoKHN0cnVjdCBrdm1f ZGlydHlfcmluZyAqcmluZywgdTMyIHNsb3QsIHU2NCBvZmZzZXQpCiAJa3ZtX2RpcnR5X2dmbl9z ZXRfZGlydGllZChlbnRyeSk7CiAJcmluZy0+ZGlydHlfaW5kZXgrKzsKIAl0cmFjZV9rdm1fZGly dHlfcmluZ19wdXNoKHJpbmcsIHNsb3QsIG9mZnNldCk7CisKKwlpZiAoa3ZtX2RpcnR5X3Jpbmdf c29mdF9mdWxsKHZjcHUpKQorCQlrdm1fbWFrZV9yZXF1ZXN0KEtWTV9SRVFfUklOR19TT0ZUX0ZV TEwsIHZjcHUpOwogfQogCiBzdHJ1Y3QgcGFnZSAqa3ZtX2RpcnR5X3JpbmdfZ2V0X3BhZ2Uoc3Ry dWN0IGt2bV9kaXJ0eV9yaW5nICpyaW5nLCB1MzIgb2Zmc2V0KQoKVGhhbmtzLAoKCU0uCgotLSAK V2l0aG91dCBkZXZpYXRpb24gZnJvbSB0aGUgbm9ybSwgcHJvZ3Jlc3MgaXMgbm90IHBvc3NpYmxl LgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwprdm1hcm0g bWFpbGluZyBsaXN0Cmt2bWFybUBsaXN0cy5jcy5jb2x1bWJpYS5lZHUKaHR0cHM6Ly9saXN0cy5j cy5jb2x1bWJpYS5lZHUvbWFpbG1hbi9saXN0aW5mby9rdm1hcm0K From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62506C3F6B0 for ; Mon, 22 Aug 2022 21:42:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238236AbiHVVmZ (ORCPT ); Mon, 22 Aug 2022 17:42:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36552 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238228AbiHVVmX (ORCPT ); Mon, 22 Aug 2022 17:42:23 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A88CB564D2; Mon, 22 Aug 2022 14:42:21 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 3ADABB81982; Mon, 22 Aug 2022 21:42:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A6F27C433C1; Mon, 22 Aug 2022 21:42:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1661204538; bh=m3afk3mljkvownoqx6MC00ioq6Fmulwr4fb2KxjYoHM=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=PYqyst0c/Rsc+ShBpyzFzAilHnFn3hycljOEIChGLUbJhQIsLJgPvnhh+7FgjHZRU nslyP/dSlL0Q4A+e0GgacIWJ65CO0qqnm0LTOvPiQ8TkudfxcnVN7Ua24PBegg6/08 HszYQM7XTYA8iW6ZSOlny9jOOPFo5Wwp/zXizabqwYB3BwNxV+lZwITw1jKtcYBT4t K+UqcBl3HhRozYVIOllwNBqIsXsg+n3ShVXuHmB+gre0VjUlYRbPe8vv6etglQy9wT ClUPKAQRNS3QYE4GHr4zqqZjEVzasMPBJR2JRO9XOZjlRZkmFm+XkGzo7tzSNDzzzZ FIUdpa3iZS+Pg== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oQFBY-004xE0-EA; Mon, 22 Aug 2022 22:42:16 +0100 Date: Mon, 22 Aug 2022 22:42:15 +0100 Message-ID: <87fshovtu0.wl-maz@kernel.org> From: Marc Zyngier To: Gavin Shan Cc: kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, peterx@redhat.com, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, catalin.marinas@arm.com, will@kernel.org, shuah@kernel.org, seanjc@google.com, drjones@redhat.com, dmatlack@google.com, bgardon@google.com, ricarkol@google.com, zhenyzha@redhat.com, shan.gavin@gmail.com Subject: Re: [PATCH v1 1/5] KVM: arm64: Enable ring-based dirty memory tracking In-Reply-To: <41fb5a1f-29a9-e6bb-9fab-4c83a2a8fce5@redhat.com> References: <20220819005601.198436-1-gshan@redhat.com> <20220819005601.198436-2-gshan@redhat.com> <87lerkwtm5.wl-maz@kernel.org> <41fb5a1f-29a9-e6bb-9fab-4c83a2a8fce5@redhat.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: gshan@redhat.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, peterx@redhat.com, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, catalin.marinas@arm.com, will@kernel.org, shuah@kernel.org, seanjc@google.com, drjones@redhat.com, dmatlack@google.com, bgardon@google.com, ricarkol@google.com, zhenyzha@redhat.com, shan.gavin@gmail.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-doc@vger.kernel.org Hi Gavin, On Mon, 22 Aug 2022 02:58:20 +0100, Gavin Shan wrote: >=20 > Hi Marc, >=20 > On 8/19/22 6:00 PM, Marc Zyngier wrote: > > On Fri, 19 Aug 2022 01:55:57 +0100, > > Gavin Shan wrote: > >>=20 > >> The ring-based dirty memory tracking has been available and enabled > >> on x86 for a while. The feature is beneficial when the number of > >> dirty pages is small in a checkpointing system or live migration > >> scenario. More details can be found from fb04a1eddb1a ("KVM: X86: > >> Implement ring-based dirty memory tracking"). > >>=20 > >> This enables the ring-based dirty memory tracking on ARM64. It's > >> notable that no extra reserved ring entries are needed on ARM64 > >> because the huge pages are always split into base pages when page > >> dirty tracking is enabled. > >=20 > > Can you please elaborate on this? Adding a per-CPU ring of course > > results in extra memory allocation, so there must be a subtle > > x86-specific detail that I'm not aware of... > >=20 >=20 > Sure. I guess it's helpful to explain how it works in next revision. > Something like below: >=20 > This enables the ring-based dirty memory tracking on ARM64. The feature > is enabled by CONFIG_HAVE_KVM_DIRTY_RING, detected and enabled by > CONFIG_HAVE_KVM_DIRTY_RING. A ring buffer is created on every vcpu and > each entry is described by 'struct kvm_dirty_gfn'. The ring buffer is > pushed by host when page becomes dirty and pulled by userspace. A vcpu > exit is forced when the ring buffer becomes full. The ring buffers on > all vcpus can be reset by ioctl command KVM_RESET_DIRTY_RINGS. >=20 > Yes, I think so. Adding a per-CPU ring results in extra memory allocation. > However, it's avoiding synchronization among multiple vcpus when dirty > pages happen on multiple vcpus. More discussion can be found from [1] Oh, I totally buy the relaxation of the synchronisation (though I doubt this will have any visible effect until we have something like Oliver's patches to allow parallel faulting). But it is the "no extra reserved ring entries are needed on ARM64" argument that I don't get yet. > > [1] https://patchwork.kernel.org/project/kvm/patch/BL2PR08MB4812F929A2760= BC40EA757CF0630@BL2PR08MB481.namprd08.prod.outlook.com/ > (comment#8 from Radim Kr=C4=8Dm=C3=A1=C5=99 on May 3, 2016, 2:11 p.m. UTC) >=20 >=20 > >>=20 > >> Signed-off-by: Gavin Shan > >> --- > >> Documentation/virt/kvm/api.rst | 2 +- > >> arch/arm64/include/uapi/asm/kvm.h | 1 + > >> arch/arm64/kvm/Kconfig | 1 + > >> arch/arm64/kvm/arm.c | 8 ++++++++ > >> 4 files changed, 11 insertions(+), 1 deletion(-) > >>=20 > >> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/a= pi.rst > >> index abd7c32126ce..19fa1ac017ed 100644 > >> --- a/Documentation/virt/kvm/api.rst > >> +++ b/Documentation/virt/kvm/api.rst > >> @@ -8022,7 +8022,7 @@ regardless of what has actually been exposed thr= ough the CPUID leaf. > >> 8.29 KVM_CAP_DIRTY_LOG_RING > >> --------------------------- > >> -:Architectures: x86 > >> +:Architectures: x86, arm64 > >> :Parameters: args[0] - size of the dirty log ring > >> KVM is capable of tracking dirty memory using ring buffers that > >> are > >> diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/ua= pi/asm/kvm.h > >> index 3bb134355874..7e04b0b8d2b2 100644 > >> --- a/arch/arm64/include/uapi/asm/kvm.h > >> +++ b/arch/arm64/include/uapi/asm/kvm.h > >> @@ -43,6 +43,7 @@ > >> #define __KVM_HAVE_VCPU_EVENTS > >> #define KVM_COALESCED_MMIO_PAGE_OFFSET 1 > >> +#define KVM_DIRTY_LOG_PAGE_OFFSET 64 > >=20 > > For context, the documentation says: > >=20 > > > > - if KVM_CAP_DIRTY_LOG_RING is available, a number of pages at > > KVM_DIRTY_LOG_PAGE_OFFSET * PAGE_SIZE. [...] > > > >=20 > > What is the reason for picking this particular value? > >=20 >=20 > It's inherited from x86. I don't think it has to be this particular > value. The value is used to distinguish the region's owners like > kvm_run, KVM_PIO_PAGE_OFFSET, KVM_COALESCED_MMIO_PAGE_OFFSET, and > KVM_DIRTY_LOG_PAGE_OFFSET. >=20 > How about to have 2 for KVM_DIRTY_LOG_PAGE_OFFSET in next revision? > The virtual area is cheap, I guess it's also nice to use x86's > pattern to have 64 for KVM_DIRTY_LOG_PAGE_OFFSET. >=20 > #define KVM_COALESCED_MMIO_PAGE_OFFSET 1 > #define KVM_DIRTY_LOG_PAGE_OFFSET 2 Given that this is just an offset in the vcpu "file", I don't think it matters that much. 64 definitely allows for some struct vcpu growth, and it doesn't hurt to be compatible with x86 (for once...). >=20 > >> #define KVM_REG_SIZE(id) > >> \ > >> (1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT)) > >> diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig > >> index 815cc118c675..0309b2d0f2da 100644 > >> --- a/arch/arm64/kvm/Kconfig > >> +++ b/arch/arm64/kvm/Kconfig > >> @@ -32,6 +32,7 @@ menuconfig KVM > >> select KVM_VFIO > >> select HAVE_KVM_EVENTFD > >> select HAVE_KVM_IRQFD > >> + select HAVE_KVM_DIRTY_RING > >> select HAVE_KVM_MSI > >> select HAVE_KVM_IRQCHIP > >> select HAVE_KVM_IRQ_ROUTING > >> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > >> index 986cee6fbc7f..3de6b9b39db7 100644 > >> --- a/arch/arm64/kvm/arm.c > >> +++ b/arch/arm64/kvm/arm.c > >> @@ -866,6 +866,14 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) > >> if (!ret) > >> ret =3D 1; > >> + /* Force vcpu exit if its dirty ring is soft-full */ > >> + if (unlikely(vcpu->kvm->dirty_ring_size && > >> + kvm_dirty_ring_soft_full(&vcpu->dirty_ring))) { > >> + vcpu->run->exit_reason =3D KVM_EXIT_DIRTY_RING_FULL; > >> + trace_kvm_dirty_ring_exit(vcpu); > >> + ret =3D 0; > >> + } > >> + > >=20 > > Why can't this be moved to kvm_vcpu_exit_request() instead? I would > > also very much like the check to be made a common helper with x86. > >=20 > > A seemingly approach would be to make this a request on dirty log > > insertion, and avoid the whole "check the log size" on every run, > > which adds pointless overhead to unsuspecting users (aka everyone). > >=20 >=20 > I though of having the check in kvm_vcpu_exit_request(). The various > exit reasons are prioritized. x86 gives KVM_EXIT_DIRTY_RING_FULL the > highest priority and ARM64 is just to follow. I don't think it really > matters. I will improve it accordingly in next revision: >=20 > - Change kvm_dirty_ring_soft_full() to something as below in dirty_ring.c >=20 > bool kvm_dirty_ring_soft_full(struct kvm_vcpu *vcpu) > { > struct kvm *kvm =3D vcpu->vcpu; > struct kvm_dirty_ring *ring =3D &vcpu->dirty_ring; >=20 > if (unlikely(kvm->dirty_ring_size && > kvm_dirty_ring_used(ring) >=3D ring->soft_limit)) { > vcpu->run->exit_reason =3D KVM_EXIT_DIRTY_RING_FULL; > trace_kvm_dirty_ring_exit(vcpu); > return true; > } >=20 > return false; > } >=20 > - Use the modified kvm_dirty_ring_soft_full() in kvm_vcpu_exit_request(). >=20 > Userspace needs KVM_EXIT_DIRTY_RING_FULL to collect the dirty log in time. > Otherwise, the dirty log in the ring buffer will be overwritten. I'm not > sure if anything else I missed? I'm fine with the above, but what I really wanted is a request from the dirty-ring insertion, instead of a check in kvm_vpcu_exit_request. Something like this (which obviously doesn't compile, but you'll get the idea): diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 986cee6fbc7f..0b41feb6fb7d 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -747,6 +747,12 @@ static int check_vcpu_requests(struct kvm_vcpu *vcpu) =20 if (kvm_check_request(KVM_REQ_SUSPEND, vcpu)) return kvm_vcpu_suspend(vcpu); + + if (kvm_check_request(KVM_REQ_RING_SOFT_FULL, vcpu)) { + vcpu->run->exit_reason =3D KVM_EXIT_DIRTY_RING_FULL; + trace_kvm_dirty_ring_exit(vcpu); + return 0; + } } =20 return 1; diff --git a/virt/kvm/dirty_ring.c b/virt/kvm/dirty_ring.c index f4c2a6eb1666..08b2f01164fa 100644 --- a/virt/kvm/dirty_ring.c +++ b/virt/kvm/dirty_ring.c @@ -149,6 +149,7 @@ int kvm_dirty_ring_reset(struct kvm *kvm, struct kvm_di= rty_ring *ring) =20 void kvm_dirty_ring_push(struct kvm_dirty_ring *ring, u32 slot, u64 offset) { + struct kvm_vcpu *vcpu =3D container_of(ring, struct kvm_vcpu, dirty_ring); struct kvm_dirty_gfn *entry; =20 /* It should never get full */ @@ -166,6 +167,9 @@ void kvm_dirty_ring_push(struct kvm_dirty_ring *ring, u= 32 slot, u64 offset) kvm_dirty_gfn_set_dirtied(entry); ring->dirty_index++; trace_kvm_dirty_ring_push(ring, slot, offset); + + if (kvm_dirty_ring_soft_full(vcpu)) + kvm_make_request(KVM_REQ_RING_SOFT_FULL, vcpu); } =20 struct page *kvm_dirty_ring_get_page(struct kvm_dirty_ring *ring, u32 offs= et) Thanks, M. --=20 Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E3325C28D13 for ; Mon, 22 Aug 2022 21:43:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kLJJItUQrGqamVhihW00uZRzRYEazSUlO1IhTVlBlJI=; b=gtUJKTX/2qGfiZ dl8atxO5OvdBtjcrSs5RWA5L1nxJsNqumt+UFzmwDMXyhu40uBRjDvSn3Vp8fbKukCZtwfAaO561I vZjviy4ckgzqEzT/8iNjjZr+3LnZCYQJwpiSO9/z4AaiF0NdoQnGmRjmrpxB+nb1/Q3V1URgH2oKq 7QgyKNHOFEFaH6nsbCwe+dFNf1iyJ77xb8zNY0widuSDTXgrbZ+Sr93vh8eiP7924VoIrCVYw2MYl e5KD2jAIRoUyESDw22s/pY8E4J+wblTfoFLSe8WhnJWAqPQsP9RNH+I+4ahven8I1VEPbNv50WEbE H6VQlWCgEsHzv165xkhA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oQFBj-00F6yj-EY; Mon, 22 Aug 2022 21:42:28 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oQFBc-00F6uG-MX for linux-arm-kernel@lists.infradead.org; Mon, 22 Aug 2022 21:42:22 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 72BDE6131B; Mon, 22 Aug 2022 21:42:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A6F27C433C1; Mon, 22 Aug 2022 21:42:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1661204538; bh=m3afk3mljkvownoqx6MC00ioq6Fmulwr4fb2KxjYoHM=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=PYqyst0c/Rsc+ShBpyzFzAilHnFn3hycljOEIChGLUbJhQIsLJgPvnhh+7FgjHZRU nslyP/dSlL0Q4A+e0GgacIWJ65CO0qqnm0LTOvPiQ8TkudfxcnVN7Ua24PBegg6/08 HszYQM7XTYA8iW6ZSOlny9jOOPFo5Wwp/zXizabqwYB3BwNxV+lZwITw1jKtcYBT4t K+UqcBl3HhRozYVIOllwNBqIsXsg+n3ShVXuHmB+gre0VjUlYRbPe8vv6etglQy9wT ClUPKAQRNS3QYE4GHr4zqqZjEVzasMPBJR2JRO9XOZjlRZkmFm+XkGzo7tzSNDzzzZ FIUdpa3iZS+Pg== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oQFBY-004xE0-EA; Mon, 22 Aug 2022 22:42:16 +0100 Date: Mon, 22 Aug 2022 22:42:15 +0100 Message-ID: <87fshovtu0.wl-maz@kernel.org> From: Marc Zyngier To: Gavin Shan Cc: kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, peterx@redhat.com, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, catalin.marinas@arm.com, will@kernel.org, shuah@kernel.org, seanjc@google.com, drjones@redhat.com, dmatlack@google.com, bgardon@google.com, ricarkol@google.com, zhenyzha@redhat.com, shan.gavin@gmail.com Subject: Re: [PATCH v1 1/5] KVM: arm64: Enable ring-based dirty memory tracking In-Reply-To: <41fb5a1f-29a9-e6bb-9fab-4c83a2a8fce5@redhat.com> References: <20220819005601.198436-1-gshan@redhat.com> <20220819005601.198436-2-gshan@redhat.com> <87lerkwtm5.wl-maz@kernel.org> <41fb5a1f-29a9-e6bb-9fab-4c83a2a8fce5@redhat.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: gshan@redhat.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, peterx@redhat.com, pbonzini@redhat.com, corbet@lwn.net, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, oliver.upton@linux.dev, catalin.marinas@arm.com, will@kernel.org, shuah@kernel.org, seanjc@google.com, drjones@redhat.com, dmatlack@google.com, bgardon@google.com, ricarkol@google.com, zhenyzha@redhat.com, shan.gavin@gmail.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220822_144220_882404_78A1055B X-CRM114-Status: GOOD ( 57.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org SGkgR2F2aW4sCgpPbiBNb24sIDIyIEF1ZyAyMDIyIDAyOjU4OjIwICswMTAwLApHYXZpbiBTaGFu IDxnc2hhbkByZWRoYXQuY29tPiB3cm90ZToKPiAKPiBIaSBNYXJjLAo+IAo+IE9uIDgvMTkvMjIg NjowMCBQTSwgTWFyYyBaeW5naWVyIHdyb3RlOgo+ID4gT24gRnJpLCAxOSBBdWcgMjAyMiAwMTo1 NTo1NyArMDEwMCwKPiA+IEdhdmluIFNoYW4gPGdzaGFuQHJlZGhhdC5jb20+IHdyb3RlOgo+ID4+ IAo+ID4+IFRoZSByaW5nLWJhc2VkIGRpcnR5IG1lbW9yeSB0cmFja2luZyBoYXMgYmVlbiBhdmFp bGFibGUgYW5kIGVuYWJsZWQKPiA+PiBvbiB4ODYgZm9yIGEgd2hpbGUuIFRoZSBmZWF0dXJlIGlz IGJlbmVmaWNpYWwgd2hlbiB0aGUgbnVtYmVyIG9mCj4gPj4gZGlydHkgcGFnZXMgaXMgc21hbGwg aW4gYSBjaGVja3BvaW50aW5nIHN5c3RlbSBvciBsaXZlIG1pZ3JhdGlvbgo+ID4+IHNjZW5hcmlv LiBNb3JlIGRldGFpbHMgY2FuIGJlIGZvdW5kIGZyb20gZmIwNGExZWRkYjFhICgiS1ZNOiBYODY6 Cj4gPj4gSW1wbGVtZW50IHJpbmctYmFzZWQgZGlydHkgbWVtb3J5IHRyYWNraW5nIikuCj4gPj4g Cj4gPj4gVGhpcyBlbmFibGVzIHRoZSByaW5nLWJhc2VkIGRpcnR5IG1lbW9yeSB0cmFja2luZyBv biBBUk02NC4gSXQncwo+ID4+IG5vdGFibGUgdGhhdCBubyBleHRyYSByZXNlcnZlZCByaW5nIGVu dHJpZXMgYXJlIG5lZWRlZCBvbiBBUk02NAo+ID4+IGJlY2F1c2UgdGhlIGh1Z2UgcGFnZXMgYXJl IGFsd2F5cyBzcGxpdCBpbnRvIGJhc2UgcGFnZXMgd2hlbiBwYWdlCj4gPj4gZGlydHkgdHJhY2tp bmcgaXMgZW5hYmxlZC4KPiA+IAo+ID4gQ2FuIHlvdSBwbGVhc2UgZWxhYm9yYXRlIG9uIHRoaXM/ IEFkZGluZyBhIHBlci1DUFUgcmluZyBvZiBjb3Vyc2UKPiA+IHJlc3VsdHMgaW4gZXh0cmEgbWVt b3J5IGFsbG9jYXRpb24sIHNvIHRoZXJlIG11c3QgYmUgYSBzdWJ0bGUKPiA+IHg4Ni1zcGVjaWZp YyBkZXRhaWwgdGhhdCBJJ20gbm90IGF3YXJlIG9mLi4uCj4gPiAKPiAKPiBTdXJlLiBJIGd1ZXNz IGl0J3MgaGVscGZ1bCB0byBleHBsYWluIGhvdyBpdCB3b3JrcyBpbiBuZXh0IHJldmlzaW9uLgo+ IFNvbWV0aGluZyBsaWtlIGJlbG93Ogo+IAo+IFRoaXMgZW5hYmxlcyB0aGUgcmluZy1iYXNlZCBk aXJ0eSBtZW1vcnkgdHJhY2tpbmcgb24gQVJNNjQuIFRoZSBmZWF0dXJlCj4gaXMgZW5hYmxlZCBi eSBDT05GSUdfSEFWRV9LVk1fRElSVFlfUklORywgZGV0ZWN0ZWQgYW5kIGVuYWJsZWQgYnkKPiBD T05GSUdfSEFWRV9LVk1fRElSVFlfUklORy4gQSByaW5nIGJ1ZmZlciBpcyBjcmVhdGVkIG9uIGV2 ZXJ5IHZjcHUgYW5kCj4gZWFjaCBlbnRyeSBpcyBkZXNjcmliZWQgYnkgJ3N0cnVjdCBrdm1fZGly dHlfZ2ZuJy4gVGhlIHJpbmcgYnVmZmVyIGlzCj4gcHVzaGVkIGJ5IGhvc3Qgd2hlbiBwYWdlIGJl Y29tZXMgZGlydHkgYW5kIHB1bGxlZCBieSB1c2Vyc3BhY2UuIEEgdmNwdQo+IGV4aXQgaXMgZm9y Y2VkIHdoZW4gdGhlIHJpbmcgYnVmZmVyIGJlY29tZXMgZnVsbC4gVGhlIHJpbmcgYnVmZmVycyBv bgo+IGFsbCB2Y3B1cyBjYW4gYmUgcmVzZXQgYnkgaW9jdGwgY29tbWFuZCBLVk1fUkVTRVRfRElS VFlfUklOR1MuCj4gCj4gWWVzLCBJIHRoaW5rIHNvLiBBZGRpbmcgYSBwZXItQ1BVIHJpbmcgcmVz dWx0cyBpbiBleHRyYSBtZW1vcnkgYWxsb2NhdGlvbi4KPiBIb3dldmVyLCBpdCdzIGF2b2lkaW5n IHN5bmNocm9uaXphdGlvbiBhbW9uZyBtdWx0aXBsZSB2Y3B1cyB3aGVuIGRpcnR5Cj4gcGFnZXMg aGFwcGVuIG9uIG11bHRpcGxlIHZjcHVzLiBNb3JlIGRpc2N1c3Npb24gY2FuIGJlIGZvdW5kIGZy b20gWzFdCgpPaCwgSSB0b3RhbGx5IGJ1eSB0aGUgcmVsYXhhdGlvbiBvZiB0aGUgc3luY2hyb25p c2F0aW9uICh0aG91Z2ggSQpkb3VidCB0aGlzIHdpbGwgaGF2ZSBhbnkgdmlzaWJsZSBlZmZlY3Qg dW50aWwgd2UgaGF2ZSBzb21ldGhpbmcgbGlrZQpPbGl2ZXIncyBwYXRjaGVzIHRvIGFsbG93IHBh cmFsbGVsIGZhdWx0aW5nKS4KCkJ1dCBpdCBpcyB0aGUgIm5vIGV4dHJhIHJlc2VydmVkIHJpbmcg ZW50cmllcyBhcmUgbmVlZGVkIG9uIEFSTTY0Igphcmd1bWVudCB0aGF0IEkgZG9uJ3QgZ2V0IHll dC4KCgo+Cj4gWzFdIGh0dHBzOi8vcGF0Y2h3b3JrLmtlcm5lbC5vcmcvcHJvamVjdC9rdm0vcGF0 Y2gvQkwyUFIwOE1CNDgxMkY5MjlBMjc2MEJDNDBFQTc1N0NGMDYzMEBCTDJQUjA4TUI0ODEubmFt cHJkMDgucHJvZC5vdXRsb29rLmNvbS8KPiAoY29tbWVudCM4IGZyb20gUmFkaW0gS3LEjW3DocWZ IG9uIE1heSAzLCAyMDE2LCAyOjExIHAubS4gVVRDKQo+IAo+IAo+ID4+IAo+ID4+IFNpZ25lZC1v ZmYtYnk6IEdhdmluIFNoYW4gPGdzaGFuQHJlZGhhdC5jb20+Cj4gPj4gLS0tCj4gPj4gICBEb2N1 bWVudGF0aW9uL3ZpcnQva3ZtL2FwaS5yc3QgICAgfCAyICstCj4gPj4gICBhcmNoL2FybTY0L2lu Y2x1ZGUvdWFwaS9hc20va3ZtLmggfCAxICsKPiA+PiAgIGFyY2gvYXJtNjQva3ZtL0tjb25maWcg ICAgICAgICAgICB8IDEgKwo+ID4+ICAgYXJjaC9hcm02NC9rdm0vYXJtLmMgICAgICAgICAgICAg IHwgOCArKysrKysrKwo+ID4+ICAgNCBmaWxlcyBjaGFuZ2VkLCAxMSBpbnNlcnRpb25zKCspLCAx IGRlbGV0aW9uKC0pCj4gPj4gCj4gPj4gZGlmZiAtLWdpdCBhL0RvY3VtZW50YXRpb24vdmlydC9r dm0vYXBpLnJzdCBiL0RvY3VtZW50YXRpb24vdmlydC9rdm0vYXBpLnJzdAo+ID4+IGluZGV4IGFi ZDdjMzIxMjZjZS4uMTlmYTFhYzAxN2VkIDEwMDY0NAo+ID4+IC0tLSBhL0RvY3VtZW50YXRpb24v dmlydC9rdm0vYXBpLnJzdAo+ID4+ICsrKyBiL0RvY3VtZW50YXRpb24vdmlydC9rdm0vYXBpLnJz dAo+ID4+IEBAIC04MDIyLDcgKzgwMjIsNyBAQCByZWdhcmRsZXNzIG9mIHdoYXQgaGFzIGFjdHVh bGx5IGJlZW4gZXhwb3NlZCB0aHJvdWdoIHRoZSBDUFVJRCBsZWFmLgo+ID4+ICAgOC4yOSBLVk1f Q0FQX0RJUlRZX0xPR19SSU5HCj4gPj4gICAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KPiA+ PiAgIC06QXJjaGl0ZWN0dXJlczogeDg2Cj4gPj4gKzpBcmNoaXRlY3R1cmVzOiB4ODYsIGFybTY0 Cj4gPj4gICA6UGFyYW1ldGVyczogYXJnc1swXSAtIHNpemUgb2YgdGhlIGRpcnR5IGxvZyByaW5n Cj4gPj4gICAgIEtWTSBpcyBjYXBhYmxlIG9mIHRyYWNraW5nIGRpcnR5IG1lbW9yeSB1c2luZyBy aW5nIGJ1ZmZlcnMgdGhhdAo+ID4+IGFyZQo+ID4+IGRpZmYgLS1naXQgYS9hcmNoL2FybTY0L2lu Y2x1ZGUvdWFwaS9hc20va3ZtLmggYi9hcmNoL2FybTY0L2luY2x1ZGUvdWFwaS9hc20va3ZtLmgK PiA+PiBpbmRleCAzYmIxMzQzNTU4NzQuLjdlMDRiMGI4ZDJiMiAxMDA2NDQKPiA+PiAtLS0gYS9h cmNoL2FybTY0L2luY2x1ZGUvdWFwaS9hc20va3ZtLmgKPiA+PiArKysgYi9hcmNoL2FybTY0L2lu Y2x1ZGUvdWFwaS9hc20va3ZtLmgKPiA+PiBAQCAtNDMsNiArNDMsNyBAQAo+ID4+ICAgI2RlZmlu ZSBfX0tWTV9IQVZFX1ZDUFVfRVZFTlRTCj4gPj4gICAgICNkZWZpbmUgS1ZNX0NPQUxFU0NFRF9N TUlPX1BBR0VfT0ZGU0VUIDEKPiA+PiArI2RlZmluZSBLVk1fRElSVFlfTE9HX1BBR0VfT0ZGU0VU IDY0Cj4gPiAKPiA+IEZvciBjb250ZXh0LCB0aGUgZG9jdW1lbnRhdGlvbiBzYXlzOgo+ID4gCj4g PiA8cXVvdGU+Cj4gPiAtIGlmIEtWTV9DQVBfRElSVFlfTE9HX1JJTkcgaXMgYXZhaWxhYmxlLCBh IG51bWJlciBvZiBwYWdlcyBhdAo+ID4gICAgS1ZNX0RJUlRZX0xPR19QQUdFX09GRlNFVCAqIFBB R0VfU0laRS4gWy4uLl0KPiA+IDwvcXVvdGU+Cj4gPiAKPiA+IFdoYXQgaXMgdGhlIHJlYXNvbiBm b3IgcGlja2luZyB0aGlzIHBhcnRpY3VsYXIgdmFsdWU/Cj4gPiAKPiAKPiBJdCdzIGluaGVyaXRl ZCBmcm9tIHg4Ni4gSSBkb24ndCB0aGluayBpdCBoYXMgdG8gYmUgdGhpcyBwYXJ0aWN1bGFyCj4g dmFsdWUuICBUaGUgdmFsdWUgaXMgdXNlZCB0byBkaXN0aW5ndWlzaCB0aGUgcmVnaW9uJ3Mgb3du ZXJzIGxpa2UKPiBrdm1fcnVuLCBLVk1fUElPX1BBR0VfT0ZGU0VULCBLVk1fQ09BTEVTQ0VEX01N SU9fUEFHRV9PRkZTRVQsIGFuZAo+IEtWTV9ESVJUWV9MT0dfUEFHRV9PRkZTRVQuCj4gCj4gSG93 IGFib3V0IHRvIGhhdmUgMiBmb3IgS1ZNX0RJUlRZX0xPR19QQUdFX09GRlNFVCBpbiBuZXh0IHJl dmlzaW9uPwo+IFRoZSB2aXJ0dWFsIGFyZWEgaXMgY2hlYXAsIEkgZ3Vlc3MgaXQncyBhbHNvIG5p Y2UgdG8gdXNlIHg4NidzCj4gcGF0dGVybiB0byBoYXZlIDY0IGZvciBLVk1fRElSVFlfTE9HX1BB R0VfT0ZGU0VULgo+IAo+ICAgICAjZGVmaW5lIEtWTV9DT0FMRVNDRURfTU1JT19QQUdFX09GRlNF VCAgIDEKPiAgICAgI2RlZmluZSBLVk1fRElSVFlfTE9HX1BBR0VfT0ZGU0VUICAgICAgICAyCgpH aXZlbiB0aGF0IHRoaXMgaXMganVzdCBhbiBvZmZzZXQgaW4gdGhlIHZjcHUgImZpbGUiLCBJIGRv bid0IHRoaW5rIGl0Cm1hdHRlcnMgdGhhdCBtdWNoLiA2NCBkZWZpbml0ZWx5IGFsbG93cyBmb3Ig c29tZSBzdHJ1Y3QgdmNwdSBncm93dGgsCmFuZCBpdCBkb2Vzbid0IGh1cnQgdG8gYmUgY29tcGF0 aWJsZSB3aXRoIHg4NiAoZm9yIG9uY2UuLi4pLgoKPiAKPiA+PiAgICAgI2RlZmluZSBLVk1fUkVH X1NJWkUoaWQpCj4gPj4gXAo+ID4+ICAgCSgxVSA8PCAoKChpZCkgJiBLVk1fUkVHX1NJWkVfTUFT SykgPj4gS1ZNX1JFR19TSVpFX1NISUZUKSkKPiA+PiBkaWZmIC0tZ2l0IGEvYXJjaC9hcm02NC9r dm0vS2NvbmZpZyBiL2FyY2gvYXJtNjQva3ZtL0tjb25maWcKPiA+PiBpbmRleCA4MTVjYzExOGM2 NzUuLjAzMDliMmQwZjJkYSAxMDA2NDQKPiA+PiAtLS0gYS9hcmNoL2FybTY0L2t2bS9LY29uZmln Cj4gPj4gKysrIGIvYXJjaC9hcm02NC9rdm0vS2NvbmZpZwo+ID4+IEBAIC0zMiw2ICszMiw3IEBA IG1lbnVjb25maWcgS1ZNCj4gPj4gICAJc2VsZWN0IEtWTV9WRklPCj4gPj4gICAJc2VsZWN0IEhB VkVfS1ZNX0VWRU5URkQKPiA+PiAgIAlzZWxlY3QgSEFWRV9LVk1fSVJRRkQKPiA+PiArCXNlbGVj dCBIQVZFX0tWTV9ESVJUWV9SSU5HCj4gPj4gICAJc2VsZWN0IEhBVkVfS1ZNX01TSQo+ID4+ICAg CXNlbGVjdCBIQVZFX0tWTV9JUlFDSElQCj4gPj4gICAJc2VsZWN0IEhBVkVfS1ZNX0lSUV9ST1VU SU5HCj4gPj4gZGlmZiAtLWdpdCBhL2FyY2gvYXJtNjQva3ZtL2FybS5jIGIvYXJjaC9hcm02NC9r dm0vYXJtLmMKPiA+PiBpbmRleCA5ODZjZWU2ZmJjN2YuLjNkZTZiOWIzOWRiNyAxMDA2NDQKPiA+ PiAtLS0gYS9hcmNoL2FybTY0L2t2bS9hcm0uYwo+ID4+ICsrKyBiL2FyY2gvYXJtNjQva3ZtL2Fy bS5jCj4gPj4gQEAgLTg2Niw2ICs4NjYsMTQgQEAgaW50IGt2bV9hcmNoX3ZjcHVfaW9jdGxfcnVu KHN0cnVjdCBrdm1fdmNwdSAqdmNwdSkKPiA+PiAgIAkJaWYgKCFyZXQpCj4gPj4gICAJCQlyZXQg PSAxOwo+ID4+ICAgKwkJLyogRm9yY2UgdmNwdSBleGl0IGlmIGl0cyBkaXJ0eSByaW5nIGlzIHNv ZnQtZnVsbCAqLwo+ID4+ICsJCWlmICh1bmxpa2VseSh2Y3B1LT5rdm0tPmRpcnR5X3Jpbmdfc2l6 ZSAmJgo+ID4+ICsJCQkgICAgIGt2bV9kaXJ0eV9yaW5nX3NvZnRfZnVsbCgmdmNwdS0+ZGlydHlf cmluZykpKSB7Cj4gPj4gKwkJCXZjcHUtPnJ1bi0+ZXhpdF9yZWFzb24gPSBLVk1fRVhJVF9ESVJU WV9SSU5HX0ZVTEw7Cj4gPj4gKwkJCXRyYWNlX2t2bV9kaXJ0eV9yaW5nX2V4aXQodmNwdSk7Cj4g Pj4gKwkJCXJldCA9IDA7Cj4gPj4gKwkJfQo+ID4+ICsKPiA+IAo+ID4gV2h5IGNhbid0IHRoaXMg YmUgbW92ZWQgdG8ga3ZtX3ZjcHVfZXhpdF9yZXF1ZXN0KCkgaW5zdGVhZD8gSSB3b3VsZAo+ID4g YWxzbyB2ZXJ5IG11Y2ggbGlrZSB0aGUgY2hlY2sgdG8gYmUgbWFkZSBhIGNvbW1vbiBoZWxwZXIg d2l0aCB4ODYuCj4gPiAKPiA+IEEgc2VlbWluZ2x5IGFwcHJvYWNoIHdvdWxkIGJlIHRvIG1ha2Ug dGhpcyBhIHJlcXVlc3Qgb24gZGlydHkgbG9nCj4gPiBpbnNlcnRpb24sIGFuZCBhdm9pZCB0aGUg d2hvbGUgImNoZWNrIHRoZSBsb2cgc2l6ZSIgb24gZXZlcnkgcnVuLAo+ID4gd2hpY2ggYWRkcyBw b2ludGxlc3Mgb3ZlcmhlYWQgdG8gdW5zdXNwZWN0aW5nIHVzZXJzIChha2EgZXZlcnlvbmUpLgo+ ID4gCj4gCj4gSSB0aG91Z2ggb2YgaGF2aW5nIHRoZSBjaGVjayBpbiBrdm1fdmNwdV9leGl0X3Jl cXVlc3QoKS4gVGhlIHZhcmlvdXMKPiBleGl0IHJlYXNvbnMgYXJlIHByaW9yaXRpemVkLiB4ODYg Z2l2ZXMgS1ZNX0VYSVRfRElSVFlfUklOR19GVUxMIHRoZQo+IGhpZ2hlc3QgcHJpb3JpdHkgYW5k IEFSTTY0IGlzIGp1c3QgdG8gZm9sbG93LiBJIGRvbid0IHRoaW5rIGl0IHJlYWxseQo+IG1hdHRl cnMuIEkgd2lsbCBpbXByb3ZlIGl0IGFjY29yZGluZ2x5IGluIG5leHQgcmV2aXNpb246Cj4gCj4g LSBDaGFuZ2Uga3ZtX2RpcnR5X3Jpbmdfc29mdF9mdWxsKCkgdG8gc29tZXRoaW5nIGFzIGJlbG93 IGluIGRpcnR5X3JpbmcuYwo+IAo+ICAgYm9vbCBrdm1fZGlydHlfcmluZ19zb2Z0X2Z1bGwoc3Ry dWN0IGt2bV92Y3B1ICp2Y3B1KQo+ICAgewo+ICAgICAgICBzdHJ1Y3Qga3ZtICprdm0gPSB2Y3B1 LT52Y3B1Owo+ICAgICAgICBzdHJ1Y3Qga3ZtX2RpcnR5X3JpbmcgKnJpbmcgPSAmdmNwdS0+ZGly dHlfcmluZzsKPiAKPiAgICAgICAgaWYgKHVubGlrZWx5KGt2bS0+ZGlydHlfcmluZ19zaXplICYm Cj4gICAgICAgICAgICAgICAgICAgICBrdm1fZGlydHlfcmluZ191c2VkKHJpbmcpID49IHJpbmct PnNvZnRfbGltaXQpKSB7Cj4gICAgICAgICAgICB2Y3B1LT5ydW4tPmV4aXRfcmVhc29uID0gS1ZN X0VYSVRfRElSVFlfUklOR19GVUxMOwo+ICAgICAgICAgICAgdHJhY2Vfa3ZtX2RpcnR5X3Jpbmdf ZXhpdCh2Y3B1KTsKPiAgICAgICAgICAgIHJldHVybiB0cnVlOwo+ICAgICAgICB9Cj4gCj4gICAg ICAgIHJldHVybiBmYWxzZTsKPiAgIH0KPiAKPiAtIFVzZSB0aGUgbW9kaWZpZWQga3ZtX2RpcnR5 X3Jpbmdfc29mdF9mdWxsKCkgaW4ga3ZtX3ZjcHVfZXhpdF9yZXF1ZXN0KCkuCj4gCj4gVXNlcnNw YWNlIG5lZWRzIEtWTV9FWElUX0RJUlRZX1JJTkdfRlVMTCB0byBjb2xsZWN0IHRoZSBkaXJ0eSBs b2cgaW4gdGltZS4KPiBPdGhlcndpc2UsIHRoZSBkaXJ0eSBsb2cgaW4gdGhlIHJpbmcgYnVmZmVy IHdpbGwgYmUgb3ZlcndyaXR0ZW4uIEknbSBub3QKPiBzdXJlIGlmIGFueXRoaW5nIGVsc2UgSSBt aXNzZWQ/CgpJJ20gZmluZSB3aXRoIHRoZSBhYm92ZSwgYnV0IHdoYXQgSSByZWFsbHkgd2FudGVk IGlzIGEgcmVxdWVzdCBmcm9tCnRoZSBkaXJ0eS1yaW5nIGluc2VydGlvbiwgaW5zdGVhZCBvZiBh IGNoZWNrIGluIGt2bV92cGN1X2V4aXRfcmVxdWVzdC4KU29tZXRoaW5nIGxpa2UgdGhpcyAod2hp Y2ggb2J2aW91c2x5IGRvZXNuJ3QgY29tcGlsZSwgYnV0IHlvdSdsbCBnZXQKdGhlIGlkZWEpOgoK ZGlmZiAtLWdpdCBhL2FyY2gvYXJtNjQva3ZtL2FybS5jIGIvYXJjaC9hcm02NC9rdm0vYXJtLmMK aW5kZXggOTg2Y2VlNmZiYzdmLi4wYjQxZmViNmZiN2QgMTAwNjQ0Ci0tLSBhL2FyY2gvYXJtNjQv a3ZtL2FybS5jCisrKyBiL2FyY2gvYXJtNjQva3ZtL2FybS5jCkBAIC03NDcsNiArNzQ3LDEyIEBA IHN0YXRpYyBpbnQgY2hlY2tfdmNwdV9yZXF1ZXN0cyhzdHJ1Y3Qga3ZtX3ZjcHUgKnZjcHUpCiAK IAkJaWYgKGt2bV9jaGVja19yZXF1ZXN0KEtWTV9SRVFfU1VTUEVORCwgdmNwdSkpCiAJCQlyZXR1 cm4ga3ZtX3ZjcHVfc3VzcGVuZCh2Y3B1KTsKKworCQlpZiAoa3ZtX2NoZWNrX3JlcXVlc3QoS1ZN X1JFUV9SSU5HX1NPRlRfRlVMTCwgdmNwdSkpIHsKKwkJCXZjcHUtPnJ1bi0+ZXhpdF9yZWFzb24g PSBLVk1fRVhJVF9ESVJUWV9SSU5HX0ZVTEw7CisJCQl0cmFjZV9rdm1fZGlydHlfcmluZ19leGl0 KHZjcHUpOworCQkJcmV0dXJuIDA7CisJCX0KIAl9CiAKIAlyZXR1cm4gMTsKZGlmZiAtLWdpdCBh L3ZpcnQva3ZtL2RpcnR5X3JpbmcuYyBiL3ZpcnQva3ZtL2RpcnR5X3JpbmcuYwppbmRleCBmNGMy YTZlYjE2NjYuLjA4YjJmMDExNjRmYSAxMDA2NDQKLS0tIGEvdmlydC9rdm0vZGlydHlfcmluZy5j CisrKyBiL3ZpcnQva3ZtL2RpcnR5X3JpbmcuYwpAQCAtMTQ5LDYgKzE0OSw3IEBAIGludCBrdm1f ZGlydHlfcmluZ19yZXNldChzdHJ1Y3Qga3ZtICprdm0sIHN0cnVjdCBrdm1fZGlydHlfcmluZyAq cmluZykKIAogdm9pZCBrdm1fZGlydHlfcmluZ19wdXNoKHN0cnVjdCBrdm1fZGlydHlfcmluZyAq cmluZywgdTMyIHNsb3QsIHU2NCBvZmZzZXQpCiB7CisJc3RydWN0IGt2bV92Y3B1ICp2Y3B1ID0g Y29udGFpbmVyX29mKHJpbmcsIHN0cnVjdCBrdm1fdmNwdSwgZGlydHlfcmluZyk7CiAJc3RydWN0 IGt2bV9kaXJ0eV9nZm4gKmVudHJ5OwogCiAJLyogSXQgc2hvdWxkIG5ldmVyIGdldCBmdWxsICov CkBAIC0xNjYsNiArMTY3LDkgQEAgdm9pZCBrdm1fZGlydHlfcmluZ19wdXNoKHN0cnVjdCBrdm1f ZGlydHlfcmluZyAqcmluZywgdTMyIHNsb3QsIHU2NCBvZmZzZXQpCiAJa3ZtX2RpcnR5X2dmbl9z ZXRfZGlydGllZChlbnRyeSk7CiAJcmluZy0+ZGlydHlfaW5kZXgrKzsKIAl0cmFjZV9rdm1fZGly dHlfcmluZ19wdXNoKHJpbmcsIHNsb3QsIG9mZnNldCk7CisKKwlpZiAoa3ZtX2RpcnR5X3Jpbmdf c29mdF9mdWxsKHZjcHUpKQorCQlrdm1fbWFrZV9yZXF1ZXN0KEtWTV9SRVFfUklOR19TT0ZUX0ZV TEwsIHZjcHUpOwogfQogCiBzdHJ1Y3QgcGFnZSAqa3ZtX2RpcnR5X3JpbmdfZ2V0X3BhZ2Uoc3Ry dWN0IGt2bV9kaXJ0eV9yaW5nICpyaW5nLCB1MzIgb2Zmc2V0KQoKVGhhbmtzLAoKCU0uCgotLSAK V2l0aG91dCBkZXZpYXRpb24gZnJvbSB0aGUgbm9ybSwgcHJvZ3Jlc3MgaXMgbm90IHBvc3NpYmxl LgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgt YXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQu b3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJt LWtlcm5lbAo=