From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from out-173.mta1.migadu.com (out-173.mta1.migadu.com [95.215.58.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 37D3C13D271 for ; Fri, 19 Apr 2024 22:23:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.173 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713565435; cv=none; b=JIzP6yBXFwpsqDLReQmvppMKmzOqtNIdCbvI+/RT5AMhejqFEHpbIsFgD9QgQtfB6tcvA7LRuisxi3FeBdxQ5Q/sDGx6Tjfesw0diKzsgzGQWRAg0z8X7IvDrKQZtzLBYYth4pxi/4DLVn8GEMr/uCycved1CF6fC9FXXbzB7nc= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713565435; c=relaxed/simple; bh=VD6QFxX1Xd3DNVIOsaev//o7h4mZlv5eWXjCnxaWhik=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=RIWnAhOqAqQuFeJLiddMhB5jRGHDhk4MICws/69LPiYiIV7slrK4WhAhIj+RWftB9YCupS2rgpOnzGrqvKW3FSCa/RPjuZYF0ZQb0tw6izdDlVj0DHNsVE8c0a3yeKGPUkeWnQdM63x7iPO+pKF7cJV+ijyCqTyAjKq9bSk9OK4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=P8isnNkf; arc=none smtp.client-ip=95.215.58.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="P8isnNkf" Date: Fri, 19 Apr 2024 22:23:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1713565432; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mthBWOOhcNSWoMZhMTAfzR7XGep2Qks53Rpg12wvEhs=; b=P8isnNkfBYXWB0pGZ4e76KozKbuIPo6rHG7Ox+d9b31tlskAlwzX5Fiy922x3848sQHdds 2boqaE4bF1mMbPeod2X267+67KV2A02qKBDa6v3fbd7lZbgPblznjJvZe8hzyomSxDd6nH 88u9gjkSLzKAR6O3WgIDpbW6x4ZMNao= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: James Houghton Cc: David Matlack , Andrew Morton , Paolo Bonzini , Yu Zhao , Marc Zyngier , Sean Christopherson , Jonathan Corbet , James Morse , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Shaoqin Huang , Gavin Shan , Ricardo Koller , Raghavendra Rao Ananta , Ryan Roberts , David Rientjes , Axel Rasmussen , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org Subject: Re: [PATCH v3 0/7] mm/kvm: Improve parallelism for access bit harvesting Message-ID: References: <20240401232946.1837665-1-jthoughton@google.com> Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Migadu-Flow: FLOW_OUT On Fri, Apr 19, 2024 at 01:57:03PM -0700, James Houghton wrote: > On Fri, Apr 12, 2024 at 11:41 AM David Matlack wrote: > > > > On 2024-04-01 11:29 PM, James Houghton wrote: > > > This patchset adds a fast path in KVM to test and clear access bits on > > > sptes without taking the mmu_lock. It also adds support for using a > > > bitmap to (1) test the access bits for many sptes in a single call to > > > mmu_notifier_test_young, and to (2) clear the access bits for many ptes > > > in a single call to mmu_notifier_clear_young. > > > > How much improvement would we get if we _just_ made test/clear_young > > lockless on x86 and hold the read-lock on arm64? And then how much > > benefit does the bitmap look-around add on top of that? Thanks David for providing the suggestion. > I don't have these results right now. For the next version I will (1) > separate the series into the locking change and the bitmap change, and > I will (2) have performance data for each change separately. It is > conceivable that the bitmap change should just be considered as a > completely separate patchset. That'd be great. Having the performance numbers will make it even more compelling, but I'd be tempted to go for the lock improvement just because it doesn't add any new complexity and leverages existing patterns in the architectures that people seem to want improvements for. The bitmap interface, OTOH, is rather complex. At least the current implementation breaks some of the isolation we have between the MMU code and the page table walker library on arm64, which I'm not ecstatic about. It _could_ be justified by a massive performance uplift over locking, but it'd have to be a sizable win. -- Thanks, Oliver From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69FABC4345F for ; Fri, 19 Apr 2024 22:24:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=uMXzdfqvKpFtgC7hcowB1D+BUouyPG7FPZA3+85QE+c=; b=lQffrBIHwXUNiP 8o68vzDx7jVA+oJk6hVdl6X06sltLQtYgm22vrkzbkIOTvMorcQLud1tUeRolieMsk1W/DKW2BQN2 AjCv+CACLYbzg9LxlOMPnAJ+3jnOdvREaBA6ZKHQbs6cUtOzlLJsMM1mAu9mUWVfFF386xxttxsLH XLRb8+d1IFUeyFIHI2lPrQ3DKqeY1tpgUcoas5AnJNdByelvPG8zkyMoP7LtNef0OiiE2KCR4fDsh 85fJ/K/JscqwCcBVO4Jg2LLkj6Z/Yjd9BepOxMCepIZ+okNfHldDJfx24OxPJkr34u2zsmq7NG75c laYtcmBNWVwdxUddmkuA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rxweL-000000078Fp-3bem; Fri, 19 Apr 2024 22:24:05 +0000 Received: from out-188.mta1.migadu.com ([95.215.58.188]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rxweJ-000000078DC-06wx for linux-arm-kernel@lists.infradead.org; Fri, 19 Apr 2024 22:24:04 +0000 Date: Fri, 19 Apr 2024 22:23:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1713565432; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mthBWOOhcNSWoMZhMTAfzR7XGep2Qks53Rpg12wvEhs=; b=P8isnNkfBYXWB0pGZ4e76KozKbuIPo6rHG7Ox+d9b31tlskAlwzX5Fiy922x3848sQHdds 2boqaE4bF1mMbPeod2X267+67KV2A02qKBDa6v3fbd7lZbgPblznjJvZe8hzyomSxDd6nH 88u9gjkSLzKAR6O3WgIDpbW6x4ZMNao= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: James Houghton Cc: David Matlack , Andrew Morton , Paolo Bonzini , Yu Zhao , Marc Zyngier , Sean Christopherson , Jonathan Corbet , James Morse , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Shaoqin Huang , Gavin Shan , Ricardo Koller , Raghavendra Rao Ananta , Ryan Roberts , David Rientjes , Axel Rasmussen , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org Subject: Re: [PATCH v3 0/7] mm/kvm: Improve parallelism for access bit harvesting Message-ID: References: <20240401232946.1837665-1-jthoughton@google.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240419_152403_247394_4AA73F09 X-CRM114-Status: GOOD ( 24.50 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gRnJpLCBBcHIgMTksIDIwMjQgYXQgMDE6NTc6MDNQTSAtMDcwMCwgSmFtZXMgSG91Z2h0b24g d3JvdGU6Cj4gT24gRnJpLCBBcHIgMTIsIDIwMjQgYXQgMTE6NDHigK9BTSBEYXZpZCBNYXRsYWNr IDxkbWF0bGFja0Bnb29nbGUuY29tPiB3cm90ZToKPiA+Cj4gPiBPbiAyMDI0LTA0LTAxIDExOjI5 IFBNLCBKYW1lcyBIb3VnaHRvbiB3cm90ZToKPiA+ID4gVGhpcyBwYXRjaHNldCBhZGRzIGEgZmFz dCBwYXRoIGluIEtWTSB0byB0ZXN0IGFuZCBjbGVhciBhY2Nlc3MgYml0cyBvbgo+ID4gPiBzcHRl cyB3aXRob3V0IHRha2luZyB0aGUgbW11X2xvY2suIEl0IGFsc28gYWRkcyBzdXBwb3J0IGZvciB1 c2luZyBhCj4gPiA+IGJpdG1hcCB0byAoMSkgdGVzdCB0aGUgYWNjZXNzIGJpdHMgZm9yIG1hbnkg c3B0ZXMgaW4gYSBzaW5nbGUgY2FsbCB0bwo+ID4gPiBtbXVfbm90aWZpZXJfdGVzdF95b3VuZywg YW5kIHRvICgyKSBjbGVhciB0aGUgYWNjZXNzIGJpdHMgZm9yIG1hbnkgcHRlcwo+ID4gPiBpbiBh IHNpbmdsZSBjYWxsIHRvIG1tdV9ub3RpZmllcl9jbGVhcl95b3VuZy4KPiA+Cj4gPiBIb3cgbXVj aCBpbXByb3ZlbWVudCB3b3VsZCB3ZSBnZXQgaWYgd2UgX2p1c3RfIG1hZGUgdGVzdC9jbGVhcl95 b3VuZwo+ID4gbG9ja2xlc3Mgb24geDg2IGFuZCBob2xkIHRoZSByZWFkLWxvY2sgb24gYXJtNjQ/ IEFuZCB0aGVuIGhvdyBtdWNoCj4gPiBiZW5lZml0IGRvZXMgdGhlIGJpdG1hcCBsb29rLWFyb3Vu ZCBhZGQgb24gdG9wIG9mIHRoYXQ/CgpUaGFua3MgRGF2aWQgZm9yIHByb3ZpZGluZyB0aGUgc3Vn Z2VzdGlvbi4KCj4gSSBkb24ndCBoYXZlIHRoZXNlIHJlc3VsdHMgcmlnaHQgbm93LiBGb3IgdGhl IG5leHQgdmVyc2lvbiBJIHdpbGwgKDEpCj4gc2VwYXJhdGUgdGhlIHNlcmllcyBpbnRvIHRoZSBs b2NraW5nIGNoYW5nZSBhbmQgdGhlIGJpdG1hcCBjaGFuZ2UsIGFuZAo+IEkgd2lsbCAoMikgaGF2 ZSBwZXJmb3JtYW5jZSBkYXRhIGZvciBlYWNoIGNoYW5nZSBzZXBhcmF0ZWx5LiBJdCBpcwo+IGNv bmNlaXZhYmxlIHRoYXQgdGhlIGJpdG1hcCBjaGFuZ2Ugc2hvdWxkIGp1c3QgYmUgY29uc2lkZXJl ZCBhcyBhCj4gY29tcGxldGVseSBzZXBhcmF0ZSBwYXRjaHNldC4KClRoYXQnZCBiZSBncmVhdC4g SGF2aW5nIHRoZSBwZXJmb3JtYW5jZSBudW1iZXJzIHdpbGwgbWFrZSBpdCBldmVuIG1vcmUKY29t cGVsbGluZywgYnV0IEknZCBiZSB0ZW1wdGVkIHRvIGdvIGZvciB0aGUgbG9jayBpbXByb3ZlbWVu dCBqdXN0CmJlY2F1c2UgaXQgZG9lc24ndCBhZGQgYW55IG5ldyBjb21wbGV4aXR5IGFuZCBsZXZl cmFnZXMgZXhpc3RpbmcgcGF0dGVybnMKaW4gdGhlIGFyY2hpdGVjdHVyZXMgdGhhdCBwZW9wbGUg c2VlbSB0byB3YW50IGltcHJvdmVtZW50cyBmb3IuCgpUaGUgYml0bWFwIGludGVyZmFjZSwgT1RP SCwgaXMgcmF0aGVyIGNvbXBsZXguIEF0IGxlYXN0IHRoZSBjdXJyZW50CmltcGxlbWVudGF0aW9u IGJyZWFrcyBzb21lIG9mIHRoZSBpc29sYXRpb24gd2UgaGF2ZSBiZXR3ZWVuIHRoZSBNTVUgY29k ZQphbmQgdGhlIHBhZ2UgdGFibGUgd2Fsa2VyIGxpYnJhcnkgb24gYXJtNjQsIHdoaWNoIEknbSBu b3QgZWNzdGF0aWMgYWJvdXQuCkl0IF9jb3VsZF8gYmUganVzdGlmaWVkIGJ5IGEgbWFzc2l2ZSBw ZXJmb3JtYW5jZSB1cGxpZnQgb3ZlciBsb2NraW5nLCBidXQKaXQnZCBoYXZlIHRvIGJlIGEgc2l6 YWJsZSB3aW4uCgotLSAKVGhhbmtzLApPbGl2ZXIKCl9fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0CmxpbnV4 LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9y Zy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK