From mboxrd@z Thu Jan 1 00:00:00 1970 From: Waiman Long Subject: Re: [PATCH v10 03/19] qspinlock: Add pending bit Date: Tue, 13 May 2014 15:47:00 -0400 Message-ID: <537276B4.10209@hp.com> References: <1399474907-22206-1-git-send-email-Waiman.Long@hp.com> <1399474907-22206-4-git-send-email-Waiman.Long@hp.com> <20140512152208.GA12309@potion.brq.redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20140512152208.GA12309@potion.brq.redhat.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org To: =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= Cc: x86@kernel.org, Gleb Natapov , Peter Zijlstra , linux-kernel@vger.kernel.org, "H. Peter Anvin" , Boris Ostrovsky , linux-arch@vger.kernel.org, kvm@vger.kernel.org, Raghavendra K T , Ingo Molnar , xen-devel@lists.xenproject.org, "Paul E. McKenney" , Rik van Riel , Konrad Rzeszutek Wilk , Scott J Norton , Paolo Bonzini , Thomas Gleixner , virtualization@lists.linux-foundation.org, Chegu Vinod , Oleg Nesterov , David Vrabel , Linus Torvalds List-Id: linux-arch.vger.kernel.org T24gMDUvMTIvMjAxNCAxMToyMiBBTSwgUmFkaW0gS3LEjW3DocWZIHdyb3RlOgo+IDIwMTQtMDUt MDcgMTE6MDEtMDQwMCwgV2FpbWFuIExvbmc6Cj4+IEZyb206IFBldGVyIFppamxzdHJhPHBldGVy ekBpbmZyYWRlYWQub3JnPgo+Pgo+PiBCZWNhdXNlIHRoZSBxc3BpbmxvY2sgbmVlZHMgdG8gdG91 Y2ggYSBzZWNvbmQgY2FjaGVsaW5lOyBhZGQgYSBwZW5kaW5nCj4+IGJpdCBhbmQgYWxsb3cgYSBz aW5nbGUgaW4td29yZCBzcGlubmVyIGJlZm9yZSB3ZSBwdW50IHRvIHRoZSBzZWNvbmQKPj4gY2Fj aGVsaW5lLgo+IEkgdGhpbmsgdGhlcmUgaXMgYW4gdW53YW50ZWQgc2NlbmFyaW8gb24gdmlydHVh bCBtYWNoaW5lczoKPiAxKSBWQ1BVIHNldHMgdGhlIHBlbmRpbmcgYml0IGFuZCBzdGFydCBzcGlu bmluZy4KPiAyKSBQZW5kaW5nIFZDUFUgZ2V0cyBkZXNjaGVkdWxlZC4KPiAgICAgIC0gd2UgaGF2 ZSBQTEUgYW5kIGxvY2sgaG9sZGVyIGlzbid0IHJ1bm5pbmcgWzFdCj4gICAgICAtIHRoZSBoeXBl cnZpc29yIHJhbmRvbWx5IHByZWVtcHRzIHVzCj4gMykgTG9jayBob2xkZXIgdW5sb2NrcyB3aGls ZSBwZW5kaW5nIFZDUFUgaXMgd2FpdGluZyBpbiBxdWV1ZS4KPiA0KSBTdWJzZXF1ZW50IGxvY2tl cnMgd2lsbCBzZWUgZnJlZSBsb2NrIHdpdGggc2V0IHBlbmRpbmcgYml0IGFuZCB3aWxsCj4gICAg IGxvb3AgaW4gdHJ5bG9jaydzICdmb3IgKDs7KScKPiAgICAgIC0gdGhlIHdvcnN0LWNhc2UgaXMg bG9jayBzdGFydmluZyBbMl0KPiAgICAgIC0gUExFIGNhbiBzYXZlIHVzIGZyb20gd2FzdGluZyB3 aG9sZSB0aW1lc2xpY2UKPgo+IFJldHJ5IHRocmVzaG9sZCBpcyB0aGUgZWFzaWVzdCBzb2x1dGlv biwgcmVnYXJkbGVzcyBvZiBpdHMgdWdsaW5lc3MgWzRdLgoKWWVzLCB0aGF0IGNhbiBiZSBhIHJl YWwgaXNzdWUuIFNvbWUgc29ydCBvZiByZXRyeSB0aHJlc2hvbGQsIGFzIHlvdSAKc2FpZCwgc2hv dWxkIGJlIGFibGUgdG8gaGFuZGxlIGl0LgoKQlRXLCB0aGUgcmVsZXZhbnQgcGF0Y2ggc2hvdWxk IGJlIDE2LzE5IHdoZXJlIHRoZSBQViBzcGlubG9jayBzdHVmZiAKc2hvdWxkIGJlIGRpc2N1c3Nl ZC4gVGhpcyBwYXRjaCBpcyBwZXJmZWN0bHkgZmluZS4KCj4gQW5vdGhlciBtaW5vciBkZXNpZ24g ZmxhdyBpcyB0aGF0IGZvcm1lcmx5IGZpcnN0IFZDUFUgZ2V0cyBhcHBlbmRlZCB0bwo+IHRoZSB0 YWlsIHdoZW4gaXQgZGVjaWRlcyB0byBxdWV1ZTsKPiBpcyB0aGUgcGVyZm9ybWFuY2UgZ2FpbiB3 b3J0aCBpdD8KPgo+IFRoYW5rcy4KClllcywgdGhlIHBlcmZvcm1hbmNlIGdhaW4gaXMgd29ydGgg aXQuIFRoZSBwcmltYXJ5IGdvYWwgaXMgdG8gYmUgbm90IAp3b3JzZSB0aGFuIHRpY2tldCBzcGlu bG9jayBpbiBsaWdodCBsb2FkIHNpdHVhdGlvbiB3aGljaCBpcyB0aGUgbW9zdCAKY29tbW9uIGNh c2UuIFRoaXMgZmVhdHVyZSBpcyBuZWVkIHRvIGFjaGlldmUgdGhhdC4KCi1Mb25nbWFuCl9fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fClZpcnR1YWxpemF0aW9u IG1haWxpbmcgbGlzdApWaXJ0dWFsaXphdGlvbkBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpo dHRwczovL2xpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby92aXJ0dWFs aXphdGlvbg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from g2t2354.austin.hp.com ([15.217.128.53]:56407 "EHLO g2t2354.austin.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753245AbaEMTrI (ORCPT ); Tue, 13 May 2014 15:47:08 -0400 Message-ID: <537276B4.10209@hp.com> Date: Tue, 13 May 2014 15:47:00 -0400 From: Waiman Long MIME-Version: 1.0 Subject: Re: [PATCH v10 03/19] qspinlock: Add pending bit References: <1399474907-22206-1-git-send-email-Waiman.Long@hp.com> <1399474907-22206-4-git-send-email-Waiman.Long@hp.com> <20140512152208.GA12309@potion.brq.redhat.com> In-Reply-To: <20140512152208.GA12309@potion.brq.redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Sender: linux-arch-owner@vger.kernel.org List-ID: To: =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= Cc: Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Peter Zijlstra , linux-arch@vger.kernel.org, x86@kernel.org, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, xen-devel@lists.xenproject.org, kvm@vger.kernel.org, Paolo Bonzini , Konrad Rzeszutek Wilk , Boris Ostrovsky , "Paul E. McKenney" , Rik van Riel , Linus Torvalds , Raghavendra K T , David Vrabel , Oleg Nesterov , Gleb Natapov , Scott J Norton , Chegu Vinod Message-ID: <20140513194700.R9rWNb9R5Kcfns1Hc4LQFHKXaEANISy7CFGp_nu95co@z> On 05/12/2014 11:22 AM, Radim Krčmář wrote: > 2014-05-07 11:01-0400, Waiman Long: >> From: Peter Zijlstra >> >> Because the qspinlock needs to touch a second cacheline; add a pending >> bit and allow a single in-word spinner before we punt to the second >> cacheline. > I think there is an unwanted scenario on virtual machines: > 1) VCPU sets the pending bit and start spinning. > 2) Pending VCPU gets descheduled. > - we have PLE and lock holder isn't running [1] > - the hypervisor randomly preempts us > 3) Lock holder unlocks while pending VCPU is waiting in queue. > 4) Subsequent lockers will see free lock with set pending bit and will > loop in trylock's 'for (;;)' > - the worst-case is lock starving [2] > - PLE can save us from wasting whole timeslice > > Retry threshold is the easiest solution, regardless of its ugliness [4]. Yes, that can be a real issue. Some sort of retry threshold, as you said, should be able to handle it. BTW, the relevant patch should be 16/19 where the PV spinlock stuff should be discussed. This patch is perfectly fine. > Another minor design flaw is that formerly first VCPU gets appended to > the tail when it decides to queue; > is the performance gain worth it? > > Thanks. Yes, the performance gain is worth it. The primary goal is to be not worse than ticket spinlock in light load situation which is the most common case. This feature is need to achieve that. -Longman