From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Date: Wed, 06 Jul 2016 18:05:58 +0800 From: xinhui MIME-Version: 1.0 Subject: Re: [PATCH v2 0/4] implement vcpu preempted check References: <1467124991-13164-1-git-send-email-xinhui.pan@linux.vnet.ibm.com> <20160706065255.GH30909@twins.programming.kicks-ass.net> In-Reply-To: <20160706065255.GH30909@twins.programming.kicks-ass.net> Content-Type: text/plain; charset="utf-8"; format="flowed" Message-Id: <577CD806.8000008@linux.vnet.ibm.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , List-Archive: To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, virtualization@lists.linux-foundation.org, linux-s390@vger.kernel.org, mingo@redhat.com, mpe@ellerman.id.au, paulus@samba.org, benh@kernel.crashing.org, paulmck@linux.vnet.ibm.com, waiman.long@hpe.com, will.deacon@arm.com, boqun.feng@gmail.com, dave@stgolabs.net, schwidefsky@de.ibm.com, pbonzini@redhat.com List-ID: Content-Transfer-Encoding: 8bit On 2016年07月06日 14:52, Peter Zijlstra wrote: > On Tue, Jun 28, 2016 at 10:43:07AM -0400, Pan Xinhui wrote: >> change fomr v1: >> a simplier definition of default vcpu_is_preempted >> skip mahcine type check on ppc, and add config. remove dedicated macro. >> add one patch to drop overload of rwsem_spin_on_owner and mutex_spin_on_owner. >> add more comments >> thanks boqun and Peter's suggestion. >> >> This patch set aims to fix lock holder preemption issues. >> >> test-case: >> perf record -a perf bench sched messaging -g 400 -p && perf report >> >> 18.09% sched-messaging [kernel.vmlinux] [k] osq_lock >> 12.28% sched-messaging [kernel.vmlinux] [k] rwsem_spin_on_owner >> 5.27% sched-messaging [kernel.vmlinux] [k] mutex_unlock >> 3.89% sched-messaging [kernel.vmlinux] [k] wait_consider_task >> 3.64% sched-messaging [kernel.vmlinux] [k] _raw_write_lock_irq >> 3.41% sched-messaging [kernel.vmlinux] [k] mutex_spin_on_owner.is >> 2.49% sched-messaging [kernel.vmlinux] [k] system_call >> >> We introduce interface bool vcpu_is_preempted(int cpu) and use it in some spin >> loops of osq_lock, rwsem_spin_on_owner and mutex_spin_on_owner. >> These spin_on_onwer variant also cause rcu stall before we apply this patch set >> > > Paolo, could you help out with an (x86) KVM interface for this? > > Waiman, could you see if you can utilize this to get rid of the > SPIN_THRESHOLD in qspinlock_paravirt? > hmm. maybe something like below. wait_node can go into pv_wait() earlier as soon as the prev cpu is preempted. but for the wait_head, as qspinlock does not record the lock_holder correctly(thanks to lock stealing), vcpu preemption check might get wrong results. Waiman, I have used one hash table to keep the lock holder in my ppc implementation patch. I think we could do something similar in generic code? diff --git a/kernel/locking/qspinlock_paravirt.h b/kernel/locking/qspinlock_paravirt.h index 74c4a86..40560e8 100644 --- a/kernel/locking/qspinlock_paravirt.h +++ b/kernel/locking/qspinlock_paravirt.h @@ -312,7 +312,8 @@ pv_wait_early(struct pv_node *prev, int loop) if ((loop & PV_PREV_CHECK_MASK) != 0) return false; - return READ_ONCE(prev->state) != vcpu_running; + return READ_ONCE(prev->state) != vcpu_running || + vcpu_is_preempted(prev->cpu); } /* From mboxrd@z Thu Jan 1 00:00:00 1970 From: xinhui Subject: Re: [PATCH v2 0/4] implement vcpu preempted check Date: Wed, 06 Jul 2016 18:05:58 +0800 Message-ID: <577CD806.8000008@linux.vnet.ibm.com> References: <1467124991-13164-1-git-send-email-xinhui.pan@linux.vnet.ibm.com> <20160706065255.GH30909@twins.programming.kicks-ass.net> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20160706065255.GH30909@twins.programming.kicks-ass.net> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org To: Peter Zijlstra Cc: linux-s390@vger.kernel.org, dave@stgolabs.net, mpe@ellerman.id.au, boqun.feng@gmail.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, waiman.long@hpe.com, virtualization@lists.linux-foundation.org, mingo@redhat.com, paulus@samba.org, benh@kernel.crashing.org, schwidefsky@de.ibm.com, pbonzini@redhat.com, paulmck@linux.vnet.ibm.com, linuxppc-dev@lists.ozlabs.org List-Id: virtualization@lists.linuxfoundation.org CgpPbiAyMDE25bm0MDfmnIgwNuaXpSAxNDo1MiwgUGV0ZXIgWmlqbHN0cmEgd3JvdGU6Cj4gT24g VHVlLCBKdW4gMjgsIDIwMTYgYXQgMTA6NDM6MDdBTSAtMDQwMCwgUGFuIFhpbmh1aSB3cm90ZToK Pj4gY2hhbmdlIGZvbXIgdjE6Cj4+IAlhIHNpbXBsaWVyIGRlZmluaXRpb24gb2YgZGVmYXVsdCB2 Y3B1X2lzX3ByZWVtcHRlZAo+PiAJc2tpcCBtYWhjaW5lIHR5cGUgY2hlY2sgb24gcHBjLCBhbmQg YWRkIGNvbmZpZy4gcmVtb3ZlIGRlZGljYXRlZCBtYWNyby4KPj4gCWFkZCBvbmUgcGF0Y2ggdG8g ZHJvcCBvdmVybG9hZCBvZiByd3NlbV9zcGluX29uX293bmVyIGFuZCBtdXRleF9zcGluX29uX293 bmVyLgo+PiAJYWRkIG1vcmUgY29tbWVudHMKPj4gCXRoYW5rcyBib3F1biBhbmQgUGV0ZXIncyBz dWdnZXN0aW9uLgo+Pgo+PiBUaGlzIHBhdGNoIHNldCBhaW1zIHRvIGZpeCBsb2NrIGhvbGRlciBw cmVlbXB0aW9uIGlzc3Vlcy4KPj4KPj4gdGVzdC1jYXNlOgo+PiBwZXJmIHJlY29yZCAtYSBwZXJm IGJlbmNoIHNjaGVkIG1lc3NhZ2luZyAtZyA0MDAgLXAgJiYgcGVyZiByZXBvcnQKPj4KPj4gMTgu MDklICBzY2hlZC1tZXNzYWdpbmcgIFtrZXJuZWwudm1saW51eF0gIFtrXSBvc3FfbG9jawo+PiAx Mi4yOCUgIHNjaGVkLW1lc3NhZ2luZyAgW2tlcm5lbC52bWxpbnV4XSAgW2tdIHJ3c2VtX3NwaW5f b25fb3duZXIKPj4gICA1LjI3JSAgc2NoZWQtbWVzc2FnaW5nICBba2VybmVsLnZtbGludXhdICBb a10gbXV0ZXhfdW5sb2NrCj4+ICAgMy44OSUgIHNjaGVkLW1lc3NhZ2luZyAgW2tlcm5lbC52bWxp bnV4XSAgW2tdIHdhaXRfY29uc2lkZXJfdGFzawo+PiAgIDMuNjQlICBzY2hlZC1tZXNzYWdpbmcg IFtrZXJuZWwudm1saW51eF0gIFtrXSBfcmF3X3dyaXRlX2xvY2tfaXJxCj4+ICAgMy40MSUgIHNj aGVkLW1lc3NhZ2luZyAgW2tlcm5lbC52bWxpbnV4XSAgW2tdIG11dGV4X3NwaW5fb25fb3duZXIu aXMKPj4gICAyLjQ5JSAgc2NoZWQtbWVzc2FnaW5nICBba2VybmVsLnZtbGludXhdICBba10gc3lz dGVtX2NhbGwKPj4KPj4gV2UgaW50cm9kdWNlIGludGVyZmFjZSBib29sIHZjcHVfaXNfcHJlZW1w dGVkKGludCBjcHUpIGFuZCB1c2UgaXQgaW4gc29tZSBzcGluCj4+IGxvb3BzIG9mIG9zcV9sb2Nr LCByd3NlbV9zcGluX29uX293bmVyIGFuZCBtdXRleF9zcGluX29uX293bmVyLgo+PiBUaGVzZSBz cGluX29uX29ud2VyIHZhcmlhbnQgYWxzbyBjYXVzZSByY3Ugc3RhbGwgYmVmb3JlIHdlIGFwcGx5 IHRoaXMgcGF0Y2ggc2V0Cj4+Cj4KPiBQYW9sbywgY291bGQgeW91IGhlbHAgb3V0IHdpdGggYW4g KHg4NikgS1ZNIGludGVyZmFjZSBmb3IgdGhpcz8KPgo+IFdhaW1hbiwgY291bGQgeW91IHNlZSBp ZiB5b3UgY2FuIHV0aWxpemUgdGhpcyB0byBnZXQgcmlkIG9mIHRoZQo+IFNQSU5fVEhSRVNIT0xE IGluIHFzcGlubG9ja19wYXJhdmlydD8KPgpobW0uIG1heWJlIHNvbWV0aGluZyBsaWtlIGJlbG93 LiB3YWl0X25vZGUgY2FuIGdvIGludG8gcHZfd2FpdCgpIGVhcmxpZXIgYXMgc29vbiBhcyB0aGUg cHJldiBjcHUgaXMgcHJlZW1wdGVkLgpidXQgZm9yIHRoZSB3YWl0X2hlYWQsIGFzIHFzcGlubG9j ayBkb2VzIG5vdCByZWNvcmQgdGhlIGxvY2tfaG9sZGVyIGNvcnJlY3RseSh0aGFua3MgdG8gbG9j ayBzdGVhbGluZyksIHZjcHUgcHJlZW1wdGlvbiBjaGVjayBtaWdodCBnZXQgd3JvbmcgcmVzdWx0 cy4KCldhaW1hbiwgSSBoYXZlIHVzZWQgb25lIGhhc2ggdGFibGUgdG8ga2VlcCB0aGUgbG9jayBo b2xkZXIgaW4gbXkgcHBjIGltcGxlbWVudGF0aW9uIHBhdGNoLiBJIHRoaW5rIHdlIGNvdWxkIGRv IHNvbWV0aGluZyBzaW1pbGFyIGluIGdlbmVyaWMgY29kZT8KCmRpZmYgLS1naXQgYS9rZXJuZWwv bG9ja2luZy9xc3BpbmxvY2tfcGFyYXZpcnQuaCBiL2tlcm5lbC9sb2NraW5nL3FzcGlubG9ja19w YXJhdmlydC5oCmluZGV4IDc0YzRhODYuLjQwNTYwZTggMTAwNjQ0Ci0tLSBhL2tlcm5lbC9sb2Nr aW5nL3FzcGlubG9ja19wYXJhdmlydC5oCisrKyBiL2tlcm5lbC9sb2NraW5nL3FzcGlubG9ja19w YXJhdmlydC5oCkBAIC0zMTIsNyArMzEyLDggQEAgcHZfd2FpdF9lYXJseShzdHJ1Y3QgcHZfbm9k ZSAqcHJldiwgaW50IGxvb3ApCiAgICAgICAgIGlmICgobG9vcCAmIFBWX1BSRVZfQ0hFQ0tfTUFT SykgIT0gMCkKICAgICAgICAgICAgICAgICByZXR1cm4gZmFsc2U7CiAgCi0gICAgICAgcmV0dXJu IFJFQURfT05DRShwcmV2LT5zdGF0ZSkgIT0gdmNwdV9ydW5uaW5nOworICAgICAgIHJldHVybiBS RUFEX09OQ0UocHJldi0+c3RhdGUpICE9IHZjcHVfcnVubmluZyB8fAorICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIHZjcHVfaXNfcHJlZW1wdGVkKHByZXYtPmNwdSk7CiAgfQogIAogIC8q CgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpWaXJ0dWFs aXphdGlvbiBtYWlsaW5nIGxpc3QKVmlydHVhbGl6YXRpb25AbGlzdHMubGludXgtZm91bmRhdGlv bi5vcmcKaHR0cHM6Ly9saXN0cy5saW51eGZvdW5kYXRpb24ub3JnL21haWxtYW4vbGlzdGluZm8v dmlydHVhbGl6YXRpb24=