From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933452AbcA0OwO (ORCPT ); Wed, 27 Jan 2016 09:52:14 -0500 Received: from mx1.redhat.com ([209.132.183.28]:39305 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932393AbcA0OwL (ORCPT ); Wed, 27 Jan 2016 09:52:11 -0500 From: Jan Stancek Subject: [BUG] scheduler doesn't balance thread to idle cpu for 3 seconds To: alex.shi@intel.com, guz.fnst@cn.fujitsu.com, peterz@infradead.org, mingo@redhat.com, jolsa@redhat.com, riel@redhat.com, linux-kernel@vger.kernel.org Message-ID: <56A8D994.6050205@redhat.com> Date: Wed, 27 Jan 2016 15:52:04 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="------------080908020801090803070409" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This is a multi-part message in MIME format. --------------080908020801090803070409 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Hello, pthread_cond_wait_1/2 [1] is rarely failing for me on 4.5.0-rc1, on x86_64 KVM guest with 2 CPUs. This test [1]: - spawns 2 SCHED_RR threads - first thread with higher priority sets alarm for 2 seconds and blocks on condition - second thread with lower priority is busy looping for 5 seconds - after 2 seconds alarm signal arrives and handler signals condition - high priority thread should resume running But rarely I see that high priority thread doesn't resume running until low priority thread completes its 5 second busy loop. Looking at traces (short version attached, long version at [2]), I see that after 2 seconds scheduler tries to wake up main thread, but it appears to do that on same CPU where SCHED_RR low prio thread is running, so nothing happens. Then scheduler makes numerous balance attempts, but main thread is not balanced to idle CPU. My guess is this started with following commit, which changed weighted_cpuload(): commit b92486cbf2aa230d00f160664858495c81d2b37b Author: Alex Shi Date: Thu Jun 20 10:18:50 2013 +0800 sched: Compute runnable load avg in cpu_load and cpu_avg_load_per_task I could reproduce it with HEAD set at above commit, I couldn't reproduce it with 3.10 kernel so far. Regards, Jan [1] https://github.com/linux-test-project/ltp/blob/master/testcases/open_posix_testsuite/functional/threads/condvar/pthread_cond_wait_1.c [2] http://jan.stancek.eu/tmp/pthread_cond_wait_failure/sched-trace1.tar.bz2 --------------080908020801090803070409 Content-Type: text/plain; charset=UTF-8; name="sched-trace1-short-v4.5-rc1" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="sched-trace1-short-v4.5-rc1" IyB0ZXN0IHN0YXJ0cyBydW5uaW5nCiAgICAgICAgICAgIGJhc2gtNTYyNiAgWzAwMV0gLi4u LiAgIDI5NC42MDkwMjE6IHNjaGVkX3Byb2Nlc3NfZm9yazogY29tbT1iYXNoIHBpZD01NjI2 IGNoaWxkX2NvbW09YmFzaCBjaGlsZF9waWQ9MTAwMDUKICAgICAgICAgICAgYmFzaC01NjI2 ICBbMDAxXSBkLi4uICAgMjk0LjYwOTAyMzogc2NoZWRfbWlncmF0ZV90YXNrOiBjb21tPWJh c2ggcGlkPTEwMDA1IHByaW89MTIwIG9yaWdfY3B1PTEgZGVzdF9jcHU9MAogICAgICAgICAg ICBiYXNoLTU2MjYgIFswMDFdIGQuLi4gICAyOTQuNjA5MDI5OiBzY2hlZF93YWtldXBfbmV3 OiBjb21tPWJhc2ggcGlkPTEwMDA1IHByaW89MTIwIHRhcmdldF9jcHU9MDAwCiAgICAgICAg ICA8aWRsZT4tMCAgICAgWzAwMF0gZC4uLiAgIDI5NC42MDkwNzM6IHNjaGVkX3N3aXRjaDog cHJldl9jb21tPXN3YXBwZXIvMCBwcmV2X3BpZD0wIHByZXZfcHJpbz0xMjAgcHJldl9zdGF0 ZT1SID09PiBuZXh0X2NvbW09YmFzaCBuZXh0X3BpZD0xMDAwNSBuZXh0X3ByaW89MTIwCiBj b25kdmFyX3B0aHJlYWQtMTAwMDUgWzAwMF0gZC5oLiAgIDI5NC42MDkzNTU6IHNjaGVkX3N0 YXRfcnVudGltZTogY29tbT1iYXNoIHBpZD0xMDAwNSBydW50aW1lPTMzMDAxNSBbbnNdIHZy dW50aW1lPTkzNzU4Njg2MCBbbnNdCiBjb25kdmFyX3B0aHJlYWQtMTAwMDUgWzAwMF0gZC4u LiAgIDI5NC42MTAyMjI6IHNjaGVkX3N0YXRfcnVudGltZTogY29tbT1jb25kdmFyX3B0aHJl YWQgcGlkPTEwMDA1IHJ1bnRpbWU9ODY2NjYyIFtuc10gdnJ1bnRpbWU9OTM4NDUzNTIyIFtu c10KIGNvbmR2YXJfcHRocmVhZC0xMDAwNSBbMDAwXSBkLi4uICAgMjk0LjYxMDIyNDogc2No ZWRfbWlncmF0ZV90YXNrOiBjb21tPWNvbmR2YXJfcHRocmVhZCBwaWQ9MTAwMDUgcHJpbz0x MjAgb3JpZ19jcHU9MCBkZXN0X2NwdT0wCiBjb25kdmFyX3B0aHJlYWQtMTAwMDUgWzAwMF0g Li4uLiAgIDI5NC42MTAyMzE6IHNjaGVkX3Byb2Nlc3NfZm9yazogY29tbT1jb25kdmFyX3B0 aHJlYWQgcGlkPTEwMDA1IGNoaWxkX2NvbW09Y29uZHZhcl9wdGhyZWFkIGNoaWxkX3BpZD0x MDAwNgojIGhpZ2ggcHJpbyB0aHJlYWQgKDEwMDA2KSBpcyBzdGFydGVkCiBjb25kdmFyX3B0 aHJlYWQtMTAwMDUgWzAwMF0gZC4uLiAgIDI5NC42MTAyMzI6IHNjaGVkX21pZ3JhdGVfdGFz azogY29tbT1jb25kdmFyX3B0aHJlYWQgcGlkPTEwMDA2IHByaW89MTIwIG9yaWdfY3B1PTAg ZGVzdF9jcHU9MAogY29uZHZhcl9wdGhyZWFkLTEwMDA1IFswMDBdIGQuLi4gICAyOTQuNjEw MjMzOiBzY2hlZF9zdGF0X3J1bnRpbWU6IGNvbW09Y29uZHZhcl9wdGhyZWFkIHBpZD0xMDAw NSBydW50aW1lPTExOTI5IFtuc10gdnJ1bnRpbWU9OTM4NDY1NDUxIFtuc10KIGNvbmR2YXJf cHRocmVhZC0xMDAwNSBbMDAwXSBkLi4uICAgMjk0LjYxMDIzODogc2NoZWRfd2FrZXVwX25l dzogY29tbT1jb25kdmFyX3B0aHJlYWQgcGlkPTEwMDA2IHByaW89MTIwIHRhcmdldF9jcHU9 MDAwCiBjb25kdmFyX3B0aHJlYWQtMTAwMDUgWzAwMF0gZC4uLiAgIDI5NC42MTAyNjc6IHNj aGVkX3N0YXRfcnVudGltZTogY29tbT1jb25kdmFyX3B0aHJlYWQgcGlkPTEwMDA1IHJ1bnRp bWU9MzQxOTAgW25zXSB2cnVudGltZT05Mzg0OTk2NDEgW25zXQogY29uZHZhcl9wdGhyZWFk LTEwMDA1IFswMDBdIGQuLi4gICAyOTQuNjEwMjY4OiBzY2hlZF9taWdyYXRlX3Rhc2s6IGNv bW09Y29uZHZhcl9wdGhyZWFkIHBpZD0xMDAwNSBwcmlvPTEyMCBvcmlnX2NwdT0wIGRlc3Rf Y3B1PTAKIGNvbmR2YXJfcHRocmVhZC0xMDAwNSBbMDAwXSAuLi4uICAgMjk0LjYxMDI3MDog c2NoZWRfcHJvY2Vzc19mb3JrOiBjb21tPWNvbmR2YXJfcHRocmVhZCBwaWQ9MTAwMDUgY2hp bGRfY29tbT1jb25kdmFyX3B0aHJlYWQgY2hpbGRfcGlkPTEwMDA3CiMgbG93IHByaW8gdGhy ZWFkICgxMDAwNykgaXMgc3RhcnRlZAogY29uZHZhcl9wdGhyZWFkLTEwMDA1IFswMDBdIGQu Li4gICAyOTQuNjEwMjcxOiBzY2hlZF9taWdyYXRlX3Rhc2s6IGNvbW09Y29uZHZhcl9wdGhy ZWFkIHBpZD0xMDAwNyBwcmlvPTEyMCBvcmlnX2NwdT0wIGRlc3RfY3B1PTEKIGNvbmR2YXJf cHRocmVhZC0xMDAwNSBbMDAwXSBkLi4uICAgMjk0LjYxMDI3Nzogc2NoZWRfd2FrZXVwX25l dzogY29tbT1jb25kdmFyX3B0aHJlYWQgcGlkPTEwMDA3IHByaW89MTIwIHRhcmdldF9jcHU9 MDAxCiBjb25kdmFyX3B0aHJlYWQtMTAwMDUgWzAwMF0gZC4uLiAgIDI5NC42MTAyOTg6IHNj aGVkX3N0YXRfcnVudGltZTogY29tbT1jb25kdmFyX3B0aHJlYWQgcGlkPTEwMDA1IHJ1bnRp bWU9MzEzNTMgW25zXSB2cnVudGltZT05Mzg1MzA5OTQgW25zXQogY29uZHZhcl9wdGhyZWFk LTEwMDA1IFswMDBdIGQuLi4gICAyOTQuNjEwMzAyOiBzY2hlZF9zd2l0Y2g6IHByZXZfY29t bT1jb25kdmFyX3B0aHJlYWQgcHJldl9waWQ9MTAwMDUgcHJldl9wcmlvPTEyMCBwcmV2X3N0 YXRlPVMgPT0+IG5leHRfY29tbT1jb25kdmFyX3B0aHJlYWQgbmV4dF9waWQ9MTAwMDYgbmV4 dF9wcmlvPTEyMAogY29uZHZhcl9wdGhyZWFkLTEwMDA2IFswMDBdIGQuLi4gICAyOTQuNjEw MzExOiBzY2hlZF9zdGF0X3J1bnRpbWU6IGNvbW09Y29uZHZhcl9wdGhyZWFkIHBpZD0xMDAw NiBydW50aW1lPTEyMzU1IFtuc10gdnJ1bnRpbWU9OTQ0NDY1ODcyIFtuc10KICAgICAgICAg IDxpZGxlPi0wICAgICBbMDAxXSBkLi4uICAgMjk0LjYxMDMxODogc2NoZWRfc3dpdGNoOiBw cmV2X2NvbW09c3dhcHBlci8xIHByZXZfcGlkPTAgcHJldl9wcmlvPTEyMCBwcmV2X3N0YXRl PVIgPT0+IG5leHRfY29tbT1jb25kdmFyX3B0aHJlYWQgbmV4dF9waWQ9MTAwMDcgbmV4dF9w cmlvPTEyMAogY29uZHZhcl9wdGhyZWFkLTEwMDA3IFswMDFdIGQuLi4gICAyOTQuNjEwMzMy OiBzY2hlZF9zdGF0X3J1bnRpbWU6IGNvbW09Y29uZHZhcl9wdGhyZWFkIHBpZD0xMDAwNyBy dW50aW1lPTU5OTEwIFtuc10gdnJ1bnRpbWU9NTgxNjk2MTE5IFtuc10KIGNvbmR2YXJfcHRo cmVhZC0xMDAwNiBbMDAwXSBkLi4uICAgMjk0LjYxMDM0MDogc2NoZWRfc3dpdGNoOiBwcmV2 X2NvbW09Y29uZHZhcl9wdGhyZWFkIHByZXZfcGlkPTEwMDA2IHByZXZfcHJpbz04OSBwcmV2 X3N0YXRlPVMgPT0+IG5leHRfY29tbT1zd2FwcGVyLzAgbmV4dF9waWQ9MCBuZXh0X3ByaW89 MTIwCiBjb25kdmFyX3B0aHJlYWQtMTAwMDcgWzAwMV0gZC4uLiAgIDI5Ni41NzEzODk6IHNj aGVkX3N3aXRjaDogcHJldl9jb21tPWNvbmR2YXJfcHRocmVhZCBwcmV2X3BpZD0xMDAwNyBw cmV2X3ByaW89OTQgcHJldl9zdGF0ZT1SID09PiBuZXh0X2NvbW09d2F0Y2hkb2cvMSBuZXh0 X3BpZD0xMyBuZXh0X3ByaW89MAogICAgICB3YXRjaGRvZy8xLTEzICAgIFswMDFdIGQuLi4g ICAyOTYuNTcxMzkzOiBzY2hlZF9taWdyYXRlX3Rhc2s6IGNvbW09Y29uZHZhcl9wdGhyZWFk IHBpZD0xMDAwNyBwcmlvPTk0IG9yaWdfY3B1PTEgZGVzdF9jcHU9MAogICAgICAgICAgPGlk bGU+LTAgICAgIFswMDBdIGQuLi4gICAyOTYuNTcxNDEzOiBzY2hlZF9zd2l0Y2g6IHByZXZf Y29tbT1zd2FwcGVyLzAgcHJldl9waWQ9MCBwcmV2X3ByaW89MTIwIHByZXZfc3RhdGU9UiA9 PT4gbmV4dF9jb21tPWNvbmR2YXJfcHRocmVhZCBuZXh0X3BpZD0xMDAwNyBuZXh0X3ByaW89 OTQKIyBhZnRlciAyIHNlY29uZHMsIGFsYXJtIHNpZ25hbCB0cmllcyB0byB3YWtlIHVwIG1h aW4gdGhyZWFkIG9uIHNhbWUgY3B1IHdoZXJlIGxvdyBwcmlvIHRocmVhZCBpcyBydW5uaW5n CiBjb25kdmFyX3B0aHJlYWQtMTAwMDcgWzAwMF0gZC5oLiAgIDI5Ni42MTAzNTI6IHNjaGVk X3dha2V1cDogY29tbT1jb25kdmFyX3B0aHJlYWQgcGlkPTEwMDA1IHByaW89MTIwIHRhcmdl dF9jcHU9MDAwCiBjb25kdmFyX3B0aHJlYWQtMTAwMDcgWzAwMF0gZC4uLiAgIDI5OS42MTA0 NDY6IHNjaGVkX3N3aXRjaDogcHJldl9jb21tPWNvbmR2YXJfcHRocmVhZCBwcmV2X3BpZD0x MDAwNyBwcmV2X3ByaW89OTQgcHJldl9zdGF0ZT14ID09PiBuZXh0X2NvbW09dHVuZWQgbmV4 dF9waWQ9MTE4MyBuZXh0X3ByaW89MTIwCiAgICB4ZnNhaWxkL2RtLTAtNDI0ICAgWzAwMF0g ZC4uLiAgIDI5OS42MTA1ODI6IHNjaGVkX3N3aXRjaDogcHJldl9jb21tPXhmc2FpbGQvZG0t MCBwcmV2X3BpZD00MjQgcHJldl9wcmlvPTEyMCBwcmV2X3N0YXRlPVMgPT0+IG5leHRfY29t bT1jb25kdmFyX3B0aHJlYWQgbmV4dF9waWQ9MTAwMDUgbmV4dF9wcmlvPTEyMAojIG1haW4g dGhyZWFkIHJ1bnMgb25seSBhZnRlciBsb3cgcHJpbyB0aHJlYWQgaXMgZG9uZQogY29uZHZh cl9wdGhyZWFkLTEwMDA1IFswMDBdIGROLi4gICAyOTkuNjEwNTk2OiBzY2hlZF93YWtldXA6 IGNvbW09Y29uZHZhcl9wdGhyZWFkIHBpZD0xMDAwNiBwcmlvPTg5IHRhcmdldF9jcHU9MDAw CiBjb25kdmFyX3B0aHJlYWQtMTAwMDUgWzAwMF0gZE4uLiAgIDI5OS42MTA1OTc6IHNjaGVk X3N0YXRfcnVudGltZTogY29tbT1jb25kdmFyX3B0aHJlYWQgcGlkPTEwMDA1IHJ1bnRpbWU9 MTM5NDQgW25zXSB2cnVudGltZT05Mzg1NDQ5MzggW25zXQogY29uZHZhcl9wdGhyZWFkLTEw MDA1IFswMDBdIGQuLi4gICAyOTkuNjEwNTk4OiBzY2hlZF9zd2l0Y2g6IHByZXZfY29tbT1j b25kdmFyX3B0aHJlYWQgcHJldl9waWQ9MTAwMDUgcHJldl9wcmlvPTEyMCBwcmV2X3N0YXRl PVIgPT0+IG5leHRfY29tbT1jb25kdmFyX3B0aHJlYWQgbmV4dF9waWQ9MTAwMDYgbmV4dF9w cmlvPTg5CiBjb25kdmFyX3B0aHJlYWQtMTAwMDYgWzAwMF0gZC4uLiAgIDI5OS42MTA2MTQ6 IHNjaGVkX3N3aXRjaDogcHJldl9jb21tPWNvbmR2YXJfcHRocmVhZCBwcmV2X3BpZD0xMDAw NiBwcmV2X3ByaW89ODkgcHJldl9zdGF0ZT14ID09PiBuZXh0X2NvbW09Y29uZHZhcl9wdGhy ZWFkIG5leHRfcGlkPTEwMDA1IG5leHRfcHJpbz0xMjAKIGNvbmR2YXJfcHRocmVhZC0xMDAw NSBbMDAwXSBkLmguICAgMjk5LjYxMTU5MTogc2NoZWRfc3RhdF9ydW50aW1lOiBjb21tPWNv bmR2YXJfcHRocmVhZCBwaWQ9MTAwMDUgcnVudGltZT05NzY3NDUgW25zXSB2cnVudGltZT05 Mzk1MjE2ODMgW25zXQogY29uZHZhcl9wdGhyZWFkLTEwMDA1IFswMDBdIGQuLi4gICAyOTku NjEyMDI5OiBzY2hlZF9zdGF0X3J1bnRpbWU6IGNvbW09Y29uZHZhcl9wdGhyZWFkIHBpZD0x MDAwNSBydW50aW1lPTQzODMxMCBbbnNdIHZydW50aW1lPTkzOTk1OTk5MyBbbnNdCiBjb25k dmFyX3B0aHJlYWQtMTAwMDUgWzAwMF0gZC4uLiAgIDI5OS42MTIwMzE6IHNjaGVkX3N3aXRj aDogcHJldl9jb21tPWNvbmR2YXJfcHRocmVhZCBwcmV2X3BpZD0xMDAwNSBwcmV2X3ByaW89 MTIwIHByZXZfc3RhdGU9eCA9PT4gbmV4dF9jb21tPXJjdW9zLzAgbmV4dF9waWQ9OSBuZXh0 X3ByaW89MTIwCg== --------------080908020801090803070409--