From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A193BE732D0 for ; Fri, 29 Sep 2023 10:21:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To: Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=lKFoxdRkLTqqeQsvh4AvyUof9cGGj7cAggKCqCouSAA=; b=UuujDqz+UFvTMM+Ch/ZrfG4xRP dK+E4ruiWX2Fh8As8cEs/Zedc8DccTUXzziA/d9RT4PCRpY5MXipbWkRSNW5XQAFjtzZfdzMhZ1wY PMn8j1IBEeJmc/3KAb6YgjyKsmCPbU6pX/qeS+8NrNNm7FiqYsuhoMeAoIUSYfCj4cJ5ED5/1NELv 4pvmlAjq7acUkTwQSM8kzhXZUcf+AwoIGqH/PH0JjHziejDT9wuNS+ipDdnHN10iRoKhEFbfFjlWR t5w3Yl9otBCb5o2XRmqolpVtYGgDOBFz0l4h7Tf6LdBXqAAnzmfTIhnlyEoqVPOIP3VGP/NOMS/QQ Ab//RzQg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qmAd2-007XGt-1Z; Fri, 29 Sep 2023 10:21:48 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qmAd1-007XGe-0D; Fri, 29 Sep 2023 10:21:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Transfer-Encoding: Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date: Sender:Reply-To:Content-ID:Content-Description; bh=lKFoxdRkLTqqeQsvh4AvyUof9cGGj7cAggKCqCouSAA=; b=gA0Cs9nTbER/aLaPoARGDChZIo DlEKlFIYmOxPXeqSq5duqn6lp0RSdXtgimVz/R7EMh/kvP/+f55x3MBatV3vZS+VvgLJ3T+Gc7Shs K9hZOM2qHLX8wbOIWSpfOnX/xc3iMqYpqoh88ZHPBfwiyz31XGMELjqzzusTQxYeRVO5yQDemr0L+ W0Z3EeDT/wGRtQOnMU/IVP5+K8sX7dg3XasOa19hrxOrA68VZSa4zDSWKSKgRD9t5FjvuGSnnIW4y cjyuRvaDSxmNkvzC0k1BQE+0Wz/OUfEsQAXB7y0tSVnv80kg+bOnddD8dRtZ7oPLs8l5WVWOjQM3F dTKwOFEA==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qmAco-0062Tw-2C; Fri, 29 Sep 2023 10:21:37 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id C867730030F; Fri, 29 Sep 2023 12:21:35 +0200 (CEST) Date: Fri, 29 Sep 2023 12:21:35 +0200 From: Peter Zijlstra To: Kuyo Chang =?utf-8?B?KOW8teW7uuaWhyk=?= Cc: "dietmar.eggemann@arm.com" , "linux-kernel@vger.kernel.org" , "linux-mediatek@lists.infradead.org" , "rostedt@goodmis.org" , wsd_upstream , "vschneid@redhat.com" , "bristot@redhat.com" , "juri.lelli@redhat.com" , "mingo@redhat.com" , "linux-arm-kernel@lists.infradead.org" , "bsegall@google.com" , "mgorman@suse.de" , "matthias.bgg@gmail.com" , "vincent.guittot@linaro.org" , "angelogioacchino.delregno@collabora.com" Subject: Re: [PATCH 1/1] sched/core: Fix stuck on completion for affine_move_task() when stopper disable Message-ID: <20230929102135.GD6282@noisy.programming.kicks-ass.net> References: <20230927033431.12406-1-kuyo.chang@mediatek.com> <20230927080850.GB21824@noisy.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-BeenThere: linux-mediatek@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-mediatek" Errors-To: linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org On Wed, Sep 27, 2023 at 03:57:35PM +0000, Kuyo Chang (張建文) wrote: > This issue occurs at CPU hotplug/set_affinity stress test. > The reproduce ratio is very low(about once a week). I'm assuming you're running an arm64 kernel with preempt_full=y (the default for arm64). Could you please test the below? diff --git a/kernel/sched/core.c b/kernel/sched/core.c index d8fd29d66b24..079a63b8a954 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2645,9 +2645,11 @@ static int migration_cpu_stop(void *data) * it. */ WARN_ON_ONCE(!pending->stop_pending); + preempt_disable(); task_rq_unlock(rq, p, &rf); stop_one_cpu_nowait(task_cpu(p), migration_cpu_stop, &pending->arg, &pending->stop_work); + preempt_enable(); return 0; } out: @@ -2967,12 +2969,13 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag complete = true; } + preempt_disable(); task_rq_unlock(rq, p, rf); - if (push_task) { stop_one_cpu_nowait(rq->cpu, push_cpu_stop, p, &rq->push_work); } + preempt_enable(); if (complete) complete_all(&pending->done); @@ -3038,12 +3041,13 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag if (flags & SCA_MIGRATE_ENABLE) p->migration_flags &= ~MDF_PUSH; + preempt_disable(); task_rq_unlock(rq, p, rf); - if (!stop_pending) { stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop, &pending->arg, &pending->stop_work); } + preempt_enable(); if (flags & SCA_MIGRATE_ENABLE) return 0; @@ -9459,6 +9461,7 @@ static void balance_push(struct rq *rq) * Temporarily drop rq->lock such that we can wake-up the stop task. * Both preemption and IRQs are still disabled. */ + preempt_disable(); raw_spin_rq_unlock(rq); stop_one_cpu_nowait(rq->cpu, __balance_push_cpu_stop, push_task, this_cpu_ptr(&push_work)); @@ -9468,6 +9471,7 @@ static void balance_push(struct rq *rq) * which kthread_is_per_cpu() and will push this task away. */ raw_spin_rq_lock(rq); + preempt_enable(); } static void balance_push_set(int cpu, bool on) From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8FB4FE7F153 for ; Fri, 29 Sep 2023 10:22:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=XIfjW8sDf1ihkJQojyrV5SIS5O0mXj9lL8JsBY4K3QY=; b=p2sVXnKkF2HU0l TmPGa6hOOUgZ/hIIthyVOLymqrSdkYu/CcOFC56VDRoSr7z95RjVTQ6Yt7gWfore58GtwmIHZRbRd 4VntHcHxIvZaBaCGn+z6Xodpk8HEF7pIjNjvGaGx97vr+1REaFtuQmRJWUDvwebB0zyzKDOIIpJso N5+TyilR2+aIY52KN0H0Mt4Gghfbbin0JK0XgyB7W5jWIDLz6ox5E1ywZbcfIY+IQRbjDyMQwXH4D bJkqs56ejpFyI3ENJP9gAFqnH7gTdM6k74PaHyQ6TaikpLnTdJmlIgNcSMRlw7Oz8XXV5h5So4FrE gS7pWxMw1TQFhFrVUDtQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qmAd2-007XGn-00; Fri, 29 Sep 2023 10:21:48 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qmAd1-007XGe-0D; Fri, 29 Sep 2023 10:21:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Transfer-Encoding: Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date: Sender:Reply-To:Content-ID:Content-Description; bh=lKFoxdRkLTqqeQsvh4AvyUof9cGGj7cAggKCqCouSAA=; b=gA0Cs9nTbER/aLaPoARGDChZIo DlEKlFIYmOxPXeqSq5duqn6lp0RSdXtgimVz/R7EMh/kvP/+f55x3MBatV3vZS+VvgLJ3T+Gc7Shs K9hZOM2qHLX8wbOIWSpfOnX/xc3iMqYpqoh88ZHPBfwiyz31XGMELjqzzusTQxYeRVO5yQDemr0L+ W0Z3EeDT/wGRtQOnMU/IVP5+K8sX7dg3XasOa19hrxOrA68VZSa4zDSWKSKgRD9t5FjvuGSnnIW4y cjyuRvaDSxmNkvzC0k1BQE+0Wz/OUfEsQAXB7y0tSVnv80kg+bOnddD8dRtZ7oPLs8l5WVWOjQM3F dTKwOFEA==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qmAco-0062Tw-2C; Fri, 29 Sep 2023 10:21:37 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id C867730030F; Fri, 29 Sep 2023 12:21:35 +0200 (CEST) Date: Fri, 29 Sep 2023 12:21:35 +0200 From: Peter Zijlstra To: Kuyo Chang =?utf-8?B?KOW8teW7uuaWhyk=?= Cc: "dietmar.eggemann@arm.com" , "linux-kernel@vger.kernel.org" , "linux-mediatek@lists.infradead.org" , "rostedt@goodmis.org" , wsd_upstream , "vschneid@redhat.com" , "bristot@redhat.com" , "juri.lelli@redhat.com" , "mingo@redhat.com" , "linux-arm-kernel@lists.infradead.org" , "bsegall@google.com" , "mgorman@suse.de" , "matthias.bgg@gmail.com" , "vincent.guittot@linaro.org" , "angelogioacchino.delregno@collabora.com" Subject: Re: [PATCH 1/1] sched/core: Fix stuck on completion for affine_move_task() when stopper disable Message-ID: <20230929102135.GD6282@noisy.programming.kicks-ass.net> References: <20230927033431.12406-1-kuyo.chang@mediatek.com> <20230927080850.GB21824@noisy.programming.kicks-ass.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gV2VkLCBTZXAgMjcsIDIwMjMgYXQgMDM6NTc6MzVQTSArMDAwMCwgS3V5byBDaGFuZyAo5by1 5bu65paHKSB3cm90ZToKCj4gVGhpcyBpc3N1ZSBvY2N1cnMgYXQgQ1BVIGhvdHBsdWcvc2V0X2Fm ZmluaXR5IHN0cmVzcyB0ZXN0Lgo+IFRoZSByZXByb2R1Y2UgcmF0aW8gaXMgdmVyeSBsb3coYWJv dXQgb25jZSBhIHdlZWspLgoKSSdtIGFzc3VtaW5nIHlvdSdyZSBydW5uaW5nIGFuIGFybTY0IGtl cm5lbCB3aXRoIHByZWVtcHRfZnVsbD15ICh0aGUKZGVmYXVsdCBmb3IgYXJtNjQpLgoKQ291bGQg eW91IHBsZWFzZSB0ZXN0IHRoZSBiZWxvdz8KCmRpZmYgLS1naXQgYS9rZXJuZWwvc2NoZWQvY29y ZS5jIGIva2VybmVsL3NjaGVkL2NvcmUuYwppbmRleCBkOGZkMjlkNjZiMjQuLjA3OWE2M2I4YTk1 NCAxMDA2NDQKLS0tIGEva2VybmVsL3NjaGVkL2NvcmUuYworKysgYi9rZXJuZWwvc2NoZWQvY29y ZS5jCkBAIC0yNjQ1LDkgKzI2NDUsMTEgQEAgc3RhdGljIGludCBtaWdyYXRpb25fY3B1X3N0b3Ao dm9pZCAqZGF0YSkKIAkJICogaXQuCiAJCSAqLwogCQlXQVJOX09OX09OQ0UoIXBlbmRpbmctPnN0 b3BfcGVuZGluZyk7CisJCXByZWVtcHRfZGlzYWJsZSgpOwogCQl0YXNrX3JxX3VubG9jayhycSwg cCwgJnJmKTsKIAkJc3RvcF9vbmVfY3B1X25vd2FpdCh0YXNrX2NwdShwKSwgbWlncmF0aW9uX2Nw dV9zdG9wLAogCQkJCSAgICAmcGVuZGluZy0+YXJnLCAmcGVuZGluZy0+c3RvcF93b3JrKTsKKwkJ cHJlZW1wdF9lbmFibGUoKTsKIAkJcmV0dXJuIDA7CiAJfQogb3V0OgpAQCAtMjk2NywxMiArMjk2 OSwxMyBAQCBzdGF0aWMgaW50IGFmZmluZV9tb3ZlX3Rhc2soc3RydWN0IHJxICpycSwgc3RydWN0 IHRhc2tfc3RydWN0ICpwLCBzdHJ1Y3QgcnFfZmxhZwogCQkJY29tcGxldGUgPSB0cnVlOwogCQl9 CiAKKwkJcHJlZW1wdF9kaXNhYmxlKCk7CiAJCXRhc2tfcnFfdW5sb2NrKHJxLCBwLCByZik7Ci0K IAkJaWYgKHB1c2hfdGFzaykgewogCQkJc3RvcF9vbmVfY3B1X25vd2FpdChycS0+Y3B1LCBwdXNo X2NwdV9zdG9wLAogCQkJCQkgICAgcCwgJnJxLT5wdXNoX3dvcmspOwogCQl9CisJCXByZWVtcHRf ZW5hYmxlKCk7CiAKIAkJaWYgKGNvbXBsZXRlKQogCQkJY29tcGxldGVfYWxsKCZwZW5kaW5nLT5k b25lKTsKQEAgLTMwMzgsMTIgKzMwNDEsMTMgQEAgc3RhdGljIGludCBhZmZpbmVfbW92ZV90YXNr KHN0cnVjdCBycSAqcnEsIHN0cnVjdCB0YXNrX3N0cnVjdCAqcCwgc3RydWN0IHJxX2ZsYWcKIAkJ aWYgKGZsYWdzICYgU0NBX01JR1JBVEVfRU5BQkxFKQogCQkJcC0+bWlncmF0aW9uX2ZsYWdzICY9 IH5NREZfUFVTSDsKIAorCQlwcmVlbXB0X2Rpc2FibGUoKTsKIAkJdGFza19ycV91bmxvY2socnEs IHAsIHJmKTsKLQogCQlpZiAoIXN0b3BfcGVuZGluZykgewogCQkJc3RvcF9vbmVfY3B1X25vd2Fp dChjcHVfb2YocnEpLCBtaWdyYXRpb25fY3B1X3N0b3AsCiAJCQkJCSAgICAmcGVuZGluZy0+YXJn LCAmcGVuZGluZy0+c3RvcF93b3JrKTsKIAkJfQorCQlwcmVlbXB0X2VuYWJsZSgpOwogCiAJCWlm IChmbGFncyAmIFNDQV9NSUdSQVRFX0VOQUJMRSkKIAkJCXJldHVybiAwOwpAQCAtOTQ1OSw2ICs5 NDYxLDcgQEAgc3RhdGljIHZvaWQgYmFsYW5jZV9wdXNoKHN0cnVjdCBycSAqcnEpCiAJICogVGVt cG9yYXJpbHkgZHJvcCBycS0+bG9jayBzdWNoIHRoYXQgd2UgY2FuIHdha2UtdXAgdGhlIHN0b3Ag dGFzay4KIAkgKiBCb3RoIHByZWVtcHRpb24gYW5kIElSUXMgYXJlIHN0aWxsIGRpc2FibGVkLgog CSAqLworCXByZWVtcHRfZGlzYWJsZSgpOwogCXJhd19zcGluX3JxX3VubG9jayhycSk7CiAJc3Rv cF9vbmVfY3B1X25vd2FpdChycS0+Y3B1LCBfX2JhbGFuY2VfcHVzaF9jcHVfc3RvcCwgcHVzaF90 YXNrLAogCQkJICAgIHRoaXNfY3B1X3B0cigmcHVzaF93b3JrKSk7CkBAIC05NDY4LDYgKzk0NzEs NyBAQCBzdGF0aWMgdm9pZCBiYWxhbmNlX3B1c2goc3RydWN0IHJxICpycSkKIAkgKiB3aGljaCBr dGhyZWFkX2lzX3Blcl9jcHUoKSBhbmQgd2lsbCBwdXNoIHRoaXMgdGFzayBhd2F5LgogCSAqLwog CXJhd19zcGluX3JxX2xvY2socnEpOworCXByZWVtcHRfZW5hYmxlKCk7CiB9CiAKIHN0YXRpYyB2 b2lkIGJhbGFuY2VfcHVzaF9zZXQoaW50IGNwdSwgYm9vbCBvbikKCl9fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBs aXN0CmxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5m cmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK