[PATCH] sched: Fix rq nr_uninterruptible count

[PATCH] sched: Fix rq nr_uninterruptible count

[zhenggy]

When an uninterrptable task is queue to a differect cpu as where
it is dequeued, the rq nr_uninterruptible will be incorrent, so
fix it.

Signed-off-by: GuoYong Zheng <zhenggy@chinatelecom.cn>
---
 kernel/sched/core.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 25b582b..cd5ef6e 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4068,6 +4068,7 @@ bool ttwu_state_match(struct task_struct *p, unsigned int state, int *success)
 {
 	unsigned long flags;
 	int cpu, success = 0;
+	struct rq *src_rq, *dst_rq;

 	preempt_disable();
 	if (p == current) {
@@ -4205,6 +4206,16 @@ bool ttwu_state_match(struct task_struct *p, unsigned int state, int *success)
 			atomic_dec(&task_rq(p)->nr_iowait);
 		}

+		if (p->sched_contributes_to_load) {
+			src_rq = cpu_rq(task_cpu(p));
+			dst_rq = cpu_rq(cpu);
+
+			double_rq_lock(src_rq, dst_rq);
+			src_rq->nr_uninterruptible--;
+			dst_rq->nr_uninterruptible++;
+			double_rq_unlock(src_rq, dst_rq);
+		}
+
 		wake_flags |= WF_MIGRATED;
 		psi_ttwu_dequeue(p);
 		set_task_cpu(p, cpu);
-- 
1.8.3.1

Re: [PATCH] sched: Fix rq nr_uninterruptible count

[Slade's Kernel Patch Bot]

On 2/28/23 02:08, zhenggy wrote:
> When an uninterrptable task is queue to a differect cpu as where
> it is dequeued, the rq nr_uninterruptible will be incorrent, so
> fix it.
> 
> Signed-off-by: GuoYong Zheng <zhenggy@chinatelecom.cn>
> ---
>  kernel/sched/core.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 25b582b..cd5ef6e 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -4068,6 +4068,7 @@ bool ttwu_state_match(struct task_struct *p, unsigned int state, int *success)
>  {
>  	unsigned long flags;
>  	int cpu, success = 0;
> +	struct rq *src_rq, *dst_rq;
> 
>  	preempt_disable();
>  	if (p == current) {
> @@ -4205,6 +4206,16 @@ bool ttwu_state_match(struct task_struct *p, unsigned int state, int *success)
>  			atomic_dec(&task_rq(p)->nr_iowait);
>  		}
> 
> +		if (p->sched_contributes_to_load) {
> +			src_rq = cpu_rq(task_cpu(p));
> +			dst_rq = cpu_rq(cpu);
> +
> +			double_rq_lock(src_rq, dst_rq);
> +			src_rq->nr_uninterruptible--;
> +			dst_rq->nr_uninterruptible++;
> +			double_rq_unlock(src_rq, dst_rq);
> +		}
> +
>  		wake_flags |= WF_MIGRATED;
>  		psi_ttwu_dequeue(p);
>  		set_task_cpu(p, cpu);

This is Slade's kernel patch bot. When scanning his mailbox, I came across this message, which appears to have an odd format for its text ("text/plain; charset=gbk"). Emailed patches should be in ASCII or UTF-8 encoding. If you're using an email client, make sure it's configured to use either of the two formats.

You can learn more in the kernel's documentation here: https://www.kernel.org/doc/html/latest/process/email-clients.html

If you believe this is an error, please address a message to Slade Watkins <srw@sladewatkins.net>.

Thank you,
-- Slade's kernel patch bot

[PATCH] sched: Fix rq nr_uninterruptible count

[zhenggy]

When an uninterrptable task is queue to a differect cpu as where
it is dequeued, the rq nr_uninterruptible will be incorrent, so
fix it.

Signed-off-by: GuoYong Zheng <zhenggy@chinatelecom.cn>
---
 kernel/sched/core.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 25b582b..cd5ef6e 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4068,6 +4068,7 @@ bool ttwu_state_match(struct task_struct *p, unsigned int state, int *success)
 {
 	unsigned long flags;
 	int cpu, success = 0;
+	struct rq *src_rq, *dst_rq;

 	preempt_disable();
 	if (p == current) {
@@ -4205,6 +4206,16 @@ bool ttwu_state_match(struct task_struct *p, unsigned int state, int *success)
 			atomic_dec(&task_rq(p)->nr_iowait);
 		}

+		if (p->sched_contributes_to_load) {
+			src_rq = cpu_rq(task_cpu(p));
+			dst_rq = cpu_rq(cpu);
+
+			double_rq_lock(src_rq, dst_rq);
+			src_rq->nr_uninterruptible--;
+			dst_rq->nr_uninterruptible++;
+			double_rq_unlock(src_rq, dst_rq);
+		}
+
 		wake_flags |= WF_MIGRATED;
 		psi_ttwu_dequeue(p);
 		set_task_cpu(p, cpu);
-- 
1.8.3.1

Re: [PATCH] sched: Fix rq nr_uninterruptible count

[Bagas Sanjaya]

[-- Attachment #1: Type: text/plain, Size: 1283 bytes --]

On Tue, Feb 28, 2023 at 03:08:58PM +0800, zhenggy wrote:
> When an uninterrptable task is queue to a differect cpu as where
> it is dequeued, the rq nr_uninterruptible will be incorrent, so
> fix it.

tl;dr (too much typos above). Would you like to "massage" the commit
mesage?

> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 25b582b..cd5ef6e 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -4068,6 +4068,7 @@ bool ttwu_state_match(struct task_struct *p, unsigned int state, int *success)
>  {
>  	unsigned long flags;
>  	int cpu, success = 0;
> +	struct rq *src_rq, *dst_rq;
> 
>  	preempt_disable();
>  	if (p == current) {
> @@ -4205,6 +4206,16 @@ bool ttwu_state_match(struct task_struct *p, unsigned int state, int *success)
>  			atomic_dec(&task_rq(p)->nr_iowait);
>  		}
> 
> +		if (p->sched_contributes_to_load) {
> +			src_rq = cpu_rq(task_cpu(p));
> +			dst_rq = cpu_rq(cpu);
> +
> +			double_rq_lock(src_rq, dst_rq);
> +			src_rq->nr_uninterruptible--;
> +			dst_rq->nr_uninterruptible++;
> +			double_rq_unlock(src_rq, dst_rq);
> +		}
> +

Can you explain what the code above does to fix the problem you
mentioned above?

Thanks.

-- 
An old man doll... just what I always wanted! - Clara

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [PATCH] sched: Fix rq nr_uninterruptible count

[Mike Galbraith]

On Tue, 2023-02-28 at 15:08 +0800, zhenggy wrote:
> When an uninterrptable task is queue to a differect cpu as where
> it is dequeued, the rq nr_uninterruptible will be incorrent, so
> fix it.

That's intentional, see 47-52 of kernel/sched/loadavg.c.

	-Mike

Re: [PATCH] sched: Fix rq nr_uninterruptible count

[Nikolay Borisov]

On 28.02.23 г. 10:46 ч., zhenggy wrote:
> When an uninterrptable task is queue to a differect cpu as where
> it is dequeued, the rq nr_uninterruptible will be incorrent, so
> fix it.
> 
> Signed-off-by: GuoYong Zheng <zhenggy@chinatelecom.cn>


    37  *  - cpu_rq()->nr_uninterruptible isn't accurately tracked 
per-CPU because
    38  *    this would add another cross-CPU cacheline miss and atomic 
operation
    39  *    to the wakeup path. Instead we increment on whatever CPU 
the task ran
    40  *    when it went into uninterruptible state and decrement on 
whatever CPU
    41  *    did the wakeup. This means that only the sum of 
nr_uninterruptible over
    42  *    all CPUs yields the correct result.

> ---
>   kernel/sched/core.c | 11 +++++++++++
>   1 file changed, 11 insertions(+)
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 25b582b..cd5ef6e 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -4068,6 +4068,7 @@ bool ttwu_state_match(struct task_struct *p, unsigned int state, int *success)
>   {
>   	unsigned long flags;
>   	int cpu, success = 0;
> +	struct rq *src_rq, *dst_rq;
> 
>   	preempt_disable();
>   	if (p == current) {
> @@ -4205,6 +4206,16 @@ bool ttwu_state_match(struct task_struct *p, unsigned int state, int *success)
>   			atomic_dec(&task_rq(p)->nr_iowait);
>   		}
> 
> +		if (p->sched_contributes_to_load) {
> +			src_rq = cpu_rq(task_cpu(p));
> +			dst_rq = cpu_rq(cpu);
> +
> +			double_rq_lock(src_rq, dst_rq);
> +			src_rq->nr_uninterruptible--;
> +			dst_rq->nr_uninterruptible++;
> +			double_rq_unlock(src_rq, dst_rq);
> +		}
> +
>   		wake_flags |= WF_MIGRATED;
>   		psi_ttwu_dequeue(p);
>   		set_task_cpu(p, cpu);