[PATCH v1 1/1] wq: Avoid using isolated cpus' timers on unbounded queue_delayed_work

[PATCH v1 1/1] wq: Avoid using isolated cpus' timers on unbounded queue_delayed_work

[Leonardo Bras]

When __queue_delayed_work() is called with WORK_CPU_UNBOUND, it means any
cpu is able to run the work, as well as any cpu timer is able to be used.

This is not good if a system does use CPU isolation, because it can take
away some valuable cpu time to:
1 - deal with the timer interrupt,
2 - schedule-out the desired task,
3 - queue work on a random workqueue, and
4 - schedule the desired task back to the cpu.

So to fix this, during __queue_delayed_work(), if both:
- Work is not cpu-bounded,
- CPU isolation is in place,
then pick a random non-isolated cpu to use both the timer and the
system per-cpu workqueue.

AS an optimization, if the current cpu is not isolated, use it's timer
instead of looking for another candidate.

Signed-off-by: Leonardo Bras <leobras@redhat.com>
---
Changes since RFC:
- Do not use the same cpu from the timer for queueing the work.
- If the current cpu is not isolated, use it's timer instead of
  looking for another candidate.

 kernel/workqueue.c | 22 ++++++++++++++++++----
 1 file changed, 18 insertions(+), 4 deletions(-)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 76e60faed8923..0a85b00bbe52d 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -1958,10 +1958,24 @@ static void __queue_delayed_work(int cpu, struct workqueue_struct *wq,
 	dwork->cpu = cpu;
 	timer->expires = jiffies + delay;
 
-	if (unlikely(cpu != WORK_CPU_UNBOUND))
-		add_timer_on(timer, cpu);
-	else
-		add_timer(timer);
+	if (likely(cpu == WORK_CPU_UNBOUND)) {
+		if (!housekeeping_enabled(HK_TYPE_TIMER)) {
+			/* Reuse the same timer */
+			add_timer(timer);
+			return;
+		}
+
+		/*
+		 * If the work is cpu-unbound, and cpu isolation is in place,
+		 * only use timers from housekeeping cpus.
+		 * If the current cpu is a housekeeping cpu, use it instead.
+		 */
+		cpu = smp_processor_id();
+		if (!housekeeping_test_cpu(cpu, HK_TYPE_TIMER))
+			cpu = housekeeping_any_cpu(HK_TYPE_TIMER);
+	}
+
+	add_timer_on(timer, cpu);
 }
 
 /**
-- 
2.43.0

Re: [PATCH v1 1/1] wq: Avoid using isolated cpus' timers on unbounded queue_delayed_work

[Tejun Heo]

Hello,

On Thu, Jan 25, 2024 at 10:03:20PM -0300, Leonardo Bras wrote:
...
> AS an optimization, if the current cpu is not isolated, use it's timer
  ^                                                           ^
  As                                                          its

> instead of looking for another candidate.

The sentence reads weird tho. It's always the same timer. We're deciding
which CPU to queue the timer on.

> @@ -1958,10 +1958,24 @@ static void __queue_delayed_work(int cpu, struct workqueue_struct *wq,
>  	dwork->cpu = cpu;
>  	timer->expires = jiffies + delay;
>  
> -	if (unlikely(cpu != WORK_CPU_UNBOUND))
> -		add_timer_on(timer, cpu);
> -	else
> -		add_timer(timer);
> +	if (likely(cpu == WORK_CPU_UNBOUND)) {
> +		if (!housekeeping_enabled(HK_TYPE_TIMER)) {
> +			/* Reuse the same timer */

This comment is confusing because it's always the same timer.

> +			add_timer(timer);
> +			return;
> +		}
> +
> +		/*
> +		 * If the work is cpu-unbound, and cpu isolation is in place,
> +		 * only use timers from housekeeping cpus.
> +		 * If the current cpu is a housekeeping cpu, use it instead.
> +		 */
> +		cpu = smp_processor_id();
> +		if (!housekeeping_test_cpu(cpu, HK_TYPE_TIMER))
> +			cpu = housekeeping_any_cpu(HK_TYPE_TIMER);
> +	}
> +
> +	add_timer_on(timer, cpu);
>  }

I find the control flow a bit difficult to follow. It's not the end of the
world to have two add_timer_on() calls. Would something like the following
be easier to read?

	if (housekeeping_enabled(HK_TYPE_TIMER)) {
		cpu = smp_processor_id();
		if (!housekeeping_test_cpu(cpu, HK_TYPE_TIMER))
			cpu = housekeeping_any_cpu(HK_TYPE_TIMER);
		add_timer_on(timer, cpu);
	} else {
		if (likely(cpu == WORK_CPU_UNBOUND))
			add_timer(timer, cpu);
		else
			add_timer_on(timer, cpu);
	}

Thanks.

-- 
tejun

Re: [PATCH v1 1/1] wq: Avoid using isolated cpus' timers on unbounded queue_delayed_work

[Leonardo Bras]

On Fri, Jan 26, 2024 at 11:49:02AM -1000, Tejun Heo wrote:
> Hello,
> 
> On Thu, Jan 25, 2024 at 10:03:20PM -0300, Leonardo Bras wrote:
> ...
> > AS an optimization, if the current cpu is not isolated, use it's timer
>   ^                                                           ^
>   As                                                          its
> 
> > instead of looking for another candidate.
> 
> The sentence reads weird tho. It's always the same timer. We're deciding
> which CPU to queue the timer on.
> 

Hello,

Thanks for pointing that out, I will improve it in the v2.



> > @@ -1958,10 +1958,24 @@ static void __queue_delayed_work(int cpu, struct workqueue_struct *wq,
> >  	dwork->cpu = cpu;
> >  	timer->expires = jiffies + delay;
> >  
> > -	if (unlikely(cpu != WORK_CPU_UNBOUND))
> > -		add_timer_on(timer, cpu);
> > -	else
> > -		add_timer(timer);
> > +	if (likely(cpu == WORK_CPU_UNBOUND)) {
> > +		if (!housekeeping_enabled(HK_TYPE_TIMER)) {
> > +			/* Reuse the same timer */
> 
> This comment is confusing because it's always the same timer.

Thanks, I will point out this being the last cpu used to handle the timer.

> 
> > +			add_timer(timer);
> > +			return;
> > +		}
> > +
> > +		/*
> > +		 * If the work is cpu-unbound, and cpu isolation is in place,
> > +		 * only use timers from housekeeping cpus.
> > +		 * If the current cpu is a housekeeping cpu, use it instead.
> > +		 */
> > +		cpu = smp_processor_id();
> > +		if (!housekeeping_test_cpu(cpu, HK_TYPE_TIMER))
> > +			cpu = housekeeping_any_cpu(HK_TYPE_TIMER);
> > +	}
> > +
> > +	add_timer_on(timer, cpu);
> >  }
> 
> I find the control flow a bit difficult to follow. It's not the end of the
> world to have two add_timer_on() calls. Would something like the following
> be easier to read?
> 
> 	if (housekeeping_enabled(HK_TYPE_TIMER)) {
> 		cpu = smp_processor_id();
> 		if (!housekeeping_test_cpu(cpu, HK_TYPE_TIMER))
> 			cpu = housekeeping_any_cpu(HK_TYPE_TIMER);
> 		add_timer_on(timer, cpu);
> 	} else {
> 		if (likely(cpu == WORK_CPU_UNBOUND))
> 			add_timer(timer, cpu);
> 		else
> 			add_timer_on(timer, cpu);
> 	}
> 
> Thanks.

I am not really against it, but for me it's kind of weird to have that many 
calls to add_timer_on() if we can avoid it. 

I would rather go with:

###
if (unlikely(cpu != WORK_CPU_UNBOUND)) {
	add_timer_on(timer, cpu);
	return;
}

if (!housekeeping_enabled(HK_TYPE_TIMER)) {
	add_timer(timer);
	return;
}

cpu = smp_processor_id();
if (!housekeeping_test_cpu(cpu, HK_TYPE_TIMER))
	cpu = housekeeping_any_cpu(HK_TYPE_TIMER);

add_timer_on(timer, cpu);
###

What do you think?

Thanks,
Leo

> 
> -- 
> tejun
> 

Re: [PATCH v1 1/1] wq: Avoid using isolated cpus' timers on unbounded queue_delayed_work

[Tejun Heo]

Hello,

On Fri, Jan 26, 2024 at 07:05:35PM -0300, Leonardo Bras wrote:
> > 	if (housekeeping_enabled(HK_TYPE_TIMER)) {
> > 		cpu = smp_processor_id();
> > 		if (!housekeeping_test_cpu(cpu, HK_TYPE_TIMER))
> > 			cpu = housekeeping_any_cpu(HK_TYPE_TIMER);
> > 		add_timer_on(timer, cpu);
> > 	} else {
> > 		if (likely(cpu == WORK_CPU_UNBOUND))
> > 			add_timer(timer, cpu);
> > 		else
> > 			add_timer_on(timer, cpu);
> > 	}
> > 
> > Thanks.
> 
> I am not really against it, but for me it's kind of weird to have that many 
> calls to add_timer_on() if we can avoid it. 
> 
> I would rather go with:
> 
> ###
> if (unlikely(cpu != WORK_CPU_UNBOUND)) {
> 	add_timer_on(timer, cpu);
> 	return;
> }
> 
> if (!housekeeping_enabled(HK_TYPE_TIMER)) {
> 	add_timer(timer);
> 	return;
> }
> 
> cpu = smp_processor_id();
> if (!housekeeping_test_cpu(cpu, HK_TYPE_TIMER))
> 	cpu = housekeeping_any_cpu(HK_TYPE_TIMER);
> 
> add_timer_on(timer, cpu);
> ###
> 
> What do you think?

Isn't that still the same number of add_timer[_on]() calls?

Thanks.

-- 
tejun

Re: [PATCH v1 1/1] wq: Avoid using isolated cpus' timers on unbounded queue_delayed_work

[Leonardo Bras]

On Mon, Jan 29, 2024 at 08:18:15AM -1000, Tejun Heo wrote:
> Hello,
> 
> On Fri, Jan 26, 2024 at 07:05:35PM -0300, Leonardo Bras wrote:
> > > 	if (housekeeping_enabled(HK_TYPE_TIMER)) {
> > > 		cpu = smp_processor_id();
> > > 		if (!housekeeping_test_cpu(cpu, HK_TYPE_TIMER))
> > > 			cpu = housekeeping_any_cpu(HK_TYPE_TIMER);
> > > 		add_timer_on(timer, cpu);
> > > 	} else {
> > > 		if (likely(cpu == WORK_CPU_UNBOUND))
> > > 			add_timer(timer, cpu);
> > > 		else
> > > 			add_timer_on(timer, cpu);
> > > 	}
> > > 
> > > Thanks.
> > 
> > I am not really against it, but for me it's kind of weird to have that many 
> > calls to add_timer_on() if we can avoid it. 
> > 
> > I would rather go with:
> > 
> > ###
> > if (unlikely(cpu != WORK_CPU_UNBOUND)) {
> > 	add_timer_on(timer, cpu);
> > 	return;
> > }
> > 
> > if (!housekeeping_enabled(HK_TYPE_TIMER)) {
> > 	add_timer(timer);
> > 	return;
> > }
> > 
> > cpu = smp_processor_id();
> > if (!housekeeping_test_cpu(cpu, HK_TYPE_TIMER))
> > 	cpu = housekeeping_any_cpu(HK_TYPE_TIMER);
> > 
> > add_timer_on(timer, cpu);
> > ###
> > 
> > What do you think?
> 
> Isn't that still the same number of add_timer[_on]() calls?

Yeah, sorry about this, what I meant was: If we are ok on calling 
add_timer_on() multiple times, I would rather go with the above version, as 
I think it's better for readability.

> 
> Thanks.

Thank you for reviewing!
Leo

> 
> -- 
> tejun
> 

Re: [PATCH v1 1/1] wq: Avoid using isolated cpus' timers on unbounded queue_delayed_work

[Tejun Heo]

On Mon, Jan 29, 2024 at 04:26:57PM -0300, Leonardo Bras wrote:
> > Isn't that still the same number of add_timer[_on]() calls?
> 
> Yeah, sorry about this, what I meant was: If we are ok on calling 
> add_timer_on() multiple times, I would rather go with the above version, as 
> I think it's better for readability.

I don't know. It looks more verbose and less clear to me in that it isn't
immediately clear that every branch ends with timer being added. But this is
really minor, so unless you have a really strong opinion against the
suggested structured, can we just do that?

Thanks.

-- 
tejun

Re: [PATCH v1 1/1] wq: Avoid using isolated cpus' timers on unbounded queue_delayed_work

[Leonardo Bras]

On Mon, Jan 29, 2024 at 10:51:09AM -1000, Tejun Heo wrote:
> On Mon, Jan 29, 2024 at 04:26:57PM -0300, Leonardo Bras wrote:
> > > Isn't that still the same number of add_timer[_on]() calls?
> > 
> > Yeah, sorry about this, what I meant was: If we are ok on calling 
> > add_timer_on() multiple times, I would rather go with the above version, as 
> > I think it's better for readability.
> 
> I don't know. It looks more verbose and less clear to me in that it isn't
> immediately clear that every branch ends with timer being added. But this is
> really minor, so unless you have a really strong opinion against the
> suggested structured, can we just do that?

Sure, we can go with the one you suggested.

Thanks!
Leo

> 
> Thanks.
> 
> -- 
> tejun
> 

Re: [PATCH v1 1/1] wq: Avoid using isolated cpus' timers on unbounded queue_delayed_work

[Leonardo Bras]

On Mon, Jan 29, 2024 at 05:54:04PM -0300, Leonardo Bras wrote:
> On Mon, Jan 29, 2024 at 10:51:09AM -1000, Tejun Heo wrote:
> > On Mon, Jan 29, 2024 at 04:26:57PM -0300, Leonardo Bras wrote:
> > > > Isn't that still the same number of add_timer[_on]() calls?
> > > 
> > > Yeah, sorry about this, what I meant was: If we are ok on calling 
> > > add_timer_on() multiple times, I would rather go with the above version, as 
> > > I think it's better for readability.
> > 
> > I don't know. It looks more verbose and less clear to me in that it isn't
> > immediately clear that every branch ends with timer being added. But this is
> > really minor, so unless you have a really strong opinion against the
> > suggested structured, can we just do that?
> 
> Sure, we can go with the one you suggested.

Hello Tejun,

While I was reworking the code with your suggestion, I found out 
that there is a difference in our suggestions:

###
        if (housekeeping_enabled(HK_TYPE_TIMER)) {
                cpu = smp_processor_id();
                if (!housekeeping_test_cpu(cpu, HK_TYPE_TIMER))
                        cpu = housekeeping_any_cpu(HK_TYPE_TIMER);
                add_timer_on(timer, cpu);
        } else {
                if (likely(cpu == WORK_CPU_UNBOUND))
                        add_timer(timer, cpu);
                else
                        add_timer_on(timer, cpu);
        }
###

In your suggestion you always check housekeeping to be sure that the timer 
is never handled on an isolated cpu.

In my suggestions, it only checks and use housekeeping cpus only
if (cpu == WORK_CPU_UNBOUND).

I was previously afraid to use the approach of your suggestion due to some 
user having the need to handle the timer on the cpu passed as parameter.

But now thinking on it again, it does make sense that the CPU which handles 
the timer has nothing to do with the one that actually runs the work
(other than possible cacheline optimizations), and your suggestion is much 
better for CPU Isolation than mine.

I will send the v2 soon.

Thanks again,
Leo

> 
> Thanks!
> Leo
> 
> > 
> > Thanks.
> > 
> > -- 
> > tejun
> >