[GIT PULL] workqueue changes for v4.2-rc1

[GIT PULL] workqueue changes for v4.2-rc1

[Tejun Heo]

Hello, Linus.

Most of the changes are around implementing and fixing fallouts from
sysfs and internal interface to limit the CPUs available to all
unbound workqueues to help isolating CPUs.  It needs more work as
ordered workqueues can roam unrestricted but still is a significant
improvement.

Thanks.

The following changes since commit b787f68c36d49bb1d9236f403813641efa74a031:

  Linux 4.1-rc1 (2015-04-26 17:59:10 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq.git for-4.2

for you to fetch changes up to 402dd89d6cdbeeaab42b810542b487017725c628:

  workqueue: fix typos in comments (2015-05-29 09:20:01 -0400)

----------------------------------------------------------------
Chen Hanxiao (1):
      workqueue: fix a typo

Chris Bainbridge (1):
      workqueue: fix trivial typo in Documentation/workqueue.txt

Frederic Weisbecker (1):
      workqueue: Create low-level unbound workqueues cpumask

Gong Zhaogang (1):
      workqueue: function name in the comment differs from the real function name

Lai Jiangshan (9):
      workqueue: split apply_workqueue_attrs() into 3 stages
      workqueue: Allow modifying low level unbound workqueue cpumask
      workqueue: wq_pool_mutex protects the attrs-installation
      workqueue: simplify wq_update_unbound_numa()
      workqueue: separate out and refactor the locking of applying attrs
      workqueue: ensure attrs changes are properly synchronized
      workqueue: remove the declaration of copy_workqueue_attrs()
      workqueue: remove the lock from wq_sysfs_prep_attrs()
      workqueue: move flush_scheduled_work() to workqueue.h

Shailendra Verma (1):
      workqueue: fix typos in comments

 Documentation/workqueue.txt |   2 +-
 include/linux/workqueue.h   |  31 ++-
 kernel/workqueue.c          | 484 +++++++++++++++++++++++++++++---------------
 3 files changed, 353 insertions(+), 164 deletions(-)

-- 
tejun

Re: [GIT PULL] workqueue changes for v4.2-rc1

[Tejun Heo]

On Fri, Jun 26, 2015 at 11:35:52AM -0400, Tejun Heo wrote:
> Hello, Linus.
> 
> Most of the changes are around implementing and fixing fallouts from
> sysfs and internal interface to limit the CPUs available to all
> unbound workqueues to help isolating CPUs.  It needs more work as
> ordered workqueues can roam unrestricted but still is a significant
> improvement.

Ooh, it isn't in mainline yet but pulling rcu tree will cause a silent
conflict with this pull request which leads to build failure.  The two
colliding commits are.

 5b95e1af8d17 ("workqueue: wq_pool_mutex protects the attrs-installation")
 eeacf8982637 ("rcu: Rename rcu_lockdep_assert() to RCU_LOCKDEP_WARN()")

The former adds rcu_lockdep_assert() usage and the latter renames and
flips it.  It can be resolved by renaming and negating the conditions
in the new usage.

Thanks.

-- 
tejun

Re: [GIT PULL] workqueue changes for v4.2-rc1

[Linus Torvalds]

On Fri, Jun 26, 2015 at 9:01 AM, Tejun Heo <tj@kernel.org> wrote:
>
> Ooh, it isn't in mainline yet but pulling rcu tree will cause a silent
> conflict with this pull request which leads to build failure.

I tend to try to do a full "make allmodconfig" build between all pull
requests (although I can optimize that a bit for very targeted pull
requests), so hopefully I'll notice and remember your note.

But just in case:

> The two colliding commits are.
>
>  5b95e1af8d17 ("workqueue: wq_pool_mutex protects the attrs-installation")
>  eeacf8982637 ("rcu: Rename rcu_lockdep_assert() to RCU_LOCKDEP_WARN()")
>
> The former adds rcu_lockdep_assert() usage and the latter renames and
> flips it.  It can be resolved by renaming and negating the conditions
> in the new usage.

it would be great if when I get the RCU pull request that introduces
that renaming, whoever sends it to me could remind me about it.

I'm assuming the pull request will come through Ingo. Ingo?

                      Linus

Re: [GIT PULL] workqueue changes for v4.2-rc1

[Stephen Rothwell]

Hi Linus,

On Fri, 26 Jun 2015 20:18:10 -0700 Linus Torvalds <torvalds@linux-foundation.org> wrote:
>
> On Fri, Jun 26, 2015 at 9:01 AM, Tejun Heo <tj@kernel.org> wrote:
> >
> > Ooh, it isn't in mainline yet but pulling rcu tree will cause a silent
> > conflict with this pull request which leads to build failure.
> 
> I tend to try to do a full "make allmodconfig" build between all pull
> requests (although I can optimize that a bit for very targeted pull
> requests), so hopefully I'll notice and remember your note.
> 
> But just in case:
> 
> > The two colliding commits are.
> >
> >  5b95e1af8d17 ("workqueue: wq_pool_mutex protects the attrs-installation")
> >  eeacf8982637 ("rcu: Rename rcu_lockdep_assert() to RCU_LOCKDEP_WARN()")
> >
> > The former adds rcu_lockdep_assert() usage and the latter renames and
> > flips it.  It can be resolved by renaming and negating the conditions
> > in the new usage.
> 
> it would be great if when I get the RCU pull request that introduces
> that renaming, whoever sends it to me could remind me about it.

I was wondering why I didn't see that in linux-next ... turns out I
did, but that rcu commit vanished after June 23 ...  I have no idea
where it went, but it has not been in the last 3 -next releases.

If it turns up again, this is the merge fix patch I was using:

From: Stephen Rothwell <sfr@canb.auug.org.au>
Date: Sat, 20 Jun 2015 19:39:43 +1000
Subject: [PATCH] workqueue: fix up for rcu_lockdep_assert() rename

Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
---
 kernel/workqueue.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 837427cc5bdf..44cd4144ebcb 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -348,10 +348,10 @@ static void workqueue_sysfs_unregister(struct workqueue_struct *wq);
 			 "sched RCU or wq->mutex should be held")
 
 #define assert_rcu_or_wq_mutex_or_pool_mutex(wq)			\
-	rcu_lockdep_assert(rcu_read_lock_sched_held() ||		\
-			   lockdep_is_held(&wq->mutex) ||		\
-			   lockdep_is_held(&wq_pool_mutex),		\
-			   "sched RCU, wq->mutex or wq_pool_mutex should be held")
+	RCU_LOCKDEP_WARN(!rcu_read_lock_sched_held() &&		\
+			 !lockdep_is_held(&wq->mutex) &&		\
+			 !lockdep_is_held(&wq_pool_mutex),		\
+			 "sched RCU, wq->mutex or wq_pool_mutex should be held")
 
 #define for_each_cpu_worker_pool(pool, cpu)				\
 	for ((pool) = &per_cpu(cpu_worker_pools, cpu)[0];		\
-- 
2.1.4

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

Re: [GIT PULL] workqueue changes for v4.2-rc1

[Paul E. McKenney]

On Sat, Jun 27, 2015 at 02:35:56PM +1000, Stephen Rothwell wrote:
> Hi Linus,
> 
> On Fri, 26 Jun 2015 20:18:10 -0700 Linus Torvalds <torvalds@linux-foundation.org> wrote:
> >
> > On Fri, Jun 26, 2015 at 9:01 AM, Tejun Heo <tj@kernel.org> wrote:
> > >
> > > Ooh, it isn't in mainline yet but pulling rcu tree will cause a silent
> > > conflict with this pull request which leads to build failure.
> > 
> > I tend to try to do a full "make allmodconfig" build between all pull
> > requests (although I can optimize that a bit for very targeted pull
> > requests), so hopefully I'll notice and remember your note.
> > 
> > But just in case:
> > 
> > > The two colliding commits are.
> > >
> > >  5b95e1af8d17 ("workqueue: wq_pool_mutex protects the attrs-installation")
> > >  eeacf8982637 ("rcu: Rename rcu_lockdep_assert() to RCU_LOCKDEP_WARN()")
> > >
> > > The former adds rcu_lockdep_assert() usage and the latter renames and
> > > flips it.  It can be resolved by renaming and negating the conditions
> > > in the new usage.
> > 
> > it would be great if when I get the RCU pull request that introduces
> > that renaming, whoever sends it to me could remind me about it.
> 
> I was wondering why I didn't see that in linux-next ... turns out I
> did, but that rcu commit vanished after June 23 ...  I have no idea
> where it went, but it has not been in the last 3 -next releases.

On that date, I moved my rcu/next branch to the commit that I sent to
Ingo in my pull request for the current merge window.  As I understand
it, during the merge window, I am not supposed to advertise commits
to -next that are not destined for that merge window.  When the merge
window closes, I will rebase the rest of the RCU commits to v4.2-rc1,
at which point an updated version of that commit will reappear.

> If it turns up again, this is the merge fix patch I was using:

Thank you, I will include this.

							Thanx, Paul

> From: Stephen Rothwell <sfr@canb.auug.org.au>
> Date: Sat, 20 Jun 2015 19:39:43 +1000
> Subject: [PATCH] workqueue: fix up for rcu_lockdep_assert() rename
> 
> Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
> ---
>  kernel/workqueue.c | 8 ++++----
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/kernel/workqueue.c b/kernel/workqueue.c
> index 837427cc5bdf..44cd4144ebcb 100644
> --- a/kernel/workqueue.c
> +++ b/kernel/workqueue.c
> @@ -348,10 +348,10 @@ static void workqueue_sysfs_unregister(struct workqueue_struct *wq);
>  			 "sched RCU or wq->mutex should be held")
> 
>  #define assert_rcu_or_wq_mutex_or_pool_mutex(wq)			\
> -	rcu_lockdep_assert(rcu_read_lock_sched_held() ||		\
> -			   lockdep_is_held(&wq->mutex) ||		\
> -			   lockdep_is_held(&wq_pool_mutex),		\
> -			   "sched RCU, wq->mutex or wq_pool_mutex should be held")
> +	RCU_LOCKDEP_WARN(!rcu_read_lock_sched_held() &&		\
> +			 !lockdep_is_held(&wq->mutex) &&		\
> +			 !lockdep_is_held(&wq_pool_mutex),		\
> +			 "sched RCU, wq->mutex or wq_pool_mutex should be held")
> 
>  #define for_each_cpu_worker_pool(pool, cpu)				\
>  	for ((pool) = &per_cpu(cpu_worker_pools, cpu)[0];		\
> -- 
> 2.1.4
> 
> -- 
> Cheers,
> Stephen Rothwell                    sfr@canb.auug.org.au
> 

Re: [GIT PULL] workqueue changes for v4.2-rc1

[Ingo Molnar]

* Linus Torvalds <torvalds@linux-foundation.org> wrote:

> On Fri, Jun 26, 2015 at 9:01 AM, Tejun Heo <tj@kernel.org> wrote:
> >
> > Ooh, it isn't in mainline yet but pulling rcu tree will cause a silent
> > conflict with this pull request which leads to build failure.
> 
> I tend to try to do a full "make allmodconfig" build between all pull
> requests (although I can optimize that a bit for very targeted pull
> requests), so hopefully I'll notice and remember your note.
> 
> But just in case:
> 
> > The two colliding commits are.
> >
> >  5b95e1af8d17 ("workqueue: wq_pool_mutex protects the attrs-installation")
> >  eeacf8982637 ("rcu: Rename rcu_lockdep_assert() to RCU_LOCKDEP_WARN()")
> >
> > The former adds rcu_lockdep_assert() usage and the latter renames and flips 
> > it.  It can be resolved by renaming and negating the conditions in the new 
> > usage.
> 
> it would be great if when I get the RCU pull request that introduces that 
> renaming, whoever sends it to me could remind me about it.
> 
> I'm assuming the pull request will come through Ingo. Ingo?

Yeah.

There was some discussion about how to warn about RCU failures precisely, so I 
think Paul yanked the new style RCU warnings for the time being. When/if they
come back I'll be careful and will remind you of semantic conflicts.

Thanks,

	Ingo

Re: [GIT PULL] workqueue changes for v4.2-rc1

[Paul E. McKenney]

On Sat, Jun 27, 2015 at 10:09:28AM +0200, Ingo Molnar wrote:
> 
> * Linus Torvalds <torvalds@linux-foundation.org> wrote:
> 
> > On Fri, Jun 26, 2015 at 9:01 AM, Tejun Heo <tj@kernel.org> wrote:
> > >
> > > Ooh, it isn't in mainline yet but pulling rcu tree will cause a silent
> > > conflict with this pull request which leads to build failure.
> > 
> > I tend to try to do a full "make allmodconfig" build between all pull
> > requests (although I can optimize that a bit for very targeted pull
> > requests), so hopefully I'll notice and remember your note.
> > 
> > But just in case:
> > 
> > > The two colliding commits are.
> > >
> > >  5b95e1af8d17 ("workqueue: wq_pool_mutex protects the attrs-installation")
> > >  eeacf8982637 ("rcu: Rename rcu_lockdep_assert() to RCU_LOCKDEP_WARN()")
> > >
> > > The former adds rcu_lockdep_assert() usage and the latter renames and flips 
> > > it.  It can be resolved by renaming and negating the conditions in the new 
> > > usage.
> > 
> > it would be great if when I get the RCU pull request that introduces that 
> > renaming, whoever sends it to me could remind me about it.
> > 
> > I'm assuming the pull request will come through Ingo. Ingo?
> 
> Yeah.
> 
> There was some discussion about how to warn about RCU failures precisely, so I 
> think Paul yanked the new style RCU warnings for the time being. When/if they
> come back I'll be careful and will remind you of semantic conflicts.

Yes, it ended up in the batch destined for v4.3.

If it would make things easier, I could easily introduce the new API in
v4.3, along with the changes visible at that time, and pull the old API
in v4.4.  That way, the conflicts appearing in v4.4 could be resolved
in the originating tree, given that the new API would then be in place
everywhere.

Either way works for me, just let me know!

							Thanx, Paul

Re: [GIT PULL] workqueue changes for v4.2-rc1

[Ingo Molnar]

* Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:

> On Sat, Jun 27, 2015 at 10:09:28AM +0200, Ingo Molnar wrote:
> > 
> > * Linus Torvalds <torvalds@linux-foundation.org> wrote:
> > 
> > > On Fri, Jun 26, 2015 at 9:01 AM, Tejun Heo <tj@kernel.org> wrote:
> > > >
> > > > Ooh, it isn't in mainline yet but pulling rcu tree will cause a silent
> > > > conflict with this pull request which leads to build failure.
> > > 
> > > I tend to try to do a full "make allmodconfig" build between all pull
> > > requests (although I can optimize that a bit for very targeted pull
> > > requests), so hopefully I'll notice and remember your note.
> > > 
> > > But just in case:
> > > 
> > > > The two colliding commits are.
> > > >
> > > >  5b95e1af8d17 ("workqueue: wq_pool_mutex protects the attrs-installation")
> > > >  eeacf8982637 ("rcu: Rename rcu_lockdep_assert() to RCU_LOCKDEP_WARN()")
> > > >
> > > > The former adds rcu_lockdep_assert() usage and the latter renames and flips 
> > > > it.  It can be resolved by renaming and negating the conditions in the new 
> > > > usage.
> > > 
> > > it would be great if when I get the RCU pull request that introduces that 
> > > renaming, whoever sends it to me could remind me about it.
> > > 
> > > I'm assuming the pull request will come through Ingo. Ingo?
> > 
> > Yeah.
> > 
> > There was some discussion about how to warn about RCU failures precisely, so I 
> > think Paul yanked the new style RCU warnings for the time being. When/if they
> > come back I'll be careful and will remind you of semantic conflicts.
> 
> Yes, it ended up in the batch destined for v4.3.
> 
> If it would make things easier, I could easily introduce the new API in
> v4.3, along with the changes visible at that time, and pull the old API
> in v4.4.  That way, the conflicts appearing in v4.4 could be resolved
> in the originating tree, given that the new API would then be in place
> everywhere.
> 
> Either way works for me, just let me know!

I think having it all in v4.3 is perfectly fine!

Thanks,

	Ingo

Re: [GIT PULL] workqueue changes for v4.2-rc1

[Paul E. McKenney]

On Mon, Jun 29, 2015 at 08:52:20AM +0200, Ingo Molnar wrote:
> 
> * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote:
> 
> > On Sat, Jun 27, 2015 at 10:09:28AM +0200, Ingo Molnar wrote:
> > > 
> > > * Linus Torvalds <torvalds@linux-foundation.org> wrote:
> > > 
> > > > On Fri, Jun 26, 2015 at 9:01 AM, Tejun Heo <tj@kernel.org> wrote:
> > > > >
> > > > > Ooh, it isn't in mainline yet but pulling rcu tree will cause a silent
> > > > > conflict with this pull request which leads to build failure.
> > > > 
> > > > I tend to try to do a full "make allmodconfig" build between all pull
> > > > requests (although I can optimize that a bit for very targeted pull
> > > > requests), so hopefully I'll notice and remember your note.
> > > > 
> > > > But just in case:
> > > > 
> > > > > The two colliding commits are.
> > > > >
> > > > >  5b95e1af8d17 ("workqueue: wq_pool_mutex protects the attrs-installation")
> > > > >  eeacf8982637 ("rcu: Rename rcu_lockdep_assert() to RCU_LOCKDEP_WARN()")
> > > > >
> > > > > The former adds rcu_lockdep_assert() usage and the latter renames and flips 
> > > > > it.  It can be resolved by renaming and negating the conditions in the new 
> > > > > usage.
> > > > 
> > > > it would be great if when I get the RCU pull request that introduces that 
> > > > renaming, whoever sends it to me could remind me about it.
> > > > 
> > > > I'm assuming the pull request will come through Ingo. Ingo?
> > > 
> > > Yeah.
> > > 
> > > There was some discussion about how to warn about RCU failures precisely, so I 
> > > think Paul yanked the new style RCU warnings for the time being. When/if they
> > > come back I'll be careful and will remind you of semantic conflicts.
> > 
> > Yes, it ended up in the batch destined for v4.3.
> > 
> > If it would make things easier, I could easily introduce the new API in
> > v4.3, along with the changes visible at that time, and pull the old API
> > in v4.4.  That way, the conflicts appearing in v4.4 could be resolved
> > in the originating tree, given that the new API would then be in place
> > everywhere.
> > 
> > Either way works for me, just let me know!
> 
> I think having it all in v4.3 is perfectly fine!

Will do!

							Thanx, Paul