* [PATCH tip/core/rcu 1/5] srcu: Abstract function name
2018-02-26 21:56 [PATCH tip/core/rcu 0/5] SRCU updates Paul E. McKenney
@ 2018-02-26 21:56 ` Paul E. McKenney
2018-02-26 21:56 ` [PATCH tip/core/rcu 2/5] srcu: Prevent sdp->srcu_gp_seq_needed_exp counter wrap Paul E. McKenney
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Paul E. McKenney @ 2018-02-26 21:56 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, jiangshanlai, dipankar, akpm, mathieu.desnoyers, josh,
tglx, peterz, rostedt, dhowells, edumazet, fweisbec, oleg,
Paul E. McKenney
This commit moves to __func__ for function names in the name of better
resilience to change.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/rcu/srcutree.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
index d5cea81378cc..01f4a8bf0e03 100644
--- a/kernel/rcu/srcutree.c
+++ b/kernel/rcu/srcutree.c
@@ -386,7 +386,7 @@ void cleanup_srcu_struct(struct srcu_struct *sp)
flush_delayed_work(&per_cpu_ptr(sp->sda, cpu)->work);
if (WARN_ON(rcu_seq_state(READ_ONCE(sp->srcu_gp_seq)) != SRCU_STATE_IDLE) ||
WARN_ON(srcu_readers_active(sp))) {
- pr_info("cleanup_srcu_struct: Active srcu_struct %p state: %d\n", sp, rcu_seq_state(READ_ONCE(sp->srcu_gp_seq)));
+ pr_info("%s: Active srcu_struct %p state: %d\n", __func__, sp, rcu_seq_state(READ_ONCE(sp->srcu_gp_seq)));
return; /* Caller forgot to stop doing call_srcu()? */
}
free_percpu(sp->sda);
--
2.5.2
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH tip/core/rcu 2/5] srcu: Prevent sdp->srcu_gp_seq_needed_exp counter wrap
2018-02-26 21:56 [PATCH tip/core/rcu 0/5] SRCU updates Paul E. McKenney
2018-02-26 21:56 ` [PATCH tip/core/rcu 1/5] srcu: Abstract function name Paul E. McKenney
@ 2018-02-26 21:56 ` Paul E. McKenney
2018-02-26 21:56 ` [PATCH tip/core/rcu 3/5] srcu: Reduce scans of srcu_data in counter wrap check Paul E. McKenney
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Paul E. McKenney @ 2018-02-26 21:56 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, jiangshanlai, dipankar, akpm, mathieu.desnoyers, josh,
tglx, peterz, rostedt, dhowells, edumazet, fweisbec, oleg,
Ildar Ismagilov, Paul E. McKenney
From: Ildar Ismagilov <devix84@gmail.com>
SRCU checks each srcu_data structure's grace-period number for counter
wrap four times per cycle by default. This frequency guarantees that
normal comparisons will detect potential wrap. However, the expedited
grace-period number is not checked. The consquences are not too horrible
(a failure to expedite a grace period when requested), but it would be
good to avoid such things. This commit therefore adds this check to
the expedited grace-period number.
Signed-off-by: Ildar Ismagilov <devix84@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/rcu/srcutree.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
index 01f4a8bf0e03..bb9607baad9b 100644
--- a/kernel/rcu/srcutree.c
+++ b/kernel/rcu/srcutree.c
@@ -579,6 +579,9 @@ static void srcu_gp_end(struct srcu_struct *sp)
if (ULONG_CMP_GE(gpseq,
sdp->srcu_gp_seq_needed + 100))
sdp->srcu_gp_seq_needed = gpseq;
+ if (ULONG_CMP_GE(gpseq,
+ sdp->srcu_gp_seq_needed_exp + 100))
+ sdp->srcu_gp_seq_needed_exp = gpseq;
spin_unlock_irqrestore_rcu_node(sdp, flags);
}
}
--
2.5.2
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH tip/core/rcu 3/5] srcu: Reduce scans of srcu_data in counter wrap check
2018-02-26 21:56 [PATCH tip/core/rcu 0/5] SRCU updates Paul E. McKenney
2018-02-26 21:56 ` [PATCH tip/core/rcu 1/5] srcu: Abstract function name Paul E. McKenney
2018-02-26 21:56 ` [PATCH tip/core/rcu 2/5] srcu: Prevent sdp->srcu_gp_seq_needed_exp counter wrap Paul E. McKenney
@ 2018-02-26 21:56 ` Paul E. McKenney
2018-02-26 21:56 ` [PATCH tip/core/rcu 4/5] srcu: Remove dead code in srcu_gp_end() Paul E. McKenney
2018-02-26 21:56 ` [PATCH tip/core/rcu 5/5] rcu: Remove SRCU throttling Paul E. McKenney
4 siblings, 0 replies; 6+ messages in thread
From: Paul E. McKenney @ 2018-02-26 21:56 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, jiangshanlai, dipankar, akpm, mathieu.desnoyers, josh,
tglx, peterz, rostedt, dhowells, edumazet, fweisbec, oleg,
Ildar Ismagilov, Paul E. McKenney
From: Ildar Ismagilov <devix84@gmail.com>
Currently, given a multi-level srcu_node tree, SRCU can scan the full
set of srcu_data structures at each level when cleaning up after a grace
period. This, though harmless otherwise, represents pointless overhead.
This commit therefore eliminates this overhead by scanning the srcu_data
structures only when traversing the leaf srcu_node structures.
Signed-off-by: Ildar Ismagilov <devix84@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/rcu/srcutree.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
index bb9607baad9b..d582a2352cdb 100644
--- a/kernel/rcu/srcutree.c
+++ b/kernel/rcu/srcutree.c
@@ -527,6 +527,7 @@ static void srcu_gp_end(struct srcu_struct *sp)
{
unsigned long cbdelay;
bool cbs;
+ bool last_lvl;
int cpu;
unsigned long flags;
unsigned long gpseq;
@@ -559,7 +560,8 @@ static void srcu_gp_end(struct srcu_struct *sp)
rcu_for_each_node_breadth_first(sp, snp) {
spin_lock_irq_rcu_node(snp);
cbs = false;
- if (snp >= sp->level[rcu_num_lvls - 1])
+ last_lvl = snp >= sp->level[rcu_num_lvls - 1];
+ if (last_lvl)
cbs = snp->srcu_have_cbs[idx] == gpseq;
snp->srcu_have_cbs[idx] = gpseq;
rcu_seq_set_state(&snp->srcu_have_cbs[idx], 1);
@@ -572,7 +574,7 @@ static void srcu_gp_end(struct srcu_struct *sp)
srcu_schedule_cbs_snp(sp, snp, mask, cbdelay);
/* Occasionally prevent srcu_data counter wrap. */
- if (!(gpseq & counter_wrap_check))
+ if (!(gpseq & counter_wrap_check) && last_lvl)
for (cpu = snp->grplo; cpu <= snp->grphi; cpu++) {
sdp = per_cpu_ptr(sp->sda, cpu);
spin_lock_irqsave_rcu_node(sdp, flags);
--
2.5.2
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH tip/core/rcu 4/5] srcu: Remove dead code in srcu_gp_end()
2018-02-26 21:56 [PATCH tip/core/rcu 0/5] SRCU updates Paul E. McKenney
` (2 preceding siblings ...)
2018-02-26 21:56 ` [PATCH tip/core/rcu 3/5] srcu: Reduce scans of srcu_data in counter wrap check Paul E. McKenney
@ 2018-02-26 21:56 ` Paul E. McKenney
2018-02-26 21:56 ` [PATCH tip/core/rcu 5/5] rcu: Remove SRCU throttling Paul E. McKenney
4 siblings, 0 replies; 6+ messages in thread
From: Paul E. McKenney @ 2018-02-26 21:56 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, jiangshanlai, dipankar, akpm, mathieu.desnoyers, josh,
tglx, peterz, rostedt, dhowells, edumazet, fweisbec, oleg,
Byungchul Park, Paul E. McKenney
From: Byungchul Park <byungchul.park@lge.com>
Of course, compilers will optimize out a dead code. Anyway, remove
any dead code for better readibility.
Signed-off-by: Byungchul Park <byungchul.park@lge.com>
Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/rcu/srcutree.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
index d582a2352cdb..f962f4428c39 100644
--- a/kernel/rcu/srcutree.c
+++ b/kernel/rcu/srcutree.c
@@ -532,7 +532,6 @@ static void srcu_gp_end(struct srcu_struct *sp)
unsigned long flags;
unsigned long gpseq;
int idx;
- int idxnext;
unsigned long mask;
struct srcu_data *sdp;
struct srcu_node *snp;
@@ -556,7 +555,6 @@ static void srcu_gp_end(struct srcu_struct *sp)
/* Initiate callback invocation as needed. */
idx = rcu_seq_ctr(gpseq) % ARRAY_SIZE(snp->srcu_have_cbs);
- idxnext = (idx + 1) % ARRAY_SIZE(snp->srcu_have_cbs);
rcu_for_each_node_breadth_first(sp, snp) {
spin_lock_irq_rcu_node(snp);
cbs = false;
--
2.5.2
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH tip/core/rcu 5/5] rcu: Remove SRCU throttling
2018-02-26 21:56 [PATCH tip/core/rcu 0/5] SRCU updates Paul E. McKenney
` (3 preceding siblings ...)
2018-02-26 21:56 ` [PATCH tip/core/rcu 4/5] srcu: Remove dead code in srcu_gp_end() Paul E. McKenney
@ 2018-02-26 21:56 ` Paul E. McKenney
4 siblings, 0 replies; 6+ messages in thread
From: Paul E. McKenney @ 2018-02-26 21:56 UTC (permalink / raw)
To: linux-kernel
Cc: mingo, jiangshanlai, dipankar, akpm, mathieu.desnoyers, josh,
tglx, peterz, rostedt, dhowells, edumazet, fweisbec, oleg,
Paul E. McKenney
The code in srcu_gp_end() inserts a delay every 0x3ff grace periods in
order to prevent SRCU grace-period work from consuming an entire CPU
when there is a long sequence of expedited SRCU grace-period requests.
However, all of SRCU's grace-period work is carried out in workqueues,
which are in turn within kthreads, which are automatically throttled as
needed by the scheduler. In particular, if there is plenty of idle time,
there is no point in throttling.
This commit therefore removes the expedited SRCU grace-period throttling.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
---
kernel/rcu/srcutree.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c
index f962f4428c39..e2796f79a597 100644
--- a/kernel/rcu/srcutree.c
+++ b/kernel/rcu/srcutree.c
@@ -596,9 +596,7 @@ static void srcu_gp_end(struct srcu_struct *sp)
ULONG_CMP_LT(gpseq, sp->srcu_gp_seq_needed)) {
srcu_gp_start(sp);
spin_unlock_irq_rcu_node(sp);
- /* Throttle expedited grace periods: Should be rare! */
- srcu_reschedule(sp, rcu_seq_ctr(gpseq) & 0x3ff
- ? 0 : SRCU_INTERVAL);
+ srcu_reschedule(sp, 0);
} else {
spin_unlock_irq_rcu_node(sp);
}
--
2.5.2
^ permalink raw reply related [flat|nested] 6+ messages in thread