From: Frederic Weisbecker <frederic@kernel.org>
To: Thomas Gleixner <tglx@linutronix.de>,
Anna-Maria Behnsen <anna-maria@linutronix.de>
Cc: LKML <linux-kernel@vger.kernel.org>,
Frederic Weisbecker <frederic@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Juri Lelli <juri.lelli@redhat.com>
Subject: [PATCH 06/12] timer: Optimize _next_timer_interrupt() level iteration
Date: Fri, 17 Jul 2020 16:05:45 +0200 [thread overview]
Message-ID: <20200717140551.29076-7-frederic@kernel.org> (raw)
In-Reply-To: <20200717140551.29076-1-frederic@kernel.org>
If a level has a timer that expires before we reach the next level,
there is no need to iterate further.
The next level is reached when the 3 lower bits of the current level are
cleared. If the next event happens before/during that, the next levels
won't give us an earlier expiration.
Tested-by: Juri Lelli <juri.lelli@redhat.com>
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Anna-Maria Behnsen <anna-maria@linutronix.de>
Cc: Juri Lelli <juri.lelli@redhat.com>
---
kernel/time/timer.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/kernel/time/timer.c b/kernel/time/timer.c
index af1c08b0b168..9abc41715fd2 100644
--- a/kernel/time/timer.c
+++ b/kernel/time/timer.c
@@ -1526,6 +1526,7 @@ static unsigned long __next_timer_interrupt(struct timer_base *base)
clk = base->clk;
for (lvl = 0; lvl < LVL_DEPTH; lvl++, offset += LVL_SIZE) {
int pos = next_pending_bucket(base, offset, clk & LVL_MASK);
+ unsigned long lvl_clk = clk & LVL_CLK_MASK;
if (pos >= 0) {
unsigned long tmp = clk + (unsigned long) pos;
@@ -1533,6 +1534,13 @@ static unsigned long __next_timer_interrupt(struct timer_base *base)
tmp <<= LVL_SHIFT(lvl);
if (time_before(tmp, next))
next = tmp;
+
+ /*
+ * If the next expiration happens before we reach
+ * the next level, no need to check further.
+ */
+ if (pos <= ((LVL_CLK_DIV - lvl_clk) & LVL_CLK_MASK))
+ break;
}
/*
* Clock for the next level. If the current level clock lower
@@ -1570,7 +1578,7 @@ static unsigned long __next_timer_interrupt(struct timer_base *base)
* So the simple check whether the lower bits of the current
* level are 0 or not is sufficient for all cases.
*/
- adj = clk & LVL_CLK_MASK ? 1 : 0;
+ adj = lvl_clk ? 1 : 0;
clk >>= LVL_CLK_SHIFT;
clk += adj;
}
--
2.26.2
next prev parent reply other threads:[~2020-07-17 14:06 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-17 14:05 [PATCH 00/11] timer: Reduce timers softirq v3 Frederic Weisbecker
2020-07-17 14:05 ` [PATCH 01/12] timer: Fix wheel index calculation on last level Frederic Weisbecker
2020-07-17 19:49 ` [tip: timers/urgent] " tip-bot2 for Frederic Weisbecker
2020-07-17 14:05 ` [PATCH 02/12] timer: Preserve higher bits of expiration on index calculation Frederic Weisbecker
2020-07-17 20:00 ` [tip: timers/core] timers: " tip-bot2 for Frederic Weisbecker
2020-07-17 14:05 ` [PATCH 03/12] timers: Use only bucket expiry for base->next_expiry value Frederic Weisbecker
2020-07-17 20:00 ` [tip: timers/core] " tip-bot2 for Anna-Maria Behnsen
2020-07-17 14:05 ` [PATCH 04/12] timer: Move trigger_dyntick_cpu() to enqueue_timer() Frederic Weisbecker
2020-07-17 20:00 ` [tip: timers/core] timers: " tip-bot2 for Frederic Weisbecker
2020-07-17 14:05 ` [PATCH 05/12] timer: Add comments about calc_index() ceiling work Frederic Weisbecker
2020-07-17 20:00 ` [tip: timers/core] timers: " tip-bot2 for Frederic Weisbecker
2020-07-17 14:05 ` Frederic Weisbecker [this message]
2020-07-17 20:00 ` [tip: timers/core] timers: Optimize _next_timer_interrupt() level iteration tip-bot2 for Frederic Weisbecker
2020-07-17 14:05 ` [PATCH 07/12] timers: Always keep track of next expiry Frederic Weisbecker
2020-07-17 20:00 ` [tip: timers/core] " tip-bot2 for Frederic Weisbecker
2020-07-17 14:05 ` [PATCH 08/12] timer: Reuse next expiry cache after nohz exit Frederic Weisbecker
2020-07-17 20:00 ` [tip: timers/core] timers: " tip-bot2 for Frederic Weisbecker
2020-07-17 14:05 ` [PATCH 09/12] timer: Expand clk forward logic beyond nohz Frederic Weisbecker
2020-07-17 20:00 ` [tip: timers/core] timers: " tip-bot2 for Frederic Weisbecker
2020-07-17 14:05 ` [PATCH 10/12] timer: Spare timer softirq until next expiry Frederic Weisbecker
2020-07-17 20:00 ` [tip: timers/core] timers: " tip-bot2 for Frederic Weisbecker
2020-07-17 14:05 ` [PATCH 11/12] timer: Remove must_forward_clk Frederic Weisbecker
2020-07-17 20:00 ` [tip: timers/core] timers: " tip-bot2 for Frederic Weisbecker
2020-07-17 14:05 ` [PATCH 12/12] timer: Lower base clock forwarding threshold Frederic Weisbecker
2020-07-17 20:00 ` [tip: timers/core] timers: " tip-bot2 for Frederic Weisbecker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200717140551.29076-7-frederic@kernel.org \
--to=frederic@kernel.org \
--cc=anna-maria@linutronix.de \
--cc=juri.lelli@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.