From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752654AbbFYAEE (ORCPT ); Wed, 24 Jun 2015 20:04:04 -0400 Received: from mga03.intel.com ([134.134.136.65]:37167 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751207AbbFYADz (ORCPT ); Wed, 24 Jun 2015 20:03:55 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.13,673,1427785200"; d="scan'208";a="594256712" Subject: [RFC][PATCH] sched: might_sleep(): do rate-limiting before sanity checks To: dave@sr71.net Cc: mingo@redhat.com, peterz@infradead.org, linux-kernel@vger.kernel.org, dave.hansen@linux.intel.com From: Dave Hansen Date: Wed, 24 Jun 2015 17:03:54 -0700 Message-Id: <20150625000354.255E7263@viggo.jf.intel.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Dave Hansen I have a dumb microbenchmark. It loops doing single-byte writes to a file. I have a few other patches to work on some things in the filesystem write path. But after those are applied, the 4th-hottest kernel function is ___might_sleep() which seems a bit silly. I narrowed the overhead down to the pushf/pop in native_save_fl() underneath the irqs_disabled() call. Those instructions must serialize something in the CPU because they seem to be way slower than they should be. In any case, we ratelimit might_sleep() checks anyway. But, we do the ratelimiting *after* we check the other conditions for might_sleep() including the (costly) irqs_disabled() call. If we flip these around and ratelimit _before_ the other checks, I see a boost in the microbenchmark. The downside here is that we end up doing more frequent updates to the global 'prev_jiffy'. But, we're still only actually updating it once per jiffy. I tested this on an 80-core system and my test scales better with this patch applied than without it, which made me feel a bit better that the global updates to 'prev_jiffy' won't be that painful in practice. Cc: Ingo Molnar Cc: Peter Zijlstra Cc: linux-kernel@vger.kernel.org Signed-off-by: Dave Hansen --- b/kernel/sched/core.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff -puN kernel/sched/core.c~might-sleep-ratelimit-first kernel/sched/core.c --- a/kernel/sched/core.c~might-sleep-ratelimit-first 2015-06-24 16:57:24.643850450 -0700 +++ b/kernel/sched/core.c 2015-06-24 16:57:24.650850764 -0700 @@ -7330,13 +7330,14 @@ void ___might_sleep(const char *file, in static unsigned long prev_jiffy; /* ratelimiting */ rcu_sleep_check(); /* WARN_ON_ONCE() by default, no rate limit reqd. */ + if (time_before(jiffies, prev_jiffy + HZ) && prev_jiffy) + return; + prev_jiffy = jiffies; + if ((preempt_count_equals(preempt_offset) && !irqs_disabled() && !is_idle_task(current)) || system_state != SYSTEM_RUNNING || oops_in_progress) return; - if (time_before(jiffies, prev_jiffy + HZ) && prev_jiffy) - return; - prev_jiffy = jiffies; printk(KERN_ERR "BUG: sleeping function called from invalid context at %s:%d\n", _