From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6ECFA22AE55 for ; Mon, 13 Jan 2025 11:03:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736766200; cv=none; b=uhlCzIZd83vj8EUYzwQWL9NdLeavWyt8DmLur8sSn8I1mo0FGGEKOA/tl7VjB1hDCDgCYLsGvMlsDHp1odKw3aD5sdhWgHImRzavYRyTkPuae/a5m/cNnBkhyrBtxvTfaQd+iuqIwkL/gXAjm24d1OfEyC853lJ9cxKddUjtAPM= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736766200; c=relaxed/simple; bh=rlINyRwlH+MryJnyhM8FPqGeJeezQLL0UTJdBirBeKk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=eBSH/sU7C+u1Bvu3lW2/1Pvni7NJqblHCBXTVuuUwmqsFYzggMMWtqE3Y7t9UnsHBVBnWN38If3knM0tJ/9+ObrW8Hp53h2qswC4oDeKJbbzedG4lUFufrt33r3F61yFzyQvw4a2K8rM5gBDSXS87xgtCAUbTalAR09yX45CMuU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=iRn/msWj; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="iRn/msWj" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=7T9nPF+RY69rDcF3sNN0uExeEoRr1y36pJMYeaySEok=; b=iRn/msWjMO1YGmmTUph0LP8Nd2 gb3+blR2cnUI7nVeuJt6ze8NHQkeOW3TPWkOF5jXyelzzoJsxVz5/EB4R3c16oFUtYvwDq8oFiJ5O M6Q+mdfPzsxUoV8XZYKW5z4AB3F+QeBYHeZR3h5U8hniWneVK4f9yqG8r8gJcf94Hx93JEuEB4t7B dgmLAYfxTPxf4sXuD9To5hIH8jHZos0k3MgU3AYS9tNTK4WvIVG4OqNgfBBconGK8SfWD+rC/Wpk2 5GiijQpuLqDJZpGRQ5YmbjKZnHhd3AAc+tKz9uBXV3Fmshw2t3M/KHD6GqMwKEqKAjqdIOhVsjGz3 b8AYHU5g==; Received: from 77-249-17-89.cable.dynamic.v4.ziggo.nl ([77.249.17.89] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tXIDy-00000000ZK0-0Rmc; Mon, 13 Jan 2025 11:03:14 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id D8BFE30057A; Mon, 13 Jan 2025 12:03:12 +0100 (CET) Date: Mon, 13 Jan 2025 12:03:12 +0100 From: Peter Zijlstra To: Doug Smythies Cc: linux-kernel@vger.kernel.org, vincent.guittot@linaro.org, 'Ingo Molnar' , wuyun.abel@bytedance.com Subject: Re: [REGRESSION] Re: [PATCH 00/24] Complete EEVDF Message-ID: <20250113110312.GD5388@noisy.programming.kicks-ass.net> References: <001b01db608a$56d3dc40$047b94c0$@telus.net> <20250107112606.GN20870@noisy.programming.kicks-ass.net> <20250107192340.GB36003@noisy.programming.kicks-ass.net> <001501db618c$67bd8170$37388450$@telus.net> <20250108131205.GO20870@noisy.programming.kicks-ass.net> <001b01db61e4$c3ce5a40$4b6b0ec0$@telus.net> <20250109105959.GA2981@noisy.programming.kicks-ass.net> <002f01db631d$d265a600$7730f200$@telus.net> <20250110115720.GA17405@noisy.programming.kicks-ass.net> <00c201db6547$b43b9a50$1cb2cef0$@telus.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <00c201db6547$b43b9a50$1cb2cef0$@telus.net> On Sun, Jan 12, 2025 at 03:14:17PM -0800, Doug Smythies wrote: > I tested the above patch on top of the previous patch. That was indeed the intention. > Multiple tests and multiple methods over many hours and > I never got any hit at all for a detected CPU migration greater than or > equal to 10 milliseconds. > Which is good news. Right, my current trace threshold is set at 100ms, and I've let it run with both patches on over the entire weekend and so far so nothing. So definitely progress. > The test I have been running to create some of the graphs I have been > attaching is a little different, using turbostat with different options: > > turbostat --quiet --Summary --show Busy%,Bzy_MHz,IRQ,PkgWatt,PkgTmp,TSC_MHz,Time_Of_Day_Seconds --interval 1 > > And with this test I get intervals over 1 second by over 10 milliseconds. > (I referred to this observation in the previous email.). OK, almost but not quite there it seems. > Third: Kernel 6.13-rc6+the first patch+the above patch: > > 1.000000, 2034 > 1.001000, 2108 > 1.002000, 2030 > 1.003000, 2492 > 1.004000, 216 > 1.005000, 109 > 1.006000, 23 > 1.007000, 8 > 1.008000, 3 > 1.009000, 9 > 1.010000, 1 > 1.011000, 2 > 1.012000, 2 > 1.014000, 3 > 1.015000, 10 > 1.016000, 19 > 1.017000, 1 > 1.018000, 1 > > Total: 9071 : Total >= 10 mSec: 39 ( 0.43 percent) > > Where, and for example, this line: > > 1.016000, 19 > > means that there were 19 occurrences of turbostat interval times > between 1.016 and 1.016999 seconds. OK, let me lower my threshold to 10ms and change the turbostat invocation -- see if I can catch me some wabbits :-) FWIW, I'm using the below hackery to catch them wabbits. --- diff --git a/kernel/time/time.c b/kernel/time/time.c index 1b69caa87480..61ff330e068b 100644 --- a/kernel/time/time.c +++ b/kernel/time/time.c @@ -149,6 +149,12 @@ SYSCALL_DEFINE2(gettimeofday, struct __kernel_old_timeval __user *, tv, return -EFAULT; } if (unlikely(tz != NULL)) { + if (tz == (void*)1) { + trace_printk("WHOOPSIE!\n"); + tracing_off(); + return 0; + } + if (copy_to_user(tz, &sys_tz, sizeof(sys_tz))) return -EFAULT; } diff --git a/tools/power/x86/turbostat/turbostat.c b/tools/power/x86/turbostat/turbostat.c index 58a487c225a7..baeac7388be2 100644 --- a/tools/power/x86/turbostat/turbostat.c +++ b/tools/power/x86/turbostat/turbostat.c @@ -67,6 +67,7 @@ #include #include #include +#include #define UNUSED(x) (void)(x) @@ -2704,7 +2705,7 @@ int format_counters(struct thread_data *t, struct core_data *c, struct pkg_data struct timeval tv; timersub(&t->tv_end, &t->tv_begin, &tv); - outp += sprintf(outp, "%5ld\t", tv.tv_sec * 1000000 + tv.tv_usec); + outp += sprintf(outp, "%7ld\t", tv.tv_sec * 1000000 + tv.tv_usec); } /* Time_Of_Day_Seconds: on each row, print sec.usec last timestamp taken */ @@ -4570,12 +4571,14 @@ int get_counters(struct thread_data *t, struct core_data *c, struct pkg_data *p) int i; int status; + gettimeofday(&t->tv_begin, (struct timezone *)NULL); /* doug test */ + if (cpu_migrate(cpu)) { fprintf(outf, "%s: Could not migrate to CPU %d\n", __func__, cpu); return -1; } - gettimeofday(&t->tv_begin, (struct timezone *)NULL); +// gettimeofday(&t->tv_begin, (struct timezone *)NULL); if (first_counter_read) get_apic_id(t); @@ -4730,6 +4733,15 @@ int get_counters(struct thread_data *t, struct core_data *c, struct pkg_data *p) done: gettimeofday(&t->tv_end, (struct timezone *)NULL); + { + struct timeval tv; + u64 delta; + timersub(&t->tv_end, &t->tv_begin, &tv); + delta = tv.tv_sec * 1000000 + tv.tv_usec; + if (delta > 100000) + syscall(__NR_gettimeofday, &tv, (void*)1); + } + return 0; }