From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932849AbdDZSNh (ORCPT ); Wed, 26 Apr 2017 14:13:37 -0400 Received: from mout.gmx.net ([212.227.15.19]:62920 "EHLO mout.gmx.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932823AbdDZSN2 (ORCPT ); Wed, 26 Apr 2017 14:13:28 -0400 Message-ID: <1493230388.4132.5.camel@gmx.de> Subject: Re: x86-tip tsc/tick gripage From: Mike Galbraith To: Peter Zijlstra Cc: Ingo Molnar , "Paul E. McKenney" , LKML , Ingo Molnar , Thomas Gleixner , Frederic Weisbecker Date: Wed, 26 Apr 2017 20:13:08 +0200 In-Reply-To: <1493209836.21594.29.camel@gmx.de> References: <1493193739.4412.30.camel@gmx.de> <1493194602.21594.4.camel@gmx.de> <20170426082137.koj7q3ftfqw3o22u@gmail.com> <1493195514.21594.5.camel@gmx.de> <1493197062.21594.8.camel@gmx.de> <20170426102617.l62cdn4gs4h5i4fw@hirez.programming.kicks-ass.net> <1493206789.21594.25.camel@gmx.de> <1493209836.21594.29.camel@gmx.de> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.16.5 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:gvSQXW2jhRRjSvfnHeUmr/cIp+hRJSubGOiRACOb8lvKDsx5BBt LDvP+qizxzjiaKhuZJQQX8348qP3qstCQCnT2RfQk5BXIsWGll4ZUPM02970F6l340/657s Op7zAIxRoZeJHjSAw39dU2Bqxn35jmceoj3BwkSPlgLL6v4a6P+q6Id8rI6DBYzGKeDIhHl c3o6wssqERq4V6PNO6IiA== X-UI-Out-Filterresults: notjunk:1;V01:K0:r2OyPodfznU=:uKx1pZUYLIhR/ravu00ky5 9HTx66hO6jLOy8L5iWAPk1O46nGOMhFNuwa1rK9YKdOXcDiP8XQzLshgluMBiNIusgjnnIqqg a7zN2yCqRXIaOMAfzX/n1FU2JEZIemFsaDNcqak1pP2VRkrQb2we+/4xmGtRaN7X81ryOTE2Q udPVBh02l9EOgxOhXi7vzCe411irxeq+awSgsijsENZ5BksRbrdc7Tfs3OJVOZidnilT5Knn6 OozSyH3vj8w+XfzvJw5cWuSEO3RiuxRZBPsxnypBFQIjuXRclEE1bHOK5ZJyMEgCMNMU6erSG 0FK03Xrn382ux3bZmxZN8cUk2YFjQX3tzzDheoXZ4Qa2H00BfoOdUZWlAsomVdbv5Ou4tRWPl xVNJnxMYRZUZFzPXnyfaTePdBUyReuMrM6BDz3Zv3wBB5KWGPSBkfp5AqLIOGO5ODCtqV413H FkL0Vk475xtZuM7vaUaVO/xSkQwXcw1mh5VHPudhDItzFXzqtWgvcP65nVEEl7mYeTJoRxXJk APJlohlPJrAjCBOcvGQgROy9LytcUusSuGWp9pe7ZBeIiLF7xjyhEPzTV/mtyEpUuLwp82mPE /izI7j8WMXjSOCcS5hfMxhbfgEdfk9IUD+HGlrCcVA0pRR9PQaiiDS9BrB3zzhiHG8QLrw9I1 bRGVwF9pPGEeP+41szAzQxh2k+iHf8ogE0s9kkFEAYL2tga3+FnVzV1MDCUM11VEOp7KRo8P3 doSdQPF9WB/gND5azmFgY2WMK0KZsaJlPET4K9/cE4h+6wu5dri6alG1DC0w78wbMyD4VQuPS apRosSU Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 2017-04-26 at 14:30 +0200, Mike Galbraith wrote: > On Wed, 2017-04-26 at 13:39 +0200, Mike Galbraith wrote: > > On Wed, 2017-04-26 at 12:26 +0200, Peter Zijlstra wrote: > > > On Wed, Apr 26, 2017 at 10:57:42AM +0200, Mike Galbraith wrote: > > > > > > > Both still lose their TSC. > > > > > > > > [ 11.982468] tsc: Refined TSC clocksource calibration: 2260.999 MHz > > > > [ 11.994275] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x20974a4d8bb, max_idle_ns: 440795246623 ns > > > > [ 13.064172] clocksource: Switched to clocksource tsc > > > > [ 240.247851] clocksource: timekeeping watchdog on CPU23: Marking clocksource 'tsc' as unstable because the skew is too large: > > > > [ 240.462501] clocksource: 'tsc' cs_now: 108fe5be09f cs_last: b90a6a0676 mask: ffffffffffffffff > > > > [ 240.675057] tsc: Marking TSC unstable due to clocksource watchdog > > > > > > > > > And they didn't use to? We don't typically write to TSC or TSC_ADJUST > > > and thus would not cause such behaviour. > > > > Nope. > > DL980 seems perfectly happy with master.today.. so off we go. ec2206b91d430da57c856869b5a37dc1e569e80d is the first bad commit commit ec2206b91d430da57c856869b5a37dc1e569e80d Author: Anna-Maria Gleixner Date: Mon Mar 20 10:34:20 2017 +0100 timer: Implement the hierarchical pull model Placing timers at enqueue time on a target CPU based on dubious heuristics does not make any sense: 1) Most timer wheel timers are canceled or rearmed before they expire. 2) The heuristics to predict which CPU will be busy when the timer expires are wrong by definition. So we waste precious cycles to place timers at enqueue time. The proper solution to this problem is to always queue the timers on the local CPU and allow the non pinned timers to be pulled onto a busy CPU at expiry time. To achieve this the timer storage has been split into local pinned and global timers. Local pinned timers are always expired on the CPU on which they have been queued. Global timers can be expired on any CPU. As long as a CPU is busy it expires both local and global timers. When a CPU goes idle it arms for the first expiring local timer. If the first expiring pinned (local) timer is before the first expiring movable timer, then no action is required because the CPU will wake up before the first movable timer expires. If the first expiring movable timer is before the first expiring pinned (local) timer, then this timer is queued into a idle timerqueue and eventually expired by some other active CPU. To avoid global locking the timerqueues are implemented as a hierarchy. The lowest level of the hierarchy holds the CPUs. The CPUs are associated to groups of 8, which are seperated per node. If more than one CPU group exist, then a second level in the hierarchy collects the groups. Depending on the size of the system more than 2 levels are required. Each group has a "migrator" which checks the timerqueue during the tick for remote expirable timers. If the last CPU in a group goes idle it reports the first expiring event in the group up to the next group(s) in the hierarchy. If the last CPU goes idle it arms its timer for the first system wide expiring timer to ensure that no timer event is missed. Signed-off-by: Anna-Maria Gleixner Signed-off-by: Thomas Gleixner :040000 040000 76086a473919931466dfeea76a17566e2bb15130 5432a1bbc5ace9796e70e91447c6f26eb9eb51be M include :040000 040000 102292e74099b7c3e0b2125a4deef728267b4d9d 24385b3c68140fbb3362f32753f296d127e22097 M kernel git bisect start # good: [ea839b41744dffe5c77b8d9842c9bb7073460901] Merge tag 'arc-4.11-final' of git://git.kernel.org/pub/scm/linux/kernel/git/vgupta/arc git bisect good ea839b41744dffe5c77b8d9842c9bb7073460901 # bad: [d02c59825bbfd2726430f410321f9d0bfc9d1ef4] Merge branch 'WIP.x86/fpu' git bisect bad d02c59825bbfd2726430f410321f9d0bfc9d1ef4 # good: [1212249de8142df8c64c4165d1dee8ed4c3d766c] Merge branch 'perf/core' git bisect good 1212249de8142df8c64c4165d1dee8ed4c3d766c # good: [28a68c6ee0837004c87845a09717a99b4e96f7d4] Merge branch 'x86-mm-for-xen' git bisect good 28a68c6ee0837004c87845a09717a99b4e96f7d4 # good: [7fd018291abbc076dc8073d48bba9614ca67b818] Merge branch 'x86/cleanups' git bisect good 7fd018291abbc076dc8073d48bba9614ca67b818 # good: [6f029d98bc76cf9aa21e73e4beba9014e7e3adbb] Merge branch 'x86/platform' git bisect good 6f029d98bc76cf9aa21e73e4beba9014e7e3adbb # bad: [4691eff5d6475014b4c04481b03aff89f5ec87c1] Merge branch 'WIP.timers' git bisect bad 4691eff5d6475014b4c04481b03aff89f5ec87c1 # good: [0605ab6fd5aa0c78133fc6611fa5e4f17e46c396] sched/wait: Disambiguate wq_entry->task_list and wq_head->task_list naming git bisect good 0605ab6fd5aa0c78133fc6611fa5e4f17e46c396 # bad: [1e1c48e6befca01abb2ed224a76fbe0ea8c73e30] timer: Always queue timers on the local CPU git bisect bad 1e1c48e6befca01abb2ed224a76fbe0ea8c73e30 # good: [c82a8b6b216e6910f36a94939de7e8793b25feaa] timer: Retrieve next expiry of pinned/non-pinned timers seperately git bisect good c82a8b6b216e6910f36a94939de7e8793b25feaa # good: [e2e1214438bb25ec4e37b8aedb3d3e502535b09b] tick/sched: Split out jiffies update helper function git bisect good e2e1214438bb25ec4e37b8aedb3d3e502535b09b # bad: [270c4e558cc26a0d89e89dda46c35ac39457a896] timer_migration: Add tracepoints git bisect bad 270c4e558cc26a0d89e89dda46c35ac39457a896 # bad: [ec2206b91d430da57c856869b5a37dc1e569e80d] timer: Implement the hierarchical pull model git bisect bad ec2206b91d430da57c856869b5a37dc1e569e80d