From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1426199AbdD3FIR (ORCPT ); Sun, 30 Apr 2017 01:08:17 -0400 Received: from mout.gmx.net ([212.227.15.15]:54862 "EHLO mout.gmx.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1032592AbdD3FII (ORCPT ); Sun, 30 Apr 2017 01:08:08 -0400 Message-ID: <1493528870.4365.5.camel@gmx.de> Subject: Re: [patch] timer: Fix timers_update_migration(), and call it in tmigr_init() From: Mike Galbraith To: paulmck@linux.vnet.ibm.com Cc: LKML , Ingo Molnar , Ingo Molnar , Thomas Gleixner , PeterZijlstra , Frederic Weisbecker Date: Sun, 30 Apr 2017 07:07:50 +0200 In-Reply-To: <1493526015.7220.6.camel@gmx.de> References: <1493195514.21594.5.camel@gmx.de> <1493197062.21594.8.camel@gmx.de> <20170426102617.l62cdn4gs4h5i4fw@hirez.programming.kicks-ass.net> <1493206789.21594.25.camel@gmx.de> <1493209836.21594.29.camel@gmx.de> <1493482000.4547.6.camel@gmx.de> <20170429180638.GI3956@linux.vnet.ibm.com> <1493490033.4339.3.camel@gmx.de> <20170429214544.GK3956@linux.vnet.ibm.com> <1493515318.5494.1.camel@gmx.de> <20170430034346.GO3956@linux.vnet.ibm.com> <1493526015.7220.6.camel@gmx.de> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.16.5 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:FJXPf5qY0JGcwUPFlbqbrv6OsbxElQYQ0dH/pzAUrqhQqBocy7R /WBkyWX0aITTPJe2Blbz2MxRI1X6aHlCHvIMMqWU9ntam9cluc/qKr/372LCh/U6H1M1/On ITgUcNvDV51Q4NGcgV1gXoyd1z/jkYObp88DMgGHNd7fwk6LJEq3oyqhq4KqFF6TTi5+idQ chwc4scRYykhIJaVub2Yg== X-UI-Out-Filterresults: notjunk:1;V01:K0:YPI2NgQWsIQ=:JAH7POfSYCjzzSn6aQhYOz ZiMIEfQ2I5YQYstbHo1KQjHBZsROztUixB5bRy7vMrbdZW3ob+fJFV1q2/sg8V6Upj3zykCul tbzu7AqRhnmp65VODWJgFfYd2YUPIScKK4Y7OmJWtjEp7t5KCq3bHQkfNi+1KZ4umR2QlvRxO uEAW7VyJCF0DWWnwPDCqBqciCqL5XRBSJFYbxeXGFlYoZpX6NqqYgjSrNDX0nMNuLN6JyhuZQ zEjiVb2ari3Sq1cQmcLpOmzq6teV54BEc10OP7DzhmKyYMSiBS2UrepD2yEBQmlwC0UI/Gi58 OZ3OzdjuzgjhbfxHfBpSfu8IyPoo/x/7dbPqsafXsXBpMg0+gD/MwrtLLonNriTWRq29cHiQE Bsj7+EKA0m0SLb2ZJxIj9nPKczUyI42x4DjtYUTZW6sRmr+7gVEQgc7rJgLI48lJi0QQHtPbT W4sn9jqPQlhYWRVdi3g2gRnN8Ka4/pxRlQ1E40BDq9yp3S9PsA1IMz3v79Zix7V13ZzHfKuq9 GVKSsuNyLc87LI9ZQG2LiWweL/tkTzSHlRD3J8uXUkWguCd4U7KNkzrQAd2tQdzVdzXaO0JcX JyeRHbl/ihKJ+WobXrrELPhnaNjExt9fqsodO2jqyQqktUFPCaXr/LH8QTvwazkyyjX2WEsNF soeBiQvqAZaPbnSUxTDHclV5aglrKsKtq+VAxvARlyfwJ1ChzSSq4O91q0PFEM5nqd6ZEd2mX GuV4cfJ/GOrYsskG+EdVoHlMCdpi7rFL+h5KemEhZ5heOaKDZKxJvbo84MWuRRHS8zhwpJqw4 OFAJle5 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, 2017-04-30 at 06:20 +0200, Mike Galbraith wrote: > On Sat, 2017-04-29 at 20:43 -0700, Paul E. McKenney wrote: > > On Sun, Apr 30, 2017 at 03:21:58AM +0200, Mike Galbraith wrote: > > > On Sat, 2017-04-29 at 14:45 -0700, Paul E. McKenney wrote: > > > > On Sat, Apr 29, 2017 at 08:20:33PM +0200, Mike Galbraith wrote: > > > > > On Sat, 2017-04-29 at 11:06 -0700, Paul E. McKenney wrote: > > > > > > > > > > > If someone will either repost a fresh series or point me at > > > > > > exactly > > > > > > the set of patches to use, I will run it through rcutorture > > > > > > again. > > > > > > > > > > Patchlet is against x86-tip/master.today. > > > > > > > > So today's (as in Saturday April 29) x86-tip/master with the > > > > following > > > > patch applied? > > > > > > Yeah. > > > > OK, will fire it up once the current set of overnight tests > > complete. > > I certainly don't want to discourage you from beating hell outta tip, > just want to make sure you know that I'm seeing zero RCU woes, only > late timer expiry (sharpening rocks/sticks to focus trace). Ah, seems a cpu shutting down the tick can race with add_timer_on(), leaving the timer stranded until some other event kicks the cpu awake. -0 [025] d..4 92.087954: tmigr_group_set_cpu_active: group=ffff88017d03d000 lvl=1 numa=-1 active=1 migrator=25 num_childs=8 parent= (null) nextevt=125916000000 evtcpu=2 -0 [025] d..4 92.087956: tmigr_group_removeevt: group=ffff88017d03d000 lvl=1 numa=-1 active=1 migrator=25 num_childs=8 parent= (null) nextevt=125916000000 evtcpu=2 -0 [025] d..3 92.087956: tmigr_group_set_cpu_active: group=ffff880179cd1000 lvl=0 numa=3 active=1 migrator=25 num_childs=8 parent=ffff88017d03d000 nextevt=316380000000 evtcpu=27 -0 [025] d..3 92.087957: tmigr_group_removeevt: group=ffff880179cd1000 lvl=0 numa=3 active=1 migrator=25 num_childs=8 parent=ffff88017d03d000 nextevt=316380000000 evtcpu=27 -0 [025] d.h3 92.087959: hrtimer_cancel: hrtimer=ffff880277455d20 -0 [025] d.h2 92.087959: hrtimer_expire_entry: hrtimer=ffff880277455d20 function=tick_sched_timer now=91100168287 -0 [025] d.h2 92.087964: hrtimer_expire_exit: hrtimer=ffff880277455d20 -0 [025] d.s3 92.087968: timer_cancel: timer=ffffffff820dd640 -0 [025] ..s2 92.087969: timer_expire_entry: timer=ffffffff820dd640 function=clocksource_watchdog now=4294915072 -0 [025] d.s4 92.087971: timer_start: timer=ffffffff820dd640 function=clocksource_watchdog expires=4294915197 [timeout=125] cpu=26 idx=218 flags= -0 [025] d.s3 92.087974: wake_up_idle_cpu: NO KICK 26 - !set_nr_and_not_polling(rq->idle) -0 [025] ..s2 92.087974: timer_expire_exit: timer=ffffffff820dd640 -0 [025] .Ns2 92.087982: tmigr_handle_remote: group=ffff880179cd1000 lvl=0 numa=3 active=1 migrator=25 num_childs=8 parent=ffff88017d03d000 nextevt=316380000000 evtcpu=27 cpu=25 -0 [025] .Ns2 92.087982: tmigr_handle_remote: group=ffff88017d03d000 lvl=1 numa=-1 active=1 migrator=25 num_childs=8 parent= (null) nextevt=125916000000 evtcpu=2 cpu=25 -0 [025] dN.3 92.087995: hrtimer_start: hrtimer=ffff880277455d20 function=tick_sched_timer expires=91104000000 softexpires=91104000000 -0 [026] dN.3 92.088009: tmigr_group_set_cpu_active: group=ffff880179cd1000 lvl=0 numa=3 active=2 migrator=25 num_childs=8 parent=ffff88017d03d000 nextevt=316380000000 evtcpu=27 -0 [026] dN.3 92.088011: tmigr_group_removeevt: group=ffff880179cd1000 lvl=0 numa=3 active=2 migrator=25 num_childs=8 parent=ffff88017d03d000 nextevt=316380000000 evtcpu=27 -0 [025] d..4 92.088016: tmigr_group_addevt: group=ffff880179cd1000 lvl=0 numa=3 active=1 migrator=25 num_childs=8 parent=ffff88017d03d000 nextevt=316380000000 evtcpu=27 -0 [026] dN.3 92.088016: hrtimer_start: hrtimer=ffff880277495d20 function=tick_sched_timer expires=91104000000 softexpires=91104000000 -0 [025] d..5 92.088017: tmigr_group_set_cpu_inactive: group=ffff88017d03d000 lvl=1 numa=-1 active=1 migrator=26 num_childs=8 parent= (null) nextevt=125916000000 evtcpu=2 -0 [025] d..4 92.088017: tmigr_group_set_cpu_inactive: group=ffff880179cd1000 lvl=0 numa=3 active=1 migrator=26 num_childs=8 parent=ffff88017d03d000 nextevt=316380000000 evtcpu=27 -0 [025] d..2 92.088018: tick_stop: success=1 dependency=NONE -0 [025] d..3 92.088018: hrtimer_cancel: hrtimer=ffff880277455d20 -0 [025] d..3 92.088020: hrtimer_start: hrtimer=ffff880277455d20 function=tick_sched_timer expires=125916000000 softexpires=125916000000 -0 [026] d..4 92.088022: tmigr_group_addevt: group=ffff880179cd1000 lvl=0 numa=3 active=0 migrator=26 num_childs=8 parent=ffff88017d03d000 nextevt=316380000000 evtcpu=27 -0 [026] d..5 92.088023: tmigr_group_addevt: group=ffff88017d03d000 lvl=1 numa=-1 active=0 migrator=26 num_childs=8 parent= (null) nextevt=125916000000 evtcpu=2 -0 [026] d..5 92.088024: tmigr_group_set_cpu_inactive: group=ffff88017d03d000 lvl=1 numa=-1 active=0 migrator=-1 num_childs=8 parent= (null) nextevt=125916000000 evtcpu=2 -0 [026] d..4 92.088025: tmigr_group_set_cpu_inactive: group=ffff880179cd1000 lvl=0 numa=3 active=0 migrator=-1 num_childs=8 parent=ffff88017d03d000 nextevt=316380000000 evtcpu=27 -0 [026] d..2 92.088025: tick_stop: success=1 dependency=NONE -0 [026] d..3 92.088026: hrtimer_cancel: hrtimer=ffff880277495d20 -0 [026] d..3 92.088027: hrtimer_start: hrtimer=ffff880277495d20 function=tick_sched_timer expires=93148000000 softexpires=93148000000 -0 [026] d..4 94.135877: tmigr_group_set_cpu_active: group=ffff88017d03d000 lvl=1 numa=-1 active=2 migrator=1 num_childs=8 parent= (null) nextevt=316380000000 evtcpu=56 -0 [026] d..4 94.135879: tmigr_group_removeevt: group=ffff88017d03d000 lvl=1 numa=-1 active=2 migrator=1 num_childs=8 parent= (null) nextevt=316380000000 evtcpu=56 -0 [026] d..3 94.135879: tmigr_group_set_cpu_active: group=ffff880179cd1000 lvl=0 numa=3 active=1 migrator=26 num_childs=8 parent=ffff88017d03d000 nextevt=316380000000 evtcpu=27 -0 [026] d..3 94.135880: tmigr_group_removeevt: group=ffff880179cd1000 lvl=0 numa=3 active=1 migrator=26 num_childs=8 parent=ffff88017d03d000 nextevt=316380000000 evtcpu=27 -0 [026] d.h3 94.135881: hrtimer_cancel: hrtimer=ffff880277495d20 -0 [026] d.h2 94.135882: hrtimer_expire_entry: hrtimer=ffff880277495d20 function=tick_sched_timer now=93148247903 -0 [026] d.h2 94.136137: hrtimer_expire_exit: hrtimer=ffff880277495d20 -0 [026] d.s3 94.136141: timer_cancel: timer=ffffffff820dd640 -0 [026] ..s2 94.136141: timer_expire_entry: timer=ffffffff820dd640 function=clocksource_watchdog now=4294915584 -0 [026] d.s4 94.136144: timer_start: timer=ffffffff820dd640 function=clocksource_watchdog expires=4294915709 [timeout=125] cpu=27 idx=219 flags= -0 [026] d.s3 94.136146: wake_up_idle_cpu: KICK 27 -0 [026] ..s1 94.136148: clocksource_watchdog: LATE by 387 ticks