From: Mike Galbraith <efault@gmx.de>
To: Sachin Sant <sachinp@linux.vnet.ibm.com>,
Matt Fleming <matt@codeblueprint.co.uk>,
Michael Ellerman <mpe@ellerman.id.au>
Cc: "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>,
peterz@infradead.org,
"linux-next@vger.kernel.org" <linux-next@vger.kernel.org>,
linux-kernel@vger.kernel.org
Subject: Re: [tip:sched/core] sched/core: Add debugging code to catch missing update_rq_clock() calls
Date: Tue, 31 Jan 2017 12:48:37 +0100 [thread overview]
Message-ID: <1485863317.4498.2.camel@gmx.de> (raw)
In-Reply-To: <33118425-8548-475A-8DED-9A2424C0BCCC@linux.vnet.ibm.com>
On Tue, 2017-01-31 at 16:30 +0530, Sachin Sant wrote:
> Trimming the cc list.
>
> > > I assume I should be worried?
> >
> > Thanks for the report. No need to worry, the bug has existed for a
> > while, this patch just turns on the warning ;-)
> >
> > The following commit queued up in tip/sched/core should fix your
> > issues (assuming you see the same callstack on all your powerpc
> > machines):
> >
> > https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/commit/?h=sched/core&id=1b1d62254df0fe42a711eb71948f915918987790
>
> I still see this warning with today’s next running inside PowerVM LPAR
> on a POWER8 box. The stack trace is different from what Michael had
> reported.
>
> Easiest way to recreate this is to Online/offline cpu’s.
(Ditto tip.today, x86_64 + hotplug stress)
[ 94.804196] ------------[ cut here ]------------
[ 94.804201] WARNING: CPU: 3 PID: 27 at kernel/sched/sched.h:804 set_next_entity+0x81c/0x910
[ 94.804201] rq->clock_update_flags < RQCF_ACT_SKIP
[ 94.804202] Modules linked in: ebtable_filter(E) ebtables(E) fuse(E) bridge(E) stp(E) llc(E) iscsi_ibft(E) iscsi_boot_sysfs(E) ip6t_REJECT(E) xt_tcpudp(E) nf_conntrack_ipv6(E) nf_defrag_ipv6(E) ip6table_raw(E) ipt_REJECT(E) iptable_raw(E) iptable_filter(E) ip6table_mangle(E) nf_conntrack_netbios_ns(E) nf_conntrack_broadcast(E) nf_conntrack_ipv4(E) nf_defrag_ipv4(E) ip_tables(E) xt_conntrack(E) nf_conntrack(E) ip6table_filter(E) ip6_tables(E) x_tables(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) coretemp(E) kvm_intel(E) kvm(E) irqbypass(E) crct10dif_pclmul(E) crc32_pclmul(E) nls_iso8859_1(E) crc32c_intel(E) nls_cp437(E) snd_hda_codec_realtek(E) snd_hda_codec_hdmi(E) snd_hda_codec_generic(E) nfsd(E) aesni_intel(E) snd_hda_intel(E) snd_hda_codec(E) snd_hwdep(E) aes_x86_64(E) snd_hda_core
(E) crypto_simd(E)
[ 94.804220] snd_pcm(E) auth_rpcgss(E) snd_timer(E) snd(E) iTCO_wdt(E) iTCO_vendor_support(E) joydev(E) nfs_acl(E) lpc_ich(E) cryptd(E) lockd(E) intel_smartconnect(E) mfd_core(E) i2c_i801(E) battery(E) glue_helper(E) mei_me(E) shpchp(E) mei(E) soundcore(E) grace(E) fan(E) thermal(E) tpm_infineon(E) pcspkr(E) sunrpc(E) efivarfs(E) sr_mod(E) cdrom(E) hid_logitech_hidpp(E) hid_logitech_dj(E) uas(E) usb_storage(E) hid_generic(E) usbhid(E) nouveau(E) wmi(E) i2c_algo_bit(E) drm_kms_helper(E) syscopyarea(E) sysfillrect(E) sysimgblt(E) fb_sys_fops(E) ahci(E) xhci_pci(E) ehci_pci(E) ttm(E) libahci(E) xhci_hcd(E) ehci_hcd(E) r8169(E) mii(E) libata(E) drm(E) usbcore(E) fjes(E) video(E) button(E) af_packet(E) sd_mod(E) vfat(E) fat(E) ext4(E) crc16(E) jbd2(E) mbcache(E) dm_mod(E) loop(E) sg(E) scsi_
mod(E) autofs4(E)
[ 94.804246] CPU: 3 PID: 27 Comm: migration/3 Tainted: G E 4.10.0-tip #15
[ 94.804247] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013
[ 94.804247] Call Trace:
[ 94.804251] ? dump_stack+0x5c/0x7c
[ 94.804253] ? __warn+0xc4/0xe0
[ 94.804255] ? warn_slowpath_fmt+0x4f/0x60
[ 94.804256] ? set_next_entity+0x81c/0x910
[ 94.804258] ? pick_next_task_fair+0x20a/0xa20
[ 94.804259] ? sched_cpu_starting+0x50/0x50
[ 94.804260] ? sched_cpu_dying+0x237/0x280
[ 94.804261] ? sched_cpu_starting+0x50/0x50
[ 94.804262] ? cpuhp_invoke_callback+0x83/0x3e0
[ 94.804263] ? take_cpu_down+0x56/0x90
[ 94.804266] ? multi_cpu_stop+0xa9/0xd0
[ 94.804267] ? cpu_stop_queue_work+0xb0/0xb0
[ 94.804268] ? cpu_stopper_thread+0x81/0x110
[ 94.804270] ? smpboot_thread_fn+0xfe/0x150
[ 94.804272] ? kthread+0xf4/0x130
[ 94.804273] ? sort_range+0x20/0x20
[ 94.804274] ? kthread_park+0x80/0x80
[ 94.804276] ? ret_from_fork+0x26/0x40
[ 94.804277] ---[ end trace b0a9e4aa1fb229bb ]---
WARNING: multiple messages have this Message-ID (diff)
From: Mike Galbraith <efault@gmx.de>
To: Sachin Sant <sachinp@linux.vnet.ibm.com>,
Matt Fleming <matt@codeblueprint.co.uk>,
Michael Ellerman <mpe@ellerman.id.au>
Cc: "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>,
peterz@infradead.org,
"linux-next@vger.kernel.org" <linux-next@vger.kernel.org>,
linux-kernel@vger.kernel.org
Subject: Re: [tip:sched/core] sched/core: Add debugging code to catch missing update_rq_clock() calls
Date: Tue, 31 Jan 2017 12:48:37 +0100 [thread overview]
Message-ID: <1485863317.4498.2.camel@gmx.de> (raw)
In-Reply-To: <33118425-8548-475A-8DED-9A2424C0BCCC@linux.vnet.ibm.com>
On Tue, 2017-01-31 at 16:30 +0530, Sachin Sant wrote:
> Trimming the cc list.
>=20
> > > I assume I should be worried?
> >=20
> > Thanks for the report. No need to worry, the bug has existed for a
> > while, this patch just turns on the warning ;-)
> >=20
> > The following commit queued up in tip/sched/core should fix your
> > issues (assuming you see the same callstack on all your powerpc
> > machines):
> >=20
> > https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/commit/?h=3Ds=
ched/core&id=3D1b1d62254df0fe42a711eb71948f915918987790
>=20
> I still see this warning with today=A2s next running inside PowerVM LPAR
> on a POWER8 box. The stack trace is different from what Michael had
> reported.
>=20
> Easiest way to recreate this is to Online/offline cpu=A2s.
(Ditto tip.today, x86_64 + hotplug stress)
[ 94.804196] ------------[ cut here ]------------
[ 94.804201] WARNING: CPU: 3 PID: 27 at kernel/sched/sched.h:804 set_next=
_entity+0x81c/0x910
[ 94.804201] rq->clock_update_flags < RQCF_ACT_SKIP
[ 94.804202] Modules linked in: ebtable_filter(E) ebtables(E) fuse(E) bri=
dge(E) stp(E) llc(E) iscsi_ibft(E) iscsi_boot_sysfs(E) ip6t_REJECT(E) xt_tc=
pudp(E) nf_conntrack_ipv6(E) nf_defrag_ipv6(E) ip6table_raw(E) ipt_REJECT(E=
) iptable_raw(E) iptable_filter(E) ip6table_mangle(E) nf_conntrack_netbios_=
ns(E) nf_conntrack_broadcast(E) nf_conntrack_ipv4(E) nf_defrag_ipv4(E) ip_t=
ables(E) xt_conntrack(E) nf_conntrack(E) ip6table_filter(E) ip6_tables(E) x=
_tables(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) coretemp(E) kvm_inte=
l(E) kvm(E) irqbypass(E) crct10dif_pclmul(E) crc32_pclmul(E) nls_iso8859_1(=
E) crc32c_intel(E) nls_cp437(E) snd_hda_codec_realtek(E) snd_hda_codec_hdmi=
(E) snd_hda_codec_generic(E) nfsd(E) aesni_intel(E) snd_hda_intel(E) snd_hd=
a_codec(E) snd_hwdep(E) aes_x86_64(E) snd_hda_core(E) crypto_simd(E)
[ 94.804220] snd_pcm(E) auth_rpcgss(E) snd_timer(E) snd(E) iTCO_wdt(E) i=
TCO_vendor_support(E) joydev(E) nfs_acl(E) lpc_ich(E) cryptd(E) lockd(E) in=
tel_smartconnect(E) mfd_core(E) i2c_i801(E) battery(E) glue_helper(E) mei_m=
e(E) shpchp(E) mei(E) soundcore(E) grace(E) fan(E) thermal(E) tpm_infineon(=
E) pcspkr(E) sunrpc(E) efivarfs(E) sr_mod(E) cdrom(E) hid_logitech_hidpp(E)=
hid_logitech_dj(E) uas(E) usb_storage(E) hid_generic(E) usbhid(E) nouveau(=
E) wmi(E) i2c_algo_bit(E) drm_kms_helper(E) syscopyarea(E) sysfillrect(E) s=
ysimgblt(E) fb_sys_fops(E) ahci(E) xhci_pci(E) ehci_pci(E) ttm(E) libahci(E=
) xhci_hcd(E) ehci_hcd(E) r8169(E) mii(E) libata(E) drm(E) usbcore(E) fjes(=
E) video(E) button(E) af_packet(E) sd_mod(E) vfat(E) fat(E) ext4(E) crc16(E=
) jbd2(E) mbcache(E) dm_mod(E) loop(E) sg(E) scsi_mod(E) autofs4(E)
[ 94.804246] CPU: 3 PID: 27 Comm: migration/3 Tainted: G E 4=
.10.0-tip #15
[ 94.804247] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/=
23/2013
[ 94.804247] Call Trace:
[ 94.804251] ? dump_stack+0x5c/0x7c
[ 94.804253] ? __warn+0xc4/0xe0
[ 94.804255] ? warn_slowpath_fmt+0x4f/0x60
[ 94.804256] ? set_next_entity+0x81c/0x910
[ 94.804258] ? pick_next_task_fair+0x20a/0xa20
[ 94.804259] ? sched_cpu_starting+0x50/0x50
[ 94.804260] ? sched_cpu_dying+0x237/0x280
[ 94.804261] ? sched_cpu_starting+0x50/0x50
[ 94.804262] ? cpuhp_invoke_callback+0x83/0x3e0
[ 94.804263] ? take_cpu_down+0x56/0x90
[ 94.804266] ? multi_cpu_stop+0xa9/0xd0
[ 94.804267] ? cpu_stop_queue_work+0xb0/0xb0
[ 94.804268] ? cpu_stopper_thread+0x81/0x110
[ 94.804270] ? smpboot_thread_fn+0xfe/0x150
[ 94.804272] ? kthread+0xf4/0x130
[ 94.804273] ? sort_range+0x20/0x20
[ 94.804274] ? kthread_park+0x80/0x80
[ 94.804276] ? ret_from_fork+0x26/0x40
[ 94.804277] ---[ end trace b0a9e4aa1fb229bb ]---
WARNING: multiple messages have this Message-ID (diff)
From: Mike Galbraith <efault@gmx.de>
To: Sachin Sant <sachinp@linux.vnet.ibm.com>,
Matt Fleming <matt@codeblueprint.co.uk>,
Michael Ellerman <mpe@ellerman.id.au>
Cc: "linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>,
peterz@infradead.org,
"linux-next@vger.kernel.org" <linux-next@vger.kernel.org>,
linux-kernel@vger.kernel.org
Subject: Re: [tip:sched/core] sched/core: Add debugging code to catch missing update_rq_clock() calls
Date: Tue, 31 Jan 2017 12:48:37 +0100 [thread overview]
Message-ID: <1485863317.4498.2.camel@gmx.de> (raw)
In-Reply-To: <33118425-8548-475A-8DED-9A2424C0BCCC@linux.vnet.ibm.com>
On Tue, 2017-01-31 at 16:30 +0530, Sachin Sant wrote:
> Trimming the cc list.
>
> > > I assume I should be worried?
> >
> > Thanks for the report. No need to worry, the bug has existed for a
> > while, this patch just turns on the warning ;-)
> >
> > The following commit queued up in tip/sched/core should fix your
> > issues (assuming you see the same callstack on all your powerpc
> > machines):
> >
> > https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/commit/?h=sched/core&id=1b1d62254df0fe42a711eb71948f915918987790
>
> I still see this warning with today’s next running inside PowerVM LPAR
> on a POWER8 box. The stack trace is different from what Michael had
> reported.
>
> Easiest way to recreate this is to Online/offline cpu’s.
(Ditto tip.today, x86_64 + hotplug stress)
[ 94.804196] ------------[ cut here ]------------
[ 94.804201] WARNING: CPU: 3 PID: 27 at kernel/sched/sched.h:804 set_next_entity+0x81c/0x910
[ 94.804201] rq->clock_update_flags < RQCF_ACT_SKIP
[ 94.804202] Modules linked in: ebtable_filter(E) ebtables(E) fuse(E) bridge(E) stp(E) llc(E) iscsi_ibft(E) iscsi_boot_sysfs(E) ip6t_REJECT(E) xt_tcpudp(E) nf_conntrack_ipv6(E) nf_defrag_ipv6(E) ip6table_raw(E) ipt_REJECT(E) iptable_raw(E) iptable_filter(E) ip6table_mangle(E) nf_conntrack_netbios_ns(E) nf_conntrack_broadcast(E) nf_conntrack_ipv4(E) nf_defrag_ipv4(E) ip_tables(E) xt_conntrack(E) nf_conntrack(E) ip6table_filter(E) ip6_tables(E) x_tables(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) coretemp(E) kvm_intel(E) kvm(E) irqbypass(E) crct10dif_pclmul(E) crc32_pclmul(E) nls_iso8859_1(E) crc32c_intel(E) nls_cp437(E) snd_hda_codec_realtek(E) snd_hda_codec_hdmi(E) snd_hda_codec_generic(E) nfsd(E) aesni_intel(E) snd_hda_intel(E) snd_hda_codec(E) snd_hwdep(E) aes_x86_64(E) snd_hda_core(E) crypto_simd(E)
[ 94.804220] snd_pcm(E) auth_rpcgss(E) snd_timer(E) snd(E) iTCO_wdt(E) iTCO_vendor_support(E) joydev(E) nfs_acl(E) lpc_ich(E) cryptd(E) lockd(E) intel_smartconnect(E) mfd_core(E) i2c_i801(E) battery(E) glue_helper(E) mei_me(E) shpchp(E) mei(E) soundcore(E) grace(E) fan(E) thermal(E) tpm_infineon(E) pcspkr(E) sunrpc(E) efivarfs(E) sr_mod(E) cdrom(E) hid_logitech_hidpp(E) hid_logitech_dj(E) uas(E) usb_storage(E) hid_generic(E) usbhid(E) nouveau(E) wmi(E) i2c_algo_bit(E) drm_kms_helper(E) syscopyarea(E) sysfillrect(E) sysimgblt(E) fb_sys_fops(E) ahci(E) xhci_pci(E) ehci_pci(E) ttm(E) libahci(E) xhci_hcd(E) ehci_hcd(E) r8169(E) mii(E) libata(E) drm(E) usbcore(E) fjes(E) video(E) button(E) af_packet(E) sd_mod(E) vfat(E) fat(E) ext4(E) crc16(E) jbd2(E) mbcache(E) dm_mod(E) loop(E) sg(E) scsi_mod(E) autofs4(E)
[ 94.804246] CPU: 3 PID: 27 Comm: migration/3 Tainted: G E 4.10.0-tip #15
[ 94.804247] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013
[ 94.804247] Call Trace:
[ 94.804251] ? dump_stack+0x5c/0x7c
[ 94.804253] ? __warn+0xc4/0xe0
[ 94.804255] ? warn_slowpath_fmt+0x4f/0x60
[ 94.804256] ? set_next_entity+0x81c/0x910
[ 94.804258] ? pick_next_task_fair+0x20a/0xa20
[ 94.804259] ? sched_cpu_starting+0x50/0x50
[ 94.804260] ? sched_cpu_dying+0x237/0x280
[ 94.804261] ? sched_cpu_starting+0x50/0x50
[ 94.804262] ? cpuhp_invoke_callback+0x83/0x3e0
[ 94.804263] ? take_cpu_down+0x56/0x90
[ 94.804266] ? multi_cpu_stop+0xa9/0xd0
[ 94.804267] ? cpu_stop_queue_work+0xb0/0xb0
[ 94.804268] ? cpu_stopper_thread+0x81/0x110
[ 94.804270] ? smpboot_thread_fn+0xfe/0x150
[ 94.804272] ? kthread+0xf4/0x130
[ 94.804273] ? sort_range+0x20/0x20
[ 94.804274] ? kthread_park+0x80/0x80
[ 94.804276] ? ret_from_fork+0x26/0x40
[ 94.804277] ---[ end trace b0a9e4aa1fb229bb ]---
next prev parent reply other threads:[~2017-01-31 11:49 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-09-21 13:38 [PATCH v2 0/7] sched: Diagnostic checks for missing rq clock updates Matt Fleming
2016-09-21 13:38 ` [PATCH v2 1/7] sched/fair: Update the rq clock before detaching tasks Matt Fleming
2016-10-03 12:49 ` Peter Zijlstra
2016-10-03 14:37 ` Matt Fleming
2016-10-03 14:42 ` Peter Zijlstra
2016-09-21 13:38 ` [PATCH v2 2/7] sched/fair: Update rq clock before waking up new task Matt Fleming
2016-09-21 13:38 ` [PATCH v2 3/7] sched/fair: Update rq clock in task_hot() Matt Fleming
2016-09-21 13:38 ` [PATCH v2 4/7] sched: Add wrappers for lockdep_(un)pin_lock() Matt Fleming
2017-01-14 12:40 ` [tip:sched/core] sched/core: " tip-bot for Matt Fleming
2016-09-21 13:38 ` [PATCH v2 5/7] sched/core: Reset RQCF_ACT_SKIP before unpinning rq->lock Matt Fleming
2017-01-14 12:41 ` [tip:sched/core] " tip-bot for Matt Fleming
2016-09-21 13:38 ` [PATCH v2 6/7] sched/fair: Push rq lock pin/unpin into idle_balance() Matt Fleming
2017-01-14 12:41 ` [tip:sched/core] " tip-bot for Matt Fleming
2016-09-21 13:38 ` [PATCH v2 7/7] sched/core: Add debug code to catch missing update_rq_clock() Matt Fleming
2016-09-21 15:58 ` Petr Mladek
2016-09-21 19:08 ` Matt Fleming
2016-09-21 19:46 ` Thomas Gleixner
2016-09-22 0:44 ` Sergey Senozhatsky
2016-09-22 8:04 ` Peter Zijlstra
2016-09-22 8:36 ` Jan Kara
2016-09-22 9:39 ` Peter Zijlstra
2016-09-22 10:17 ` Peter Zijlstra
2017-01-14 12:44 ` [tip:sched/core] sched/core: Add debugging code to catch missing update_rq_clock() calls tip-bot for Matt Fleming
2017-01-30 21:24 ` Michael Ellerman
2017-01-30 21:24 ` Michael Ellerman
2017-01-30 21:34 ` Matt Fleming
2017-01-31 8:35 ` Michael Ellerman
2017-01-31 8:35 ` Michael Ellerman
2017-01-31 11:00 ` Sachin Sant
2017-01-31 11:00 ` Sachin Sant
2017-01-31 11:00 ` Sachin Sant
2017-01-31 11:48 ` Mike Galbraith [this message]
2017-01-31 11:48 ` Mike Galbraith
2017-01-31 11:48 ` Mike Galbraith
2017-01-31 17:22 ` Ross Zwisler
2017-01-31 17:22 ` Ross Zwisler
2017-02-02 15:55 ` Peter Zijlstra
2017-02-02 22:01 ` Matt Fleming
2017-02-03 3:05 ` Mike Galbraith
2017-02-03 4:33 ` Sachin Sant
2017-02-03 4:33 ` Sachin Sant
2017-02-03 8:53 ` Peter Zijlstra
2017-02-03 8:53 ` Peter Zijlstra
2017-02-03 11:04 ` Sachin Sant
2017-02-03 11:04 ` Sachin Sant
2017-02-03 12:59 ` Mike Galbraith
2017-02-03 13:37 ` Peter Zijlstra
2017-02-03 13:52 ` Mike Galbraith
2017-02-03 15:44 ` Paul E. McKenney
2017-02-03 15:54 ` Paul E. McKenney
2017-02-06 6:23 ` Sachin Sant
2017-02-06 6:23 ` Sachin Sant
2017-02-06 15:10 ` Paul E. McKenney
2017-02-06 15:14 ` Paul E. McKenney
2017-02-03 13:04 ` Borislav Petkov
2017-02-22 9:03 ` Wanpeng Li
2017-02-24 9:16 ` [tip:sched/urgent] sched/core: Fix update_rq_clock() splat on hotplug (and suspend/resume) tip-bot for Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1485863317.4498.2.camel@gmx.de \
--to=efault@gmx.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-next@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=matt@codeblueprint.co.uk \
--cc=mpe@ellerman.id.au \
--cc=peterz@infradead.org \
--cc=sachinp@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.