All of lore.kernel.org
 help / color / mirror / Atom feed
From: Quentin Perret <qperret@google.com>
To: linux-kernel@vger.kernel.org
Cc: peterz@infradead.org, aaron.lwe@gmail.com,
	valentin.schneider@arm.com, mingo@kernel.org, pauld@redhat.com,
	jdesfossez@digitalocean.com, naravamudan@digitalocean.com,
	vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
	juri.lelli@redhat.com, rostedt@goodmis.org, bsegall@google.com,
	mgorman@suse.de, kernel-team@android.com, john.stultz@linaro.org
Subject: NULL pointer dereference in pick_next_task_fair
Date: Mon, 28 Oct 2019 17:46:03 +0000	[thread overview]
Message-ID: <20191028174603.GA246917@google.com> (raw)

Hi all,

The following issue has been observed with 5.4-rc*-based android
kernels both on an x86 VM and native arm64:

---8<---
[  222.515957] BUG: kernel NULL pointer dereference, address: 0000000000000040
[  222.518871] #PF: supervisor read access in kernel mode
[  222.518871] #PF: error_code(0x0000) - not-present page
[  222.518871] PGD 800000007cdf7067 P4D 800000007cdf7067 PUD 7c868067 PMD 0 
[  222.518871] Oops: 0000 [#1] PREEMPT SMP PTI
[  222.518871] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 5.4.0-rc4-mainline-g6a9b34166c05 #1
[  222.518871] Hardware name: ChromiumOS crosvm, BIOS 0 
[  222.518871] RIP: 0010:set_next_entity+0xf/0xf0
[  222.518871] Code: 42 0b e5 85 31 c0 e8 90 1a fc ff 0f 0b eb 9b 66 90 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 57 41 56 53 48 89 f3 49 89 fe <83> 7e 40 00 74 3d 4c 89 f7 48 89 de e8 80 f9 ff ff 4c 8d 7b 18 4d
[  222.518871] RSP: 0018:ffffb95040073de8 EFLAGS: 00010046
[  222.518871] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff85ec0c30
[  222.518871] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff9ec44c728500
[  222.518871] RBP: ffffb95040073e00 R08: 0000000000000000 R09: 0000000000000000
[  222.518871] R10: 0000000000000000 R11: ffffffff84ef0a30 R12: 0000000000000000
[  222.518871] R13: 0000000000000000 R14: ffff9ec44c728500 R15: ffffb95040073e60
[  222.518871] FS:  0000000000000000(0000) GS:ffff9ec44c700000(0000) knlGS:0000000000000000
[  222.518871] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  222.518871] CR2: 0000000000000040 CR3: 000000007c87a002 CR4: 00000000003606e0
[  222.518871] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  222.518871] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  222.518871] Call Trace:
[  222.518871]  pick_next_task_fair+0xe8/0x410
[  222.518871]  ? rcu_note_context_switch+0x3ef/0x520
[  222.518871]  ? finish_task_switch+0x10d/0x250
[  222.518871]  __schedule+0x1f8/0x6e0
[  222.518871]  schedule_idle+0x27/0x40
[  222.518871]  do_idle+0x21c/0x240
[  222.518871]  cpu_startup_entry+0x25/0x30
[  222.518871]  start_secondary+0x18d/0x1b0
[  222.518871]  secondary_startup_64+0xa4/0xb0
[  222.518871] Modules linked in: vmw_vsock_virtio_transport vmw_vsock_virtio_transport_common vsock_diag vsock can_raw can snd_intel8x0 snd_ac97_codec ac97_bus virtio_input virtio_pci virtio_mmio virtio_crypto mmc_block mmc_core dummy_cpufreq rtc_test dummy_hcd can_dev vcan virtio_net net_failover failover virt_wifi virtio_blk virtio_gpu ttm virtio_rng virtio_ring virtio 8250_of crypto_engine
[  222.518871] CR2: 0000000000000040
[  222.518871] ---[ end trace ae741809965b22f2 ]---
[  222.518871] RIP: 0010:set_next_entity+0xf/0xf0
[  222.518871] Code: 42 0b e5 85 31 c0 e8 90 1a fc ff 0f 0b eb 9b 66 90 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 57 41 56 53 48 89 f3 49 89 fe <83> 7e 40 00 74 3d 4c 89 f7 48 89 de e8 80 f9 ff ff 4c 8d 7b 18 4d
[  222.518871] RSP: 0018:ffffb95040073de8 EFLAGS: 00010046
[  222.518871] RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffff85ec0c30
[  222.518871] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff9ec44c728500
[  222.518871] RBP: ffffb95040073e00 R08: 0000000000000000 R09: 0000000000000000
[  222.518871] R10: 0000000000000000 R11: ffffffff84ef0a30 R12: 0000000000000000
[  222.518871] R13: 0000000000000000 R14: ffff9ec44c728500 R15: ffffb95040073e60
[  222.518871] FS:  0000000000000000(0000) GS:ffff9ec44c700000(0000) knlGS:0000000000000000
[  222.518871] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  222.518871] CR2: 0000000000000040 CR3: 000000007c87a002 CR4: 00000000003606e0
[  222.518871] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  222.518871] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  222.518871] Kernel panic - not syncing: Fatal exception
[  222.518871] Kernel Offset: 0x3e00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
--->8---

The issue is very transient and relatively hard to reproduce.

After digging a bit, the offending commit seems to be:

    67692435c411 ("sched: Rework pick_next_task() slow-path")

By 'offending' I mean that reverting it makes the issue go away. The
issue comes from the fact that pick_next_entity() returns a NULL se in
the 'simple' path of pick_next_task_fair(), which causes obvious
problems in the subsequent call to set_next_entity().

I'll dig more, but if anybody understands the issue in the meatime feel
free to send me a patch to try out :)

Thanks,
Quentin

             reply	other threads:[~2019-10-28 17:46 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-28 17:46 Quentin Perret [this message]
2019-10-28 21:49 ` NULL pointer dereference in pick_next_task_fair Peter Zijlstra
2019-10-29 11:34 ` Peter Zijlstra
2019-10-29 11:50   ` Quentin Perret
2019-10-30 22:50     ` Ram Muthiah
2019-10-31  1:33       ` Valentin Schneider
2019-10-31 10:54         ` Valentin Schneider
2019-10-31 14:24           ` Valentin Schneider
2019-10-31 22:15       ` Valentin Schneider
2019-11-06 12:05 ` Peter Zijlstra
2019-11-06 13:08   ` Peter Zijlstra
2019-11-06 15:04     ` Qais Yousef
2019-11-06 16:57       ` Peter Zijlstra
2019-11-06 17:26         ` Qais Yousef
2019-11-06 15:51   ` Kirill Tkhai
2019-11-06 16:54     ` Peter Zijlstra
2019-11-06 17:27       ` Peter Zijlstra
2019-11-07  8:36         ` Kirill Tkhai
2019-11-07 13:26           ` Peter Zijlstra
2019-11-07 15:12             ` Kirill Tkhai
2019-11-07 15:42               ` Peter Zijlstra
2019-11-07 15:53                 ` Kirill Tkhai
2019-11-07 15:38             ` Quentin Perret
2019-11-07 18:43               ` Peter Zijlstra
2019-11-07 19:27                 ` Quentin Perret
2019-11-07 19:31                   ` Peter Zijlstra
2019-11-07 19:42                     ` Quentin Perret
2019-11-07 19:29                 ` Peter Zijlstra
2019-11-08 11:02                   ` Quentin Perret
2019-11-08 11:47                     ` Valentin Schneider
2019-11-08 11:58                       ` Quentin Perret
2019-11-08 12:00                     ` Peter Zijlstra
2019-11-08 12:15                       ` Quentin Perret
2019-11-08 12:35                         ` Peter Zijlstra
2019-11-08 12:24                       ` Peter Zijlstra
2019-11-08 11:55                   ` Peter Zijlstra
2019-11-08 12:52                     ` Peter Zijlstra
2019-11-07 16:09             ` Qais Yousef

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191028174603.GA246917@google.com \
    --to=qperret@google.com \
    --cc=aaron.lwe@gmail.com \
    --cc=bsegall@google.com \
    --cc=dietmar.eggemann@arm.com \
    --cc=jdesfossez@digitalocean.com \
    --cc=john.stultz@linaro.org \
    --cc=juri.lelli@redhat.com \
    --cc=kernel-team@android.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@suse.de \
    --cc=mingo@kernel.org \
    --cc=naravamudan@digitalocean.com \
    --cc=pauld@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=valentin.schneider@arm.com \
    --cc=vincent.guittot@linaro.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.