From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christian Borntraeger Subject: Re: [PATCH 2/2] KVM: Create debugfs dir and stat files for each VM Date: Fri, 27 Nov 2015 09:54:49 +0100 Message-ID: <56581A59.4080705@de.ibm.com> References: <56576FD6.40506@de.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Cc: frankja@linux.vnet.ibm.com, kvm@vger.kernel.org, pbonzini@redhat.com, pmorel@linux.vnet.ibm.com, dan.carpenter@oracle.com, "Kevin's boot bot" To: Tyler Baker Return-path: Received: from e06smtp07.uk.ibm.com ([195.75.94.103]:43860 "EHLO e06smtp07.uk.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753931AbbK0Iyy (ORCPT ); Fri, 27 Nov 2015 03:54:54 -0500 Received: from localhost by e06smtp07.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 27 Nov 2015 08:54:53 -0000 Received: from b06cxnps4074.portsmouth.uk.ibm.com (d06relay11.portsmouth.uk.ibm.com [9.149.109.196]) by d06dlp01.portsmouth.uk.ibm.com (Postfix) with ESMTP id D636D17D805D for ; Fri, 27 Nov 2015 08:55:16 +0000 (GMT) Received: from d06av10.portsmouth.uk.ibm.com (d06av10.portsmouth.uk.ibm.com [9.149.37.251]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id tAR8soRF57409684 for ; Fri, 27 Nov 2015 08:54:50 GMT Received: from d06av10.portsmouth.uk.ibm.com (localhost [127.0.0.1]) by d06av10.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id tAR7spgS026214 for ; Fri, 27 Nov 2015 00:54:51 -0700 In-Reply-To: <56576FD6.40506@de.ibm.com> Sender: kvm-owner@vger.kernel.org List-ID: On 11/26/2015 09:47 PM, Christian Borntraeger wrote: > On 11/26/2015 05:17 PM, Tyler Baker wrote: >> Hi Christian, >> >> The kernelci.org bot recently has been reporting kvm guest boot >> failures[1] on various arm64 platforms in next-20151126. The bot >> bisected[2] the failures to the commit in -next titled "KVM: Create >> debugfs dir and stat files for each VM". I confirmed by reverting this >> commit on top of next-20151126 it resolves the boot issue. >> >> In this test case the host and guest are booted with the same kernel. >> The host is booted over nfs, installs qemu (qemu-system arm64 2.4.0), >> and launches a guest. The host is booting fine, but when the guest is >> launched it errors with "Failed to retrieve host CPU features!". I >> checked the host logs, and found an "Unable to handle kernel paging >> request" splat[3] which occurs when the guest is attempting to start. >> >> I scanned the patch in question but nothing obvious jumped out at me, >> any thoughts? > > Not really. > Do you have processing running that do read the files in /sys/kernel/debug/kvm/* ? > > If I read the arm oops message correctly it oopsed inside > __srcu_read_lock. there is actually nothing in there that can oops, > except the access to the preempt count. I am just guessing right now, > but maybe the preempt variable is no longer available (as the process > is gone). As long as a debugfs file is open, we hold a reference to > the kvm, which holds a reference to the mm, so the mm might be killed > after the process. But this is supposed to work, so maybe its something > different. An objdump of __srcu_read_lock might help. Hmm, the preempt thing is done in srcu_read_lock, but the crash is in __srcu_read_lock. This function gets the srcu struct from mmu_notifier.c, which must be present and is initialized during boot. int __srcu_read_lock(struct srcu_struct *sp) { int idx; idx = READ_ONCE(sp->completed) & 0x1; __this_cpu_inc(sp->per_cpu_ref->c[idx]); smp_mb(); /* B */ /* Avoid leaking the critical section. */ __this_cpu_inc(sp->per_cpu_ref->seq[idx]); return idx; } Looking at the code I have no clue why the patch does make a difference. Can you try to get an objdump -S for__Srcu_read_lock? > > I will drop it from my tree until we understand the problem > > Christian >