From mboxrd@z Thu Jan  1 00:00:00 1970
From: Gleb Natapov <gleb@redhat.com>
Subject: Re: [nvmx testing, linux on linux] Disabling EPT in L1 renders L2
 stuck on boot
Date: Mon, 7 Oct 2013 18:18:02 +0300
Message-ID: <20131007151802.GB9127@redhat.com>
References: <CAOaxAcbLEDpF4OWBZYV-0LjZ3ifHRq6qmbUoqR-8dcS5JbP=TA@mail.gmail.com>
 <CAOaxAcZu4VeecK+2A5czZ8km1Y26qzys7EBNdbgSHw-nurh=ZA@mail.gmail.com>
 <20131004093859.GF31244@redhat.com>
 <CAOaxAcatqVOKCJ06=ckkKBAkk_Vjsxqe1TaVgUXB8MsXHhLZyA@mail.gmail.com>
 <20131004130550.GA12257@redhat.com>
 <20131004130817.GB12257@redhat.com>
 <CAOaxAca2tfZxtjBwS43AGBR3yTaKb+fC8DpoW3bvn=RXtfJp8w@mail.gmail.com>
 <CAOaxAcbh8nfpV_fUh1kf+BTrE+tHF-GV1=uwKtFnP=hyWrc4Ug@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: "kvm@vger.kernel.org" <kvm@vger.kernel.org>
To: Kashyap Chamarthy <kashyap.cv@gmail.com>
Return-path: <kvm-owner@vger.kernel.org>
Received: from mx1.redhat.com ([209.132.183.28]:21058 "EHLO mx1.redhat.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1751567Ab3JIMrL (ORCPT <rfc822;kvm@vger.kernel.org>);
	Wed, 9 Oct 2013 08:47:11 -0400
Content-Disposition: inline
In-Reply-To: <CAOaxAcbh8nfpV_fUh1kf+BTrE+tHF-GV1=uwKtFnP=hyWrc4Ug@mail.gmail.com>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Mon, Oct 07, 2013 at 06:29:30PM +0530, Kashyap Chamarthy wrote:
> Gleb, so I just did a trace of KVM MMU to try to understand why L2 is
> stuck with shadow on EPT
> 
> Ensure, EPT is enabled on L0 & disabled on L1
> 
> On L0:
> ---------
>   $ cat /sys/module/kvm_intel/parameters/ept
>   Y
> 
> On L1
> ---------
>   $ cat /sys/module/kvm_intel/parameters/ept
>   N
> 
> Build and install trace-cmd on L1:
> 
>   $ git clone git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/trace-cmd.git
> trace-cmd
>   $ make && sudo make-install
> 
> Run trace command (on L1) to capture kvmmmu event:
> 
>   $ trace-cmd record -b 2000 -e kvmmmu
You need to trace kvm events too and make buffer bigger
trace-cmd record -b 100000 -e kvmmmu -e kvm.

>   /sys/kernel/debug/tracing/events/kvmmmu/filter
>   /sys/kernel/debug/tracing/events/*/kvmmmu/filter
>   Hit Ctrl^C to stop recording
> 
> Boot L2 guest:
> 
>   $ virsh start nguest-01 --console
> 
> After a minute or two, ctl-C the trace-cmd, generate the report:
> 
>   $ trace-cmd report
>   version = 6
>   trace-cmd: No such file or directory
>     function is_writable_pte not defined
>   CPU 1 is empty
>   CPU 2 is empty
>   CPU 3 is empty
>   cpus=4
>    qemu-system-x86-2554  [000] 90667.995345:
> kvm_mmu_invalidate_zap_all_pages: kvm-mmu-valid-gen 0 used_pages 0
>    qemu-system-x86-2557  [000] 90668.078978: kvm_mmu_get_page:     new
> sp gfn 0 0/2 q0 direct --- !pge !nxe root 0 sync
>    qemu-system-x86-2557  [000] 90668.078980: kvm_mmu_get_page:     new
> sp gfn 40000 0/2 q0 direct --- !pge !nxe root 0 sync
>    qemu-system-x86-2557  [000] 90668.078980: kvm_mmu_get_page:     new
> sp gfn 80000 0/2 q0 direct --- !pge !nxe root 0 sync
>    qemu-system-x86-2557  [000] 90668.078981: kvm_mmu_get_page:     new
> sp gfn c0000 0/2 q0 direct --- !pge !nxe root 0 sync
>    qemu-system-x86-2557  [000] 90668.079025: kvm_mmu_get_page:     new
> sp gfn 0 0/1 q0 direct --- !pge !nxe root 0 sync
Is this all?

> 
> Side question:  I wonder why would trace-cmd say "trace-cmd: No such
> file or directory"?
> 
Ignore it.

--
			Gleb.