From mboxrd@z Thu Jan  1 00:00:00 1970
From: Radim =?utf-8?B?S3LEjW3DocWZ?= <rkrcmar@redhat.com>
Subject: Re: [PATCH] KVM: VMX: fix vmwrite to invalid VMCS
Date: Tue, 7 Jul 2015 15:50:13 +0200
Message-ID: <20150707135012.GA26862@potion.brq.redhat.com>
References: <1435931368-27730-1-git-send-email-rkrcmar@redhat.com>
 <5596A679.1050202@redhat.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: QUOTED-PRINTABLE
Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
	stable@vger.kernel.org, Yang Zhang <yang.z.zhang@intel.com>,
	Liang Li <liang.z.li@intel.com>
To: Paolo Bonzini <pbonzini@redhat.com>
Return-path: <stable-owner@vger.kernel.org>
Content-Disposition: inline
In-Reply-To: <5596A679.1050202@redhat.com>
Sender: stable-owner@vger.kernel.org
List-Id: kvm.vger.kernel.org

2015-07-03 17:12+0200, Paolo Bonzini:
> On 03/07/2015 15:49, Radim Kr=C4=8Dm=C3=A1=C5=99 wrote:
>> fpu_activate is called outside of vcpu_load(), which means it should=
 not
>> touch VMCS, but fpu_activate needs to.  Avoid the call by moving it =
to a
>> point where we know that the guest needs eager FPU and VMCS is loade=
d.
>>=20
>> This will get rid of the following trace
>>=20
>>  vmwrite error: reg 6800 value 0 (err 1)
>>   [<ffffffff8162035b>] dump_stack+0x19/0x1b
>>   [<ffffffffa046c701>] vmwrite_error+0x2c/0x2e [kvm_intel]
>>   [<ffffffffa045f26f>] vmcs_writel+0x1f/0x30 [kvm_intel]
>>   [<ffffffffa04617e5>] vmx_fpu_activate.part.61+0x45/0xb0 [kvm_intel=
]
>>   [<ffffffffa0461865>] vmx_fpu_activate+0x15/0x20 [kvm_intel]
>>   [<ffffffffa0560b91>] kvm_arch_vcpu_create+0x51/0x70 [kvm]
>>   [<ffffffffa0548011>] kvm_vm_ioctl+0x1c1/0x760 [kvm]
>>   [<ffffffff8118b55a>] ? handle_mm_fault+0x49a/0xec0
>>   [<ffffffff811e47d5>] do_vfs_ioctl+0x2e5/0x4c0
>>   [<ffffffff8127abbe>] ? file_has_perm+0xae/0xc0
>>   [<ffffffff811e4a51>] SyS_ioctl+0xa1/0xc0
>>   [<ffffffff81630949>] system_call_fastpath+0x16/0x1b
>>=20
>> (Note: we also unconditionally activate FPU in vmx_vcpu_reset(), so =
the
>>  removed code added nothing.)
>>=20
>> Fixes: c447e76b4cab ("kvm/fpu: Enable eager restore kvm FPU for MPX"=
)
>> Cc: <stable@vger.kernel.org>
>> Reported-by: Vlastimil Holer <vlastimil.holer@gmail.com>
>> Signed-off-by: Radim Kr=C4=8Dm=C3=A1=C5=99 <rkrcmar@redhat.com>
>=20
> Andrey reported offlist that the bug went away by reverting 1cde293. =
 So
> the patch would at least need a new commit message. :)
>=20
> I'm putting it on hold.

I think it's a different bug than the one Andrey reproduced
(https://bugzilla.kernel.org/show_bug.cgi?id=3D100671).
I'll send a v2 that cleans up the code and makes the commit message
clearer, unless you find the reasoning below unsound.

This bug is specific to 'kvm_arch_vcpu_create()' and Vlastimil Holer hi=
t
it on RHEL 7.2 (278.el7) kernel that didn't have 1cde2930e154
("sched/preempt: Add static_key() to preempt_notifiers").

The commit message does not base on tracing (I haven't reproduced it),
but I couldn't make sense out of this bug otherwise.
I think it happens just because other VCPU preempted the new one betwee=
n
vmx_vcpu_put()+put_cpu() and the end of kvm_x86_ops->fpu_activate(), so
vmwrite accessed different VMCS.  The code in kvm_vm_ioctl_create_vcpu(=
)
that made me think so:

  vcpu =3D kvm_arch_vcpu_create(id) {
    vcpu =3D kvm_x86_ops->vcpu_create(kvm, id) {
      vmx =3D kmem_cache_zalloc(kvm_vcpu_cache, GFP_KERNEL);
      kvm_vcpu_init(&vmx->vcpu, kvm, id);
      vmx->loaded_vmcs =3D &vmx->vmcs01;
      vmx->loaded_vmcs->vmcs =3D alloc_vmcs();
      loaded_vmcs_init(vmx->loaded_vmcs);

      // disabling preemption and activating VMCS
      cpu =3D get_cpu();
      vmx_vcpu_load(&vmx->vcpu, cpu);

      vmx_vcpu_setup(vmx);

      // abandoning VMCS and enabling preemption
      vmx_vcpu_put(&vmx->vcpu);
      put_cpu();

      return &vmx->vcpu;
    }

    // enabled preemption and undefined current VMCS
    kvm_x86_ops->fpu_activate(vcpu);
    return vcpu;
  }

  preempt_notifier_init(&vcpu->preempt_notifier, &kvm_preempt_ops);
  kvm_arch_vcpu_setup(vcpu) {
    vcpu_load(vcpu);
    ...
  }