From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DE09C4332F for ; Tue, 2 Nov 2021 15:53:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E10D561050 for ; Tue, 2 Nov 2021 15:53:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234624AbhKBPzr (ORCPT ); Tue, 2 Nov 2021 11:55:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33022 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231361AbhKBPzq (ORCPT ); Tue, 2 Nov 2021 11:55:46 -0400 Received: from mail-pf1-x430.google.com (mail-pf1-x430.google.com [IPv6:2607:f8b0:4864:20::430]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C5F4C061714 for ; Tue, 2 Nov 2021 08:53:10 -0700 (PDT) Received: by mail-pf1-x430.google.com with SMTP id s5so8384738pfg.2 for ; Tue, 02 Nov 2021 08:53:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=uXG46CNq1yzX39v2pBF3aSKlrNsVM6oau9TO8Y5/ynw=; b=pAs2y+ijAdLSwVnN/iXoGJIDLS2TO7pznZDqrBgEiymfJ5fMb7bmsUqig8SFz/Xfz9 j6el+hpRpb/l8ztcGtA143m4jBw95nEruKEKLZflgyH0d2cKE1Vm3WPi9wOqO5i1+Got XoLGG8E6DHHL/sQ7ohtI+bDXXVanHnKjoPPgFat2aSGVzvixVFf5qEt0Ofs/Rn/gfo/M z0d3eH1FlZFlKYHdt3NTZ5BlT9ik94kSiECQBj3WqPjPXKf+mh8TnEQje+pknFsZ5NZd i4H1RfdoBxpT3Fcp/BBPgnShbHpk6nXoYfhmhoGPoZS/pZcuC6RVsf3d2+Y8t8vBgnBj 9oPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=uXG46CNq1yzX39v2pBF3aSKlrNsVM6oau9TO8Y5/ynw=; b=boIdJPKFC6RSizdanRJd7Dh6UGygkV8Il2sstopD20byN4o3F7YeXCPaxW+Qr0dgMu Vwb1CD7lTssGK0tmiHr9CjEgFVRxKgzjgntxcGFfWxC/vjvVljzIa7PYui7BIOHtPcDN ZCNrYUA5DMu6rE6f+DAAppBWbjR6dBXUqLB2u7nxTZ7p6TLAB4qjE0A/idkWoIlEbmMs 8oJWVYVcFDnESjlBt+jMRSFXj0fBIjtdeSoUp7MAW28CEXK9UGpU4f6XvMUDBzNtj22B YJsPt/6uAxepe/AllfzG0UtJLGC3ENdqA8JzO2YiH2G6K0alrzUescxgGe3cpWq6n/yS RdUg== X-Gm-Message-State: AOAM530zMuOT2mZPffOkt7jryu/0NnzfA8c+1oavl36F2bZUprZp/MfQ khGXNcSf9TNAL9VLOLlBe7UEuQ== X-Google-Smtp-Source: ABdhPJxNtDKLBpxoiRVYVgHQ8HO9yQL5vu6uWpAHtSkKWriSHm706e6zFB2nNeZItuHLc1BNhEWGrA== X-Received: by 2002:a63:e24b:: with SMTP id y11mr28060654pgj.452.1635868389352; Tue, 02 Nov 2021 08:53:09 -0700 (PDT) Received: from google.com (157.214.185.35.bc.googleusercontent.com. [35.185.214.157]) by smtp.gmail.com with ESMTPSA id nr14sm2839945pjb.24.2021.11.02.08.53.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Nov 2021 08:53:08 -0700 (PDT) Date: Tue, 2 Nov 2021 15:53:04 +0000 From: Sean Christopherson To: Vitaly Kuznetsov Cc: Maxim Levitsky , Paolo Bonzini , kvm@vger.kernel.org, Kieran Bingham , Jan Kiszka , Andrew Jones , Jonathan Corbet , Ingo Molnar , Thomas Gleixner , "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" , Johannes Berg , Wanpeng Li , "H. Peter Anvin" , Jessica Yu , Jim Mattson , Joerg Roedel , Yang Weijiang , linux-kernel@vger.kernel.org, Borislav Petkov , "open list:KERNEL SELFTEST FRAMEWORK" , "open list:DOCUMENTATION" , Shuah Khan , Andrew Morton Subject: Re: [PATCH v3 6/6] KVM: selftests: test KVM_GUESTDBG_BLOCKIRQ Message-ID: References: <20210811122927.900604-1-mlevitsk@redhat.com> <20210811122927.900604-7-mlevitsk@redhat.com> <137f2dcc-75d2-9d71-e259-dd66d43ad377@redhat.com> <87sfwfkhk5.fsf@vitty.brq.redhat.com> <87k0hqkf6p.fsf@vitty.brq.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87k0hqkf6p.fsf@vitty.brq.redhat.com> Precedence: bulk List-ID: X-Mailing-List: linux-doc@vger.kernel.org On Tue, Nov 02, 2021, Vitaly Kuznetsov wrote: > Sean Christopherson writes: > > I haven't verified on hardware, but my guess is that this code in vmx_vcpu_run() > > > > /* When single-stepping over STI and MOV SS, we must clear the > > * corresponding interruptibility bits in the guest state. Otherwise > > * vmentry fails as it then expects bit 14 (BS) in pending debug > > * exceptions being set, but that's not correct for the guest debugging > > * case. */ > > if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP) > > vmx_set_interrupt_shadow(vcpu, 0); > > > > interacts badly with APICv=1. It will kill the STI shadow and cause the IRQ in > > vmcs.GUEST_RVI to be recognized when it (micro-)architecturally should not. My > > head is going in circles trying to sort out what would actually happen. Maybe > > comment out that and/or disable APICv to see if either one makes the test pass? > > > > Interestingly, > > loading 'kvm-intel' with 'enable_apicv=0' makes the test pass, however, > commenting out "vmx_set_interrupt_shadow()" as suggested gives a > different result (with enable_apicv=1): > > # ./x86_64/debug_regs > ==== Test Assertion Failure ==== > x86_64/debug_regs.c:179: run->exit_reason == KVM_EXIT_DEBUG && run->debug.arch.exception == DB_VECTOR && run->debug.arch.pc == target_rip && run->debug.arch.dr6 == target_dr6 > pid=16352 tid=16352 errno=0 - Success > 1 0x0000000000402b33: main at debug_regs.c:179 (discriminator 10) > 2 0x00007f36401bd554: ?? ??:0 > 3 0x00000000004023a9: _start at ??:? > SINGLE_STEP[1]: exit 9 exception -2147483615 rip 0x1 (should be 0x4024d9) dr6 0xffff4ff0 (should be 0xffff4ff0) Exit 9 is KVM_EXIT_FAIL_ENTRY, which in this case VM-Entry likely failed due to invalid guest state because there was STI blocking with single-step enabled but no pending BS #DB: Bit 14 (BS) must be 1 if the TF flag (bit 8) in the RFLAGS field is 1 and the BTF flag (bit 1) in the IA32_DEBUGCTL field is 0. Which is precisely what that hack-a-fix avoids. There isn't really a clean solution for legacy single-step, AFAIK the only way to avoid this would be to switch KVM_GUESTDBG_SINGLESTEP to use MTF. But that mess is a red herring, the test fails with the same signature with APICv=1 if the STI is replaced by PUSHF+BTS+POPFD (to avoid the STI shadow). We all missed this key detail from Vitaly's report: SINGLE_STEP[1]: exit 8 exception 1 rip 0x402a25 (should be 0x402a27) dr6 0xffff4ff0 (should be 0xffff4ff0) ^^^^^^ Exit '8' is KVM_EXIT_SHUTDOWN, i.e. the arrival of the IRQ hosed the guest because the test doesn't invoke vm_init_descriptor_tables() to install event handlers. The "exception 1" shows up because the run page isn't sanitized by the test, i.e. it's stale data that happens to match. So I would fully expect this test to fail with AVIC=1. The problem is that KVM_GUESTDBG_BLOCKIRQ does absolutely nothing to handle APICv interrupts. And even if KVM does something to fudge that behavior in the emulated local APIC, the test will then fail miserably virtual IPIs (currently AVIC only). I stand by my original comment that "Deviating this far from architectural behavior will end in tears at some point." Rather than try to "fix" APICv, I vote to instead either reject KVM_GUESTDBG_BLOCKIRQ if APICv=1, or log a debug message saying that KVM_GUESTDBG_BLOCKIRQ is ineffective with APICv=1.