[RFC PATCH v8 00/35] AMD: Add Secure AVIC Guest Support

[RFC PATCH v8 00/35] AMD: Add Secure AVIC Guest Support

[Neeraj Upadhyay]

Introduction
------------

Secure AVIC is a new hardware feature in the AMD64 architecture to
allow SEV-SNP guests to prevent the hypervisor from generating
unexpected interrupts to a vCPU or otherwise violate architectural
assumptions around APIC behavior.

One of the significant differences from AVIC or emulated x2APIC is that
Secure AVIC uses a guest-owned and managed APIC backing page. It also
introduces additional fields in both the VMCB and the Secure AVIC backing
page to aid the guest in limiting which interrupt vectors can be injected
into the guest.

Guest APIC Backing Page
-----------------------
Each vCPU has a guest-allocated APIC backing page of size 4K, which
maintains APIC state for that vCPU. The x2APIC MSRs are mapped at
their corresposing x2APIC MMIO offset within the guest APIC backing
page. All x2APIC accesses by guest or Secure AVIC hardware operate
on this backing page. The backing page should be pinned and NPT entry
for it should be always mapped while the corresponding vCPU is running.


MSR Accesses
------------
Secure AVIC only supports x2APIC MSR accesses. xAPIC MMIO offset based
accesses are not supported.

Some of the MSR accesses such as ICR writes (with shorthand equal to
self), SELF_IPI, EOI, TPR writes are accelerated by Secure AVIC
hardware. Other MSR accesses generate a #VC exception. The #VC
exception handler reads/writes to the guest APIC backing page.
As guest APIC backing page is accessible to the guest, the Secure
AVIC driver code optimizes APIC register access by directly
reading/writing to the guest APIC backing page (instead of taking
the #VC exception route).

In addition to the architected MSRs, following new fields are added to
the guest APIC backing page which can be modified directly by the
guest:

a. ALLOWED_IRR

ALLOWED_IRR reg offset indicates the interrupt vectors which the guest
allows the hypervisor to send. The combination of host-controlled
REQUESTED_IRR vectors (part of VMCB) and ALLOWED_IRR is used by
hardware to update the IRR vectors of the Guest APIC backing page.

#Offset        #bits        Description
204h           31:0         Guest allowed vectors 0-31
214h           31:0         Guest allowed vectors 32-63
...
274h           31:0         Guest allowed vectors 224-255

ALLOWED_IRR is meant to be used specifically for vectors that the
hypervisor is allowed to inject, such as device interrupts.  Interrupt
vectors used exclusively by the guest itself (like IPI vectors) should
not be allowed to be injected into the guest for security reasons.

b. NMI Request
 
#Offset        #bits        Description
278h           0            Set by Guest to request Virtual NMI

Guest need to set NMI Request register to allow the Hypervisor to
inject vNMI to it.

LAPIC Timer Support
-------------------
LAPIC timer is emulated by the hypervisor. So, APIC_LVTT, APIC_TMICT and
APIC_TDCR, APIC_TMCCT APIC registers are not read/written to the guest
APIC backing page and are communicated to the hypervisor using SVM_EXIT_MSR
VMGEXIT. 

IPI Support
-----------
Only SELF_IPI is accelerated by Secure AVIC hardware. Other IPIs require
writing (from the Secure AVIC driver) to the IRR vector of the target CPU
backing page and then issuing VMGEXIT for the hypervisor to notify the
target vCPU.

KEXEC Support
-------------
Secure AVIC enabled guest can kexec to another kernel which has Secure
AVIC enabled, as the Hypervisor has Secure AVIC feature bit set in the
sev_status.

Open Points
-----------

The Secure AVIC driver only supports physical destination mode. If
logical destination mode need to be supported, then a separate x2apic
driver would be required for supporting logical destination mode.


Testing
-------

This series is based on top of commit 262fcdc7c5e8 Revert "sched/numa: add statistics of numa balance task" of tip/tip master branch.

Host Secure AVIC support patch series is at [1].

Qemu support patch is at [2].

QEMU commandline for testing Secure AVIC enabled guest:

qemu-system-x86_64 <...> -object sev-snp-guest,id=sev0,policy=0xb0000,cbitpos=51,
reduced-phys-bits=1,allowed-sev-features=true,secure-avic=true

Following tests are done:

1) Boot to Prompt using initramfs and ubuntu fs.
2) Verified timer and IPI as part of the guest bootup.
3) Verified long run SCF TORTURE IPI test.

[1] https://github.com/AMDESE/linux-kvm/tree/savic-host-latest
[2] https://github.com/AMDESE/qemu/tree/secure-avic

Changes since v7

v7: https://lore.kernel.org/lkml/20250610175424.209796-1-Neeraj.Upadhyay@amd.com/

   - Commit log updates.
   - Applied Reviewed-by and Acked-by.
   - Combined few patches.

Changes since v6

v6: https://lore.kernel.org/lkml/20250514071803.209166-1-Neeraj.Upadhyay@amd.com/

  - Restructured the patches to split out function/macro rename into
    separate patches.
  - Update commit logs with more details on impact to kvm.ko text size.
  - Updated the new macros in patch "x86/apic: KVM: Deduplicate APIC vector =>
    register+bit math" to type cast macro parameter to unsigned int.
    This ensures better code generation for cases where signed int is
    passed to these macros. With this update, below patches have been
    removed in this version:

    x86/apic: Change apic_*_vector() vector param to unsigned
    x86/apic: Change get/set reg operations reg param to unsigned

  - Added Tianyu's Reviewed-by's.

Changes since v5

v5: https://lore.kernel.org/lkml/20250429061004.205839-1-Neeraj.Upadhyay@amd.com/

  - Add back RFC tag due to new changes to share code between KVM's
    lapic emulation and Secure AVIC.
  - Minor optimizations to the apic bitwise ops and set/get reg
    operations.
  - Other misc fixes, cleanups and refactoring due to code sharing with
    KVM lapic implementation.

Change since v4

v4: https://lore.kernel.org/lkml/20250417091708.215826-1-Neeraj.Upadhyay@amd.com/

  - Add separate patch for update_vector() apic callback addition.
  - Add a cleanup patch for moving apic_update_irq_cfg() calls to
    apic_update_vector().
  - Cleaned up change logs.
  - Rebased to latest tip/tip master. Resolved merge conflicts due to
    sev code movement to sev-startup.c in mainline.
  - Other misc cleanups.

Change since v3

v3: https://lore.kernel.org/lkml/20250401113616.204203-1-Neeraj.Upadhyay@amd.com/

  - Move KVM updates to a separate patch.
  - Cleanups to use guard().
  - Refactored IPI callbacks addition.
  - Misc cleanups.

Change since v2

v2: https://lore.kernel.org/lkml/20250226090525.231882-1-Neeraj.Upadhyay@amd.com/

  - Removed RFC tag.
  - Change config rule to not select AMD_SECURE_AVIC config if
    AMD_MEM_ENCRYPT config is enabled.
  - Fix broken backing page GFP_KERNEL allocation in setup_local_APIC().
    Use alloc_percpu() for APIC backing pages allocation during Secure
    AVIC driver probe.
  - Remove code to check for duplicate APIC_ID returned by the
    Hypervisor. Topology evaluation code already does that during boot.
  - Fix missing update_vector() callback invocation during vector
    cleanup paths. Invoke update_vector() during setup and tearing down
    of a vector.
  - Reuse find_highest_vector() from kvm/lapic.c.
  - Change savic_register_gpa/savic_unregister_gpa() interface to be
    invoked only for the local CPU.
  - Misc cleanups.

Change since v1

v1: https://lore.kernel.org/lkml/20240913113705.419146-1-Neeraj.Upadhyay@amd.com/

  - Added Kexec support.
  - Instead of doing a 2M aligned allocation for backing pages,
    allocate individual PAGE_SIZE pages for vCPUs.
  - Instead of reading Extended Topology Enumeration CPUID, APIC_ID
    value is read from Hv and updated in APIC backing page. Hv returned
    ID is checked for any duplicates.
  - Propagate all LVT* register reads and writes to Hv.
  - Check that Secure AVIC control MSR is not intercepted by Hv.
  - Fix EOI handling for level-triggered interrupts.
  - Misc cleanups and commit log updates.

Kishon Vijay Abraham I (2):
  x86/sev: Initialize VGIF for secondary VCPUs for Secure AVIC
  x86/sev: Enable NMI support for Secure AVIC

Neeraj Upadhyay (32):
  KVM: x86: Open code setting/clearing of bits in the ISR
  KVM: x86: Remove redundant parentheses around 'bitmap'
  KVM: x86: Rename VEC_POS/REG_POS macro usages
  KVM: x86: Change lapic regs base address to void pointer
  KVM: x86: Rename find_highest_vector()
  KVM: x86: Rename lapic get/set_reg() helpers
  KVM: x86: Rename lapic get/set_reg64() helpers
  KVM: x86: Rename lapic set/clear vector helpers
  x86/apic: KVM: Move apic_find_highest_vector() to a common header
  x86/apic: KVM: Move lapic get/set helpers to common code
  x86/apic: KVM: Move lapic set/clear_vector() helpers to common code
  x86/apic: KVM: Move apic_test)vector() to common code
  x86/apic: Rename 'reg_off' to 'reg'
  x86/apic: Unionize apic regs for 32bit/64bit access w/o type casting
  x86/apic: Simplify bitwise operations on apic bitmap
  x86/apic: Move apic_update_irq_cfg() calls to apic_update_vector()
  x86/apic: Add new driver for Secure AVIC
  x86/apic: Initialize Secure AVIC APIC backing page
  x86/apic: Populate .read()/.write() callbacks of Secure AVIC driver
  x86/apic: Initialize APIC ID for Secure AVIC
  x86/apic: Add update_vector() callback for apic drivers
  x86/apic: Add update_vector() callback for Secure AVIC
  x86/apic: Add support to send IPI for Secure AVIC
  x86/apic: Support LAPIC timer for Secure AVIC
  x86/apic: Add support to send NMI IPI for Secure AVIC
  x86/apic: Allow NMI to be injected from hypervisor for Secure AVIC
  x86/apic: Read and write LVT* APIC registers from HV for SAVIC guests
  x86/apic: Handle EOI writes for Secure AVIC guests
  x86/apic: Add kexec support for Secure AVIC
  x86/apic: Enable Secure AVIC in Control MSR
  x86/sev: Prevent SECURE_AVIC_CONTROL MSR interception for Secure AVIC
    guests
  x86/sev: Indicate SEV-SNP guest supports Secure AVIC

Sean Christopherson (1):
  x86/apic: KVM: Deduplicate APIC vector => register+bit math

 arch/x86/Kconfig                    |  13 +
 arch/x86/boot/compressed/sev.c      |  10 +-
 arch/x86/coco/core.c                |   3 +
 arch/x86/coco/sev/core.c            | 103 +++++++
 arch/x86/coco/sev/vc-handle.c       |  20 +-
 arch/x86/include/asm/apic.h         | 103 ++++++-
 arch/x86/include/asm/apicdef.h      |   2 +
 arch/x86/include/asm/msr-index.h    |   9 +-
 arch/x86/include/asm/sev-internal.h |   2 +
 arch/x86/include/asm/sev.h          |   8 +
 arch/x86/include/uapi/asm/svm.h     |   4 +
 arch/x86/kernel/apic/Makefile       |   1 +
 arch/x86/kernel/apic/apic.c         |   8 +
 arch/x86/kernel/apic/vector.c       |  33 ++-
 arch/x86/kernel/apic/x2apic_savic.c | 437 ++++++++++++++++++++++++++++
 arch/x86/kvm/lapic.c                |  95 ++----
 arch/x86/kvm/lapic.h                |  24 +-
 include/linux/cc_platform.h         |   8 +
 18 files changed, 770 insertions(+), 113 deletions(-)
 create mode 100644 arch/x86/kernel/apic/x2apic_savic.c


base-commit: 262fcdc7c5e8ee4b0978259ffdcd82a628f82f6d
-- 
2.34.1

[RFC PATCH v8 01/35] KVM: x86: Open code setting/clearing of bits in the ISR

[Neeraj Upadhyay]

Remove __apic_test_and_set_vector() and __apic_test_and_clear_vector(),
because the _only_ register that's safe to modify with a non-atomic
operation is ISR, because KVM isn't running the vCPU, i.e. hardware can't
service an IRQ or process an EOI for the relevant (virtual) APIC.

No functional change intended.

Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - Removed "inline" from apic_vector_to_isr().
 - Commit log updates.

 arch/x86/kvm/lapic.c | 19 +++++++------------
 1 file changed, 7 insertions(+), 12 deletions(-)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 73418dc0ebb2..013e8681247f 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -125,16 +125,6 @@ bool kvm_apic_pending_eoi(struct kvm_vcpu *vcpu, int vector)
 		apic_test_vector(vector, apic->regs + APIC_IRR);
 }
 
-static inline int __apic_test_and_set_vector(int vec, void *bitmap)
-{
-	return __test_and_set_bit(VEC_POS(vec), (bitmap) + REG_POS(vec));
-}
-
-static inline int __apic_test_and_clear_vector(int vec, void *bitmap)
-{
-	return __test_and_clear_bit(VEC_POS(vec), (bitmap) + REG_POS(vec));
-}
-
 __read_mostly DEFINE_STATIC_KEY_FALSE(kvm_has_noapic_vcpu);
 EXPORT_SYMBOL_GPL(kvm_has_noapic_vcpu);
 
@@ -744,9 +734,14 @@ void kvm_apic_clear_irr(struct kvm_vcpu *vcpu, int vec)
 }
 EXPORT_SYMBOL_GPL(kvm_apic_clear_irr);
 
+static void *apic_vector_to_isr(int vec, struct kvm_lapic *apic)
+{
+	return apic->regs + APIC_ISR + REG_POS(vec);
+}
+
 static inline void apic_set_isr(int vec, struct kvm_lapic *apic)
 {
-	if (__apic_test_and_set_vector(vec, apic->regs + APIC_ISR))
+	if (__test_and_set_bit(VEC_POS(vec), apic_vector_to_isr(vec, apic)))
 		return;
 
 	/*
@@ -789,7 +784,7 @@ static inline int apic_find_highest_isr(struct kvm_lapic *apic)
 
 static inline void apic_clear_isr(int vec, struct kvm_lapic *apic)
 {
-	if (!__apic_test_and_clear_vector(vec, apic->regs + APIC_ISR))
+	if (!__test_and_clear_bit(VEC_POS(vec), apic_vector_to_isr(vec, apic)))
 		return;
 
 	/*
-- 
2.34.1

[RFC PATCH v8 02/35] KVM: x86: Remove redundant parentheses around 'bitmap'

[Neeraj Upadhyay]

When doing pointer arithmetic in apic_test_vector() and
kvm_lapic_{set|clear}_vector(), remove the unnecessary
parentheses surrounding the 'bitmap' parameter.

No functional change intended.

Reviewed-by: Borislav Petkov (AMD) <bp@alien8.de>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - Applied Boris's Reviewed-by.
 - Commit shortlog update.

 arch/x86/kvm/lapic.c | 2 +-
 arch/x86/kvm/lapic.h | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 013e8681247f..533daf6dd1b1 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -114,7 +114,7 @@ static __always_inline void kvm_lapic_set_reg64(struct kvm_lapic *apic,
 
 static inline int apic_test_vector(int vec, void *bitmap)
 {
-	return test_bit(VEC_POS(vec), (bitmap) + REG_POS(vec));
+	return test_bit(VEC_POS(vec), bitmap + REG_POS(vec));
 }
 
 bool kvm_apic_pending_eoi(struct kvm_vcpu *vcpu, int vector)
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index 4ce30db65828..1638a3da383a 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -150,12 +150,12 @@ u64 kvm_lapic_readable_reg_mask(struct kvm_lapic *apic);
 
 static inline void kvm_lapic_clear_vector(int vec, void *bitmap)
 {
-	clear_bit(VEC_POS(vec), (bitmap) + REG_POS(vec));
+	clear_bit(VEC_POS(vec), bitmap + REG_POS(vec));
 }
 
 static inline void kvm_lapic_set_vector(int vec, void *bitmap)
 {
-	set_bit(VEC_POS(vec), (bitmap) + REG_POS(vec));
+	set_bit(VEC_POS(vec), bitmap + REG_POS(vec));
 }
 
 static inline void kvm_lapic_set_irr(int vec, struct kvm_lapic *apic)
-- 
2.34.1

[RFC PATCH v8 03/35] x86/apic: KVM: Deduplicate APIC vector => register+bit math

[Neeraj Upadhyay]

From: Sean Christopherson <seanjc@google.com>

Consolidate KVM's {REG,VEC}_POS() macros and lapic_vector_set_in_irr()'s
open coded equivalent logic in anticipation of the kernel gaining more
usage of vector => reg+bit lookups.

Use lapic_vector_set_in_irr()'s math as using divides for both the bit
number and register offset makes it easier to connect the dots, and for at
least one user, fixup_irqs(), "/ 32 * 0x10" generates ever so slightly
better code with gcc-14 (shaves a whole 3 bytes from the code stream):

((v) >> 5) << 4:
  c1 ef 05           shr    $0x5,%edi
  c1 e7 04           shl    $0x4,%edi
  81 c7 00 02 00 00  add    $0x200,%edi

(v) / 32 * 0x10:
  c1 ef 05           shr    $0x5,%edi
  83 c7 20           add    $0x20,%edi
  c1 e7 04           shl    $0x4,%edi

Keep KVM's tersely named macros as "wrappers" to avoid unnecessary churn
in KVM, and because the shorter names yield more readable code overall in
KVM.

The new macros type cast the vector parameter to "unsigned int". This is
required from better code generation for cases where an "int" is passed
to these macros in KVM code.

int v;

((v) >> 5) << 4:

  c1 f8 05    sar    $0x5,%eax
  c1 e0 04    shl    $0x4,%eax

((v) / 32 * 0x10):

  85 ff       test   %edi,%edi
  8d 47 1f    lea    0x1f(%rdi),%eax
  0f 49 c7    cmovns %edi,%eax
  c1 f8 05    sar    $0x5,%eax
  c1 e0 04    shl    $0x4,%eax

((unsigned int)(v) / 32 * 0x10):

  c1 f8 05    sar    $0x5,%eax
  c1 e0 04    shl    $0x4,%eax

(v) & (32 - 1):

  89 f8       mov    %edi,%eax
  83 e0 1f    and    $0x1f,%eax

(v) % 32

  89 fa       mov    %edi,%edx
  c1 fa 1f    sar    $0x1f,%edx
  c1 ea 1b    shr    $0x1b,%edx
  8d 04 17    lea    (%rdi,%rdx,1),%eax
  83 e0 1f    and    $0x1f,%eax
  29 d0       sub    %edx,%eax

(unsigned int)(v) % 32:

  89 f8       mov    %edi,%eax
  83 e0 1f    and    $0x1f,%eax

Overall kvm.ko text size is impacted if "unsigned int" is not used.

Bin      Orig     New (w/o unsigned int)  New (w/ unsigned int)

lapic.o  28580        28772                 28580
kvm.o    670810       671002                670810
kvm.ko   708079       708271                708079

No functional change intended.

[Neeraj: Type cast vec macro param to "unsigned int", provide data
         in commit log on "unsigned int" requirement]

Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/include/asm/apic.h | 7 +++++--
 arch/x86/kvm/lapic.h        | 4 ++--
 2 files changed, 7 insertions(+), 4 deletions(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index 23d86c9750b9..c84d4e86fe4e 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -488,11 +488,14 @@ static inline void apic_setup_apic_calls(void) { }
 
 extern void apic_ack_irq(struct irq_data *data);
 
+#define APIC_VECTOR_TO_BIT_NUMBER(v) ((unsigned int)(v) % 32)
+#define APIC_VECTOR_TO_REG_OFFSET(v) ((unsigned int)(v) / 32 * 0x10)
+
 static inline bool lapic_vector_set_in_irr(unsigned int vector)
 {
-	u32 irr = apic_read(APIC_IRR + (vector / 32 * 0x10));
+	u32 irr = apic_read(APIC_IRR + APIC_VECTOR_TO_REG_OFFSET(vector));
 
-	return !!(irr & (1U << (vector % 32)));
+	return !!(irr & (1U << APIC_VECTOR_TO_BIT_NUMBER(vector)));
 }
 
 static inline bool is_vector_pending(unsigned int vector)
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index 1638a3da383a..56369d331bfc 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -145,8 +145,8 @@ void kvm_lapic_exit(void);
 
 u64 kvm_lapic_readable_reg_mask(struct kvm_lapic *apic);
 
-#define VEC_POS(v) ((v) & (32 - 1))
-#define REG_POS(v) (((v) >> 5) << 4)
+#define VEC_POS(v) APIC_VECTOR_TO_BIT_NUMBER(v)
+#define REG_POS(v) APIC_VECTOR_TO_REG_OFFSET(v)
 
 static inline void kvm_lapic_clear_vector(int vec, void *bitmap)
 {
-- 
2.34.1

[RFC PATCH v8 04/35] KVM: x86: Rename VEC_POS/REG_POS macro usages

[Neeraj Upadhyay]

In preparation for moving most of the KVM's lapic helpers which
use VEC_POS/REG_POS macros to common APIC header for use in Secure
AVIC APIC driver, rename all VEC_POS/REG_POS macro usages to
APIC_VECTOR_TO_BIT_NUMBER/APIC_VECTOR_TO_REG_OFFSET and remove
VEC_POS/REG_POS.

While at it, clean up line wrap in find_highest_vector().

No functional change intended.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - Commit log update.

 arch/x86/kvm/lapic.c | 15 +++++++--------
 arch/x86/kvm/lapic.h |  7 ++-----
 2 files changed, 9 insertions(+), 13 deletions(-)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 533daf6dd1b1..1dbc1643c675 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -114,7 +114,7 @@ static __always_inline void kvm_lapic_set_reg64(struct kvm_lapic *apic,
 
 static inline int apic_test_vector(int vec, void *bitmap)
 {
-	return test_bit(VEC_POS(vec), bitmap + REG_POS(vec));
+	return test_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
 }
 
 bool kvm_apic_pending_eoi(struct kvm_vcpu *vcpu, int vector)
@@ -621,9 +621,8 @@ static int find_highest_vector(void *bitmap)
 	int vec;
 	u32 *reg;
 
-	for (vec = MAX_APIC_VECTOR - APIC_VECTORS_PER_REG;
-	     vec >= 0; vec -= APIC_VECTORS_PER_REG) {
-		reg = bitmap + REG_POS(vec);
+	for (vec = MAX_APIC_VECTOR - APIC_VECTORS_PER_REG; vec >= 0; vec -= APIC_VECTORS_PER_REG) {
+		reg = bitmap + APIC_VECTOR_TO_REG_OFFSET(vec);
 		if (*reg)
 			return __fls(*reg) + vec;
 	}
@@ -638,7 +637,7 @@ static u8 count_vectors(void *bitmap)
 	u8 count = 0;
 
 	for (vec = 0; vec < MAX_APIC_VECTOR; vec += APIC_VECTORS_PER_REG) {
-		reg = bitmap + REG_POS(vec);
+		reg = bitmap + APIC_VECTOR_TO_REG_OFFSET(vec);
 		count += hweight32(*reg);
 	}
 
@@ -736,12 +735,12 @@ EXPORT_SYMBOL_GPL(kvm_apic_clear_irr);
 
 static void *apic_vector_to_isr(int vec, struct kvm_lapic *apic)
 {
-	return apic->regs + APIC_ISR + REG_POS(vec);
+	return apic->regs + APIC_ISR + APIC_VECTOR_TO_REG_OFFSET(vec);
 }
 
 static inline void apic_set_isr(int vec, struct kvm_lapic *apic)
 {
-	if (__test_and_set_bit(VEC_POS(vec), apic_vector_to_isr(vec, apic)))
+	if (__test_and_set_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), apic_vector_to_isr(vec, apic)))
 		return;
 
 	/*
@@ -784,7 +783,7 @@ static inline int apic_find_highest_isr(struct kvm_lapic *apic)
 
 static inline void apic_clear_isr(int vec, struct kvm_lapic *apic)
 {
-	if (!__test_and_clear_bit(VEC_POS(vec), apic_vector_to_isr(vec, apic)))
+	if (!__test_and_clear_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), apic_vector_to_isr(vec, apic)))
 		return;
 
 	/*
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index 56369d331bfc..eb9bda52948c 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -145,17 +145,14 @@ void kvm_lapic_exit(void);
 
 u64 kvm_lapic_readable_reg_mask(struct kvm_lapic *apic);
 
-#define VEC_POS(v) APIC_VECTOR_TO_BIT_NUMBER(v)
-#define REG_POS(v) APIC_VECTOR_TO_REG_OFFSET(v)
-
 static inline void kvm_lapic_clear_vector(int vec, void *bitmap)
 {
-	clear_bit(VEC_POS(vec), bitmap + REG_POS(vec));
+	clear_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
 }
 
 static inline void kvm_lapic_set_vector(int vec, void *bitmap)
 {
-	set_bit(VEC_POS(vec), bitmap + REG_POS(vec));
+	set_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
 }
 
 static inline void kvm_lapic_set_irr(int vec, struct kvm_lapic *apic)
-- 
2.34.1

[RFC PATCH v8 05/35] KVM: x86: Change lapic regs base address to void pointer

[Neeraj Upadhyay]

Change APIC base address from "char *" to "void *" in KVM
lapic's set/get helper functions. Pointer arithmetic for "void *"
and "char *" operate identically. With "void *" there is less
of a chance of doing the wrong thing, e.g. neglecting to cast and
reading a byte instead of the desired APIC register size.

No functional change intended.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - Commit log update.

 arch/x86/kvm/lapic.c | 6 +++---
 arch/x86/kvm/lapic.h | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 1dbc1643c675..3be5f0db892c 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -79,7 +79,7 @@ module_param(lapic_timer_advance, bool, 0444);
 static int kvm_lapic_msr_read(struct kvm_lapic *apic, u32 reg, u64 *data);
 static int kvm_lapic_msr_write(struct kvm_lapic *apic, u32 reg, u64 data);
 
-static inline void __kvm_lapic_set_reg(char *regs, int reg_off, u32 val)
+static inline void __kvm_lapic_set_reg(void *regs, int reg_off, u32 val)
 {
 	*((u32 *) (regs + reg_off)) = val;
 }
@@ -89,7 +89,7 @@ static inline void kvm_lapic_set_reg(struct kvm_lapic *apic, int reg_off, u32 va
 	__kvm_lapic_set_reg(apic->regs, reg_off, val);
 }
 
-static __always_inline u64 __kvm_lapic_get_reg64(char *regs, int reg)
+static __always_inline u64 __kvm_lapic_get_reg64(void *regs, int reg)
 {
 	BUILD_BUG_ON(reg != APIC_ICR);
 	return *((u64 *) (regs + reg));
@@ -100,7 +100,7 @@ static __always_inline u64 kvm_lapic_get_reg64(struct kvm_lapic *apic, int reg)
 	return __kvm_lapic_get_reg64(apic->regs, reg);
 }
 
-static __always_inline void __kvm_lapic_set_reg64(char *regs, int reg, u64 val)
+static __always_inline void __kvm_lapic_set_reg64(void *regs, int reg, u64 val)
 {
 	BUILD_BUG_ON(reg != APIC_ICR);
 	*((u64 *) (regs + reg)) = val;
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index eb9bda52948c..7ce89bf0b974 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -165,7 +165,7 @@ static inline void kvm_lapic_set_irr(int vec, struct kvm_lapic *apic)
 	apic->irr_pending = true;
 }
 
-static inline u32 __kvm_lapic_get_reg(char *regs, int reg_off)
+static inline u32 __kvm_lapic_get_reg(void *regs, int reg_off)
 {
 	return *((u32 *) (regs + reg_off));
 }
-- 
2.34.1

[RFC PATCH v8 06/35] KVM: x86: Rename find_highest_vector()

[Neeraj Upadhyay]

In preparation for moving kvm-internal find_highest_vector() to
apic.h for use in Secure AVIC APIC driver, rename find_highest_vector()
to apic_find_highest_vector() as part of the APIC API.

No functional change intended.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - Commit log update.

 arch/x86/kvm/lapic.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 3be5f0db892c..d71878a3748c 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -616,7 +616,7 @@ static const unsigned int apic_lvt_mask[KVM_APIC_MAX_NR_LVT_ENTRIES] = {
 	[LVT_CMCI] = LVT_MASK | APIC_MODE_MASK
 };
 
-static int find_highest_vector(void *bitmap)
+static int apic_find_highest_vector(void *bitmap)
 {
 	int vec;
 	u32 *reg;
@@ -695,7 +695,7 @@ EXPORT_SYMBOL_GPL(kvm_apic_update_irr);
 
 static inline int apic_search_irr(struct kvm_lapic *apic)
 {
-	return find_highest_vector(apic->regs + APIC_IRR);
+	return apic_find_highest_vector(apic->regs + APIC_IRR);
 }
 
 static inline int apic_find_highest_irr(struct kvm_lapic *apic)
@@ -775,7 +775,7 @@ static inline int apic_find_highest_isr(struct kvm_lapic *apic)
 	if (likely(apic->highest_isr_cache != -1))
 		return apic->highest_isr_cache;
 
-	result = find_highest_vector(apic->regs + APIC_ISR);
+	result = apic_find_highest_vector(apic->regs + APIC_ISR);
 	ASSERT(result == -1 || result >= 16);
 
 	return result;
-- 
2.34.1

[RFC PATCH v8 07/35] KVM: x86: Rename lapic get/set_reg() helpers

[Neeraj Upadhyay]

In preparation for moving kvm-internal __kvm_lapic_set_reg(),
__kvm_lapic_get_reg() to apic.h for use in Secure AVIC APIC driver,
rename them as part of the APIC API.

No functional change intended.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - Commit log updates.

 arch/x86/kvm/lapic.c | 13 ++++++-------
 arch/x86/kvm/lapic.h |  4 ++--
 2 files changed, 8 insertions(+), 9 deletions(-)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index d71878a3748c..da48e5bb1818 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -79,14 +79,14 @@ module_param(lapic_timer_advance, bool, 0444);
 static int kvm_lapic_msr_read(struct kvm_lapic *apic, u32 reg, u64 *data);
 static int kvm_lapic_msr_write(struct kvm_lapic *apic, u32 reg, u64 data);
 
-static inline void __kvm_lapic_set_reg(void *regs, int reg_off, u32 val)
+static inline void apic_set_reg(void *regs, int reg_off, u32 val)
 {
 	*((u32 *) (regs + reg_off)) = val;
 }
 
 static inline void kvm_lapic_set_reg(struct kvm_lapic *apic, int reg_off, u32 val)
 {
-	__kvm_lapic_set_reg(apic->regs, reg_off, val);
+	apic_set_reg(apic->regs, reg_off, val);
 }
 
 static __always_inline u64 __kvm_lapic_get_reg64(void *regs, int reg)
@@ -3078,12 +3078,12 @@ static int kvm_apic_state_fixup(struct kvm_vcpu *vcpu,
 
 		if (!kvm_x86_ops.x2apic_icr_is_split) {
 			if (set) {
-				icr = __kvm_lapic_get_reg(s->regs, APIC_ICR) |
-				      (u64)__kvm_lapic_get_reg(s->regs, APIC_ICR2) << 32;
+				icr = apic_get_reg(s->regs, APIC_ICR) |
+				      (u64)apic_get_reg(s->regs, APIC_ICR2) << 32;
 				__kvm_lapic_set_reg64(s->regs, APIC_ICR, icr);
 			} else {
 				icr = __kvm_lapic_get_reg64(s->regs, APIC_ICR);
-				__kvm_lapic_set_reg(s->regs, APIC_ICR2, icr >> 32);
+				apic_set_reg(s->regs, APIC_ICR2, icr >> 32);
 			}
 		}
 	}
@@ -3099,8 +3099,7 @@ int kvm_apic_get_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s)
 	 * Get calculated timer current count for remaining timer period (if
 	 * any) and store it in the returned register set.
 	 */
-	__kvm_lapic_set_reg(s->regs, APIC_TMCCT,
-			    __apic_read(vcpu->arch.apic, APIC_TMCCT));
+	apic_set_reg(s->regs, APIC_TMCCT, __apic_read(vcpu->arch.apic, APIC_TMCCT));
 
 	return kvm_apic_state_fixup(vcpu, s, false);
 }
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index 7ce89bf0b974..a49e4c21db35 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -165,14 +165,14 @@ static inline void kvm_lapic_set_irr(int vec, struct kvm_lapic *apic)
 	apic->irr_pending = true;
 }
 
-static inline u32 __kvm_lapic_get_reg(void *regs, int reg_off)
+static inline u32 apic_get_reg(void *regs, int reg_off)
 {
 	return *((u32 *) (regs + reg_off));
 }
 
 static inline u32 kvm_lapic_get_reg(struct kvm_lapic *apic, int reg_off)
 {
-	return __kvm_lapic_get_reg(apic->regs, reg_off);
+	return apic_get_reg(apic->regs, reg_off);
 }
 
 DECLARE_STATIC_KEY_FALSE(kvm_has_noapic_vcpu);
-- 
2.34.1

[RFC PATCH v8 08/35] KVM: x86: Rename lapic get/set_reg64() helpers

[Neeraj Upadhyay]

In preparation for moving kvm-internal __kvm_lapic_set_reg64(),
__kvm_lapic_get_reg64() to apic.h for use in Secure AVIC APIC driver,
rename them as part of the APIC API.

No functional change intended.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - Commit log update.

 arch/x86/kvm/lapic.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index da48e5bb1818..06d33919c47d 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -89,7 +89,7 @@ static inline void kvm_lapic_set_reg(struct kvm_lapic *apic, int reg_off, u32 va
 	apic_set_reg(apic->regs, reg_off, val);
 }
 
-static __always_inline u64 __kvm_lapic_get_reg64(void *regs, int reg)
+static __always_inline u64 apic_get_reg64(void *regs, int reg)
 {
 	BUILD_BUG_ON(reg != APIC_ICR);
 	return *((u64 *) (regs + reg));
@@ -97,10 +97,10 @@ static __always_inline u64 __kvm_lapic_get_reg64(void *regs, int reg)
 
 static __always_inline u64 kvm_lapic_get_reg64(struct kvm_lapic *apic, int reg)
 {
-	return __kvm_lapic_get_reg64(apic->regs, reg);
+	return apic_get_reg64(apic->regs, reg);
 }
 
-static __always_inline void __kvm_lapic_set_reg64(void *regs, int reg, u64 val)
+static __always_inline void apic_set_reg64(void *regs, int reg, u64 val)
 {
 	BUILD_BUG_ON(reg != APIC_ICR);
 	*((u64 *) (regs + reg)) = val;
@@ -109,7 +109,7 @@ static __always_inline void __kvm_lapic_set_reg64(void *regs, int reg, u64 val)
 static __always_inline void kvm_lapic_set_reg64(struct kvm_lapic *apic,
 						int reg, u64 val)
 {
-	__kvm_lapic_set_reg64(apic->regs, reg, val);
+	apic_set_reg64(apic->regs, reg, val);
 }
 
 static inline int apic_test_vector(int vec, void *bitmap)
@@ -3080,9 +3080,9 @@ static int kvm_apic_state_fixup(struct kvm_vcpu *vcpu,
 			if (set) {
 				icr = apic_get_reg(s->regs, APIC_ICR) |
 				      (u64)apic_get_reg(s->regs, APIC_ICR2) << 32;
-				__kvm_lapic_set_reg64(s->regs, APIC_ICR, icr);
+				apic_set_reg64(s->regs, APIC_ICR, icr);
 			} else {
-				icr = __kvm_lapic_get_reg64(s->regs, APIC_ICR);
+				icr = apic_get_reg64(s->regs, APIC_ICR);
 				apic_set_reg(s->regs, APIC_ICR2, icr >> 32);
 			}
 		}
-- 
2.34.1

[RFC PATCH v8 09/35] KVM: x86: Rename lapic set/clear vector helpers

[Neeraj Upadhyay]

In preparation for moving kvm-internal kvm_lapic_set_vector(),
kvm_lapic_clear_vector() to apic.h for use in Secure AVIC APIC driver,
rename them as part of the APIC API.

No functional change intended.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - Commit log updates.

 arch/x86/kvm/lapic.c | 10 ++++------
 arch/x86/kvm/lapic.h |  6 +++---
 2 files changed, 7 insertions(+), 9 deletions(-)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 06d33919c47d..069f3fe58def 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -718,10 +718,10 @@ static inline int apic_find_highest_irr(struct kvm_lapic *apic)
 static inline void apic_clear_irr(int vec, struct kvm_lapic *apic)
 {
 	if (unlikely(apic->apicv_active)) {
-		kvm_lapic_clear_vector(vec, apic->regs + APIC_IRR);
+		apic_clear_vector(vec, apic->regs + APIC_IRR);
 	} else {
 		apic->irr_pending = false;
-		kvm_lapic_clear_vector(vec, apic->regs + APIC_IRR);
+		apic_clear_vector(vec, apic->regs + APIC_IRR);
 		if (apic_search_irr(apic) != -1)
 			apic->irr_pending = true;
 	}
@@ -1326,11 +1326,9 @@ static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode,
 
 		if (apic_test_vector(vector, apic->regs + APIC_TMR) != !!trig_mode) {
 			if (trig_mode)
-				kvm_lapic_set_vector(vector,
-						     apic->regs + APIC_TMR);
+				apic_set_vector(vector, apic->regs + APIC_TMR);
 			else
-				kvm_lapic_clear_vector(vector,
-						       apic->regs + APIC_TMR);
+				apic_clear_vector(vector, apic->regs + APIC_TMR);
 		}
 
 		kvm_x86_call(deliver_interrupt)(apic, delivery_mode,
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index a49e4c21db35..c7babae8af83 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -145,19 +145,19 @@ void kvm_lapic_exit(void);
 
 u64 kvm_lapic_readable_reg_mask(struct kvm_lapic *apic);
 
-static inline void kvm_lapic_clear_vector(int vec, void *bitmap)
+static inline void apic_clear_vector(int vec, void *bitmap)
 {
 	clear_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
 }
 
-static inline void kvm_lapic_set_vector(int vec, void *bitmap)
+static inline void apic_set_vector(int vec, void *bitmap)
 {
 	set_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
 }
 
 static inline void kvm_lapic_set_irr(int vec, struct kvm_lapic *apic)
 {
-	kvm_lapic_set_vector(vec, apic->regs + APIC_IRR);
+	apic_set_vector(vec, apic->regs + APIC_IRR);
 	/*
 	 * irr_pending must be true if any interrupt is pending; set it after
 	 * APIC_IRR to avoid race with apic_clear_irr
-- 
2.34.1

[RFC PATCH v8 10/35] x86/apic: KVM: Move apic_find_highest_vector() to a common header

[Neeraj Upadhyay]

In preparation for using apic_find_highest_vector() in Secure AVIC
guest APIC driver, move it and associated macros to apic.h.

No functional change intended.

Acked-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - Squash marking apic_find_highest_vector() inline in this patch.
 - Applied Sean's Acked-by.

 arch/x86/include/asm/apic.h | 22 ++++++++++++++++++++++
 arch/x86/kvm/lapic.c        | 18 +-----------------
 2 files changed, 23 insertions(+), 17 deletions(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index c84d4e86fe4e..c7355bcbfd60 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -503,6 +503,28 @@ static inline bool is_vector_pending(unsigned int vector)
 	return lapic_vector_set_in_irr(vector) || pi_pending_this_cpu(vector);
 }
 
+#define MAX_APIC_VECTOR			256
+#define APIC_VECTORS_PER_REG		32
+
+/*
+ * Vector states are maintained by APIC in 32-bit registers that are
+ * 16 bytes aligned. The status of each vector is kept in a single
+ * bit.
+ */
+static inline int apic_find_highest_vector(void *bitmap)
+{
+	int vec;
+	u32 *reg;
+
+	for (vec = MAX_APIC_VECTOR - APIC_VECTORS_PER_REG; vec >= 0; vec -= APIC_VECTORS_PER_REG) {
+		reg = bitmap + APIC_VECTOR_TO_REG_OFFSET(vec);
+		if (*reg)
+			return __fls(*reg) + vec;
+	}
+
+	return -1;
+}
+
 /*
  * Warm reset vector position:
  */
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 069f3fe58def..018abf2ff890 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -27,6 +27,7 @@
 #include <linux/export.h>
 #include <linux/math64.h>
 #include <linux/slab.h>
+#include <asm/apic.h>
 #include <asm/processor.h>
 #include <asm/mce.h>
 #include <asm/msr.h>
@@ -55,9 +56,6 @@
 /* 14 is the version for Xeon and Pentium 8.4.8*/
 #define APIC_VERSION			0x14UL
 #define LAPIC_MMIO_LENGTH		(1 << 12)
-/* followed define is not in apicdef.h */
-#define MAX_APIC_VECTOR			256
-#define APIC_VECTORS_PER_REG		32
 
 /*
  * Enable local APIC timer advancement (tscdeadline mode only) with adaptive
@@ -616,20 +614,6 @@ static const unsigned int apic_lvt_mask[KVM_APIC_MAX_NR_LVT_ENTRIES] = {
 	[LVT_CMCI] = LVT_MASK | APIC_MODE_MASK
 };
 
-static int apic_find_highest_vector(void *bitmap)
-{
-	int vec;
-	u32 *reg;
-
-	for (vec = MAX_APIC_VECTOR - APIC_VECTORS_PER_REG; vec >= 0; vec -= APIC_VECTORS_PER_REG) {
-		reg = bitmap + APIC_VECTOR_TO_REG_OFFSET(vec);
-		if (*reg)
-			return __fls(*reg) + vec;
-	}
-
-	return -1;
-}
-
 static u8 count_vectors(void *bitmap)
 {
 	int vec;
-- 
2.34.1

[RFC PATCH v8 11/35] x86/apic: KVM: Move lapic get/set helpers to common code

[Neeraj Upadhyay]

Move the apic_get_reg(), apic_set_reg(), apic_get_reg64() and
apic_set_reg64() helper functions to apic.h in order to reuse them in the
Secure AVIC guest APIC driver in later patches to read/write registers
from/to the APIC backing page.

No functional change intended.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - Squash 64-bit accessors movement to this patch.

 arch/x86/include/asm/apic.h | 22 ++++++++++++++++++++++
 arch/x86/kvm/lapic.c        | 17 -----------------
 arch/x86/kvm/lapic.h        |  7 ++-----
 3 files changed, 24 insertions(+), 22 deletions(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index c7355bcbfd60..b8b5fe875bde 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -525,6 +525,28 @@ static inline int apic_find_highest_vector(void *bitmap)
 	return -1;
 }
 
+static inline u32 apic_get_reg(void *regs, int reg_off)
+{
+	return *((u32 *) (regs + reg_off));
+}
+
+static inline void apic_set_reg(void *regs, int reg_off, u32 val)
+{
+	*((u32 *) (regs + reg_off)) = val;
+}
+
+static __always_inline u64 apic_get_reg64(void *regs, int reg)
+{
+	BUILD_BUG_ON(reg != APIC_ICR);
+	return *((u64 *) (regs + reg));
+}
+
+static __always_inline void apic_set_reg64(void *regs, int reg, u64 val)
+{
+	BUILD_BUG_ON(reg != APIC_ICR);
+	*((u64 *) (regs + reg)) = val;
+}
+
 /*
  * Warm reset vector position:
  */
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index 018abf2ff890..c7c609171a40 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -77,33 +77,16 @@ module_param(lapic_timer_advance, bool, 0444);
 static int kvm_lapic_msr_read(struct kvm_lapic *apic, u32 reg, u64 *data);
 static int kvm_lapic_msr_write(struct kvm_lapic *apic, u32 reg, u64 data);
 
-static inline void apic_set_reg(void *regs, int reg_off, u32 val)
-{
-	*((u32 *) (regs + reg_off)) = val;
-}
-
 static inline void kvm_lapic_set_reg(struct kvm_lapic *apic, int reg_off, u32 val)
 {
 	apic_set_reg(apic->regs, reg_off, val);
 }
 
-static __always_inline u64 apic_get_reg64(void *regs, int reg)
-{
-	BUILD_BUG_ON(reg != APIC_ICR);
-	return *((u64 *) (regs + reg));
-}
-
 static __always_inline u64 kvm_lapic_get_reg64(struct kvm_lapic *apic, int reg)
 {
 	return apic_get_reg64(apic->regs, reg);
 }
 
-static __always_inline void apic_set_reg64(void *regs, int reg, u64 val)
-{
-	BUILD_BUG_ON(reg != APIC_ICR);
-	*((u64 *) (regs + reg)) = val;
-}
-
 static __always_inline void kvm_lapic_set_reg64(struct kvm_lapic *apic,
 						int reg, u64 val)
 {
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index c7babae8af83..174df6996404 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -4,6 +4,8 @@
 
 #include <kvm/iodev.h>
 
+#include <asm/apic.h>
+
 #include <linux/kvm_host.h>
 
 #include "hyperv.h"
@@ -165,11 +167,6 @@ static inline void kvm_lapic_set_irr(int vec, struct kvm_lapic *apic)
 	apic->irr_pending = true;
 }
 
-static inline u32 apic_get_reg(void *regs, int reg_off)
-{
-	return *((u32 *) (regs + reg_off));
-}
-
 static inline u32 kvm_lapic_get_reg(struct kvm_lapic *apic, int reg_off)
 {
 	return apic_get_reg(apic->regs, reg_off);
-- 
2.34.1

[RFC PATCH v8 12/35] x86/apic: KVM: Move lapic set/clear_vector() helpers to common code

[Neeraj Upadhyay]

Move apic_clear_vector() and apic_set_vector() helper functions to
apic.h in order to reuse them in the Secure AVIC guest APIC driver
in later patches to atomically set/clear vectors in the APIC backing
page.

No functional change intended.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - Commit log update.

 arch/x86/include/asm/apic.h | 10 ++++++++++
 arch/x86/kvm/lapic.h        | 10 ----------
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index b8b5fe875bde..c6d1c51f71ec 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -547,6 +547,16 @@ static __always_inline void apic_set_reg64(void *regs, int reg, u64 val)
 	*((u64 *) (regs + reg)) = val;
 }
 
+static inline void apic_clear_vector(int vec, void *bitmap)
+{
+	clear_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
+}
+
+static inline void apic_set_vector(int vec, void *bitmap)
+{
+	set_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
+}
+
 /*
  * Warm reset vector position:
  */
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index 174df6996404..31284ec61a6a 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -147,16 +147,6 @@ void kvm_lapic_exit(void);
 
 u64 kvm_lapic_readable_reg_mask(struct kvm_lapic *apic);
 
-static inline void apic_clear_vector(int vec, void *bitmap)
-{
-	clear_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
-}
-
-static inline void apic_set_vector(int vec, void *bitmap)
-{
-	set_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
-}
-
 static inline void kvm_lapic_set_irr(int vec, struct kvm_lapic *apic)
 {
 	apic_set_vector(vec, apic->regs + APIC_IRR);
-- 
2.34.1

[RFC PATCH v8 13/35] x86/apic: KVM: Move apic_test)vector() to common code

[Neeraj Upadhyay]

Move apic_test_vector() to apic.h in order to reuse it in the Secure AVIC
guest APIC driver in later patches to test vector state in the APIC
backing page.

No functional change intended.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
 arch/x86/include/asm/apic.h | 5 +++++
 arch/x86/kvm/lapic.c        | 5 -----
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index c6d1c51f71ec..34e9b43d8940 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -557,6 +557,11 @@ static inline void apic_set_vector(int vec, void *bitmap)
 	set_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
 }
 
+static inline int apic_test_vector(int vec, void *bitmap)
+{
+	return test_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
+}
+
 /*
  * Warm reset vector position:
  */
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index c7c609171a40..bcb7bf9c0fb5 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -93,11 +93,6 @@ static __always_inline void kvm_lapic_set_reg64(struct kvm_lapic *apic,
 	apic_set_reg64(apic->regs, reg, val);
 }
 
-static inline int apic_test_vector(int vec, void *bitmap)
-{
-	return test_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
-}
-
 bool kvm_apic_pending_eoi(struct kvm_vcpu *vcpu, int vector)
 {
 	struct kvm_lapic *apic = vcpu->arch.apic;
-- 
2.34.1

[RFC PATCH v8 14/35] x86/apic: Rename 'reg_off' to 'reg'

[Neeraj Upadhyay]

Rename the 'reg_off' parameter of apic_{set|get}_reg() to 'reg' to
match other usages in apic.h.

No functional change intended.

Reviewed-by: Tianyu Lan <tiala@microsoft.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/include/asm/apic.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index 34e9b43d8940..07ba4935e873 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -525,14 +525,14 @@ static inline int apic_find_highest_vector(void *bitmap)
 	return -1;
 }
 
-static inline u32 apic_get_reg(void *regs, int reg_off)
+static inline u32 apic_get_reg(void *regs, int reg)
 {
-	return *((u32 *) (regs + reg_off));
+	return *((u32 *) (regs + reg));
 }
 
-static inline void apic_set_reg(void *regs, int reg_off, u32 val)
+static inline void apic_set_reg(void *regs, int reg, u32 val)
 {
-	*((u32 *) (regs + reg_off)) = val;
+	*((u32 *) (regs + reg)) = val;
 }
 
 static __always_inline u64 apic_get_reg64(void *regs, int reg)
-- 
2.34.1

[RFC PATCH v8 15/35] x86/apic: Unionize apic regs for 32bit/64bit access w/o type casting

[Neeraj Upadhyay]

Define apic_page construct to unionize APIC register 32bit/64bit
accesses and use it in apic_{get|set}*() to avoid using type
casting.

As Secure AVIC APIC driver requires accessing APIC page at byte
offsets (to get an APIC register's bitmap start address), support
byte access granularity in apic_page (in addition to 32-bit and
64-bit accesses).

One caveat of this change is that the generated code is slighly
larger. Below is the code generation for apic_get_reg() using
gcc-14.2:

- Without change:

apic_get_reg:

89 f6       mov    %esi,%esi
8b 04 37    mov    (%rdi,%rsi,1),%eax
c3          ret

- With change:

apic_get_reg:

c1 ee 02    shr    $0x2,%esi
8b 04 b7    mov    (%rdi,%rsi,4),%eax
c3          ret

lapic.o text size change is shown below:

Obj        Old-size      New-size

lapic.o    28800         28832

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - Commit log update.

 arch/x86/include/asm/apic.h | 25 +++++++++++++++++++++----
 1 file changed, 21 insertions(+), 4 deletions(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index 07ba4935e873..b7cbe9ba363e 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -525,26 +525,43 @@ static inline int apic_find_highest_vector(void *bitmap)
 	return -1;
 }
 
+struct apic_page {
+	union {
+		u64     regs64[PAGE_SIZE / 8];
+		u32     regs[PAGE_SIZE / 4];
+		u8      bytes[PAGE_SIZE];
+	};
+} __aligned(PAGE_SIZE);
+
 static inline u32 apic_get_reg(void *regs, int reg)
 {
-	return *((u32 *) (regs + reg));
+	struct apic_page *ap = regs;
+
+	return ap->regs[reg / 4];
 }
 
 static inline void apic_set_reg(void *regs, int reg, u32 val)
 {
-	*((u32 *) (regs + reg)) = val;
+	struct apic_page *ap = regs;
+
+	ap->regs[reg / 4] = val;
 }
 
 static __always_inline u64 apic_get_reg64(void *regs, int reg)
 {
+	struct apic_page *ap = regs;
+
 	BUILD_BUG_ON(reg != APIC_ICR);
-	return *((u64 *) (regs + reg));
+
+	return ap->regs64[reg / 8];
 }
 
 static __always_inline void apic_set_reg64(void *regs, int reg, u64 val)
 {
+	struct apic_page *ap = regs;
+
 	BUILD_BUG_ON(reg != APIC_ICR);
-	*((u64 *) (regs + reg)) = val;
+	ap->regs64[reg / 8] = val;
 }
 
 static inline void apic_clear_vector(int vec, void *bitmap)
-- 
2.34.1

[RFC PATCH v8 16/35] x86/apic: Simplify bitwise operations on APIC bitmap

[Neeraj Upadhyay]

Use 'regs' as a contiguous linear bitmap for bitwise operations in
apic_{set|clear|test}_vector(). This makes the code simpler by eliminating
the need to determine the offset of the 32-bit register and the vector bit
location within that register prior to performing bitwise operations.

This change results in slight increase in generated code size for
gcc-14.2.

- Without change

apic_set_vector:

89 f8             mov    %edi,%eax
83 e7 1f          and    $0x1f,%edi
c1 e8 05          shr    $0x5,%eax
c1 e0 04          shl    $0x4,%eax
48 01 c6          add    %rax,%rsi
f0 48 0f ab 3e    lock bts %rdi,(%rsi)
c3                ret

- With change

apic_set_vector:

89 f8             mov    %edi,%eax
c1 e8 05          shr    $0x5,%eax
8d 04 40          lea    (%rax,%rax,2),%eax
c1 e0 05          shl    $0x5,%eax
01 f8             add    %edi,%eax
89 c0             mov    %eax,%eax
f0 48 0f ab 3e    lock bts %rax,(%rsi)
c3                ret

But, lapic.o text size (bytes) decreases with this change:

Obj        Old-size      New-size

lapic.o    28832         28768

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - Commit log update.

 arch/x86/include/asm/apic.h | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index b7cbe9ba363e..f91d23757375 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -564,19 +564,28 @@ static __always_inline void apic_set_reg64(void *regs, int reg, u64 val)
 	ap->regs64[reg / 8] = val;
 }
 
+static inline unsigned int get_vec_bit(unsigned int vec)
+{
+	/*
+	 * The registers are 32-bit wide and 16-byte aligned.
+	 * Compensate for the resulting bit number spacing.
+	 */
+	return vec + 96 * (vec / 32);
+}
+
 static inline void apic_clear_vector(int vec, void *bitmap)
 {
-	clear_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
+	clear_bit(get_vec_bit(vec), bitmap);
 }
 
 static inline void apic_set_vector(int vec, void *bitmap)
 {
-	set_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
+	set_bit(get_vec_bit(vec), bitmap);
 }
 
 static inline int apic_test_vector(int vec, void *bitmap)
 {
-	return test_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
+	return test_bit(get_vec_bit(vec), bitmap);
 }
 
 /*
-- 
2.34.1

[RFC PATCH v8 17/35] x86/apic: Move apic_update_irq_cfg() calls to apic_update_vector()

[Neeraj Upadhyay]

All callers of apic_update_vector() also call apic_update_irq_cfg()
after it. So, simplify the code by moving all such apic_update_irq_cfg()
calls to apic_update_vector().

No functional change intended.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/kernel/apic/vector.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
index 93069b13d3af..a947b46a8b64 100644
--- a/arch/x86/kernel/apic/vector.c
+++ b/arch/x86/kernel/apic/vector.c
@@ -183,6 +183,7 @@ static void apic_update_vector(struct irq_data *irqd, unsigned int newvec,
 	apicd->cpu = newcpu;
 	BUG_ON(!IS_ERR_OR_NULL(per_cpu(vector_irq, newcpu)[newvec]));
 	per_cpu(vector_irq, newcpu)[newvec] = desc;
+	apic_update_irq_cfg(irqd, newvec, newcpu);
 }
 
 static void vector_assign_managed_shutdown(struct irq_data *irqd)
@@ -261,7 +262,6 @@ assign_vector_locked(struct irq_data *irqd, const struct cpumask *dest)
 	if (vector < 0)
 		return vector;
 	apic_update_vector(irqd, vector, cpu);
-	apic_update_irq_cfg(irqd, vector, cpu);
 
 	return 0;
 }
@@ -338,7 +338,7 @@ assign_managed_vector(struct irq_data *irqd, const struct cpumask *dest)
 	if (vector < 0)
 		return vector;
 	apic_update_vector(irqd, vector, cpu);
-	apic_update_irq_cfg(irqd, vector, cpu);
+
 	return 0;
 }
 
-- 
2.34.1

[RFC PATCH v8 18/35] x86/apic: Add new driver for Secure AVIC

[Neeraj Upadhyay]

The Secure AVIC feature provides SEV-SNP guests hardware acceleration
for performance sensitive APIC accesses while securely managing the
guest-owned APIC state through the use of a private APIC backing page.
This helps prevent hypervisor from generating unexpected interrupts for
a vCPU or otherwise violate architectural assumptions around APIC
behavior.

Add a new x2APIC driver that will serve as the base of the Secure AVIC
support. It is initially the same as the x2APIC phys driver (without
IPI callbacks), but will be modified as features of Secure AVIC are
implemented.

As the new driver does not implement Secure AVIC features yet, if the
hypervisor sets the Secure AVIC bit in SEV_STATUS, maintain the existing
behavior to enforce the guest termination.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Reviewed-by: Tianyu Lan <tiala@microsoft.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/Kconfig                    | 13 ++++++
 arch/x86/boot/compressed/sev.c      |  1 +
 arch/x86/coco/core.c                |  3 ++
 arch/x86/coco/sev/core.c            |  1 +
 arch/x86/include/asm/msr-index.h    |  4 +-
 arch/x86/kernel/apic/Makefile       |  1 +
 arch/x86/kernel/apic/x2apic_savic.c | 63 +++++++++++++++++++++++++++++
 include/linux/cc_platform.h         |  8 ++++
 8 files changed, 93 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kernel/apic/x2apic_savic.c

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 5b3362af7d65..368292309568 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -486,6 +486,19 @@ config X86_X2APIC
 
 	  If in doubt, say Y.
 
+config AMD_SECURE_AVIC
+	bool "AMD Secure AVIC"
+	depends on AMD_MEM_ENCRYPT && X86_X2APIC
+	help
+	  Enable this to get AMD Secure AVIC support on guests that have this feature.
+
+	  AMD Secure AVIC provides hardware acceleration for performance sensitive
+	  APIC accesses and support for managing guest owned APIC state for SEV-SNP
+	  guests. Secure AVIC does not support xapic mode. It has functional
+	  dependency on x2apic being enabled in the guest.
+
+	  If you don't know what to do here, say N.
+
 config X86_POSTED_MSI
 	bool "Enable MSI and MSI-x delivery by posted interrupts"
 	depends on X86_64 && IRQ_REMAP
diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c
index fd1b67dfea22..74e083feb2d9 100644
--- a/arch/x86/boot/compressed/sev.c
+++ b/arch/x86/boot/compressed/sev.c
@@ -235,6 +235,7 @@ bool sev_es_check_ghcb_fault(unsigned long address)
 				 MSR_AMD64_SNP_VMSA_REG_PROT |		\
 				 MSR_AMD64_SNP_RESERVED_BIT13 |		\
 				 MSR_AMD64_SNP_RESERVED_BIT15 |		\
+				 MSR_AMD64_SNP_SECURE_AVIC |		\
 				 MSR_AMD64_SNP_RESERVED_MASK)
 
 /*
diff --git a/arch/x86/coco/core.c b/arch/x86/coco/core.c
index d4610af68114..989ca9f72ba3 100644
--- a/arch/x86/coco/core.c
+++ b/arch/x86/coco/core.c
@@ -104,6 +104,9 @@ static bool noinstr amd_cc_platform_has(enum cc_attr attr)
 	case CC_ATTR_HOST_SEV_SNP:
 		return cc_flags.host_sev_snp;
 
+	case CC_ATTR_SNP_SECURE_AVIC:
+		return sev_status & MSR_AMD64_SNP_SECURE_AVIC;
+
 	default:
 		return false;
 	}
diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index fc59ce78c477..a19691436ea6 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -79,6 +79,7 @@ static const char * const sev_status_feat_names[] = {
 	[MSR_AMD64_SNP_IBS_VIRT_BIT]		= "IBSVirt",
 	[MSR_AMD64_SNP_VMSA_REG_PROT_BIT]	= "VMSARegProt",
 	[MSR_AMD64_SNP_SMT_PROT_BIT]		= "SMTProt",
+	[MSR_AMD64_SNP_SECURE_AVIC_BIT]		= "SecureAVIC",
 };
 
 /*
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 7490bb5c0776..045c0d7e160b 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -698,7 +698,9 @@
 #define MSR_AMD64_SNP_VMSA_REG_PROT	BIT_ULL(MSR_AMD64_SNP_VMSA_REG_PROT_BIT)
 #define MSR_AMD64_SNP_SMT_PROT_BIT	17
 #define MSR_AMD64_SNP_SMT_PROT		BIT_ULL(MSR_AMD64_SNP_SMT_PROT_BIT)
-#define MSR_AMD64_SNP_RESV_BIT		18
+#define MSR_AMD64_SNP_SECURE_AVIC_BIT	18
+#define MSR_AMD64_SNP_SECURE_AVIC	BIT_ULL(MSR_AMD64_SNP_SECURE_AVIC_BIT)
+#define MSR_AMD64_SNP_RESV_BIT		19
 #define MSR_AMD64_SNP_RESERVED_MASK	GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT)
 #define MSR_AMD64_RMP_BASE		0xc0010132
 #define MSR_AMD64_RMP_END		0xc0010133
diff --git a/arch/x86/kernel/apic/Makefile b/arch/x86/kernel/apic/Makefile
index 52d1808ee360..581db89477f9 100644
--- a/arch/x86/kernel/apic/Makefile
+++ b/arch/x86/kernel/apic/Makefile
@@ -18,6 +18,7 @@ ifeq ($(CONFIG_X86_64),y)
 # APIC probe will depend on the listing order here
 obj-$(CONFIG_X86_NUMACHIP)	+= apic_numachip.o
 obj-$(CONFIG_X86_UV)		+= x2apic_uv_x.o
+obj-$(CONFIG_AMD_SECURE_AVIC)	+= x2apic_savic.o
 obj-$(CONFIG_X86_X2APIC)	+= x2apic_phys.o
 obj-$(CONFIG_X86_X2APIC)	+= x2apic_cluster.o
 obj-y				+= apic_flat_64.o
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
new file mode 100644
index 000000000000..bea844f28192
--- /dev/null
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -0,0 +1,63 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * AMD Secure AVIC Support (SEV-SNP Guests)
+ *
+ * Copyright (C) 2024 Advanced Micro Devices, Inc.
+ *
+ * Author: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
+ */
+
+#include <linux/cc_platform.h>
+
+#include <asm/apic.h>
+#include <asm/sev.h>
+
+#include "local.h"
+
+static int savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
+{
+	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
+}
+
+static int savic_probe(void)
+{
+	if (!cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
+		return 0;
+
+	if (!x2apic_mode) {
+		pr_err("Secure AVIC enabled in non x2APIC mode\n");
+		snp_abort();
+		/* unreachable */
+	}
+
+	return 1;
+}
+
+static struct apic apic_x2apic_savic __ro_after_init = {
+
+	.name				= "secure avic x2apic",
+	.probe				= savic_probe,
+	.acpi_madt_oem_check		= savic_acpi_madt_oem_check,
+
+	.dest_mode_logical		= false,
+
+	.disable_esr			= 0,
+
+	.cpu_present_to_apicid		= default_cpu_present_to_apicid,
+
+	.max_apic_id			= UINT_MAX,
+	.x2apic_set_max_apicid		= true,
+	.get_apic_id			= x2apic_get_apic_id,
+
+	.calc_dest_apicid		= apic_default_calc_apicid,
+
+	.nmi_to_offline_cpu		= true,
+
+	.read				= native_apic_msr_read,
+	.write				= native_apic_msr_write,
+	.eoi				= native_apic_msr_eoi,
+	.icr_read			= native_x2apic_icr_read,
+	.icr_write			= native_x2apic_icr_write,
+};
+
+apic_driver(apic_x2apic_savic);
diff --git a/include/linux/cc_platform.h b/include/linux/cc_platform.h
index 0bf7d33a1048..7fcec025c5e0 100644
--- a/include/linux/cc_platform.h
+++ b/include/linux/cc_platform.h
@@ -96,6 +96,14 @@ enum cc_attr {
 	 * enabled to run SEV-SNP guests.
 	 */
 	CC_ATTR_HOST_SEV_SNP,
+
+	/**
+	 * @CC_ATTR_SNP_SECURE_AVIC: Secure AVIC mode is active.
+	 *
+	 * The host kernel is running with the necessary features enabled
+	 * to run SEV-SNP guests with full Secure AVIC capabilities.
+	 */
+	CC_ATTR_SNP_SECURE_AVIC,
 };
 
 #ifdef CONFIG_ARCH_HAS_CC_PLATFORM
-- 
2.34.1

[RFC PATCH v8 19/35] x86/apic: Initialize Secure AVIC APIC backing page

[Neeraj Upadhyay]

With Secure AVIC, the APIC backing page is owned and managed by guest.
Allocate and initialize APIC backing page for all guest CPUs.

The NPT entry for a vCPU's APIC backing page must always be present
when the vCPU is running, in order for Secure AVIC to function. A
VMEXIT_BUSY is returned on VMRUN and the vCPU cannot be resumed if
the NPT entry for the APIC backing page is not present. To handle this,
notify GPA of the vCPU's APIC backing page to the hypervisor by using the
SVM_VMGEXIT_SECURE_AVIC GHCB protocol event. Before executing VMRUN,
the hypervisor makes use of this information to make sure the APIC backing
page is mapped in NPT.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/coco/sev/core.c            | 22 ++++++++++++++++++++
 arch/x86/include/asm/apic.h         |  1 +
 arch/x86/include/asm/sev.h          |  2 ++
 arch/x86/include/uapi/asm/svm.h     |  4 ++++
 arch/x86/kernel/apic/apic.c         |  3 +++
 arch/x86/kernel/apic/x2apic_savic.c | 32 +++++++++++++++++++++++++++++
 6 files changed, 64 insertions(+)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index a19691436ea6..0c59ea82fa99 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -1085,6 +1085,28 @@ int __init sev_es_efi_map_ghcbs_cas(pgd_t *pgd)
 	return 0;
 }
 
+enum es_result savic_register_gpa(u64 gpa)
+{
+	struct ghcb_state state;
+	struct es_em_ctxt ctxt;
+	enum es_result res;
+	struct ghcb *ghcb;
+
+	guard(irqsave)();
+
+	ghcb = __sev_get_ghcb(&state);
+	vc_ghcb_invalidate(ghcb);
+
+	ghcb_set_rax(ghcb, SVM_VMGEXIT_SAVIC_SELF_GPA);
+	ghcb_set_rbx(ghcb, gpa);
+	res = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_SAVIC,
+				  SVM_VMGEXIT_SAVIC_REGISTER_GPA, 0);
+
+	__sev_put_ghcb(&state);
+
+	return res;
+}
+
 static void snp_register_per_cpu_ghcb(void)
 {
 	struct sev_es_runtime_data *data;
diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index f91d23757375..184cae6e786b 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -305,6 +305,7 @@ struct apic {
 
 	/* Probe, setup and smpboot functions */
 	int	(*probe)(void);
+	void	(*setup)(void);
 	int	(*acpi_madt_oem_check)(char *oem_id, char *oem_table_id);
 
 	void	(*init_apic_ldr)(void);
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index 89075ff19afa..8e5083b46607 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -533,6 +533,7 @@ int snp_svsm_vtpm_send_command(u8 *buffer);
 
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
+enum es_result savic_register_gpa(u64 gpa);
 
 static __always_inline void vc_ghcb_invalidate(struct ghcb *ghcb)
 {
@@ -605,6 +606,7 @@ static inline int snp_send_guest_request(struct snp_msg_desc *mdesc,
 static inline int snp_svsm_vtpm_send_command(u8 *buffer) { return -ENODEV; }
 static inline void __init snp_secure_tsc_prepare(void) { }
 static inline void __init snp_secure_tsc_init(void) { }
+static inline enum es_result savic_register_gpa(u64 gpa) { return ES_UNSUPPORTED; }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
diff --git a/arch/x86/include/uapi/asm/svm.h b/arch/x86/include/uapi/asm/svm.h
index 9c640a521a67..650e3256ea7d 100644
--- a/arch/x86/include/uapi/asm/svm.h
+++ b/arch/x86/include/uapi/asm/svm.h
@@ -118,6 +118,10 @@
 #define SVM_VMGEXIT_AP_CREATE			1
 #define SVM_VMGEXIT_AP_DESTROY			2
 #define SVM_VMGEXIT_SNP_RUN_VMPL		0x80000018
+#define SVM_VMGEXIT_SAVIC			0x8000001a
+#define SVM_VMGEXIT_SAVIC_REGISTER_GPA		0
+#define SVM_VMGEXIT_SAVIC_UNREGISTER_GPA	1
+#define SVM_VMGEXIT_SAVIC_SELF_GPA		~0ULL
 #define SVM_VMGEXIT_HV_FEATURES			0x8000fffd
 #define SVM_VMGEXIT_TERM_REQUEST		0x8000fffe
 #define SVM_VMGEXIT_TERM_REASON(reason_set, reason_code)	\
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index d73ba5a7b623..36f1326fea2e 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -1503,6 +1503,9 @@ static void setup_local_APIC(void)
 		return;
 	}
 
+	if (apic->setup)
+		apic->setup();
+
 	/*
 	 * If this comes from kexec/kcrash the APIC might be enabled in
 	 * SPIV. Soft disable it before doing further initialization.
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index bea844f28192..a2747ab9200a 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -8,17 +8,44 @@
  */
 
 #include <linux/cc_platform.h>
+#include <linux/percpu-defs.h>
 
 #include <asm/apic.h>
 #include <asm/sev.h>
 
 #include "local.h"
 
+static struct apic_page __percpu *apic_page __ro_after_init;
+
 static int savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 {
 	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
 }
 
+static void savic_setup(void)
+{
+	void *backing_page;
+	enum es_result res;
+	unsigned long gpa;
+
+	backing_page = this_cpu_ptr(apic_page);
+	gpa = __pa(backing_page);
+
+	/*
+	 * The NPT entry for a vCPU's APIC backing page must always be
+	 * present when the vCPU is running in order for Secure AVIC to
+	 * function. A VMEXIT_BUSY is returned on VMRUN and the vCPU cannot
+	 * be resumed if the NPT entry for the APIC backing page is not
+	 * present. Notify GPA of the vCPU's APIC backing page to the
+	 * hypervisor by calling savic_register_gpa(). Before executing
+	 * VMRUN, the hypervisor makes use of this information to make sure
+	 * the APIC backing page is mapped in NPT.
+	 */
+	res = savic_register_gpa(gpa);
+	if (res != ES_OK)
+		snp_abort();
+}
+
 static int savic_probe(void)
 {
 	if (!cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
@@ -30,6 +57,10 @@ static int savic_probe(void)
 		/* unreachable */
 	}
 
+	apic_page = alloc_percpu(struct apic_page);
+	if (!apic_page)
+		snp_abort();
+
 	return 1;
 }
 
@@ -38,6 +69,7 @@ static struct apic apic_x2apic_savic __ro_after_init = {
 	.name				= "secure avic x2apic",
 	.probe				= savic_probe,
 	.acpi_madt_oem_check		= savic_acpi_madt_oem_check,
+	.setup				= savic_setup,
 
 	.dest_mode_logical		= false,
 
-- 
2.34.1

[RFC PATCH v8 20/35] x86/apic: Populate .read()/.write() callbacks of Secure AVIC driver

[Neeraj Upadhyay]

Add read() and write() APIC callback functions to read and write x2APIC
registers directly from the guest APIC backing page of a vCPU.

The x2APIC registers are mapped at an offset within the guest APIC
backing page which is same as their x2APIC MMIO offset. Secure AVIC
adds new registers such as ALLOWED_IRRs (which are at 4-byte offset
within the IRR register offset range) and NMI_REQ to the APIC register
space.

When Secure AVIC is enabled, guest's rdmsr/wrmsr of APIC registers
result in VC exception (for non-accelerated register accesses) with
error code VMEXIT_AVIC_NOACCEL. The VC exception handler can read/write
the x2APIC register in the guest APIC backing page to complete the
rdmsr/wrmsr. Since doing this would increase the latency of accessing
x2APIC registers, instead of doing rdmsr/wrmsr based reg accesses
and handling reads/writes in VC exception, directly read/write APIC
registers from/to the guest APIC backing page of the vCPU in read()
and write() callbacks of the Secure AVIC APIC driver.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/include/asm/apicdef.h      |   2 +
 arch/x86/kernel/apic/x2apic_savic.c | 113 +++++++++++++++++++++++++++-
 2 files changed, 113 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/apicdef.h b/arch/x86/include/asm/apicdef.h
index 094106b6a538..be39a543fbe5 100644
--- a/arch/x86/include/asm/apicdef.h
+++ b/arch/x86/include/asm/apicdef.h
@@ -135,6 +135,8 @@
 #define		APIC_TDR_DIV_128	0xA
 #define	APIC_EFEAT	0x400
 #define	APIC_ECTRL	0x410
+#define APIC_SEOI	0x420
+#define APIC_IER	0x480
 #define APIC_EILVTn(n)	(0x500 + 0x10 * n)
 #define		APIC_EILVT_NR_AMD_K8	1	/* # of extended interrupts */
 #define		APIC_EILVT_NR_AMD_10H	4
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index a2747ab9200a..186e69a5e169 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -9,6 +9,7 @@
 
 #include <linux/cc_platform.h>
 #include <linux/percpu-defs.h>
+#include <linux/align.h>
 
 #include <asm/apic.h>
 #include <asm/sev.h>
@@ -22,6 +23,114 @@ static int savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
 }
 
+#define SAVIC_ALLOWED_IRR	0x204
+
+static u32 savic_read(u32 reg)
+{
+	struct apic_page *ap = this_cpu_ptr(apic_page);
+
+	/*
+	 * When Secure AVIC is enabled, rdmsr/wrmsr of APIC registers
+	 * result in VC exception (for non-accelerated register accesses)
+	 * with VMEXIT_AVIC_NOACCEL error code. The VC exception handler
+	 * can read/write the x2APIC register in the guest APIC backing page.
+	 * Since doing this would increase the latency of accessing x2APIC
+	 * registers, instead of doing rdmsr/wrmsr based accesses and
+	 * handling apic register reads/writes in VC exception, the read()
+	 * and write() callbacks directly read/write APIC register from/to
+	 * the vCPU APIC backing page.
+	 */
+	switch (reg) {
+	case APIC_LVTT:
+	case APIC_TMICT:
+	case APIC_TMCCT:
+	case APIC_TDCR:
+	case APIC_ID:
+	case APIC_LVR:
+	case APIC_TASKPRI:
+	case APIC_ARBPRI:
+	case APIC_PROCPRI:
+	case APIC_LDR:
+	case APIC_SPIV:
+	case APIC_ESR:
+	case APIC_LVTTHMR:
+	case APIC_LVTPC:
+	case APIC_LVT0:
+	case APIC_LVT1:
+	case APIC_LVTERR:
+	case APIC_EFEAT:
+	case APIC_ECTRL:
+	case APIC_SEOI:
+	case APIC_IER:
+	case APIC_EILVTn(0) ... APIC_EILVTn(3):
+		return apic_get_reg(ap, reg);
+	case APIC_ICR:
+		return (u32) apic_get_reg64(ap, reg);
+	case APIC_ISR ... APIC_ISR + 0x70:
+	case APIC_TMR ... APIC_TMR + 0x70:
+		if (WARN_ONCE(!IS_ALIGNED(reg, 16),
+			      "APIC reg read offset 0x%x not aligned at 16 bytes", reg))
+			return 0;
+		return apic_get_reg(ap, reg);
+	/* IRR and ALLOWED_IRR offset range */
+	case APIC_IRR ... APIC_IRR + 0x74:
+		/*
+		 * Either aligned at 16 bytes for valid IRR reg offset or a
+		 * valid Secure AVIC ALLOWED_IRR offset.
+		 */
+		if (WARN_ONCE(!(IS_ALIGNED(reg, 16) ||
+				IS_ALIGNED(reg - SAVIC_ALLOWED_IRR, 16)),
+			      "Misaligned IRR/ALLOWED_IRR APIC reg read offset 0x%x", reg))
+			return 0;
+		return apic_get_reg(ap, reg);
+	default:
+		pr_err("Permission denied: read of Secure AVIC reg offset 0x%x\n", reg);
+		return 0;
+	}
+}
+
+#define SAVIC_NMI_REQ		0x278
+
+static void savic_write(u32 reg, u32 data)
+{
+	struct apic_page *ap = this_cpu_ptr(apic_page);
+
+	switch (reg) {
+	case APIC_LVTT:
+	case APIC_LVT0:
+	case APIC_LVT1:
+	case APIC_TMICT:
+	case APIC_TDCR:
+	case APIC_SELF_IPI:
+	case APIC_TASKPRI:
+	case APIC_EOI:
+	case APIC_SPIV:
+	case SAVIC_NMI_REQ:
+	case APIC_ESR:
+	case APIC_LVTTHMR:
+	case APIC_LVTPC:
+	case APIC_LVTERR:
+	case APIC_ECTRL:
+	case APIC_SEOI:
+	case APIC_IER:
+	case APIC_EILVTn(0) ... APIC_EILVTn(3):
+		apic_set_reg(ap, reg, data);
+		break;
+	case APIC_ICR:
+		apic_set_reg64(ap, reg, (u64) data);
+		break;
+	/* ALLOWED_IRR offsets are writable */
+	case SAVIC_ALLOWED_IRR ... SAVIC_ALLOWED_IRR + 0x70:
+		if (IS_ALIGNED(reg - SAVIC_ALLOWED_IRR, 16)) {
+			apic_set_reg(ap, reg, data);
+			break;
+		}
+		fallthrough;
+	default:
+		pr_err("Permission denied: write to Secure AVIC reg offset 0x%x\n", reg);
+	}
+}
+
 static void savic_setup(void)
 {
 	void *backing_page;
@@ -85,8 +194,8 @@ static struct apic apic_x2apic_savic __ro_after_init = {
 
 	.nmi_to_offline_cpu		= true,
 
-	.read				= native_apic_msr_read,
-	.write				= native_apic_msr_write,
+	.read				= savic_read,
+	.write				= savic_write,
 	.eoi				= native_apic_msr_eoi,
 	.icr_read			= native_x2apic_icr_read,
 	.icr_write			= native_x2apic_icr_write,
-- 
2.34.1

[RFC PATCH v8 21/35] x86/apic: Initialize APIC ID for Secure AVIC

[Neeraj Upadhyay]

Initialize the APIC ID in the Secure AVIC APIC backing page with
the APIC_ID msr value read from Hypervisor. CPU topology evaluation
later during boot would catch and report any duplicate APIC ID for
two CPUs.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/kernel/apic/x2apic_savic.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index 186e69a5e169..618643e7242f 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -131,6 +131,18 @@ static void savic_write(u32 reg, u32 data)
 	}
 }
 
+static void init_apic_page(struct apic_page *ap)
+{
+	u32 apic_id;
+
+	/*
+	 * Before Secure AVIC is enabled, APIC msr reads are intercepted.
+	 * APIC_ID msr read returns the value from the Hypervisor.
+	 */
+	apic_id = native_apic_msr_read(APIC_ID);
+	apic_set_reg(ap, APIC_ID, apic_id);
+}
+
 static void savic_setup(void)
 {
 	void *backing_page;
@@ -138,6 +150,7 @@ static void savic_setup(void)
 	unsigned long gpa;
 
 	backing_page = this_cpu_ptr(apic_page);
+	init_apic_page(backing_page);
 	gpa = __pa(backing_page);
 
 	/*
-- 
2.34.1

[RFC PATCH v8 22/35] x86/apic: Add update_vector() callback for apic drivers

[Neeraj Upadhyay]

Add an update_vector() callback to allow APIC drivers to perform
driver specific operations on external vector allocation/teardown
on a CPU. This callback will be used in subsequent commits by Secure
AVIC APIC driver to configure the vectors which a guest vCPU allows
the hypervisor to send to it.

As system vectors have fixed vector assignments and are not dynamically
allocated, add apic_update_vector() public api to facilitate
update_vector() callback invocation for them. This will be used for
Secure AVIC enabled guests to allow the hypervisor to inject system
vectors which are emulated by the hypervisor such as APIC timer vector
and HYPERVISOR_CALLBACK_VECTOR.

While at it, cleanup line break in apic_update_irq_cfg().

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/include/asm/apic.h   |  9 +++++++++
 arch/x86/kernel/apic/vector.c | 29 ++++++++++++++++++-----------
 2 files changed, 27 insertions(+), 11 deletions(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index 184cae6e786b..9c74d1faf3e0 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -318,6 +318,8 @@ struct apic {
 	/* wakeup secondary CPU using 64-bit wakeup point */
 	int	(*wakeup_secondary_cpu_64)(u32 apicid, unsigned long start_eip, unsigned int cpu);
 
+	void	(*update_vector)(unsigned int cpu, unsigned int vector, bool set);
+
 	char	*name;
 };
 
@@ -471,6 +473,12 @@ static __always_inline bool apic_id_valid(u32 apic_id)
 	return apic_id <= apic->max_apic_id;
 }
 
+static __always_inline void apic_update_vector(unsigned int cpu, unsigned int vector, bool set)
+{
+	if (apic->update_vector)
+		apic->update_vector(cpu, vector, set);
+}
+
 #else /* CONFIG_X86_LOCAL_APIC */
 
 static inline u32 apic_read(u32 reg) { return 0; }
@@ -482,6 +490,7 @@ static inline void apic_wait_icr_idle(void) { }
 static inline u32 safe_apic_wait_icr_idle(void) { return 0; }
 static inline void apic_native_eoi(void) { WARN_ON_ONCE(1); }
 static inline void apic_setup_apic_calls(void) { }
+static inline void apic_update_vector(unsigned int cpu, unsigned int vector, bool set) { }
 
 #define apic_update_callback(_callback, _fn) do { } while (0)
 
diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
index a947b46a8b64..655eeb808ebc 100644
--- a/arch/x86/kernel/apic/vector.c
+++ b/arch/x86/kernel/apic/vector.c
@@ -134,13 +134,21 @@ static void apic_update_irq_cfg(struct irq_data *irqd, unsigned int vector,
 
 	apicd->hw_irq_cfg.vector = vector;
 	apicd->hw_irq_cfg.dest_apicid = apic->calc_dest_apicid(cpu);
+
+	apic_update_vector(cpu, vector, true);
+
 	irq_data_update_effective_affinity(irqd, cpumask_of(cpu));
-	trace_vector_config(irqd->irq, vector, cpu,
-			    apicd->hw_irq_cfg.dest_apicid);
+	trace_vector_config(irqd->irq, vector, cpu, apicd->hw_irq_cfg.dest_apicid);
 }
 
-static void apic_update_vector(struct irq_data *irqd, unsigned int newvec,
-			       unsigned int newcpu)
+static void apic_free_vector(unsigned int cpu, unsigned int vector, bool managed)
+{
+	apic_update_vector(cpu, vector, false);
+	irq_matrix_free(vector_matrix, cpu, vector, managed);
+}
+
+static void apic_chipd_update_vector(struct irq_data *irqd, unsigned int newvec,
+				     unsigned int newcpu)
 {
 	struct apic_chip_data *apicd = apic_chip_data(irqd);
 	struct irq_desc *desc = irq_data_to_desc(irqd);
@@ -174,8 +182,7 @@ static void apic_update_vector(struct irq_data *irqd, unsigned int newvec,
 		apicd->prev_cpu = apicd->cpu;
 		WARN_ON_ONCE(apicd->cpu == newcpu);
 	} else {
-		irq_matrix_free(vector_matrix, apicd->cpu, apicd->vector,
-				managed);
+		apic_free_vector(apicd->cpu, apicd->vector, managed);
 	}
 
 setnew:
@@ -261,7 +268,7 @@ assign_vector_locked(struct irq_data *irqd, const struct cpumask *dest)
 	trace_vector_alloc(irqd->irq, vector, resvd, vector);
 	if (vector < 0)
 		return vector;
-	apic_update_vector(irqd, vector, cpu);
+	apic_chipd_update_vector(irqd, vector, cpu);
 
 	return 0;
 }
@@ -337,7 +344,7 @@ assign_managed_vector(struct irq_data *irqd, const struct cpumask *dest)
 	trace_vector_alloc_managed(irqd->irq, vector, vector);
 	if (vector < 0)
 		return vector;
-	apic_update_vector(irqd, vector, cpu);
+	apic_chipd_update_vector(irqd, vector, cpu);
 
 	return 0;
 }
@@ -357,7 +364,7 @@ static void clear_irq_vector(struct irq_data *irqd)
 			   apicd->prev_cpu);
 
 	per_cpu(vector_irq, apicd->cpu)[vector] = VECTOR_SHUTDOWN;
-	irq_matrix_free(vector_matrix, apicd->cpu, vector, managed);
+	apic_free_vector(apicd->cpu, vector, managed);
 	apicd->vector = 0;
 
 	/* Clean up move in progress */
@@ -366,7 +373,7 @@ static void clear_irq_vector(struct irq_data *irqd)
 		return;
 
 	per_cpu(vector_irq, apicd->prev_cpu)[vector] = VECTOR_SHUTDOWN;
-	irq_matrix_free(vector_matrix, apicd->prev_cpu, vector, managed);
+	apic_free_vector(apicd->prev_cpu, vector, managed);
 	apicd->prev_vector = 0;
 	apicd->move_in_progress = 0;
 	hlist_del_init(&apicd->clist);
@@ -905,7 +912,7 @@ static void free_moved_vector(struct apic_chip_data *apicd)
 	 *    affinity mask comes online.
 	 */
 	trace_vector_free_moved(apicd->irq, cpu, vector, managed);
-	irq_matrix_free(vector_matrix, cpu, vector, managed);
+	apic_free_vector(cpu, vector, managed);
 	per_cpu(vector_irq, cpu)[vector] = VECTOR_UNUSED;
 	hlist_del_init(&apicd->clist);
 	apicd->prev_vector = 0;
-- 
2.34.1

[RFC PATCH v8 23/35] x86/apic: Add update_vector() callback for Secure AVIC

[Neeraj Upadhyay]

Add update_vector() callback to set/clear ALLOWED_IRR field in
a vCPU's APIC backing page for vectors which are emulated by the
hypervisor.

The ALLOWED_IRR field indicates the interrupt vectors which the
guest allows the hypervisor to inject (typically for emulated devices).
Interrupt vectors used exclusively by the guest itself and the vectors
which are not emulated by the hypervisor, such as IPI vectors, should
not be set by the guest in the ALLOWED_IRR fields.

As clearing/setting state of a vector will also be used in subsequent
commits for other APIC regs (such as APIC_IRR update for sending IPI),
add a common update_vector() in Secure AVIC driver.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/kernel/apic/x2apic_savic.c | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index 618643e7242f..2e6b62041968 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -23,6 +23,24 @@ static int savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
 }
 
+static inline void *get_reg_bitmap(unsigned int cpu, unsigned int offset)
+{
+	struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
+
+	return &ap->bytes[offset];
+}
+
+static inline void update_vector(unsigned int cpu, unsigned int offset,
+				 unsigned int vector, bool set)
+{
+	void *bitmap = get_reg_bitmap(cpu, offset);
+
+	if (set)
+		apic_set_vector(vector, bitmap);
+	else
+		apic_clear_vector(vector, bitmap);
+}
+
 #define SAVIC_ALLOWED_IRR	0x204
 
 static u32 savic_read(u32 reg)
@@ -131,6 +149,11 @@ static void savic_write(u32 reg, u32 data)
 	}
 }
 
+static void savic_update_vector(unsigned int cpu, unsigned int vector, bool set)
+{
+	update_vector(cpu, SAVIC_ALLOWED_IRR, vector, set);
+}
+
 static void init_apic_page(struct apic_page *ap)
 {
 	u32 apic_id;
@@ -212,6 +235,8 @@ static struct apic apic_x2apic_savic __ro_after_init = {
 	.eoi				= native_apic_msr_eoi,
 	.icr_read			= native_x2apic_icr_read,
 	.icr_write			= native_x2apic_icr_write,
+
+	.update_vector			= savic_update_vector,
 };
 
 apic_driver(apic_x2apic_savic);
-- 
2.34.1

[RFC PATCH v8 24/35] x86/apic: Add support to send IPI for Secure AVIC

[Neeraj Upadhyay]

With Secure AVIC only Self-IPI is accelerated. To handle all the
other IPIs, add new callbacks for sending IPI. These callbacks write
to the IRR of the target guest vCPU's APIC backing page and issue
GHCB protocol MSR write event for the hypervisor to notify the
target vCPU about the new interrupt request.

For Secure AVIC GHCB APIC MSR writes, reuse GHCB msr handling code in
vc_handle_msr() by exposing a sev-internal sev_es_ghcb_handle_msr().

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/coco/sev/core.c            |  28 ++++++
 arch/x86/coco/sev/vc-handle.c       |  11 ++-
 arch/x86/include/asm/sev-internal.h |   2 +
 arch/x86/include/asm/sev.h          |   2 +
 arch/x86/kernel/apic/x2apic_savic.c | 139 +++++++++++++++++++++++++++-
 5 files changed, 174 insertions(+), 8 deletions(-)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index 0c59ea82fa99..221a0fc0c387 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -1085,6 +1085,34 @@ int __init sev_es_efi_map_ghcbs_cas(pgd_t *pgd)
 	return 0;
 }
 
+void savic_ghcb_msr_write(u32 reg, u64 value)
+{
+	u64 msr = APIC_BASE_MSR + (reg >> 4);
+	struct pt_regs regs = {
+		.cx = msr,
+		.ax = lower_32_bits(value),
+		.dx = upper_32_bits(value)
+	};
+	struct es_em_ctxt ctxt = { .regs = &regs };
+	struct ghcb_state state;
+	enum es_result res;
+	struct ghcb *ghcb;
+
+	guard(irqsave)();
+
+	ghcb = __sev_get_ghcb(&state);
+	vc_ghcb_invalidate(ghcb);
+
+	res = sev_es_ghcb_handle_msr(ghcb, &ctxt, true);
+	if (res != ES_OK) {
+		pr_err("Secure AVIC msr (0x%llx) write returned error (%d)\n", msr, res);
+		/* MSR writes should never fail. Any failure is fatal error for SNP guest */
+		snp_abort();
+	}
+
+	__sev_put_ghcb(&state);
+}
+
 enum es_result savic_register_gpa(u64 gpa)
 {
 	struct ghcb_state state;
diff --git a/arch/x86/coco/sev/vc-handle.c b/arch/x86/coco/sev/vc-handle.c
index faf1fce89ed4..fc770cc9117d 100644
--- a/arch/x86/coco/sev/vc-handle.c
+++ b/arch/x86/coco/sev/vc-handle.c
@@ -401,14 +401,10 @@ static enum es_result __vc_handle_secure_tsc_msrs(struct pt_regs *regs, bool wri
 	return ES_OK;
 }
 
-static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
+enum es_result sev_es_ghcb_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt, bool write)
 {
 	struct pt_regs *regs = ctxt->regs;
 	enum es_result ret;
-	bool write;
-
-	/* Is it a WRMSR? */
-	write = ctxt->insn.opcode.bytes[1] == 0x30;
 
 	switch (regs->cx) {
 	case MSR_SVSM_CAA:
@@ -438,6 +434,11 @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
 	return ret;
 }
 
+static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
+{
+	return sev_es_ghcb_handle_msr(ghcb, ctxt, ctxt->insn.opcode.bytes[1] == 0x30);
+}
+
 static void __init vc_early_forward_exception(struct es_em_ctxt *ctxt)
 {
 	int trapnr = ctxt->fi.vector;
diff --git a/arch/x86/include/asm/sev-internal.h b/arch/x86/include/asm/sev-internal.h
index 3dfd306d1c9e..6876655183a6 100644
--- a/arch/x86/include/asm/sev-internal.h
+++ b/arch/x86/include/asm/sev-internal.h
@@ -97,6 +97,8 @@ static __always_inline void sev_es_wr_ghcb_msr(u64 val)
 	native_wrmsr(MSR_AMD64_SEV_ES_GHCB, low, high);
 }
 
+enum es_result sev_es_ghcb_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt, bool write);
+
 void snp_register_ghcb_early(unsigned long paddr);
 bool sev_es_negotiate_protocol(void);
 bool sev_es_check_cpu_features(void);
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index 8e5083b46607..e849e616dd24 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -534,6 +534,7 @@ int snp_svsm_vtpm_send_command(u8 *buffer);
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
 enum es_result savic_register_gpa(u64 gpa);
+void savic_ghcb_msr_write(u32 reg, u64 value);
 
 static __always_inline void vc_ghcb_invalidate(struct ghcb *ghcb)
 {
@@ -607,6 +608,7 @@ static inline int snp_svsm_vtpm_send_command(u8 *buffer) { return -ENODEV; }
 static inline void __init snp_secure_tsc_prepare(void) { }
 static inline void __init snp_secure_tsc_init(void) { }
 static inline enum es_result savic_register_gpa(u64 gpa) { return ES_UNSUPPORTED; }
+static inline void savic_ghcb_msr_write(u32 reg, u64 value) { }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index 2e6b62041968..2a95e549ff68 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -8,6 +8,7 @@
  */
 
 #include <linux/cc_platform.h>
+#include <linux/cpumask.h>
 #include <linux/percpu-defs.h>
 #include <linux/align.h>
 
@@ -109,6 +110,74 @@ static u32 savic_read(u32 reg)
 
 #define SAVIC_NMI_REQ		0x278
 
+static inline void self_ipi_reg_write(unsigned int vector)
+{
+	/*
+	 * Secure AVIC hardware accelerates guest's MSR write to SELF_IPI
+	 * register. It updates the IRR in the APIC backing page, evaluates
+	 * the new IRR for interrupt injection and continues with guest
+	 * code execution.
+	 */
+	native_apic_msr_write(APIC_SELF_IPI, vector);
+}
+
+static void send_ipi_dest(unsigned int cpu, unsigned int vector)
+{
+	update_vector(cpu, APIC_IRR, vector, true);
+}
+
+static void send_ipi_allbut(unsigned int vector)
+{
+	unsigned int cpu, src_cpu;
+
+	guard(irqsave)();
+
+	src_cpu = raw_smp_processor_id();
+
+	for_each_cpu(cpu, cpu_online_mask) {
+		if (cpu == src_cpu)
+			continue;
+		send_ipi_dest(cpu, vector);
+	}
+}
+
+static inline void self_ipi(unsigned int vector)
+{
+	u32 icr_low = APIC_SELF_IPI | vector;
+
+	native_x2apic_icr_write(icr_low, 0);
+}
+
+static void savic_icr_write(u32 icr_low, u32 icr_high)
+{
+	struct apic_page *ap = this_cpu_ptr(apic_page);
+	unsigned int dsh, vector;
+	u64 icr_data;
+
+	dsh = icr_low & APIC_DEST_ALLBUT;
+	vector = icr_low & APIC_VECTOR_MASK;
+
+	switch (dsh) {
+	case APIC_DEST_SELF:
+		self_ipi(vector);
+		break;
+	case APIC_DEST_ALLINC:
+		self_ipi(vector);
+		fallthrough;
+	case APIC_DEST_ALLBUT:
+		send_ipi_allbut(vector);
+		break;
+	default:
+		send_ipi_dest(icr_high, vector);
+		break;
+	}
+
+	icr_data = ((u64)icr_high) << 32 | icr_low;
+	if (dsh != APIC_DEST_SELF)
+		savic_ghcb_msr_write(APIC_ICR, icr_data);
+	apic_set_reg64(ap, APIC_ICR, icr_data);
+}
+
 static void savic_write(u32 reg, u32 data)
 {
 	struct apic_page *ap = this_cpu_ptr(apic_page);
@@ -119,7 +188,6 @@ static void savic_write(u32 reg, u32 data)
 	case APIC_LVT1:
 	case APIC_TMICT:
 	case APIC_TDCR:
-	case APIC_SELF_IPI:
 	case APIC_TASKPRI:
 	case APIC_EOI:
 	case APIC_SPIV:
@@ -135,7 +203,10 @@ static void savic_write(u32 reg, u32 data)
 		apic_set_reg(ap, reg, data);
 		break;
 	case APIC_ICR:
-		apic_set_reg64(ap, reg, (u64) data);
+		savic_icr_write(data, 0);
+		break;
+	case APIC_SELF_IPI:
+		self_ipi_reg_write(data);
 		break;
 	/* ALLOWED_IRR offsets are writable */
 	case SAVIC_ALLOWED_IRR ... SAVIC_ALLOWED_IRR + 0x70:
@@ -149,6 +220,61 @@ static void savic_write(u32 reg, u32 data)
 	}
 }
 
+static void send_ipi(u32 dest, unsigned int vector, unsigned int dsh)
+{
+	unsigned int icr_low;
+
+	icr_low = __prepare_ICR(dsh, vector, APIC_DEST_PHYSICAL);
+	savic_icr_write(icr_low, dest);
+}
+
+static void savic_send_ipi(int cpu, int vector)
+{
+	u32 dest = per_cpu(x86_cpu_to_apicid, cpu);
+
+	send_ipi(dest, vector, 0);
+}
+
+static void send_ipi_mask(const struct cpumask *mask, unsigned int vector, bool excl_self)
+{
+	unsigned int cpu, this_cpu;
+
+	guard(irqsave)();
+
+	this_cpu = raw_smp_processor_id();
+
+	for_each_cpu(cpu, mask) {
+		if (excl_self && cpu == this_cpu)
+			continue;
+		send_ipi(per_cpu(x86_cpu_to_apicid, cpu), vector, 0);
+	}
+}
+
+static void savic_send_ipi_mask(const struct cpumask *mask, int vector)
+{
+	send_ipi_mask(mask, vector, false);
+}
+
+static void savic_send_ipi_mask_allbutself(const struct cpumask *mask, int vector)
+{
+	send_ipi_mask(mask, vector, true);
+}
+
+static void savic_send_ipi_allbutself(int vector)
+{
+	send_ipi(0, vector, APIC_DEST_ALLBUT);
+}
+
+static void savic_send_ipi_all(int vector)
+{
+	send_ipi(0, vector, APIC_DEST_ALLINC);
+}
+
+static void savic_send_ipi_self(int vector)
+{
+	self_ipi_reg_write(vector);
+}
+
 static void savic_update_vector(unsigned int cpu, unsigned int vector, bool set)
 {
 	update_vector(cpu, SAVIC_ALLOWED_IRR, vector, set);
@@ -228,13 +354,20 @@ static struct apic apic_x2apic_savic __ro_after_init = {
 
 	.calc_dest_apicid		= apic_default_calc_apicid,
 
+	.send_IPI			= savic_send_ipi,
+	.send_IPI_mask			= savic_send_ipi_mask,
+	.send_IPI_mask_allbutself	= savic_send_ipi_mask_allbutself,
+	.send_IPI_allbutself		= savic_send_ipi_allbutself,
+	.send_IPI_all			= savic_send_ipi_all,
+	.send_IPI_self			= savic_send_ipi_self,
+
 	.nmi_to_offline_cpu		= true,
 
 	.read				= savic_read,
 	.write				= savic_write,
 	.eoi				= native_apic_msr_eoi,
 	.icr_read			= native_x2apic_icr_read,
-	.icr_write			= native_x2apic_icr_write,
+	.icr_write			= savic_icr_write,
 
 	.update_vector			= savic_update_vector,
 };
-- 
2.34.1

[RFC PATCH v8 25/35] x86/apic: Support LAPIC timer for Secure AVIC

[Neeraj Upadhyay]

Secure AVIC requires LAPIC timer to be emulated by the hypervisor.
KVM already supports emulating LAPIC timer using hrtimers. In order
to emulate LAPIC timer, APIC_LVTT, APIC_TMICT and APIC_TDCR register
values need to be propagated to the hypervisor for arming the timer.
APIC_TMCCT register value has to be read from the hypervisor, which
is required for calibrating the APIC timer. So, read/write all APIC
timer registers from/to the hypervisor.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/coco/sev/core.c            | 26 ++++++++++++++++++++++++++
 arch/x86/include/asm/sev.h          |  2 ++
 arch/x86/kernel/apic/apic.c         |  2 ++
 arch/x86/kernel/apic/x2apic_savic.c |  7 +++++--
 4 files changed, 35 insertions(+), 2 deletions(-)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index 221a0fc0c387..3f64ed6bd1e6 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -1085,6 +1085,32 @@ int __init sev_es_efi_map_ghcbs_cas(pgd_t *pgd)
 	return 0;
 }
 
+u64 savic_ghcb_msr_read(u32 reg)
+{
+	u64 msr = APIC_BASE_MSR + (reg >> 4);
+	struct pt_regs regs = { .cx = msr };
+	struct es_em_ctxt ctxt = { .regs = &regs };
+	struct ghcb_state state;
+	enum es_result res;
+	struct ghcb *ghcb;
+
+	guard(irqsave)();
+
+	ghcb = __sev_get_ghcb(&state);
+	vc_ghcb_invalidate(ghcb);
+
+	res = sev_es_ghcb_handle_msr(ghcb, &ctxt, false);
+	if (res != ES_OK) {
+		pr_err("Secure AVIC msr (0x%llx) read returned error (%d)\n", msr, res);
+		/* MSR read failures are treated as fatal errors */
+		snp_abort();
+	}
+
+	__sev_put_ghcb(&state);
+
+	return regs.ax | regs.dx << 32;
+}
+
 void savic_ghcb_msr_write(u32 reg, u64 value)
 {
 	u64 msr = APIC_BASE_MSR + (reg >> 4);
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index e849e616dd24..d10ca66aa684 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -534,6 +534,7 @@ int snp_svsm_vtpm_send_command(u8 *buffer);
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
 enum es_result savic_register_gpa(u64 gpa);
+u64 savic_ghcb_msr_read(u32 reg);
 void savic_ghcb_msr_write(u32 reg, u64 value);
 
 static __always_inline void vc_ghcb_invalidate(struct ghcb *ghcb)
@@ -609,6 +610,7 @@ static inline void __init snp_secure_tsc_prepare(void) { }
 static inline void __init snp_secure_tsc_init(void) { }
 static inline enum es_result savic_register_gpa(u64 gpa) { return ES_UNSUPPORTED; }
 static inline void savic_ghcb_msr_write(u32 reg, u64 value) { }
+static inline u64 savic_ghcb_msr_read(u32 reg) { return 0; }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 36f1326fea2e..69b1084da8f4 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -592,6 +592,8 @@ static void setup_APIC_timer(void)
 						0xF, ~0UL);
 	} else
 		clockevents_register_device(levt);
+
+	apic_update_vector(smp_processor_id(), LOCAL_TIMER_VECTOR, true);
 }
 
 /*
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index 2a95e549ff68..e5bf717db1bc 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -64,6 +64,7 @@ static u32 savic_read(u32 reg)
 	case APIC_TMICT:
 	case APIC_TMCCT:
 	case APIC_TDCR:
+		return savic_ghcb_msr_read(reg);
 	case APIC_ID:
 	case APIC_LVR:
 	case APIC_TASKPRI:
@@ -184,10 +185,12 @@ static void savic_write(u32 reg, u32 data)
 
 	switch (reg) {
 	case APIC_LVTT:
-	case APIC_LVT0:
-	case APIC_LVT1:
 	case APIC_TMICT:
 	case APIC_TDCR:
+		savic_ghcb_msr_write(reg, data);
+		break;
+	case APIC_LVT0:
+	case APIC_LVT1:
 	case APIC_TASKPRI:
 	case APIC_EOI:
 	case APIC_SPIV:
-- 
2.34.1

[RFC PATCH v8 26/35] x86/sev: Initialize VGIF for secondary VCPUs for Secure AVIC

[Neeraj Upadhyay]

From: Kishon Vijay Abraham I <kvijayab@amd.com>

Secure AVIC requires VGIF to be configured in VMSA. Configure
for secondary CPUs (the configuration for boot CPU is done by
the hypervisor).

Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/coco/sev/core.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index 3f64ed6bd1e6..e341d6239326 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -951,6 +951,9 @@ static int wakeup_cpu_via_vmgexit(u32 apic_id, unsigned long start_ip, unsigned
 	vmsa->x87_ftw		= AP_INIT_X87_FTW_DEFAULT;
 	vmsa->x87_fcw		= AP_INIT_X87_FCW_DEFAULT;
 
+	if (cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
+		vmsa->vintr_ctrl	|= V_GIF_MASK;
+
 	/* SVME must be set. */
 	vmsa->efer		= EFER_SVME;
 
-- 
2.34.1

[RFC PATCH v8 27/35] x86/apic: Add support to send NMI IPI for Secure AVIC

[Neeraj Upadhyay]

Secure AVIC has introduced a new field in the APIC backing page
"NmiReq" that has to be set by the guest to request a NMI IPI
through APIC_ICR write.

Add support to set NmiReq appropriately to send NMI IPI.

Sending NMI IPI also requires Virtual NMI feature to be enabled
in VINTRL_CTRL field in the VMSA. However, this would be added by
a later commit after adding support for injecting NMI from the
hypervisor.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/kernel/apic/x2apic_savic.c | 28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index e5bf717db1bc..66fa4b8d76ef 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -122,12 +122,19 @@ static inline void self_ipi_reg_write(unsigned int vector)
 	native_apic_msr_write(APIC_SELF_IPI, vector);
 }
 
-static void send_ipi_dest(unsigned int cpu, unsigned int vector)
+static void send_ipi_dest(unsigned int cpu, unsigned int vector, bool nmi)
 {
+	if (nmi) {
+		struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
+
+		apic_set_reg(ap, SAVIC_NMI_REQ, 1);
+		return;
+	}
+
 	update_vector(cpu, APIC_IRR, vector, true);
 }
 
-static void send_ipi_allbut(unsigned int vector)
+static void send_ipi_allbut(unsigned int vector, bool nmi)
 {
 	unsigned int cpu, src_cpu;
 
@@ -138,14 +145,17 @@ static void send_ipi_allbut(unsigned int vector)
 	for_each_cpu(cpu, cpu_online_mask) {
 		if (cpu == src_cpu)
 			continue;
-		send_ipi_dest(cpu, vector);
+		send_ipi_dest(cpu, vector, nmi);
 	}
 }
 
-static inline void self_ipi(unsigned int vector)
+static inline void self_ipi(unsigned int vector, bool nmi)
 {
 	u32 icr_low = APIC_SELF_IPI | vector;
 
+	if (nmi)
+		icr_low |= APIC_DM_NMI;
+
 	native_x2apic_icr_write(icr_low, 0);
 }
 
@@ -154,22 +164,24 @@ static void savic_icr_write(u32 icr_low, u32 icr_high)
 	struct apic_page *ap = this_cpu_ptr(apic_page);
 	unsigned int dsh, vector;
 	u64 icr_data;
+	bool nmi;
 
 	dsh = icr_low & APIC_DEST_ALLBUT;
 	vector = icr_low & APIC_VECTOR_MASK;
+	nmi = ((icr_low & APIC_DM_FIXED_MASK) == APIC_DM_NMI);
 
 	switch (dsh) {
 	case APIC_DEST_SELF:
-		self_ipi(vector);
+		self_ipi(vector, nmi);
 		break;
 	case APIC_DEST_ALLINC:
-		self_ipi(vector);
+		self_ipi(vector, nmi);
 		fallthrough;
 	case APIC_DEST_ALLBUT:
-		send_ipi_allbut(vector);
+		send_ipi_allbut(vector, nmi);
 		break;
 	default:
-		send_ipi_dest(icr_high, vector);
+		send_ipi_dest(icr_high, vector, nmi);
 		break;
 	}
 
-- 
2.34.1

[RFC PATCH v8 28/35] x86/apic: Allow NMI to be injected from hypervisor for Secure AVIC

[Neeraj Upadhyay]

Secure AVIC requires "AllowedNmi" bit in the Secure AVIC Control MSR
to be set for NMI to be injected from hypervisor. Set "AllowedNmi"
bit in Secure AVIC Control MSR to allow NMI interrupts to be injected
from hypervisor.

Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/include/asm/msr-index.h    | 3 +++
 arch/x86/kernel/apic/x2apic_savic.c | 6 ++++++
 2 files changed, 9 insertions(+)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 045c0d7e160b..a3a2b99d5745 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -702,6 +702,9 @@
 #define MSR_AMD64_SNP_SECURE_AVIC	BIT_ULL(MSR_AMD64_SNP_SECURE_AVIC_BIT)
 #define MSR_AMD64_SNP_RESV_BIT		19
 #define MSR_AMD64_SNP_RESERVED_MASK	GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT)
+#define MSR_AMD64_SECURE_AVIC_CONTROL	0xc0010138
+#define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT 1
+#define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI BIT_ULL(MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT)
 #define MSR_AMD64_RMP_BASE		0xc0010132
 #define MSR_AMD64_RMP_END		0xc0010133
 #define MSR_AMD64_RMP_CFG		0xc0010136
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index 66fa4b8d76ef..583b57636f21 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -19,6 +19,11 @@
 
 static struct apic_page __percpu *apic_page __ro_after_init;
 
+static inline void savic_wr_control_msr(u64 val)
+{
+	native_wrmsr(MSR_AMD64_SECURE_AVIC_CONTROL, lower_32_bits(val), upper_32_bits(val));
+}
+
 static int savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 {
 	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
@@ -330,6 +335,7 @@ static void savic_setup(void)
 	res = savic_register_gpa(gpa);
 	if (res != ES_OK)
 		snp_abort();
+	savic_wr_control_msr(gpa | MSR_AMD64_SECURE_AVIC_ALLOWEDNMI);
 }
 
 static int savic_probe(void)
-- 
2.34.1

[RFC PATCH v8 29/35] x86/sev: Enable NMI support for Secure AVIC

[Neeraj Upadhyay]

From: Kishon Vijay Abraham I <kvijayab@amd.com>

Now that support to send NMI IPI and support to inject NMI from
the hypervisor has been added, set V_NMI_ENABLE in VINTR_CTRL
field of VMSA to enable NMI for Secure AVIC guests.

Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/coco/sev/core.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index e341d6239326..d7c53b3eeaa9 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -952,7 +952,7 @@ static int wakeup_cpu_via_vmgexit(u32 apic_id, unsigned long start_ip, unsigned
 	vmsa->x87_fcw		= AP_INIT_X87_FCW_DEFAULT;
 
 	if (cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
-		vmsa->vintr_ctrl	|= V_GIF_MASK;
+		vmsa->vintr_ctrl	|= (V_GIF_MASK | V_NMI_ENABLE_MASK);
 
 	/* SVME must be set. */
 	vmsa->efer		= EFER_SVME;
-- 
2.34.1

[RFC PATCH v8 30/35] x86/apic: Read and write LVT* APIC registers from HV for SAVIC guests

[Neeraj Upadhyay]

Hypervisor need information about the current state of LVT registers
for device emulation and NMI. So, forward reads and write of these
registers to the hypervisor for Secure AVIC enabled guests.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/kernel/apic/x2apic_savic.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index 583b57636f21..0fecc295874e 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -69,6 +69,11 @@ static u32 savic_read(u32 reg)
 	case APIC_TMICT:
 	case APIC_TMCCT:
 	case APIC_TDCR:
+	case APIC_LVTTHMR:
+	case APIC_LVTPC:
+	case APIC_LVT0:
+	case APIC_LVT1:
+	case APIC_LVTERR:
 		return savic_ghcb_msr_read(reg);
 	case APIC_ID:
 	case APIC_LVR:
@@ -78,11 +83,6 @@ static u32 savic_read(u32 reg)
 	case APIC_LDR:
 	case APIC_SPIV:
 	case APIC_ESR:
-	case APIC_LVTTHMR:
-	case APIC_LVTPC:
-	case APIC_LVT0:
-	case APIC_LVT1:
-	case APIC_LVTERR:
 	case APIC_EFEAT:
 	case APIC_ECTRL:
 	case APIC_SEOI:
@@ -204,18 +204,18 @@ static void savic_write(u32 reg, u32 data)
 	case APIC_LVTT:
 	case APIC_TMICT:
 	case APIC_TDCR:
-		savic_ghcb_msr_write(reg, data);
-		break;
 	case APIC_LVT0:
 	case APIC_LVT1:
+	case APIC_LVTTHMR:
+	case APIC_LVTPC:
+	case APIC_LVTERR:
+		savic_ghcb_msr_write(reg, data);
+		break;
 	case APIC_TASKPRI:
 	case APIC_EOI:
 	case APIC_SPIV:
 	case SAVIC_NMI_REQ:
 	case APIC_ESR:
-	case APIC_LVTTHMR:
-	case APIC_LVTPC:
-	case APIC_LVTERR:
 	case APIC_ECTRL:
 	case APIC_SEOI:
 	case APIC_IER:
-- 
2.34.1

[RFC PATCH v8 31/35] x86/apic: Handle EOI writes for Secure AVIC guests

[Neeraj Upadhyay]

Secure AVIC accelerates guest's EOI msr writes for edge-triggered
interrupts.

For level-triggered interrupts, EOI msr writes trigger VC exception
with SVM_EXIT_AVIC_UNACCELERATED_ACCESS error code. To complete EOI
handling, the VC exception handler would need to trigger a GHCB protocol
MSR write event to notify the hypervisor about completion of the
level-triggered interrupt. Hypervisor notification is required for
cases like emulated IOAPIC, to complete and clear interrupt in the
IOAPIC's interrupt state.

However, VC exception handling adds extra performance overhead for
APIC register writes. In addition, for Secure AVIC, some unaccelerated
APIC register msr writes are trapped, whereas others are faulted. This
results in additional complexity in VC exception handling for unacclerated
APIC msr accesses. So, directly do a GHCB protocol based APIC EOI msr write
from apic->eoi() callback for level-triggered interrupts.

Use wrmsr for edge-triggered interrupts, so that hardware re-evaluates
any pending interrupt which can be delivered to guest vCPU. For level-
triggered interrupts, re-evaluation happens on return from VMGEXIT
corresponding to the GHCB event for APIC EOI msr write.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/kernel/apic/x2apic_savic.c | 35 ++++++++++++++++++++++++++++-
 1 file changed, 34 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index 0fecc295874e..a527d7e4477c 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -300,6 +300,39 @@ static void savic_update_vector(unsigned int cpu, unsigned int vector, bool set)
 	update_vector(cpu, SAVIC_ALLOWED_IRR, vector, set);
 }
 
+static void savic_eoi(void)
+{
+	unsigned int cpu;
+	void *bitmap;
+	int vec;
+
+	cpu = raw_smp_processor_id();
+	bitmap = get_reg_bitmap(cpu, APIC_ISR);
+	vec = apic_find_highest_vector(bitmap);
+	if (WARN_ONCE(vec == -1, "EOI write while no active interrupt in APIC_ISR"))
+		return;
+
+	bitmap = get_reg_bitmap(cpu, APIC_TMR);
+
+	/* Is level-triggered interrupt? */
+	if (apic_test_vector(vec, bitmap)) {
+		update_vector(cpu, APIC_ISR, vec, false);
+		/*
+		 * Propagate the EOI write to hv for level-triggered interrupts.
+		 * Return to guest from GHCB protocol event takes care of
+		 * re-evaluating interrupt state.
+		 */
+		savic_ghcb_msr_write(APIC_EOI, 0);
+	} else {
+		/*
+		 * Hardware clears APIC_ISR and re-evaluates the interrupt state
+		 * to determine if there is any pending interrupt which can be
+		 * delivered to CPU.
+		 */
+		native_apic_msr_eoi();
+	}
+}
+
 static void init_apic_page(struct apic_page *ap)
 {
 	u32 apic_id;
@@ -386,7 +419,7 @@ static struct apic apic_x2apic_savic __ro_after_init = {
 
 	.read				= savic_read,
 	.write				= savic_write,
-	.eoi				= native_apic_msr_eoi,
+	.eoi				= savic_eoi,
 	.icr_read			= native_x2apic_icr_read,
 	.icr_write			= savic_icr_write,
 
-- 
2.34.1

[RFC PATCH v8 32/35] x86/apic: Add kexec support for Secure AVIC

[Neeraj Upadhyay]

Add a apic->teardown() callback to disable Secure AVIC before
rebooting into the new kernel. This ensures that the new
kernel does not access the old APIC backing page which was
allocated by the previous kernel. Such accesses can happen
if there are any APIC accesses done during guest boot before
Secure AVIC driver probe is done by the new kernel (as Secure
AVIC would have remained enabled in the Secure AVIC control
msr).

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/coco/sev/core.c            | 23 +++++++++++++++++++++++
 arch/x86/include/asm/apic.h         |  1 +
 arch/x86/include/asm/sev.h          |  2 ++
 arch/x86/kernel/apic/apic.c         |  3 +++
 arch/x86/kernel/apic/x2apic_savic.c |  8 ++++++++
 5 files changed, 37 insertions(+)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index d7c53b3eeaa9..da7fc7913a00 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -1164,6 +1164,29 @@ enum es_result savic_register_gpa(u64 gpa)
 	return res;
 }
 
+enum es_result savic_unregister_gpa(u64 *gpa)
+{
+	struct ghcb_state state;
+	struct es_em_ctxt ctxt;
+	enum es_result res;
+	struct ghcb *ghcb;
+
+	guard(irqsave)();
+
+	ghcb = __sev_get_ghcb(&state);
+	vc_ghcb_invalidate(ghcb);
+
+	ghcb_set_rax(ghcb, SVM_VMGEXIT_SAVIC_SELF_GPA);
+	res = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_SAVIC,
+				  SVM_VMGEXIT_SAVIC_UNREGISTER_GPA, 0);
+	if (gpa && res == ES_OK)
+		*gpa = ghcb->save.rbx;
+
+	__sev_put_ghcb(&state);
+
+	return res;
+}
+
 static void snp_register_per_cpu_ghcb(void)
 {
 	struct sev_es_runtime_data *data;
diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index 9c74d1faf3e0..e8a32a3eea86 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -306,6 +306,7 @@ struct apic {
 	/* Probe, setup and smpboot functions */
 	int	(*probe)(void);
 	void	(*setup)(void);
+	void	(*teardown)(void);
 	int	(*acpi_madt_oem_check)(char *oem_id, char *oem_table_id);
 
 	void	(*init_apic_ldr)(void);
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index d10ca66aa684..35877c32b528 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -534,6 +534,7 @@ int snp_svsm_vtpm_send_command(u8 *buffer);
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
 enum es_result savic_register_gpa(u64 gpa);
+enum es_result savic_unregister_gpa(u64 *gpa);
 u64 savic_ghcb_msr_read(u32 reg);
 void savic_ghcb_msr_write(u32 reg, u64 value);
 
@@ -609,6 +610,7 @@ static inline int snp_svsm_vtpm_send_command(u8 *buffer) { return -ENODEV; }
 static inline void __init snp_secure_tsc_prepare(void) { }
 static inline void __init snp_secure_tsc_init(void) { }
 static inline enum es_result savic_register_gpa(u64 gpa) { return ES_UNSUPPORTED; }
+static inline enum es_result savic_unregister_gpa(u64 *gpa) { return ES_UNSUPPORTED; }
 static inline void savic_ghcb_msr_write(u32 reg, u64 value) { }
 static inline u64 savic_ghcb_msr_read(u32 reg) { return 0; }
 
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 69b1084da8f4..badd6a42bced 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -1170,6 +1170,9 @@ void disable_local_APIC(void)
 	if (!apic_accessible())
 		return;
 
+	if (apic->teardown)
+		apic->teardown();
+
 	apic_soft_disable();
 
 #ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index a527d7e4477c..417ea676c37e 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -345,6 +345,13 @@ static void init_apic_page(struct apic_page *ap)
 	apic_set_reg(ap, APIC_ID, apic_id);
 }
 
+static void savic_teardown(void)
+{
+	/* Disable Secure AVIC */
+	native_wrmsr(MSR_AMD64_SECURE_AVIC_CONTROL, 0, 0);
+	savic_unregister_gpa(NULL);
+}
+
 static void savic_setup(void)
 {
 	void *backing_page;
@@ -395,6 +402,7 @@ static struct apic apic_x2apic_savic __ro_after_init = {
 	.probe				= savic_probe,
 	.acpi_madt_oem_check		= savic_acpi_madt_oem_check,
 	.setup				= savic_setup,
+	.teardown			= savic_teardown,
 
 	.dest_mode_logical		= false,
 
-- 
2.34.1

[RFC PATCH v8 33/35] x86/apic: Enable Secure AVIC in Control MSR

[Neeraj Upadhyay]

With all the pieces in place now, enable Secure AVIC in Secure
AVIC Control MSR. Any access to x2APIC MSRs are emulated by
the hypervisor before Secure AVIC is enabled in the control MSR.
Post Secure AVIC enablement, all x2APIC MSR accesses (whether
accelerated by AVIC hardware or trapped as VC exception) operate
on vCPU's APIC backing page.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/include/asm/msr-index.h    | 2 ++
 arch/x86/kernel/apic/x2apic_savic.c | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index a3a2b99d5745..9561184ff989 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -703,6 +703,8 @@
 #define MSR_AMD64_SNP_RESV_BIT		19
 #define MSR_AMD64_SNP_RESERVED_MASK	GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT)
 #define MSR_AMD64_SECURE_AVIC_CONTROL	0xc0010138
+#define MSR_AMD64_SECURE_AVIC_EN_BIT	0
+#define MSR_AMD64_SECURE_AVIC_EN	BIT_ULL(MSR_AMD64_SECURE_AVIC_EN_BIT)
 #define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT 1
 #define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI BIT_ULL(MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT)
 #define MSR_AMD64_RMP_BASE		0xc0010132
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index 417ea676c37e..2849f2354bf9 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -375,7 +375,7 @@ static void savic_setup(void)
 	res = savic_register_gpa(gpa);
 	if (res != ES_OK)
 		snp_abort();
-	savic_wr_control_msr(gpa | MSR_AMD64_SECURE_AVIC_ALLOWEDNMI);
+	savic_wr_control_msr(gpa | MSR_AMD64_SECURE_AVIC_EN | MSR_AMD64_SECURE_AVIC_ALLOWEDNMI);
 }
 
 static int savic_probe(void)
-- 
2.34.1

[RFC PATCH v8 34/35] x86/sev: Prevent SECURE_AVIC_CONTROL MSR interception for Secure AVIC guests

[Neeraj Upadhyay]

The SECURE_AVIC_CONTROL MSR holds the GPA of the guest APIC backing
page and bitfields to control enablement of Secure AVIC and NMI by
guest vCPUs. This MSR is populated by the guest and the hypervisor
should not intercept it. A #VC exception will be generated otherwise.
If this occurs and Secure AVIC is enabled, terminate guest execution.

Reviewed-by: Tianyu Lan <tiala@microsoft.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/coco/sev/vc-handle.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/x86/coco/sev/vc-handle.c b/arch/x86/coco/sev/vc-handle.c
index fc770cc9117d..e856a5e18670 100644
--- a/arch/x86/coco/sev/vc-handle.c
+++ b/arch/x86/coco/sev/vc-handle.c
@@ -414,6 +414,15 @@ enum es_result sev_es_ghcb_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt
 		if (sev_status & MSR_AMD64_SNP_SECURE_TSC)
 			return __vc_handle_secure_tsc_msrs(regs, write);
 		break;
+	case MSR_AMD64_SECURE_AVIC_CONTROL:
+		/*
+		 * AMD64_SECURE_AVIC_CONTROL should not be intercepted when
+		 * Secure AVIC is enabled. Terminate the Secure AVIC guest
+		 * if the interception is enabled.
+		 */
+		if (cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
+			return ES_VMM_ERROR;
+		break;
 	default:
 		break;
 	}
-- 
2.34.1

[RFC PATCH v8 35/35] x86/sev: Indicate SEV-SNP guest supports Secure AVIC

[Neeraj Upadhyay]

Now that Secure AVIC support is added in the guest, indicate SEV-SNP
guest supports Secure AVIC feature if AMD_SECURE_AVIC config is
enabled.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v7:
 - No change.

 arch/x86/boot/compressed/sev.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c
index 74e083feb2d9..048d3e8839c3 100644
--- a/arch/x86/boot/compressed/sev.c
+++ b/arch/x86/boot/compressed/sev.c
@@ -238,13 +238,20 @@ bool sev_es_check_ghcb_fault(unsigned long address)
 				 MSR_AMD64_SNP_SECURE_AVIC |		\
 				 MSR_AMD64_SNP_RESERVED_MASK)
 
+#ifdef CONFIG_AMD_SECURE_AVIC
+#define SNP_FEATURE_SECURE_AVIC		MSR_AMD64_SNP_SECURE_AVIC
+#else
+#define SNP_FEATURE_SECURE_AVIC		0
+#endif
+
 /*
  * SNP_FEATURES_PRESENT is the mask of SNP features that are implemented
  * by the guest kernel. As and when a new feature is implemented in the
  * guest kernel, a corresponding bit should be added to the mask.
  */
 #define SNP_FEATURES_PRESENT	(MSR_AMD64_SNP_DEBUG_SWAP |	\
-				 MSR_AMD64_SNP_SECURE_TSC)
+				 MSR_AMD64_SNP_SECURE_TSC |	\
+				 SNP_FEATURE_SECURE_AVIC)
 
 u64 snp_get_unsupported_features(u64 status)
 {
-- 
2.34.1

Re: [RFC PATCH v8 01/35] KVM: x86: Open code setting/clearing of bits in the ISR

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> Remove __apic_test_and_set_vector() and __apic_test_and_clear_vector(),
> because the _only_ register that's safe to modify with a non-atomic
> operation is ISR, because KVM isn't running the vCPU, i.e. hardware can't
> service an IRQ or process an EOI for the relevant (virtual) APIC.
> 
> No functional change intended.
> 
> Suggested-by: Sean Christopherson <seanjc@google.com>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---

Acked-by: Sean Christopherson <seanjc@google.com>

Re: [RFC PATCH v8 04/35] KVM: x86: Rename VEC_POS/REG_POS macro usages

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> In preparation for moving most of the KVM's lapic helpers which
> use VEC_POS/REG_POS macros to common APIC header for use in Secure
> AVIC APIC driver, rename all VEC_POS/REG_POS macro usages to
> APIC_VECTOR_TO_BIT_NUMBER/APIC_VECTOR_TO_REG_OFFSET and remove
> VEC_POS/REG_POS.
> 
> While at it, clean up line wrap in find_highest_vector().
> 
> No functional change intended.
> 
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---

Acked-by: Sean Christopherson <seanjc@google.com>

Re: [RFC PATCH v8 05/35] KVM: x86: Change lapic regs base address to void pointer

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> Change APIC base address from "char *" to "void *" in KVM
> lapic's set/get helper functions. Pointer arithmetic for "void *"
> and "char *" operate identically. With "void *" there is less
> of a chance of doing the wrong thing, e.g. neglecting to cast and
> reading a byte instead of the desired APIC register size.
> 
> No functional change intended.
> 
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---

Acked-by: Sean Christopherson <seanjc@google.com>

Re: [RFC PATCH v8 06/35] KVM: x86: Rename find_highest_vector()

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> In preparation for moving kvm-internal find_highest_vector() to
> apic.h for use in Secure AVIC APIC driver, rename find_highest_vector()
> to apic_find_highest_vector() as part of the APIC API.
> 
> No functional change intended.
> 
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---

Acked-by: Sean Christopherson <seanjc@google.com>

Re: [RFC PATCH v8 07/35] KVM: x86: Rename lapic get/set_reg() helpers

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> In preparation for moving kvm-internal __kvm_lapic_set_reg(),
> __kvm_lapic_get_reg() to apic.h for use in Secure AVIC APIC driver,
> rename them as part of the APIC API.
> 
> No functional change intended.
> 
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---

Acked-by: Sean Christopherson <seanjc@google.com>

Re: [RFC PATCH v8 08/35] KVM: x86: Rename lapic get/set_reg64() helpers

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> In preparation for moving kvm-internal __kvm_lapic_set_reg64(),
> __kvm_lapic_get_reg64() to apic.h for use in Secure AVIC APIC driver,
> rename them as part of the APIC API.
> 
> No functional change intended.
> 
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---

Acked-by: Sean Christopherson <seanjc@google.com>

Re: [RFC PATCH v8 09/35] KVM: x86: Rename lapic set/clear vector helpers

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> In preparation for moving kvm-internal kvm_lapic_set_vector(),
> kvm_lapic_clear_vector() to apic.h for use in Secure AVIC APIC driver,
> rename them as part of the APIC API.
> 
> No functional change intended.
> 
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---

Acked-by: Sean Christopherson <seanjc@google.com>

Re: [RFC PATCH v8 11/35] x86/apic: KVM: Move lapic get/set helpers to common code

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> Move the apic_get_reg(), apic_set_reg(), apic_get_reg64() and
> apic_set_reg64() helper functions to apic.h in order to reuse them in the
> Secure AVIC guest APIC driver in later patches to read/write registers
> from/to the APIC backing page.
> 
> No functional change intended.
> 
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---

Acked-by: Sean Christopherson <seanjc@google.com>

Re: [RFC PATCH v8 12/35] x86/apic: KVM: Move lapic set/clear_vector() helpers to common code

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> Move apic_clear_vector() and apic_set_vector() helper functions to
> apic.h in order to reuse them in the Secure AVIC guest APIC driver
> in later patches to atomically set/clear vectors in the APIC backing
> page.
> 
> No functional change intended.
> 
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---

Acked-by: Sean Christopherson <seanjc@google.com>

Re: [RFC PATCH v8 13/35] x86/apic: KVM: Move apic_test)vector() to common code

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> Move apic_test_vector() to apic.h in order to reuse it in the Secure AVIC
> guest APIC driver in later patches to test vector state in the APIC
> backing page.
> 
> No functional change intended.
> 
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---

Acked-by: Sean Christopherson <seanjc@google.com>

Re: [RFC PATCH v8 04/35] KVM: x86: Rename VEC_POS/REG_POS macro usages

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> @@ -736,12 +735,12 @@ EXPORT_SYMBOL_GPL(kvm_apic_clear_irr);
>  
>  static void *apic_vector_to_isr(int vec, struct kvm_lapic *apic)
>  {
> -	return apic->regs + APIC_ISR + REG_POS(vec);
> +	return apic->regs + APIC_ISR + APIC_VECTOR_TO_REG_OFFSET(vec);
>  }
>  
>  static inline void apic_set_isr(int vec, struct kvm_lapic *apic)
>  {
> -	if (__test_and_set_bit(VEC_POS(vec), apic_vector_to_isr(vec, apic)))
> +	if (__test_and_set_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), apic_vector_to_isr(vec, apic)))
>  		return;
>  
>  	/*
> @@ -784,7 +783,7 @@ static inline int apic_find_highest_isr(struct kvm_lapic *apic)
>  
>  static inline void apic_clear_isr(int vec, struct kvm_lapic *apic)
>  {
> -	if (!__test_and_clear_bit(VEC_POS(vec), apic_vector_to_isr(vec, apic)))
> +	if (!__test_and_clear_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), apic_vector_to_isr(vec, apic)))
>  		return;
>  
>  	/*

Almost forgot.  I'd prefer to wrap these two, i.e.

	if (__test_and_set_bit(APIC_VECTOR_TO_BIT_NUMBER(vec),
			       apic_vector_to_isr(vec, apic)))
		return;

and

	if (!__test_and_clear_bit(APIC_VECTOR_TO_BIT_NUMBER(vec),
				  apic_vector_to_isr(vec, apic)))
		return;

Re: [RFC PATCH v8 15/35] x86/apic: Unionize apic regs for 32bit/64bit access w/o type casting

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> Define apic_page construct to unionize APIC register 32bit/64bit
> accesses and use it in apic_{get|set}*() to avoid using type
> casting.
> 
> As Secure AVIC APIC driver requires accessing APIC page at byte

No, it does not.  Literally all two callers of get_reg_bitmap(), the only user
of ->bytes, immediately cast the return to a "void *".

And you most definitely don't need a common, unionized struct to be able to reference
a byte offset, just define a "struct secure_apic_page".

> offsets (to get an APIC register's bitmap start address), support
> byte access granularity in apic_page (in addition to 32-bit and
> 64-bit accesses).
> 
> One caveat of this change is that the generated code is slighly
> larger. Below is the code generation for apic_get_reg() using
> gcc-14.2:
> 
> - Without change:
> 
> apic_get_reg:
> 
> 89 f6       mov    %esi,%esi
> 8b 04 37    mov    (%rdi,%rsi,1),%eax
> c3          ret
> 
> - With change:
> 
> apic_get_reg:
> 
> c1 ee 02    shr    $0x2,%esi
> 8b 04 b7    mov    (%rdi,%rsi,4),%eax
> c3          ret
> 
> lapic.o text size change is shown below:
> 
> Obj        Old-size      New-size
> 
> lapic.o    28800         28832
> 
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - Commit log update.
> 
>  arch/x86/include/asm/apic.h | 25 +++++++++++++++++++++----
>  1 file changed, 21 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
> index 07ba4935e873..b7cbe9ba363e 100644
> --- a/arch/x86/include/asm/apic.h
> +++ b/arch/x86/include/asm/apic.h
> @@ -525,26 +525,43 @@ static inline int apic_find_highest_vector(void *bitmap)
>  	return -1;
>  }
>  
> +struct apic_page {
> +	union {
> +		u64     regs64[PAGE_SIZE / 8];
> +		u32     regs[PAGE_SIZE / 4];
> +		u8      bytes[PAGE_SIZE];
> +	};
> +} __aligned(PAGE_SIZE);
> +
>  static inline u32 apic_get_reg(void *regs, int reg)
>  {
> -	return *((u32 *) (regs + reg));
> +	struct apic_page *ap = regs;
> +
> +	return ap->regs[reg / 4];
>  }

NAK.

I really, *really* don't like this patch.  IMO, the casting code is more "obvious"
and thus easier to follow.  And there is still casting going on, i.e. to a
"struct apic_page".

_If_ we want to go this route, then all of the open coded literals need to be
replaced with sizeof().  But I'd still very strongly prefer we not do this in
the first place.

Jumping ahead a bit, I also recommend the secure AVIC stuff name its global
varaible "secure_apic_page", because just "apic_page" could result in avoidable
collisions.

There are also a number of extraneous local variables in x2apic_savic.c, some of
which are actively dangerous.  E.g. using a local "bitmap" in savic_eoi() makes
it possible to reuse a pointer and access the wrong bitmap.

E.g.

---
 arch/x86/include/asm/apic.h         | 40 ++++--------------
 arch/x86/kernel/apic/x2apic_savic.c | 65 +++++++++++------------------
 2 files changed, 32 insertions(+), 73 deletions(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index e8a32a3eea86..a26e66d66444 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -536,67 +536,41 @@ static inline int apic_find_highest_vector(void *bitmap)
 	return -1;
 }
 
-struct apic_page {
-	union {
-		u64     regs64[PAGE_SIZE / 8];
-		u32     regs[PAGE_SIZE / 4];
-		u8      bytes[PAGE_SIZE];
-	};
-} __aligned(PAGE_SIZE);
-
 static inline u32 apic_get_reg(void *regs, int reg)
 {
-	struct apic_page *ap = regs;
-
-	return ap->regs[reg / 4];
+	return *((u32 *) (regs + reg));
 }
 
 static inline void apic_set_reg(void *regs, int reg, u32 val)
 {
-	struct apic_page *ap = regs;
-
-	ap->regs[reg / 4] = val;
+	*((u32 *) (regs + reg)) = val;
 }
 
 static __always_inline u64 apic_get_reg64(void *regs, int reg)
 {
-	struct apic_page *ap = regs;
-
 	BUILD_BUG_ON(reg != APIC_ICR);
-
-	return ap->regs64[reg / 8];
+	return *((u64 *) (regs + reg));
 }
 
 static __always_inline void apic_set_reg64(void *regs, int reg, u64 val)
 {
-	struct apic_page *ap = regs;
-
 	BUILD_BUG_ON(reg != APIC_ICR);
-	ap->regs64[reg / 8] = val;
-}
-
-static inline unsigned int get_vec_bit(unsigned int vec)
-{
-	/*
-	 * The registers are 32-bit wide and 16-byte aligned.
-	 * Compensate for the resulting bit number spacing.
-	 */
-	return vec + 96 * (vec / 32);
+	*((u64 *) (regs + reg)) = val;
 }
 
 static inline void apic_clear_vector(int vec, void *bitmap)
 {
-	clear_bit(get_vec_bit(vec), bitmap);
+	clear_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
 }
 
 static inline void apic_set_vector(int vec, void *bitmap)
 {
-	set_bit(get_vec_bit(vec), bitmap);
+	set_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
 }
 
 static inline int apic_test_vector(int vec, void *bitmap)
 {
-	return test_bit(get_vec_bit(vec), bitmap);
+	return test_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
 }
 
 /*
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index 2849f2354bf9..99d5f6104bc2 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -17,7 +17,11 @@
 
 #include "local.h"
 
-static struct apic_page __percpu *apic_page __ro_after_init;
+struct secure_apic_page {
+	u8 *regs[PAGE_SIZE];
+} __aligned(PAGE_SIZE);
+
+static struct secure_apic_page __percpu *secure_apic_page __ro_after_init;
 
 static inline void savic_wr_control_msr(u64 val)
 {
@@ -31,9 +35,7 @@ static int savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 
 static inline void *get_reg_bitmap(unsigned int cpu, unsigned int offset)
 {
-	struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
-
-	return &ap->bytes[offset];
+	return &per_cpu_ptr(secure_apic_page, cpu)->regs[offset];
 }
 
 static inline void update_vector(unsigned int cpu, unsigned int offset,
@@ -51,7 +53,7 @@ static inline void update_vector(unsigned int cpu, unsigned int offset,
 
 static u32 savic_read(u32 reg)
 {
-	struct apic_page *ap = this_cpu_ptr(apic_page);
+	void *ap = this_cpu_ptr(secure_apic_page);
 
 	/*
 	 * When Secure AVIC is enabled, rdmsr/wrmsr of APIC registers
@@ -129,14 +131,10 @@ static inline void self_ipi_reg_write(unsigned int vector)
 
 static void send_ipi_dest(unsigned int cpu, unsigned int vector, bool nmi)
 {
-	if (nmi) {
-		struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
-
-		apic_set_reg(ap, SAVIC_NMI_REQ, 1);
-		return;
-	}
-
-	update_vector(cpu, APIC_IRR, vector, true);
+	if (nmi)
+		apic_set_reg(per_cpu_ptr(secure_apic_page, cpu), SAVIC_NMI_REQ, 1);
+	else
+		update_vector(cpu, APIC_IRR, vector, true);
 }
 
 static void send_ipi_allbut(unsigned int vector, bool nmi)
@@ -166,7 +164,6 @@ static inline void self_ipi(unsigned int vector, bool nmi)
 
 static void savic_icr_write(u32 icr_low, u32 icr_high)
 {
-	struct apic_page *ap = this_cpu_ptr(apic_page);
 	unsigned int dsh, vector;
 	u64 icr_data;
 	bool nmi;
@@ -193,12 +190,12 @@ static void savic_icr_write(u32 icr_low, u32 icr_high)
 	icr_data = ((u64)icr_high) << 32 | icr_low;
 	if (dsh != APIC_DEST_SELF)
 		savic_ghcb_msr_write(APIC_ICR, icr_data);
-	apic_set_reg64(ap, APIC_ICR, icr_data);
+	apic_set_reg64(this_cpu_ptr(secure_apic_page), APIC_ICR, icr_data);
 }
 
 static void savic_write(u32 reg, u32 data)
 {
-	struct apic_page *ap = this_cpu_ptr(apic_page);
+	struct secure_apic_page *ap = this_cpu_ptr(secure_apic_page);
 
 	switch (reg) {
 	case APIC_LVTT:
@@ -303,19 +300,15 @@ static void savic_update_vector(unsigned int cpu, unsigned int vector, bool set)
 static void savic_eoi(void)
 {
 	unsigned int cpu;
-	void *bitmap;
 	int vec;
 
 	cpu = raw_smp_processor_id();
-	bitmap = get_reg_bitmap(cpu, APIC_ISR);
-	vec = apic_find_highest_vector(bitmap);
+	vec = apic_find_highest_vector(get_reg_bitmap(cpu, APIC_ISR));
 	if (WARN_ONCE(vec == -1, "EOI write while no active interrupt in APIC_ISR"))
 		return;
 
-	bitmap = get_reg_bitmap(cpu, APIC_TMR);
-
 	/* Is level-triggered interrupt? */
-	if (apic_test_vector(vec, bitmap)) {
+	if (apic_test_vector(vec, get_reg_bitmap(cpu, APIC_TMR))) {
 		update_vector(cpu, APIC_ISR, vec, false);
 		/*
 		 * Propagate the EOI write to hv for level-triggered interrupts.
@@ -333,18 +326,6 @@ static void savic_eoi(void)
 	}
 }
 
-static void init_apic_page(struct apic_page *ap)
-{
-	u32 apic_id;
-
-	/*
-	 * Before Secure AVIC is enabled, APIC msr reads are intercepted.
-	 * APIC_ID msr read returns the value from the Hypervisor.
-	 */
-	apic_id = native_apic_msr_read(APIC_ID);
-	apic_set_reg(ap, APIC_ID, apic_id);
-}
-
 static void savic_teardown(void)
 {
 	/* Disable Secure AVIC */
@@ -354,13 +335,17 @@ static void savic_teardown(void)
 
 static void savic_setup(void)
 {
-	void *backing_page;
+	struct secure_apic_page *ap = this_cpu_ptr(secure_apic_page);
 	enum es_result res;
 	unsigned long gpa;
 
-	backing_page = this_cpu_ptr(apic_page);
-	init_apic_page(backing_page);
-	gpa = __pa(backing_page);
+	/*
+	 * Before Secure AVIC is enabled, APIC msr reads are intercepted.
+	 * APIC_ID msr read returns the value from the Hypervisor.
+	 */
+	apic_set_reg(ap, APIC_ID, native_apic_msr_read(APIC_ID));
+
+	gpa = __pa(ap);
 
 	/*
 	 * The NPT entry for a vCPU's APIC backing page must always be
@@ -389,8 +374,8 @@ static int savic_probe(void)
 		/* unreachable */
 	}
 
-	apic_page = alloc_percpu(struct apic_page);
-	if (!apic_page)
+	secure_apic_page = alloc_percpu(struct secure_apic_page);
+	if (!secure_apic_page)
 		snp_abort();
 
 	return 1;

base-commit: 620bd94fb00da8482556057cea765656b8263b71
--

Re: [RFC PATCH v8 16/35] x86/apic: Simplify bitwise operations on APIC bitmap

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> Use 'regs' as a contiguous linear bitmap for bitwise operations in
> apic_{set|clear|test}_vector(). This makes the code simpler by eliminating

That's very debatable.  I don't find this code to be any simpler.  Quite the
opposite; it adds yet another open coded math exercise, which is so "simple"
that it warrants its own comment to explain what it's doing.

I'm not dead set against this, but I'd strongly prefer to drop this patch.

> diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
> index b7cbe9ba363e..f91d23757375 100644
> --- a/arch/x86/include/asm/apic.h
> +++ b/arch/x86/include/asm/apic.h
> @@ -564,19 +564,28 @@ static __always_inline void apic_set_reg64(void *regs, int reg, u64 val)
>  	ap->regs64[reg / 8] = val;
>  }
>  
> +static inline unsigned int get_vec_bit(unsigned int vec)
> +{
> +	/*
> +	 * The registers are 32-bit wide and 16-byte aligned.
> +	 * Compensate for the resulting bit number spacing.
> +	 */
> +	return vec + 96 * (vec / 32);
> +}
> +
>  static inline void apic_clear_vector(int vec, void *bitmap)
>  {
> -	clear_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
> +	clear_bit(get_vec_bit(vec), bitmap);
>  }
>  
>  static inline void apic_set_vector(int vec, void *bitmap)
>  {
> -	set_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
> +	set_bit(get_vec_bit(vec), bitmap);
>  }
>  
>  static inline int apic_test_vector(int vec, void *bitmap)
>  {
> -	return test_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), bitmap + APIC_VECTOR_TO_REG_OFFSET(vec));
> +	return test_bit(get_vec_bit(vec), bitmap);
>  }
>  
>  /*
> -- 
> 2.34.1
> 

Re: [RFC PATCH v8 00/35] AMD: Add Secure AVIC Guest Support

[Sean Christopherson]

On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> Kishon Vijay Abraham I (2):
>   x86/sev: Initialize VGIF for secondary VCPUs for Secure AVIC
>   x86/sev: Enable NMI support for Secure AVIC
> 
> Neeraj Upadhyay (32):
>   KVM: x86: Open code setting/clearing of bits in the ISR
>   KVM: x86: Remove redundant parentheses around 'bitmap'
>   KVM: x86: Rename VEC_POS/REG_POS macro usages
>   KVM: x86: Change lapic regs base address to void pointer
>   KVM: x86: Rename find_highest_vector()
>   KVM: x86: Rename lapic get/set_reg() helpers
>   KVM: x86: Rename lapic get/set_reg64() helpers
>   KVM: x86: Rename lapic set/clear vector helpers
>   x86/apic: KVM: Move apic_find_highest_vector() to a common header
>   x86/apic: KVM: Move lapic get/set helpers to common code
>   x86/apic: KVM: Move lapic set/clear_vector() helpers to common code
>   x86/apic: KVM: Move apic_test)vector() to common code
>   x86/apic: Rename 'reg_off' to 'reg'
>   x86/apic: Unionize apic regs for 32bit/64bit access w/o type casting
>   x86/apic: Simplify bitwise operations on apic bitmap
>   x86/apic: Move apic_update_irq_cfg() calls to apic_update_vector()
>   x86/apic: Add new driver for Secure AVIC
>   x86/apic: Initialize Secure AVIC APIC backing page
>   x86/apic: Populate .read()/.write() callbacks of Secure AVIC driver
>   x86/apic: Initialize APIC ID for Secure AVIC
>   x86/apic: Add update_vector() callback for apic drivers
>   x86/apic: Add update_vector() callback for Secure AVIC
>   x86/apic: Add support to send IPI for Secure AVIC
>   x86/apic: Support LAPIC timer for Secure AVIC
>   x86/apic: Add support to send NMI IPI for Secure AVIC
>   x86/apic: Allow NMI to be injected from hypervisor for Secure AVIC
>   x86/apic: Read and write LVT* APIC registers from HV for SAVIC guests
>   x86/apic: Handle EOI writes for Secure AVIC guests
>   x86/apic: Add kexec support for Secure AVIC
>   x86/apic: Enable Secure AVIC in Control MSR
>   x86/sev: Prevent SECURE_AVIC_CONTROL MSR interception for Secure AVIC
>     guests
>   x86/sev: Indicate SEV-SNP guest supports Secure AVIC
> 
> Sean Christopherson (1):
>   x86/apic: KVM: Deduplicate APIC vector => register+bit math

Boris, do you anticipate taking this entire series for 6.17?  If not, I'd be more
than happy to grab all of the KVM => x86/apic renames and code movement for 6.17,
e.g. to avoid complications if a conflicting KVM change comes along.  I can throw
them in a dedicated topic branch so that you could ingest the dependency prior to
6.17-rc1 if necessary.

I.e. these:

  x86/apic: Rename 'reg_off' to 'reg'
  x86/apic: KVM: Move apic_test)vector() to common code
  x86/apic: KVM: Move lapic set/clear_vector() helpers to common code
  x86/apic: KVM: Move lapic get/set helpers to common code
  x86/apic: KVM: Move apic_find_highest_vector() to a common header
  KVM: x86: Rename lapic set/clear vector helpers
  KVM: x86: Rename lapic get/set_reg64() helpers
  KVM: x86: Rename lapic get/set_reg() helpers
  KVM: x86: Rename find_highest_vector()
  KVM: x86: Change lapic regs base address to void pointer
  KVM: x86: Rename VEC_POS/REG_POS macro usages
  x86/apic: KVM: Deduplicate APIC vector => register+bit math
  KVM: x86: Remove redundant parentheses around 'bitmap'
  KVM: x86: Open code setting/clearing of bits in the ISR

Re: [RFC PATCH v8 00/35] AMD: Add Secure AVIC Guest Support

[Borislav Petkov]

On Wed, Jul 09, 2025 at 07:41:30AM -0700, Sean Christopherson wrote:
> Boris, do you anticipate taking this entire series for 6.17?  If not, I'd be more
> than happy to grab all of the KVM => x86/apic renames and code movement for 6.17,
> e.g. to avoid complications if a conflicting KVM change comes along.  I can throw
> them in a dedicated topic branch so that you could ingest the dependency prior to
> 6.17-rc1 if necessary.
 
> I.e. these:

Yah, I'd feel much more at ease if you took the KVM cleanups so that the
patchset is slimmed down and then we cat concentrate on reviewing the
remaining pile. I haven't gone through it, I know tglx did look at this and
with vacations upcoming we might not be ready for the merge window...

I can see how far I can get but you could give me that topic branch just in
case and I'll see what I can stick ontop if/when I get to it.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [RFC PATCH v8 04/35] KVM: x86: Rename VEC_POS/REG_POS macro usages

[Neeraj Upadhyay]

On 7/9/2025 7:39 PM, Sean Christopherson wrote:
> On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
>> @@ -736,12 +735,12 @@ EXPORT_SYMBOL_GPL(kvm_apic_clear_irr);
>>  
>>  static void *apic_vector_to_isr(int vec, struct kvm_lapic *apic)
>>  {
>> -	return apic->regs + APIC_ISR + REG_POS(vec);
>> +	return apic->regs + APIC_ISR + APIC_VECTOR_TO_REG_OFFSET(vec);
>>  }
>>  
>>  static inline void apic_set_isr(int vec, struct kvm_lapic *apic)
>>  {
>> -	if (__test_and_set_bit(VEC_POS(vec), apic_vector_to_isr(vec, apic)))
>> +	if (__test_and_set_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), apic_vector_to_isr(vec, apic)))
>>  		return;
>>  
>>  	/*
>> @@ -784,7 +783,7 @@ static inline int apic_find_highest_isr(struct kvm_lapic *apic)
>>  
>>  static inline void apic_clear_isr(int vec, struct kvm_lapic *apic)
>>  {
>> -	if (!__test_and_clear_bit(VEC_POS(vec), apic_vector_to_isr(vec, apic)))
>> +	if (!__test_and_clear_bit(APIC_VECTOR_TO_BIT_NUMBER(vec), apic_vector_to_isr(vec, apic)))
>>  		return;
>>  
>>  	/*
> 
> Almost forgot.  I'd prefer to wrap these two, i.e.
> 
> 	if (__test_and_set_bit(APIC_VECTOR_TO_BIT_NUMBER(vec),
> 			       apic_vector_to_isr(vec, apic)))
> 		return;
> 
> and
> 
> 	if (!__test_and_clear_bit(APIC_VECTOR_TO_BIT_NUMBER(vec),
> 				  apic_vector_to_isr(vec, apic)))
> 		return;

Ok. I have updated it for next version here:

https://github.com/AMDESE/linux-kvm/commits/savic-guest-latest
commit 862ee49


- Neeraj

Re: [RFC PATCH v8 15/35] x86/apic: Unionize apic regs for 32bit/64bit access w/o type casting

[Neeraj Upadhyay]

> 
> NAK.
> 
> I really, *really* don't like this patch.  IMO, the casting code is more "obvious"
> and thus easier to follow.  And there is still casting going on, i.e. to a
> "struct apic_page".
> 
> _If_ we want to go this route, then all of the open coded literals need to be
> replaced with sizeof().  But I'd still very strongly prefer we not do this in
> the first place.
> 
> Jumping ahead a bit, I also recommend the secure AVIC stuff name its global
> varaible "secure_apic_page", because just "apic_page" could result in avoidable
> collisions.
> 
> There are also a number of extraneous local variables in x2apic_savic.c, some of
> which are actively dangerous.  E.g. using a local "bitmap" in savic_eoi() makes
> it possible to reuse a pointer and access the wrong bitmap.
> 

Thanks for the reviews, inputs and suggested cleanups! I have addressed them for v9 at

https://github.com/AMDESE/linux-kvm/commits/savic-guest-latest

I have changed

    struct secure_apic_page {
	u8 *regs[PAGE_SIZE];
    } __aligned(PAGE_SIZE);


to

    struct secure_apic_page {
	u8 regs[PAGE_SIZE];
    } __aligned(PAGE_SIZE);


... and changed 

    struct secure_apic_page *ap = this_cpu_ptr(secure_apic_page);

to 

    void *ap = this_cpu_ptr(secure_apic_page);

in savic_write(), savic_setup()


- Neeraj

Re: [RFC PATCH v8 00/35] AMD: Add Secure AVIC Guest Support

[Sean Christopherson]

On Wed, 09 Jul 2025 09:02:07 +0530, Neeraj Upadhyay wrote:
> Introduction
> ------------
> 
> Secure AVIC is a new hardware feature in the AMD64 architecture to
> allow SEV-SNP guests to prevent the hypervisor from generating
> unexpected interrupts to a vCPU or otherwise violate architectural
> assumptions around APIC behavior.
> 
> [...]

Applied the KVM refactorings and code movement to kvm-x86 apic.

Tip tree folks, please holler if you object to any of these patches, i.e. if
you want to bikeshed some names. :-)  I've thrown these in a dedicated topic
branch, but I'll hold off on creating an "official" stable tag for a few days
to try to avoid having to carry fixups (hopefully none are needed).

[01/35] KVM: x86: Open code setting/clearing of bits in the ISR
        https://github.com/kvm-x86/linux/commit/ac48017020a5
[02/35] KVM: x86: Remove redundant parentheses around 'bitmap'
        https://github.com/kvm-x86/linux/commit/3fb7b83e2a72
[03/35] x86/apic: KVM: Deduplicate APIC vector => register+bit math
        https://github.com/kvm-x86/linux/commit/dc98e3bd494b
[04/35] KVM: x86: Rename VEC_POS/REG_POS macro usages
        https://github.com/kvm-x86/linux/commit/9cbb5fd156d7
[05/35] KVM: x86: Change lapic regs base address to void pointer
        https://github.com/kvm-x86/linux/commit/e2fa7905b293
[06/35] KVM: x86: Rename find_highest_vector()
        https://github.com/kvm-x86/linux/commit/bdaccfe4e517
[07/35] KVM: x86: Rename lapic get/set_reg() helpers
        https://github.com/kvm-x86/linux/commit/b9bd231913cf
[08/35] KVM: x86: Rename lapic get/set_reg64() helpers
        https://github.com/kvm-x86/linux/commit/9c23bc4fec2b
[09/35] KVM: x86: Rename lapic set/clear vector helpers
        https://github.com/kvm-x86/linux/commit/b5f8980f29ce
[10/35] x86/apic: KVM: Move apic_find_highest_vector() to a common header
        https://github.com/kvm-x86/linux/commit/39e81633f65e
[11/35] x86/apic: KVM: Move lapic get/set helpers to common code
        https://github.com/kvm-x86/linux/commit/3d3a9083da1e
[12/35] x86/apic: KVM: Move lapic set/clear_vector() helpers to common code
        https://github.com/kvm-x86/linux/commit/fe954bcd577e
[13/35] x86/apic: KVM: Move apic_test)vector() to common code
        https://github.com/kvm-x86/linux/commit/17776e6c203b
[14/35] x86/apic: Rename 'reg_off' to 'reg'
        https://github.com/kvm-x86/linux/commit/b95a9d313642

--
https://github.com/kvm-x86/linux/tree/next

Re: [RFC PATCH v8 15/35] x86/apic: Unionize apic regs for 32bit/64bit access w/o type casting

[Borislav Petkov]

On Thu, Jul 10, 2025 at 09:13:11AM +0530, Neeraj Upadhyay wrote:
>     struct secure_apic_page {
> 	u8 *regs[PAGE_SIZE];
>     } __aligned(PAGE_SIZE);
> 
> 
> to
> 
>     struct secure_apic_page {

secure_apic_page or secure_aVic_page?

I mean, what is a secure APIC?

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [RFC PATCH v8 15/35] x86/apic: Unionize apic regs for 32bit/64bit access w/o type casting

[Neeraj Upadhyay]

On 7/12/2025 8:51 PM, Borislav Petkov wrote:
> On Thu, Jul 10, 2025 at 09:13:11AM +0530, Neeraj Upadhyay wrote:
>>     struct secure_apic_page {
>> 	u8 *regs[PAGE_SIZE];
>>     } __aligned(PAGE_SIZE);
>>
>>
>> to
>>
>>     struct secure_apic_page {
> 
> secure_apic_page or secure_aVic_page?
>

In v8, the struct was named "apic_page". Sean's suggested to use "secure_apic_page"
to avoid name conflicts with other apic code. APM calls it "guest APIC backing page" -
guest-owned page containing APIC state.
 
> I mean, what is a secure APIC?
> 

It was more to imply like secure APIC-page rather than Secure-APIC page. I will change
it to secure_avic_page or savic_apic_page, if one of these looks cleaner. Please suggest.


- Neeraj

Re: [RFC PATCH v8 15/35] x86/apic: Unionize apic regs for 32bit/64bit access w/o type casting

[Borislav Petkov]

On Sat, Jul 12, 2025 at 10:38:08PM +0530, Neeraj Upadhyay wrote:
> It was more to imply like secure APIC-page rather than Secure-APIC page. I will change
> it to secure_avic_page or savic_apic_page, if one of these looks cleaner. Please suggest.

If the page belongs to the guest's secure AVIC machinery then it should be
called secure_avic_page to avoid confusion. Or at least have a comment above
it explaining what it is.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [RFC PATCH v8 15/35] x86/apic: Unionize apic regs for 32bit/64bit access w/o type casting

[Neeraj Upadhyay]

On 7/13/2025 12:16 AM, Borislav Petkov wrote:
> On Sat, Jul 12, 2025 at 10:38:08PM +0530, Neeraj Upadhyay wrote:
>> It was more to imply like secure APIC-page rather than Secure-APIC page. I will change
>> it to secure_avic_page or savic_apic_page, if one of these looks cleaner. Please suggest.
> 
> If the page belongs to the guest's secure AVIC machinery then it should be
> called secure_avic_page to avoid confusion. Or at least have a comment above
> it explaining what it is.
> 

Ok. I will change this to secure_avic_page in next version. Thanks for taking a look at
it during weekend.


- Neeraj

Re: [RFC PATCH v8 16/35] x86/apic: Simplify bitwise operations on APIC bitmap

[Borislav Petkov]

On Wed, Jul 09, 2025 at 07:35:41AM -0700, Sean Christopherson wrote:
> On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
> > Use 'regs' as a contiguous linear bitmap for bitwise operations in
> > apic_{set|clear|test}_vector(). This makes the code simpler by eliminating
> 
> That's very debatable.  I don't find this code to be any simpler.  Quite the
> opposite; it adds yet another open coded math exercise, which is so "simple"
> that it warrants its own comment to explain what it's doing.
> 
> I'm not dead set against this, but I'd strongly prefer to drop this patch.

> > +static inline unsigned int get_vec_bit(unsigned int vec)
> > +{
> > +	/*
> > +	 * The registers are 32-bit wide and 16-byte aligned.
> > +	 * Compensate for the resulting bit number spacing.
> > +	 */
> > +	return vec + 96 * (vec / 32);

I kinda agree. The naked 96 doesn't tell me anything. If we do this, the
explaination of what this thing does should be crystal clear, perhaps even
with an example. And the naked numbers need to be defines with proper names.

Also:

>     This change results in slight increase in generated code size for
>     gcc-14.2.
>     
>     - Without change

What is the asm supposed to tell me?

The new change gets a LEA which is noticeable or so?

The generated code size increase is, what, a couple of bytes? Who cares?

We add asm to commit messages when it is really important. Doesn't seem so to
me here but maybe I'm missing an angle...

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [RFC PATCH v8 16/35] x86/apic: Simplify bitwise operations on APIC bitmap

[Neeraj Upadhyay]

On 7/14/2025 4:22 PM, Borislav Petkov wrote:
> On Wed, Jul 09, 2025 at 07:35:41AM -0700, Sean Christopherson wrote:
>> On Wed, Jul 09, 2025, Neeraj Upadhyay wrote:
>>> Use 'regs' as a contiguous linear bitmap for bitwise operations in
>>> apic_{set|clear|test}_vector(). This makes the code simpler by eliminating
>>
>> That's very debatable.  I don't find this code to be any simpler.  Quite the
>> opposite; it adds yet another open coded math exercise, which is so "simple"
>> that it warrants its own comment to explain what it's doing.
>>
>> I'm not dead set against this, but I'd strongly prefer to drop this patch.
> 
>>> +static inline unsigned int get_vec_bit(unsigned int vec)
>>> +{
>>> +	/*
>>> +	 * The registers are 32-bit wide and 16-byte aligned.
>>> +	 * Compensate for the resulting bit number spacing.
>>> +	 */
>>> +	return vec + 96 * (vec / 32);
> 
> I kinda agree. The naked 96 doesn't tell me anything. If we do this, the
> explaination of what this thing does should be crystal clear, perhaps even
> with an example. And the naked numbers need to be defines with proper names.
> 

Ok. I have removed this change from the current series.

https://github.com/AMDESE/linux-kvm/commits/savic-guest-latest/

> Also:
> 
>>     This change results in slight increase in generated code size for
>>     gcc-14.2.
>>     
>>     - Without change
> 
> What is the asm supposed to tell me?
> 

Intent was to show that the functional impact (perf/code-size) is not
noticeable.

> The new change gets a LEA which is noticeable or so?
> 

No, not noticeable.

> The generated code size increase is, what, a couple of bytes? Who cares?
> 
> We add asm to commit messages when it is really important. Doesn't seem so to
> me here but maybe I'm missing an angle...
> 

Ok. This commit was aimed at simplifying (which folks find debatable) the usage
of bitmap ops and to match how bitmap operations are typically used in other code.
The intent of adding asm was to show that functional impact is low (while
providing "simplicity").


- Neeraj

> Thx.
> 

Re: [RFC PATCH v8 15/35] x86/apic: Unionize apic regs for 32bit/64bit access w/o type casting

[Sean Christopherson]

On Sat, Jul 12, 2025, Borislav Petkov wrote:
> On Sat, Jul 12, 2025 at 10:38:08PM +0530, Neeraj Upadhyay wrote:
> > It was more to imply like secure APIC-page rather than Secure-APIC page. I will change
> > it to secure_avic_page or savic_apic_page, if one of these looks cleaner. Please suggest.
> 
> If the page belongs to the guest's secure AVIC machinery then it should be
> called secure_avic_page to avoid confusion. Or at least have a comment above
> it explaining what it is.

secure_avic_page works for me.  I have no real opinion on the name, I suggested
prepending "secure" purely to avoid creating a too-generic "struct apic_page".

Re: [RFC PATCH v8 19/35] x86/apic: Initialize Secure AVIC APIC backing page

[Tianyu Lan]

On Wed, Jul 9, 2025 at 11:40 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> With Secure AVIC, the APIC backing page is owned and managed by guest.
> Allocate and initialize APIC backing page for all guest CPUs.
>
> The NPT entry for a vCPU's APIC backing page must always be present
> when the vCPU is running, in order for Secure AVIC to function. A
> VMEXIT_BUSY is returned on VMRUN and the vCPU cannot be resumed if
> the NPT entry for the APIC backing page is not present. To handle this,
> notify GPA of the vCPU's APIC backing page to the hypervisor by using the
> SVM_VMGEXIT_SECURE_AVIC GHCB protocol event. Before executing VMRUN,
> the hypervisor makes use of this information to make sure the APIC backing
> page is mapped in NPT.
>
> Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.

Reviewed-by: Tianyu Lan <tiala@microsoft.com>

-- 
Thanks
Tianyu Lan

Re: [RFC PATCH v8 20/35] x86/apic: Populate .read()/.write() callbacks of Secure AVIC driver

[Tianyu Lan]

On Wed, Jul 9, 2025 at 11:40 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> Add read() and write() APIC callback functions to read and write x2APIC
> registers directly from the guest APIC backing page of a vCPU.
>
> The x2APIC registers are mapped at an offset within the guest APIC
> backing page which is same as their x2APIC MMIO offset. Secure AVIC
> adds new registers such as ALLOWED_IRRs (which are at 4-byte offset
> within the IRR register offset range) and NMI_REQ to the APIC register
> space.
>
> When Secure AVIC is enabled, guest's rdmsr/wrmsr of APIC registers
> result in VC exception (for non-accelerated register accesses) with
> error code VMEXIT_AVIC_NOACCEL. The VC exception handler can read/write
> the x2APIC register in the guest APIC backing page to complete the
> rdmsr/wrmsr. Since doing this would increase the latency of accessing
> x2APIC registers, instead of doing rdmsr/wrmsr based reg accesses
> and handling reads/writes in VC exception, directly read/write APIC
> registers from/to the guest APIC backing page of the vCPU in read()
> and write() callbacks of the Secure AVIC APIC driver.
>
> Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.

Reviewed-by: Tianyu Lan <tiala@microsoft.com>

Re: [RFC PATCH v8 21/35] x86/apic: Initialize APIC ID for Secure AVIC

[Tianyu Lan]

On Wed, Jul 9, 2025 at 11:40 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> Initialize the APIC ID in the Secure AVIC APIC backing page with
> the APIC_ID msr value read from Hypervisor. CPU topology evaluation
> later during boot would catch and report any duplicate APIC ID for
> two CPUs.
>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.
>

Reviewed-by: Tianyu Lan <tiala@microsoft.com>

>  arch/x86/kernel/apic/x2apic_savic.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
>
> diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
> index 186e69a5e169..618643e7242f 100644
> --- a/arch/x86/kernel/apic/x2apic_savic.c
> +++ b/arch/x86/kernel/apic/x2apic_savic.c
> @@ -131,6 +131,18 @@ static void savic_write(u32 reg, u32 data)
>         }
>  }
>
> +static void init_apic_page(struct apic_page *ap)
> +{
> +       u32 apic_id;
> +
> +       /*
> +        * Before Secure AVIC is enabled, APIC msr reads are intercepted.
> +        * APIC_ID msr read returns the value from the Hypervisor.
> +        */
> +       apic_id = native_apic_msr_read(APIC_ID);
> +       apic_set_reg(ap, APIC_ID, apic_id);
> +}
> +
>  static void savic_setup(void)
>  {
>         void *backing_page;
> @@ -138,6 +150,7 @@ static void savic_setup(void)
>         unsigned long gpa;
>
>         backing_page = this_cpu_ptr(apic_page);
> +       init_apic_page(backing_page);
>         gpa = __pa(backing_page);
>
>         /*
> --
> 2.34.1
>
>


-- 
Thanks
Tianyu Lan

Re: [RFC PATCH v8 23/35] x86/apic: Add update_vector() callback for Secure AVIC

[Tianyu Lan]

On Wed, Jul 9, 2025 at 11:44 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> Add update_vector() callback to set/clear ALLOWED_IRR field in
> a vCPU's APIC backing page for vectors which are emulated by the
> hypervisor.
>
> The ALLOWED_IRR field indicates the interrupt vectors which the
> guest allows the hypervisor to inject (typically for emulated devices).
> Interrupt vectors used exclusively by the guest itself and the vectors
> which are not emulated by the hypervisor, such as IPI vectors, should
> not be set by the guest in the ALLOWED_IRR fields.
>
> As clearing/setting state of a vector will also be used in subsequent
> commits for other APIC regs (such as APIC_IRR update for sending IPI),
> add a common update_vector() in Secure AVIC driver.
>
> Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.

Reviewed-by: Tianyu Lan <tiala@microsoft.com>
-- 
Thanks
Tianyu Lan

[tip: x86/cleanups] x86/apic: Move apic_update_irq_cfg() call to apic_update_vector()

[tip-bot2 for Neeraj Upadhyay]

The following commit has been merged into the x86/cleanups branch of tip:

Commit-ID:     7f2b41ac3f29f682cde113f1d0b4b43d261902fe
Gitweb:        https://git.kernel.org/tip/7f2b41ac3f29f682cde113f1d0b4b43d261902fe
Author:        Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
AuthorDate:    Wed, 09 Jul 2025 09:02:24 +05:30
Committer:     Borislav Petkov (AMD) <bp@alien8.de>
CommitterDate: Tue, 15 Jul 2025 11:54:09 +02:00

x86/apic: Move apic_update_irq_cfg() call to apic_update_vector()

All callers of apic_update_vector() also call apic_update_irq_cfg() after it.
So, move the apic_update_irq_cfg() call to apic_update_vector().

No functional change intended.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250709033242.267892-18-Neeraj.Upadhyay@amd.com
---
 arch/x86/kernel/apic/vector.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
index 93069b1..a947b46 100644
--- a/arch/x86/kernel/apic/vector.c
+++ b/arch/x86/kernel/apic/vector.c
@@ -183,6 +183,7 @@ setnew:
 	apicd->cpu = newcpu;
 	BUG_ON(!IS_ERR_OR_NULL(per_cpu(vector_irq, newcpu)[newvec]));
 	per_cpu(vector_irq, newcpu)[newvec] = desc;
+	apic_update_irq_cfg(irqd, newvec, newcpu);
 }
 
 static void vector_assign_managed_shutdown(struct irq_data *irqd)
@@ -261,7 +262,6 @@ assign_vector_locked(struct irq_data *irqd, const struct cpumask *dest)
 	if (vector < 0)
 		return vector;
 	apic_update_vector(irqd, vector, cpu);
-	apic_update_irq_cfg(irqd, vector, cpu);
 
 	return 0;
 }
@@ -338,7 +338,7 @@ assign_managed_vector(struct irq_data *irqd, const struct cpumask *dest)
 	if (vector < 0)
 		return vector;
 	apic_update_vector(irqd, vector, cpu);
-	apic_update_irq_cfg(irqd, vector, cpu);
+
 	return 0;
 }
 

Re: [RFC PATCH v8 24/35] x86/apic: Add support to send IPI for Secure AVIC

[Tianyu Lan]

On Wed, Jul 9, 2025 at 11:42 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> With Secure AVIC only Self-IPI is accelerated. To handle all the
> other IPIs, add new callbacks for sending IPI. These callbacks write
> to the IRR of the target guest vCPU's APIC backing page and issue
> GHCB protocol MSR write event for the hypervisor to notify the
> target vCPU about the new interrupt request.
>
> For Secure AVIC GHCB APIC MSR writes, reuse GHCB msr handling code in
> vc_handle_msr() by exposing a sev-internal sev_es_ghcb_handle_msr().
>
> Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.

Reviewed-by: Tianyu Lan <tiala@microsoft.com>
-- 
Thanks
Tianyu Lan

Re: [RFC PATCH v8 25/35] x86/apic: Support LAPIC timer for Secure AVIC

[Tianyu Lan]

On Wed, Jul 9, 2025 at 11:42 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> Secure AVIC requires LAPIC timer to be emulated by the hypervisor.
> KVM already supports emulating LAPIC timer using hrtimers. In order
> to emulate LAPIC timer, APIC_LVTT, APIC_TMICT and APIC_TDCR register
> values need to be propagated to the hypervisor for arming the timer.
> APIC_TMCCT register value has to be read from the hypervisor, which
> is required for calibrating the APIC timer. So, read/write all APIC
> timer registers from/to the hypervisor.
>
> Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.

Reviewed-by: Tianyu Lan <tiala@microsoft.com>
--
Thanks
Tianyu Lan

Re: [RFC PATCH v8 26/35] x86/sev: Initialize VGIF for secondary VCPUs for Secure AVIC

[Tianyu Lan]

On Wed, Jul 9, 2025 at 11:42 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> From: Kishon Vijay Abraham I <kvijayab@amd.com>
>
> Secure AVIC requires VGIF to be configured in VMSA. Configure
> for secondary CPUs (the configuration for boot CPU is done by
> the hypervisor).
>
> Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.


Reviewed-by: Tianyu Lan <tiala@microsoft.com>
--
Thanks
Tianyu Lan

Re: [RFC PATCH v8 27/35] x86/apic: Add support to send NMI IPI for Secure AVIC

[Tianyu Lan]

On Wed, Jul 9, 2025 at 11:45 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> Secure AVIC has introduced a new field in the APIC backing page
> "NmiReq" that has to be set by the guest to request a NMI IPI
> through APIC_ICR write.
>
> Add support to set NmiReq appropriately to send NMI IPI.
>
> Sending NMI IPI also requires Virtual NMI feature to be enabled
> in VINTRL_CTRL field in the VMSA. However, this would be added by
> a later commit after adding support for injecting NMI from the
> hypervisor.
>
> Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.
Reviewed-by: Tianyu Lan <tiala@microsoft.com>
--
Thanks
Tianyu Lan

Re: [RFC PATCH v8 28/35] x86/apic: Allow NMI to be injected from hypervisor for Secure AVIC

[Tianyu Lan]

On Wed, Jul 9, 2025 at 11:43 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> Secure AVIC requires "AllowedNmi" bit in the Secure AVIC Control MSR
> to be set for NMI to be injected from hypervisor. Set "AllowedNmi"
> bit in Secure AVIC Control MSR to allow NMI interrupts to be injected
> from hypervisor.
>
> Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.

Reviewed-by: Tianyu Lan <tiala@microsoft.com>

Re: [RFC PATCH v8 29/35] x86/sev: Enable NMI support for Secure AVIC

[Tianyu Lan]

On Wed, Jul 9, 2025 at 11:43 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> From: Kishon Vijay Abraham I <kvijayab@amd.com>
>
> Now that support to send NMI IPI and support to inject NMI from
> the hypervisor has been added, set V_NMI_ENABLE in VINTR_CTRL
> field of VMSA to enable NMI for Secure AVIC guests.
>
> Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.

Reviewed-by: Tianyu Lan <tiala@microsoft.com>

--
Thanks
Tianyu Lan

Re: [RFC PATCH v8 30/35] x86/apic: Read and write LVT* APIC registers from HV for SAVIC guests

[Tianyu Lan]

On Wed, Jul 9, 2025 at 11:43 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> Hypervisor need information about the current state of LVT registers
> for device emulation and NMI. So, forward reads and write of these
> registers to the hypervisor for Secure AVIC enabled guests.
>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.

Reviewed-by: Tianyu Lan <tiala@microsoft.com>

--
Thanks
Tianyu Lan

Re: [RFC PATCH v8 31/35] x86/apic: Handle EOI writes for Secure AVIC guests

[Tianyu Lan]

On Wed, Jul 9, 2025 at 11:44 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> Secure AVIC accelerates guest's EOI msr writes for edge-triggered
> interrupts.
>
> For level-triggered interrupts, EOI msr writes trigger VC exception
> with SVM_EXIT_AVIC_UNACCELERATED_ACCESS error code. To complete EOI
> handling, the VC exception handler would need to trigger a GHCB protocol
> MSR write event to notify the hypervisor about completion of the
> level-triggered interrupt. Hypervisor notification is required for
> cases like emulated IOAPIC, to complete and clear interrupt in the
> IOAPIC's interrupt state.
>
> However, VC exception handling adds extra performance overhead for
> APIC register writes. In addition, for Secure AVIC, some unaccelerated
> APIC register msr writes are trapped, whereas others are faulted. This
> results in additional complexity in VC exception handling for unacclerated
> APIC msr accesses. So, directly do a GHCB protocol based APIC EOI msr write
> from apic->eoi() callback for level-triggered interrupts.
>
> Use wrmsr for edge-triggered interrupts, so that hardware re-evaluates
> any pending interrupt which can be delivered to guest vCPU. For level-
> triggered interrupts, re-evaluation happens on return from VMGEXIT
> corresponding to the GHCB event for APIC EOI msr write.
>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.

Reviewed-by: Tianyu Lan <tiala@microsoft.com>

-- 
Thanks
Tianyu Lan

Re: [RFC PATCH v8 33/35] x86/apic: Enable Secure AVIC in Control MSR

[Tianyu Lan]

[-- Attachment #1: Type: text/plain, Size: 633 bytes --]

On Wed, Jul 9, 2025 at 11:45 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> With all the pieces in place now, enable Secure AVIC in Secure
> AVIC Control MSR. Any access to x2APIC MSRs are emulated by
> the hypervisor before Secure AVIC is enabled in the control MSR.
> Post Secure AVIC enablement, all x2APIC MSR accesses (whether
> accelerated by AVIC hardware or trapped as VC exception) operate
> on vCPU's APIC backing page.
>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.

Reviewed-by: Tianyu Lan <tiala@microsoft.com>

-- 
Thanks
Tianyu Lan

[-- Attachment #2: image.png --]
[-- Type: image/png, Size: 1136562 bytes --]

Re: [RFC PATCH v8 35/35] x86/sev: Indicate SEV-SNP guest supports Secure AVIC

[Tianyu Lan]

On Wed, Jul 9, 2025 at 11:45 AM Neeraj Upadhyay <Neeraj.Upadhyay@amd.com> wrote:
>
> Now that Secure AVIC support is added in the guest, indicate SEV-SNP
> guest supports Secure AVIC feature if AMD_SECURE_AVIC config is
> enabled.
>
> Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
> Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
> ---
> Changes since v7:
>  - No change.

Reviewed-by: Tianyu Lan <tiala@microsoft.com>
-- 
Thanks
Tianyu Lan