From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-il1-f202.google.com (mail-il1-f202.google.com [209.85.166.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B6662918FF for ; Wed, 14 May 2025 19:22:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.202 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747250530; cv=none; b=aV7RjS6EURkuGzVqHV/wsBlR3JVvM4BnNJnUZtbyHbIOudEvMn6vI/aV1y/i/bD/UIuqybKBZvyDd8YTaAqJp29wbPcNVNqvA9aapzF1mAIyT+dHcxAI+6CgKIfgJRomxNx6hwLPZoCcVAOMUZDssnqDwKvnMWAZlOmB9vd9J9s= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747250530; c=relaxed/simple; bh=lOdxofayiKe/1Vjymgyk3XUfVBPe4mBc2wD0vdOwXzo=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=IumjUo+YMV1dtNFBYvXuqhDXZavb2Y9jbakp3mHgA519yUNOds9NFEOdXaRvrZDipqlAYEGLol0qiakNV/8+OfAok+1l7AVjpRv8lek1nx7lZDPAEpiM8qnTC7hctySYxEesMVM7++G+XNERof8/r6NwHwXxTHbSVnBO6MAv/nQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--rananta.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=WBYcxldx; arc=none smtp.client-ip=209.85.166.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--rananta.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="WBYcxldx" Received: by mail-il1-f202.google.com with SMTP id e9e14a558f8ab-3da7584d395so2523715ab.3 for ; Wed, 14 May 2025 12:22:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747250526; x=1747855326; darn=lists.linux.dev; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=W3MauCW+urlix1RrycVNh0XN4aZXJqzxAZQYVw1ClTM=; b=WBYcxldxOMNUpge11+qpQHtQXYb5ke7DuMra2WCfPhcElruArcSuh1FuK170TFb+9s OqcSlTdtvWUWcraPeKle1dt5fwkn7gBf9D+suQv9rKNmMXRkoIA178qTgfx8pHj4cNGD lDPN/qKZjxAttqisW8eG+gHNaR1W/UoPsnv0Edo+awUdK+wDzjSSRolVoZHJTKlinG6l uhLD1Nr4g6UIEZDEA0I7oZX1HK8yqcWvq0pqbOBmPofmDzYi3ZEmuRK9kjfIOOLfQIxc aLFtlmiY0Sdv7xqWu8Tk1mfPizOC0SKDtOGfONpIdyd40UEJ4NdqG/f6/QvocJpxauuP Ferg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747250526; x=1747855326; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=W3MauCW+urlix1RrycVNh0XN4aZXJqzxAZQYVw1ClTM=; b=wLO1WdB2qmgNZ7yF82w6+4pX71COryjdnHxNPkx5utuc82YhA2U4tx3TWpm8o2dTe8 NSChHfv+s9bvww7JMulthY1zMa5hSDrWzzpgJL3mjQJvdL4lxWQ+/t8MDMq8w2pxbHXw wydR597QGR7YKarVErUMC4Mb5orBN3bSfS2HO5xr8b4VruoflhmNEf0uivsvlQegx00t zNi6hV3EwA+kTDFjgRFcg6yMQzDEo84OZtCz+o8liCcXeROa9+54TLIJVvQf9oiWv45h 8SyoQbkRl+qn8+x5+x8z4uY17OptGzkljbrcTDozv+bps198HgEEBJa+NsO2sXn8aWPJ b8Ug== X-Forwarded-Encrypted: i=1; AJvYcCX/0sUiq1d6Qs6rip0hOwVhO7KprPUT6qJWq289ScVxrBlx2QBCHKP87pAqwhgnQrcT6k14pMk=@lists.linux.dev X-Gm-Message-State: AOJu0YwXZgNw6AGe0JkjBfpS4LfF1Ln08UjiE6IFFpBO9+xu7pBYB0uP yeRfLMG4L3k5PNzaz6cOmgUtQrDn4EiG7ds/AbCWzXMq+uS9PNr9YB6GfiK6gVEs8hUV3uUHT+o 7RoW3Jg== X-Google-Smtp-Source: AGHT+IEy09eku3jD263yCQYKQ2rr3/0cu+Cf3UzUC6jmBnOz4zk6Mpi7GLuEdzeaZ4KU1bn0EY+ScGzWk8y2 X-Received: from ilsx3.prod.google.com ([2002:a05:6e02:743:b0:3d9:367f:729a]) (user=rananta job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6e02:3784:b0:3d9:5d50:e3b1 with SMTP id e9e14a558f8ab-3db6f7f1597mr60756865ab.18.1747250526441; Wed, 14 May 2025 12:22:06 -0700 (PDT) Date: Wed, 14 May 2025 19:21:57 +0000 In-Reply-To: <20250514192159.1751538-1-rananta@google.com> Precedence: bulk X-Mailing-List: kvmarm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250514192159.1751538-1-rananta@google.com> X-Mailer: git-send-email 2.49.0.1101.gccaa498523-goog Message-ID: <20250514192159.1751538-2-rananta@google.com> Subject: [PATCH 1/3] kvm: arm64: Add support for KVM_DEV_ARM_VGIC_CONFIG_GICV4 attr From: Raghavendra Rao Ananta To: Oliver Upton , Marc Zyngier Cc: Raghavendra Rao Anata , Mingwei Zhang , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Content-Type: text/plain; charset="UTF-8" When kvm-arm.vgic_v4_enable=1, KVM adds support for direct interrupt injection by default to all the VMs in the system, aka GICv4. A shortcoming of the GIC architecture is that there's an absolute limit on the number of vPEs that can be tracked by the ITS. It is possible that an operator is running a mix of VMs on a system, only wanting to provide a specific class of VMs with hardware interrupt injection support. To support this, introduce a GIC attribute, KVM_DEV_ARM_VGIC_CONFIG_GICV4, for the userspace to enable or disable vGICv4 for a given VM. Make the interface backward compatible by leaving vGICv4 enabled by default. Suggested-by: Oliver Upton Signed-off-by: Raghavendra Rao Ananta --- arch/arm64/include/uapi/asm/kvm.h | 7 +++++ arch/arm64/kvm/vgic/vgic-init.c | 3 +++ arch/arm64/kvm/vgic/vgic-its.c | 2 +- arch/arm64/kvm/vgic/vgic-kvm-device.c | 39 +++++++++++++++++++++++++++ arch/arm64/kvm/vgic/vgic-mmio-v3.c | 12 ++++----- arch/arm64/kvm/vgic/vgic-v3.c | 16 +++++++++-- arch/arm64/kvm/vgic/vgic-v4.c | 8 +++--- include/kvm/arm_vgic.h | 5 ++++ 8 files changed, 79 insertions(+), 13 deletions(-) diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index af9d9acaf997..6762683f7e0f 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -428,6 +428,13 @@ enum { #define KVM_DEV_ARM_ITS_RESTORE_TABLES 2 #define KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES 3 #define KVM_DEV_ARM_ITS_CTRL_RESET 4 +#define KVM_DEV_ARM_VGIC_CONFIG_GICV4 5 + +enum { + KVM_DEV_ARM_VGIC_CONFIG_GICV4_UNAVAILABLE = 0, + KVM_DEV_ARM_VGIC_CONFIG_GICV4_DISABLE, + KVM_DEV_ARM_VGIC_CONFIG_GICV4_ENABLE, +}; /* Device Control API on vcpu fd */ #define KVM_ARM_VCPU_PMU_V3_CTRL 0 diff --git a/arch/arm64/kvm/vgic/vgic-init.c b/arch/arm64/kvm/vgic/vgic-init.c index 1f33e71c2a73..cd345df2271f 100644 --- a/arch/arm64/kvm/vgic/vgic-init.c +++ b/arch/arm64/kvm/vgic/vgic-init.c @@ -132,6 +132,9 @@ int kvm_vgic_create(struct kvm *kvm, u32 type) kvm->arch.vgic.in_kernel = true; kvm->arch.vgic.vgic_model = type; + kvm->arch.vgic.gicv4_config = kvm_vgic_global_state.has_gicv4 ? + KVM_DEV_ARM_VGIC_CONFIG_GICV4_ENABLE : + KVM_DEV_ARM_VGIC_CONFIG_GICV4_UNAVAILABLE; kvm->arch.vgic.vgic_dist_base = VGIC_ADDR_UNDEF; diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index fb96802799c6..bba635e4e851 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -2242,7 +2242,7 @@ static int vgic_its_save_itt(struct vgic_its *its, struct its_device *device) * have direct access to that state without GICv4.1. * Let's simply fail the save operation... */ - if (ite->irq->hw && !kvm_vgic_global_state.has_gicv4_1) + if (ite->irq->hw && !kvm_vm_has_gicv4_1(its->dev->kvm)) return -EACCES; ret = vgic_its_save_ite(its, device, ite, gpa); diff --git a/arch/arm64/kvm/vgic/vgic-kvm-device.c b/arch/arm64/kvm/vgic/vgic-kvm-device.c index 359094f68c23..f03b80fc816e 100644 --- a/arch/arm64/kvm/vgic/vgic-kvm-device.c +++ b/arch/arm64/kvm/vgic/vgic-kvm-device.c @@ -279,6 +279,33 @@ static int vgic_set_common_attr(struct kvm_device *dev, unlock_all_vcpus(dev->kvm); mutex_unlock(&dev->kvm->lock); return r; + case KVM_DEV_ARM_VGIC_CONFIG_GICV4: { + u8 __user *uaddr = (u8 __user *)(long)attr->addr; + u8 val; + + if (!kvm_vgic_global_state.has_gicv4) + return -ENXIO; + + if (get_user(val, uaddr)) + return -EFAULT; + + if (vgic_initialized(dev->kvm) && + val != dev->kvm->arch.vgic.gicv4_config) + return -EBUSY; + + switch (val) { + case KVM_DEV_ARM_VGIC_CONFIG_GICV4_ENABLE: + case KVM_DEV_ARM_VGIC_CONFIG_GICV4_DISABLE: + mutex_lock(&dev->kvm->arch.config_lock); + dev->kvm->arch.vgic.gicv4_config = val; + mutex_unlock(&dev->kvm->arch.config_lock); + break; + default: + return -EINVAL; + } + + return 0; + } } break; } @@ -309,6 +336,16 @@ static int vgic_get_common_attr(struct kvm_device *dev, r = put_user(dev->kvm->arch.vgic.mi_intid, uaddr); break; } + case KVM_DEV_ARM_VGIC_GRP_CTRL: { + switch (attr->attr) { + case KVM_DEV_ARM_VGIC_CONFIG_GICV4: { + u8 __user *uaddr = (u8 __user *)(long)attr->addr; + + r = put_user(dev->kvm->arch.vgic.gicv4_config, uaddr); + break; + } + } + } } return r; @@ -684,6 +721,8 @@ static int vgic_v3_has_attr(struct kvm_device *dev, return 0; case KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES: return 0; + case KVM_DEV_ARM_VGIC_CONFIG_GICV4: + return 0; } } return -ENXIO; diff --git a/arch/arm64/kvm/vgic/vgic-mmio-v3.c b/arch/arm64/kvm/vgic/vgic-mmio-v3.c index ae4c0593d114..66b365f59c51 100644 --- a/arch/arm64/kvm/vgic/vgic-mmio-v3.c +++ b/arch/arm64/kvm/vgic/vgic-mmio-v3.c @@ -50,8 +50,8 @@ bool vgic_has_its(struct kvm *kvm) bool vgic_supports_direct_msis(struct kvm *kvm) { - return (kvm_vgic_global_state.has_gicv4_1 || - (kvm_vgic_global_state.has_gicv4 && vgic_has_its(kvm))); + return kvm_vm_has_gicv4(kvm) && + (kvm_vgic_global_state.has_gicv4_1 || vgic_has_its(kvm)); } /* @@ -86,7 +86,7 @@ static unsigned long vgic_mmio_read_v3_misc(struct kvm_vcpu *vcpu, } break; case GICD_TYPER2: - if (kvm_vgic_global_state.has_gicv4_1 && gic_cpuif_has_vsgi()) + if (kvm_vm_has_gicv4_1(vcpu->kvm) && gic_cpuif_has_vsgi()) value = GICD_TYPER2_nASSGIcap; break; case GICD_IIDR: @@ -119,7 +119,7 @@ static void vgic_mmio_write_v3_misc(struct kvm_vcpu *vcpu, dist->enabled = val & GICD_CTLR_ENABLE_SS_G1; /* Not a GICv4.1? No HW SGIs */ - if (!kvm_vgic_global_state.has_gicv4_1 || !gic_cpuif_has_vsgi()) + if (!kvm_vm_has_gicv4_1(vcpu->kvm) || !gic_cpuif_has_vsgi()) val &= ~GICD_CTLR_nASSGIreq; /* Dist stays enabled? nASSGIreq is RO */ @@ -133,7 +133,7 @@ static void vgic_mmio_write_v3_misc(struct kvm_vcpu *vcpu, if (is_hwsgi != dist->nassgireq) vgic_v4_configure_vsgis(vcpu->kvm); - if (kvm_vgic_global_state.has_gicv4_1 && + if (kvm_vm_has_gicv4_1(vcpu->kvm) && was_enabled != dist->enabled) kvm_make_all_cpus_request(vcpu->kvm, KVM_REQ_RELOAD_GICv4); else if (!was_enabled && dist->enabled) @@ -178,7 +178,7 @@ static int vgic_mmio_uaccess_write_v3_misc(struct kvm_vcpu *vcpu, } case GICD_CTLR: /* Not a GICv4.1? No HW SGIs */ - if (!kvm_vgic_global_state.has_gicv4_1) + if (!kvm_vm_has_gicv4_1(vcpu->kvm)) val &= ~GICD_CTLR_nASSGIreq; dist->enabled = val & GICD_CTLR_ENABLE_SS_G1; diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c index b9ad7c42c5b0..bc8cb9184be9 100644 --- a/arch/arm64/kvm/vgic/vgic-v3.c +++ b/arch/arm64/kvm/vgic/vgic-v3.c @@ -20,6 +20,18 @@ static bool common_trap; static bool dir_trap; static bool gicv4_enable; +int kvm_vm_has_gicv4(struct kvm *kvm) +{ + return kvm->arch.vgic.gicv4_config == + KVM_DEV_ARM_VGIC_CONFIG_GICV4_ENABLE; +} + +int kvm_vm_has_gicv4_1(struct kvm *kvm) +{ + return (kvm_vm_has_gicv4(kvm) && + kvm_vgic_global_state.has_gicv4_1); +} + void vgic_v3_set_underflow(struct kvm_vcpu *vcpu) { struct vgic_v3_cpu_if *cpuif = &vcpu->arch.vgic_cpu.vgic_v3; @@ -404,7 +416,7 @@ int vgic_v3_save_pending_tables(struct kvm *kvm) * The above vgic initialized check also ensures that the allocation * and enabling of the doorbells have already been done. */ - if (kvm_vgic_global_state.has_gicv4_1) { + if (kvm_vm_has_gicv4_1(kvm)) { unmap_all_vpes(kvm); vlpi_avail = true; } @@ -581,7 +593,7 @@ int vgic_v3_map_resources(struct kvm *kvm) return -EBUSY; } - if (kvm_vgic_global_state.has_gicv4_1) + if (kvm_vm_has_gicv4_1(kvm)) vgic_v4_configure_vsgis(kvm); return 0; diff --git a/arch/arm64/kvm/vgic/vgic-v4.c b/arch/arm64/kvm/vgic/vgic-v4.c index c7de6154627c..814d54f4ce13 100644 --- a/arch/arm64/kvm/vgic/vgic-v4.c +++ b/arch/arm64/kvm/vgic/vgic-v4.c @@ -86,7 +86,7 @@ static irqreturn_t vgic_v4_doorbell_handler(int irq, void *info) struct kvm_vcpu *vcpu = info; /* We got the message, no need to fire again */ - if (!kvm_vgic_global_state.has_gicv4_1 && + if (!kvm_vm_has_gicv4_1(vcpu->kvm) && !irqd_irq_disabled(&irq_to_desc(irq)->irq_data)) disable_irq_nosync(irq); @@ -245,7 +245,7 @@ int vgic_v4_init(struct kvm *kvm) lockdep_assert_held(&kvm->arch.config_lock); - if (!kvm_vgic_global_state.has_gicv4) + if (!kvm_vm_has_gicv4(kvm)) return 0; /* Nothing to see here... move along. */ if (dist->its_vm.vpes) @@ -286,7 +286,7 @@ int vgic_v4_init(struct kvm *kvm) * On GICv4.1, the doorbell is managed in HW and must * be left enabled. */ - if (kvm_vgic_global_state.has_gicv4_1) + if (kvm_vm_has_gicv4_1(kvm)) irq_flags &= ~IRQ_NOAUTOEN; irq_set_status_flags(irq, irq_flags); @@ -392,7 +392,7 @@ int vgic_v4_load(struct kvm_vcpu *vcpu) * doorbell interrupt that would still be pending. This is a * GICv4.0 only "feature"... */ - if (!kvm_vgic_global_state.has_gicv4_1) + if (!kvm_vm_has_gicv4_1(vcpu->kvm)) err = irq_set_irqchip_state(vpe->irq, IRQCHIP_STATE_PENDING, false); return err; diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index 714cef854c1c..8883dc677674 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -296,6 +296,8 @@ struct vgic_dist { * else. */ struct its_vm its_vm; + + u8 gicv4_config; }; struct vgic_v2_cpu_if { @@ -447,4 +449,7 @@ bool vgic_state_is_nested(struct kvm_vcpu *vcpu); void kvm_vgic_cpu_up(void); void kvm_vgic_cpu_down(void); +int kvm_vm_has_gicv4(struct kvm *kvm); +int kvm_vm_has_gicv4_1(struct kvm *kvm); + #endif /* __KVM_ARM_VGIC_H */ -- 2.49.0.1101.gccaa498523-goog