Re: KVM CPU hotplug notifier triggers BUG_ON on arm64

From: Oliver Upton <oliver.upton@linux.dev>
To: Marc Zyngier <maz@kernel.org>
Cc: Kristina Martsenko <kristina.martsenko@arm.com>,
	isaku.yamahata@intel.com, seanjc@google.com, pbonzini@redhat.com,
	kvmarm@lists.linux.dev, kvm@vger.kernel.org,
	linux-arm-kernel@lists.infradead.org,
	James Morse <james.morse@arm.com>
Subject: Re: KVM CPU hotplug notifier triggers BUG_ON on arm64
Date: Mon, 3 Jul 2023 09:02:30 -0700	[thread overview]
Message-ID: <ZKLxFveKKvoQs5RV@thinky-boi> (raw)
In-Reply-To: <867crhxr9l.wl-maz@kernel.org>

Hey Marc,

On Mon, Jul 03, 2023 at 10:45:26AM +0100, Marc Zyngier wrote:
> On Sat, 01 Jul 2023 18:42:28 +0100, Oliver Upton <oliver.upton@linux.dev> wrote:
> > Well, one way to hack around the problem would be to just cram
> > preempt_{disable,enable}() into kvm_arch_hardware_disable(), but that's
> > kinda gross in the context of cpuhp which isn't migratable in the first
> > place. Let me have a look...

Heh, I should've mentioned I'm on holiday until Thursday.

> An alternative would be to replace the preemptible() checks with a one
> that looks at the migration state, but I'm not sure that's much better
> (it certainly looks more costly).
> 
> There is also the fact that most of our per-CPU accessors are already
> using preemption disabling, and this code has a bunch of them. So I'm
> not sure there is a lot to be gained from not disabling preemption
> upfront.
> 
> Anyway, as I was able to reproduce the issue under NV, I tested the
> hack below. If anything, I expect it to be a reasonable fix for
> 6.3/6.4, and until we come up with a better approach.

Yeah, I'm fine with a hack like this. Do you want to send this out as a
patch?

--
Thanks,
Oliver

> diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c
> index aaeae1145359..a28c4ffe4932 100644
> --- a/arch/arm64/kvm/arm.c
> +++ b/arch/arm64/kvm/arm.c
> @@ -1894,8 +1894,17 @@ static void _kvm_arch_hardware_enable(void *discard)
>  
>  int kvm_arch_hardware_enable(void)
>  {
> -	int was_enabled = __this_cpu_read(kvm_arm_hardware_enabled);
> +	int was_enabled;
>  
> +	/*
> +	 * Most calls to this function are made with migration
> +	 * disabled, but not with preemption disabled. The former is
> +	 * enough to ensure correctness, but most of the helpers
> +	 * expect the later and will throw a tantrum otherwise.
> +	 */
> +	preempt_disable();
> +
> +	was_enabled = __this_cpu_read(kvm_arm_hardware_enabled);
>  	_kvm_arch_hardware_enable(NULL);
>  
>  	if (!was_enabled) {
> @@ -1903,6 +1912,8 @@ int kvm_arch_hardware_enable(void)
>  		kvm_timer_cpu_up();
>  	}
>  
> +	preempt_enable();
> +
>  	return 0;
>  }
>  
> 
> 
> 
> -- 
> Without deviation from the norm, progress is not possible.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel