Re: [PATCH v2 2/4] KVM: x86: move sev_lock/unlock_vcpus_for_migration to kvm_main.c

[Peter Zijlstra <peterz@infradead.org>]

On Wed, Apr 16, 2025 at 07:48:00PM +0200, Paolo Bonzini wrote:
> On 4/10/25 10:16, Peter Zijlstra wrote:
> > On Tue, Apr 08, 2025 at 09:41:34PM -0400, Maxim Levitsky wrote:
> > > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> > > index 69782df3617f..71c0d8c35b4b 100644
> > > --- a/virt/kvm/kvm_main.c
> > > +++ b/virt/kvm/kvm_main.c
> > > @@ -1368,6 +1368,77 @@ static int kvm_vm_release(struct inode *inode, struct file *filp)
> > >   	return 0;
> > >   }
> > > +
> > > +/*
> > > + * Lock all VM vCPUs.
> > > + * Can be used nested (to lock vCPUS of two VMs for example)
> > > + */
> > > +int kvm_lock_all_vcpus_nested(struct kvm *kvm, bool trylock, unsigned int role)
> > > +{
> > > +	struct kvm_vcpu *vcpu;
> > > +	unsigned long i, j;
> > > +
> > > +	lockdep_assert_held(&kvm->lock);
> > > +
> > > +	kvm_for_each_vcpu(i, vcpu, kvm) {
> > > +
> > > +		if (trylock && !mutex_trylock_nested(&vcpu->mutex, role))
> > > +			goto out_unlock;
> > > +		else if (!trylock && mutex_lock_killable_nested(&vcpu->mutex, role))
> > > +			goto out_unlock;
> > > +
> > > +#ifdef CONFIG_PROVE_LOCKING
> > > +		if (!i)
> > > +			/*
> > > +			 * Reset the role to one that avoids colliding with
> > > +			 * the role used for the first vcpu mutex.
> > > +			 */
> > > +			role = MAX_LOCK_DEPTH - 1;
> > > +		else
> > > +			mutex_release(&vcpu->mutex.dep_map, _THIS_IP_);
> > > +#endif
> > > +	}
> > 
> > This code is all sorts of terrible.
> > 
> > Per the lockdep_assert_held() above, you serialize all these locks by
> > holding that lock, this means you can be using the _nest_lock()
> > annotation.
> > 
> > Also, the original code didn't have this trylock nonsense, and the
> > Changelog doesn't mention this -- in fact the Changelog claims no
> > change, which is patently false.
> > 
> > Anyway, please write like:
> > 
> > 	kvm_for_each_vcpu(i, vcpu, kvm) {
> > 		if (mutex_lock_killable_nest_lock(&vcpu->mutex, &kvm->lock))
> > 			goto unlock;
> > 	}
> > 
> > 	return 0;
> > 
> > unlock:
> > 
> > 	kvm_for_each_vcpu(j, vcpu, kvm) {
> > 		if (j == i)
> > 			break;
> > 
> > 		mutex_unlock(&vcpu->mutex);
> > 	}
> > 	return -EINTR;
> > 
> > And yes, you'll have to add mutex_lock_killable_nest_lock(), but that
> > should be trivial.
> 
> If I understand correctly, that would be actually
> _mutex_lock_killable_nest_lock() plus a wrapper macro.  But yes,
> that is easy so it sounds good.
> 
> For the ARM case, which is the actual buggy one (it was complaining
> about too high a depth) it still needs mutex_trylock_nest_lock();
> the nest_lock is needed to avoid bumping the depth on every
> mutex_trylock().

Got a link to the ARM code in question ? And I'm assuming you're talking
about task_struct::lockdep_depth ? The nest lock annotation does not
in fact increment depth beyond one of each type. It does a refcount like
thing.