From mboxrd@z Thu Jan  1 00:00:00 1970
From: Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [RFC PATCH] VFIO: Add a parameter to force nonthread IRQ
Date: Wed, 28 Oct 2015 01:44:55 +0100
Message-ID: <56301A87.9030907@redhat.com>
References: <1445908801-14732-1-git-send-email-yunhong.jiang@linux.intel.com>
 <1445917034.8018.220.camel@redhat.com>
 <20151027063501.GA22054@jnakajim-build> <562F43F8.1040101@redhat.com>
 <20151027212648.GA22916@jnakajim-build>
Mime-Version: 1.0
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 8bit
Cc: Alex Williamson <alex.williamson@redhat.com>, kvm@vger.kernel.org,
	linux-kernel@vger.kernel.org
To: Yunhong Jiang <yunhong.jiang@linux.intel.com>
Return-path: <linux-kernel-owner@vger.kernel.org>
In-Reply-To: <20151027212648.GA22916@jnakajim-build>
Sender: linux-kernel-owner@vger.kernel.org
List-Id: kvm.vger.kernel.org



On 27/10/2015 22:26, Yunhong Jiang wrote:
>> > On RT kernels however can you call eventfd_signal from interrupt
>> > context?  You cannot call spin_lock_irqsave (which can sleep) from a
>> > non-threaded interrupt handler, can you?  You would need a raw spin lock.
> Thanks for pointing this out. Yes, we can't call spin_lock_irqsave on RT 
> kernel. Will do this way on next patch. But not sure if it's overkill to use 
> raw_spinlock there since the eventfd_signal is used by other caller also.

No, I don't think you can use raw_spinlock there.  The problem is not
just eventfd_signal, it is especially wake_up_locked_poll.  You cannot
convert the whole workqueue infrastructure to use raw_spinlock.

Alex, would it make sense to use the IRQ bypass infrastructure always,
not just for VT-d, to do the MSI injection directly from the VFIO
interrupt handler and bypass the eventfd?  Basically this would add an
RCU-protected list of consumers matching the token to struct
irq_bypass_producer, and a

	int (*inject)(struct irq_bypass_consumer *);

callback to struct irq_bypass_consumer.  If any callback returns true,
the eventfd is not signaled.  The KVM implementation would be like this
(compare with virt/kvm/eventfd.c):

	/* Extracted out of irqfd_wakeup */
	static int
	irqfd_wakeup_pollin(struct kvm_kernel_irqfd *irqfd)
	{
		...
	}

	/* Extracted out of irqfd_wakeup */
	static int
	irqfd_wakeup_pollhup(struct kvm_kernel_irqfd *irqfd)
	{
		...
	}

	static int
	irqfd_wakeup(wait_queue_t *wait, unsigned mode, int sync,
		     void *key)
	{
	        struct _irqfd *irqfd = container_of(wait,
			struct _irqfd, wait);
	        unsigned long flags = (unsigned long)key;

		if (flags & POLLIN)
			irqfd_wakeup_pollin(irqfd);
		if (flags & POLLHUP)
			irqfd_wakeup_pollhup(irqfd);

		return 0;
	}

	static int kvm_arch_irq_bypass_inject(
		struct irq_bypass_consumer *cons)
	{
		struct kvm_kernel_irqfd *irqfd =
			container_of(cons, struct kvm_kernel_irqfd,
				     consumer);	

		irqfd_wakeup_pollin(irqfd);
	}

Or do you think it would be a hack?  The latency improvement might
actually be even better than what Yunhong is already reporting.

Paolo