Re: [PATCH v3 2/3] uio_pci_generic: add MSI/MSI-X support

[Gleb Natapov <gleb@scylladb.com>]

On Thu, Oct 08, 2015 at 08:39:10PM +0300, Michael S. Tsirkin wrote:
> On Thu, Oct 08, 2015 at 08:01:21PM +0300, Gleb Natapov wrote:
> > On Thu, Oct 08, 2015 at 07:43:04PM +0300, Michael S. Tsirkin wrote:
> > > On Thu, Oct 08, 2015 at 04:28:34PM +0300, Gleb Natapov wrote:
> > > > On Thu, Oct 08, 2015 at 04:20:04PM +0300, Michael S. Tsirkin wrote:
> > > > > On Thu, Oct 08, 2015 at 03:27:37PM +0300, Gleb Natapov wrote:
> > > > > > On Thu, Oct 08, 2015 at 03:06:07PM +0300, Michael S. Tsirkin wrote:
> > > > > > > On Thu, Oct 08, 2015 at 12:44:09PM +0300, Avi Kivity wrote:
> > > > > > > > 
> > > > > > > > 
> > > > > > > > On 10/08/2015 12:16 PM, Michael S. Tsirkin wrote:
> > > > > > > > >On Thu, Oct 08, 2015 at 11:46:30AM +0300, Avi Kivity wrote:
> > > > > > > > >>
> > > > > > > > >>On 10/08/2015 10:32 AM, Michael S. Tsirkin wrote:
> > > > > > > > >>>On Thu, Oct 08, 2015 at 08:33:45AM +0300, Avi Kivity wrote:
> > > > > > > > >>>>It is good practice to defend against root oopsing the kernel, but in some
> > > > > > > > >>>>cases it cannot be achieved.
> > > > > > > > >>>Absolutely. That's one of the issues with these patches. They don't even
> > > > > > > > >>>try where it's absolutely possible.
> > > > > > > > >>>
> > > > > > > > >>Are you referring to blocking the maps of the msix BAR areas?
> > > > > > > > >For example. There are more. I listed some of the issues on the mailing
> > > > > > > > >list, and I might have missed some.  VFIO has code to address all this,
> > > > > > > > >people should share code to avoid duplication, or at least read it
> > > > > > > > >to understand the issues.
> > > > > > > > 
> > > > > > > > All but one of those are unrelated to the patch that adds msix support.
> > > > > > > 
> > > > > > > They are related because msix support enables bus mastering.  Without it
> > > > > > > device is passive and can't harm anyone. With it, suddently you need to
> > > > > > > be very careful with the device to avoid corrupting kernel memory.
> > > > > > > 
> > > > > > Most (if not all) uio_pci_generic users enable pci bus mastering. The
> > > > > > fact that they do that without even tainting the kernel like the patch
> > > > > > does make current situation much worse that with the patch.
> > > > > 
> > > > > It isn't worse. It's a sane interface. Whoever enables bus mastering
> > > > > must be careful.  If userspace enables bus mastering then userspace
> > > > > needs to be very careful with the device to avoid corrupting kernel
> > > > > memory.  If kernel does it, it's kernel's responsibility.
> > > > > 
> > > > Although this definition of sanity sounds strange to me, but lets
> > > > flow with it for the sake of this email: would it be OK if proposed
> > > > interface refused to work if bus mastering is not already enabled by
> > > > userspace?
> > > 
> > > An interface could be acceptable if there's a fallback where it
> > > works without BM but slower (e.g. poll pending bits).
> > > 
> > OK.
> > 
> > > But not the proposed one.
> > >
> > Why? Greg is against ioctl interface so it will be reworked, by besides
> > that what is wrong with the concept of binding msi-x interrupt to
> > eventfd?
> 
> It's not the binding. Managing msi-x just needs more than the puny
> 2 ioctls to get # of vectors and set eventfd.
> 
> It interacts in strange ways with reset, and with PM, and ...
> 
Sorry, I need examples of what you mean. DMA also "interacts in strange
ways with reset, and with PM, and ..." and it does not have any special
handling anywhere in uio-generic. So what special properties msi-x posses
which are not part of a dma. We already agreed that if enabling of bus
mastering is done by userspace all the responsibilities pertaining to
it are also lay in userspace.

> > > Really, there's more to making msi-x work with
> > > userspace drivers than this patch. As I keep telling people, you would
> > > basically reimplement vfio/pci. Go over it, and see for yourself.
> > > Almost everything it does is relevant for msi-x.  It's just wrong to
> > > duplicate so much code.
> > > 
> > The patch is tested and works with msi-x. Restricting access to msi-x
> > registers that vfio does is not relevant here.
> 
> It works *for you* with a specific userspace application. I have no idea
> how you tested it, and what does the userspace in question do.  But it
> seems pretty clear that there are a ton of very reasonable things that
> one can do with a device and that break when you enable MSI-X.
> 
I do not follow. What things break when you enable MSI-X and why would
you enable MSI-X if things that previously worked breaks for you.
Look we cannot work with such vague statements, please be more specific
about issues that needs to be addressed. So far I got two:

 1. kernel should not enable pci bust mastering, leave it to userspace
to do before configuring msi-x
 2. if bus mustering is disabled then poll for interrupts

--
			Gleb.