Re: [v4 2/3] vfio_register_notifier: also register on the group notifier

[Alex Williamson <alex.williamson@redhat.com>]

On Thu, 17 Nov 2016 13:24:59 +0800
Jike Song <jike.song@intel.com> wrote:

> On 11/17/2016 03:45 AM, Alex Williamson wrote:
> > Perhaps calling it a filter is not correct, I was thinking that a
> > vendor driver would register the notifier with a set of required
> > events.  The driver would need to handle/ignore additional events
> > outside of the required mask.  There are certainly some complications
> > to this model though that I hadn't thought all the way through until
> > now.  For instance what if we add event XYZ in the future and the
> > vendor driver adds this to their required mask.  If we run that on an
> > older kernel where the vfio infrastructure doesn't know about that
> > event, the vendor driver needs to fail, or at least know that event is
> > not supported and retry with a set of supported events.
> > 
> > There's another problem with my proposal too, we can't put a single
> > notifier_block on multiple notifier_block heads, that just doesn't
> > work.  So we probably need to separate a group notifier from an iommu
> > notifier, the vendor driver will need to register for each.
> > 
> > Maybe we end up with something like:
> > 
> > int vfio_register_notifier(struct device *dev,
> > 			   vfio_notify_type_t type,
> > 			   unsigned long *required_events,
> > 			   struct notifier_block *nb);
> > 
> > typedef unsigned short vfio_notify_type_t;
> > enum vfio_notify_type {
> > 	VFIO_IOMMU_NOTIFY = (__force vfio_notify_type_t)0,
> > 	VFIO_GROUP_NOTIFY = (__force vfio_notify_type_t)1,
> > };
> > 
> > (stealing this from pci_dev_flags_t, hope it works)
> > 
> > A VFIO_GROUP_NOTIFY would add the notifier_block to the vfio_group, a
> > VFIO_IOMMU_NOTIFY would add it to vfio_iommu.  Each would have their
> > own unique set of events and each would compare their supported events
> > to the required events by the caller.  Supported events would be
> > cleared from the callers required_events arg.  If required_events still
> > has bits set, the notifier_block is not registered, an error is
> > returned, and the caller can identify the unsupported events by the
> > remaining bits in the required_events arg.  Can that work?  Thanks,  
> 
> Let me summarize the discussion:
> 
> - There should be 2 notifier heads, one in vfio_iommu another in vfio_group;
> - vfio_{un}register_notifier() has the type specified in parameter
> - In vfio_register_notifier, maybe:
> 
> 	static vfio_iommu_register_notifier() {..}
> 	static vfio_group_register_notifier() {..}
> 	int vfio_register_notififer(type..
> 	{
> 		if (type == VFIO_IOMMU_NOTIFY)
> 			return vfio_iommu_register_notifier();
> 		if (type == VFIO_GROUP_NOTIFY)
> 			return vfio_group_register_notifier();
> 	}
> 
> 
> 
> What's more, if we still want registration to be done in mdev framework,
> we should change parent_ops:
> 
> - rename 'notifier' to 'iommu_notifier'
> - add "group_notifier"
> - Add a group_events and a iommu_events to indicate what events vendor is
>   interested in, respectively
> 
> or otherwise don't touch it from mdev framework at all?

I think we should remove the notifier from the mdev framework and have
the vendor drivers call vfio_{un}register_notifier() directly.  Note:

 - vfio_group_release() should be modified to remove any notifier
   blocks remaining to prevent a stale call chain for the next user.

 - vfio_sanity_check_pfn_list() should be modified to WARN_ON remaining
   notifier blocks on the vfio_iommu (ie. vendor drivers will need to
   actively remove iommu notifiers since the vfio_iommu can persist
   beyond the attachment of the mdev group, the WARN_ON will promote a
   proactive approach to surfacing such issues).

I'd like to get Kirti's current series in linux-next ASAP, so please
submit a follow-on series to make these changes.  I hope we can get
that finalized and added on top of Kirti's series before the v4.10
merge window opens. Thanks,

Alex