Re: [Qemu-devel] [PATCH 0/6] PCI hotplug improvements

[Gleb Natapov <gleb@redhat.com>]

On Wed, Mar 07, 2012 at 12:51:48PM -0700, Alex Williamson wrote:
> On Wed, 2012-03-07 at 20:59 +0200, Gleb Natapov wrote:
> > On Wed, Mar 07, 2012 at 10:20:49AM -0700, Alex Williamson wrote:
> > > On Wed, 2012-03-07 at 14:43 +0200, Gleb Natapov wrote:
> > > > On Tue, Mar 06, 2012 at 05:13:36PM -0700, Alex Williamson wrote:
> > > > > Here's a re-work of the patch that added _STA for the purpose of
> > > > > using it as an ack from the guest.  Instead of that, add a notifier
> > > > > for device access.  Once the guest reads from device config space,
> > > > > it owns it.  Until that point, we can remove it directly.  As pointed
> > > > > out by MST, this passes test b) below, which the _STA method would not.
> > > > > As a bonus, no bios change is required for this.  Patches 5 & 6 are
> > > > > just cleanups that can be applied independently.  Thanks,
> > > > > 
> > > > While I agree with Michael that using _STA as ack is a hack I think
> > > > this approach is not less of a hack. It is unlikely that this is how it
> > > > work on bare metal and we should follow real HW if possible.
> > > 
> > > The test below is the only thing that proved to me it was less of a
> > > hack.  Introducing a _LCK method for a slot may be another way to do
> > > this.  Unfortunately it's not required that the OSPM call _LCK and it's
> > > not mentioned in the msft document referenced previously.
> > >  
> > I do not understand where this requirement, that device_del should work
> > if non-acpi guest is running, is coming from? Because if there is no
> > such requirement then the hack is not needed.
> 
> Where do I file my TPS report? ;)  This is simply trying to add some
> definition to "when is a hot attach completed".  Once we know that, we
> can consider the device owned by the guest after that point.  Prior to
> that, we can allow the cancellation of a hot attach, by directly
> removing the device.  After that point, we have to ask permission from
> the guest or deal with surprise removal.
> 
That just heuristic based on observation of several guests, no? device_add
sends sci interrupt to a guest. How do we know that some guest will not
get upset if it will not find promised device after getting the interrupt
and executing Notify()?

> There are two use cases I know of that make the non-ACPI device_del, or
> really device_add cancellation, useful.  The first is what we've been
> discussing, that not all guests will support ACPI based PCI hotplug and
> devices can be lost to the guest until shutdown, including assigned
I do not know how non ACPI based PCI hotplug works. I can only assume
that they have a way to notify device that it can be removed. The patch
series is about ACPI though. The patch series also does not solve lost
device problem since guest can rescan the bus and claim device forever.
You showed this in your original mail yourself.

> devices.  The second is something we more commonly run into in testing,
> that between and 'add' & 'del' (or del & add), there's some undefined
> delay required to prevent us stepping on ourselves.  For instance, if we
> do an 'add' immediately followed by a 'del', we clear the 'up' register,
> set the 'down' register and hope that the guest removes a device that it
> never knew existed.  This code allows that to work as expected, removing
That's QEMU problem and it should be solved in QEMU. What if we will not
clear 'up' bit and let guest process both events add and del?

> the device even thought the guest never saw it.  In the other direction
> (del->add), PCI won't create a device in a slot that's already occupied,
> so we never actually get to the race there.  Overall, an improvement in
> usability IMHO.
> 
> Once we define an end point for addition, we could actually take it a
> step further and add a timeout parameter to device_add, where if the
> guest hasn't taken the device before the timeout, we automatically
> remove it and device_add returns error.  We might consider doing the
> same for device_del.  Thanks,
>
IMO end point of addition should be sending of sci interrupt After that
device should not be removed without guest participation. We can provide
forced device_del that yanks device anyway for the cases when device was
erroneously added while non-ACPI guest were running.

--
			Gleb.