Re: [PATCH RFC] virtio-pci: new config layout: using memory BAR

["Michael S. Tsirkin" <mst@redhat.com>]

On Tue, Jun 11, 2013 at 11:03:50AM +0300, Gleb Natapov wrote:
> On Tue, Jun 11, 2013 at 11:02:26AM +0300, Michael S. Tsirkin wrote:
> > On Tue, Jun 11, 2013 at 10:53:48AM +0300, Gleb Natapov wrote:
> > > On Tue, Jun 11, 2013 at 10:10:47AM +0300, Michael S. Tsirkin wrote:
> > > > On Thu, Jun 06, 2013 at 10:02:14AM -0500, Anthony Liguori wrote:
> > > > > Gleb Natapov <gleb@redhat.com> writes:
> > > > > 
> > > > > > On Wed, Jun 05, 2013 at 07:41:17PM -0500, Anthony Liguori wrote:
> > > > > >> "H. Peter Anvin" <hpa@zytor.com> writes:
> > > > > >> 
> > > > > >> > On 06/05/2013 03:08 PM, Anthony Liguori wrote:
> > > > > >> >>>
> > > > > >> >>> Definitely an option.  However, we want to be able to boot from native
> > > > > >> >>> devices, too, so having an I/O BAR (which would not be used by the OS
> > > > > >> >>> driver) should still at the very least be an option.
> > > > > >> >> 
> > > > > >> >> What makes it so difficult to work with an MMIO bar for PCI-e?
> > > > > >> >> 
> > > > > >> >> With legacy PCI, tracking allocation of MMIO vs. PIO is pretty straight
> > > > > >> >> forward.  Is there something special about PCI-e here?
> > > > > >> >> 
> > > > > >> >
> > > > > >> > It's not tracking allocation.  It is that accessing memory above 1 MiB
> > > > > >> > is incredibly painful in the BIOS environment, which basically means
> > > > > >> > MMIO is inaccessible.
> > > > > >> 
> > > > > >> Oh, you mean in real mode.
> > > > > >> 
> > > > > >> SeaBIOS runs the virtio code in 32-bit mode with a flat memory layout.
> > > > > >> There are loads of ASSERT32FLAT()s in the code to make sure of this.
> > > > > >> 
> > > > > > Well, not exactly. Initialization is done in 32bit, but disk
> > > > > > reads/writes are done in 16bit mode since it should work from int13
> > > > > > interrupt handler. The only way I know to access MMIO bars from 16 bit
> > > > > > is to use SMM which we do not have in KVM.
> > > > > 
> > > > > Ah, if it's just the dataplane operations then there's another solution.
> > > > > 
> > > > > We can introduce a virtqueue flag that asks the backend to poll for new
> > > > > requests.  Then SeaBIOS can add the request to the queue and not worry
> > > > > about kicking or reading the ISR.
> > > > 
> > > > This will pin a host CPU.
> > > > If we do something timer based it will likely
> > > > both increase host CPU utilization and slow device down.
> > > > 
> > > > If we didn't care about performance at all we could
> > > > do config cycles for signalling, which is much
> > > > more elegant than polling in host, but I don't think
> > > > that's the case.
> > > > 
> > > I wouldn't call BIOS int13 interface performance critical.
> > 
> > So the plan always was to
> > - add an MMIO BAR
> > - add a register for pci-config based access to devices
> > 
> > hpa felt performance does matter there but didn't clarify why ...
> > 
> You do not what to make it too slow obviously, this is interface that is
> used to load OS during boot.

And possibly installation?

> > > > > 
> > > > > SeaBIOS is polling for completion anyway.
> > > > 
> > > > I think that's different because a disk will normally respond
> > > > quickly. So it polls a bit, but then it stops as
> > > > there are no outstanding requests.
> > > > 
> > > > -- 
> > > > MST
> > > 
> > > --
> > > 			Gleb.
> 
> --
> 			Gleb.