RE: [PATCH] SCSI driver for VMware's virtual HBA.

[Alok Kataria <akataria@vmware.com>]

Hi Chetan,

> > 
> > On Fri, 2009-08-28 at 14:18 -0700, Chetan.Loke@Emulex.Com wrote:
> > > Alok,
> > >
> > >
> > > > +static int __devinit pvscsi_allocate_rings(struct pvscsi_adapter
> > > > *adapter)
> > > > +{
> > > > +	adapter->rings_state = pci_alloc_consistent(adapter->dev, PAGE_SIZE,
> > > > +						    &adapter->ringStatePA);
> > > > +	adapter->req_ring = pci_alloc_consistent(adapter->dev,
> > > > +						 adapter->req_pages * PAGE_SIZE,
> > > > +						 &adapter->reqRingPA);
> > > > +	adapter->cmp_ring = pci_alloc_consistent(adapter->dev,
> > > > +						 adapter->cmp_pages * PAGE_SIZE,
> > > > +						 &adapter->cmpRingPA);
> 
> 
> > >
> > > I understand the emulation etc. But I see that this function isn't
> > > allocating multiple rings right? Didn't see any performance benefits?
> > >
> > 
> > The function is allocating multiple rings, actually 2 of them for the
> > IO path, request and completion rings (req_ring, cmp_ring).
> > 
> > Let me know if you were asking something else.
> > 
> 
> I was using 'Ring' loosely.
> So, Ring == [req_path,cmpl_path]
> Example - R0[req-path,cmpl-path]
> 
> But the code doesn't allocate R0,R1,...,RN.
> 
> Existing code - 
> 
>       --------
> 	Ring-0
> 	--------
> 	  Req 
> 	   +
> 	   |
>  	   +
>         Cmpl     
> 
> 
> Why not multiple rings as shown below - 
> 
>       --------	--------
> 	Ring-0	Ring-M
> 	--------	--------
> 	  Req 	  Req
> 	   +		   +
> 	   |		   |
>  	   +		   +
>         Cmpl     	  Cmpl
> 
> 
> 
> Creating multiple rings @ the guest level buys nothing in terms of
> performance? 

We implement single (pair of) rings in the hypervisor too. And haven't
seen that as a bottleneck in our experiments, yet. And IMO it only makes
sense to support that once their are devices which support multiple
request queues out their.

It would be interesting to understand your view point on this as well.
Have you seen any interesting performance data with multiple rings
support, in the emulation software that you are aware of ?  

> This is also related to how the hypervisor will schedule the
> world-interrupts. If 'a' pv-driver has multiple interrupt lines
> enabled then will the interrupts/ISR's get routed on different PCPU's
> for a vmworld? Or is there a limitation?

Right now since we support a single completion ring only one vector will
be used to deliver intr's. 
Though its important to note that pvscsi does support MSI-X and hence
can support enabling multiple vectors once the need arises.

Thanks,
Alok
> 
> 
> Chetan
>