Re: large page size virtio issues

[Hollis Blanchard <hollisb@us.ibm.com>]

On Wed, 2008-11-05 at 23:16 +1100, Rusty Russell wrote:
> On Wednesday 05 November 2008 09:14:20 Hollis Blanchard wrote:
> > Hi Rusty, I'm using a patch that changes the Linux base page size to
> > 64K. (This is actually pretty common in ppc64 world, but I happen to be
> > trying it on ppc32.)
> >
> > I'm seeing a problem with virtio. I think at least part of it can be
> > explained by qemu's TARGET_PAGE_BITS=12, and the guest's
> > PAGE_SHIFT=16. The guest allocates the queue, then passes the pfn (pa
> >
> > >> PAGE_SHIFT) to the virtio backend (vp_find_vq()). The backend then
> >
> > calculates the pa as pfn << TARGET_PAGE_BITS.
> >
> > I have to run right now, but quickly changing qemu TARGET_PAGE_BITS to
> > 16 got me a little further but still didn't work. Any thoughts?
> 
> I see Anthony hardwired page size into the queue activation ABI for 
> virtio_pci.  I think that this should be an actual 4096 (or 12) rather than 
> depending on guest page size:
> 
> virtio_pci.c:247:
> 	iowrite32(virt_to_phys(info->queue) >> PAGE_SHIFT,
> 		  vp_dev->ioaddr + VIRTIO_PCI_QUEUE_PFN);

Actually there's an additional complication: PAGE_SIZE is entrenched in
the layout of the ring structure itself (large "struct vring" comment in
include/linux/virtio_ring.h). Callers of vring_size() demonstrate this
problem.

I believe the vring is split across two pages so that it could be
directly mapped by two untrusted guests: the producer would have RW
access to the descriptors and the "available" fields, while the consumer
would have only RO access. I don't think this is yet implemented; is it
still planned?

It looks like vring_size() et al were carefully written to allow
arbitrary page sizes, so for now I'll assume that a struct vring that is
contained completely within a single guest mapping is OK and work up a
patch.

It's worth noting that the PFN_SHIFT question (in the other thread) is a
separate issue. I could have PFN_SHIFT\x10 but define VRING_PAGE_SIZE=4K
for internal alignment. I'm a little nervous about making PPC the only
arch with VRING_PAGE_SIZE=1K, since that might affect my vring layout in
"unusual" ways if it expands beyond 1K but stays inside 4K.

-- 
Hollis Blanchard
IBM Linux Technology Center