[Qemu-devel] PCI device assign problems!

[Qemu-devel] PCI device assign problems!

[GaoYi]

[-- Attachment #1: Type: text/plain, Size: 658 bytes --]

Hi,

   I am trying to run a PCI card as a pass-throughed device. The PCI driver
works well on physical PC but failed on a full-virtualized PC. The KVM
confiuration is OK as a pass-throughed network card works. I debuged the
PCI driver and found that the some registers of this PCI card should be
filled with the physical address of DMA regions. However, the PCI card can
only get the virtualized physical address.
   So for a PCI device, it cannot be passthroughed if it requires some
physical address to be filled into some registers, right? If it is so, how
to make it work as a passthroughed device?
   Appreciated if someone can provide some help.

 Yi

[-- Attachment #2: Type: text/html, Size: 685 bytes --]

Re: [Qemu-devel] PCI device assign problems!

[Alex Williamson]

On Mon, 2013-03-04 at 18:13 +0800, GaoYi wrote:
> Hi,
> 
>    I am trying to run a PCI card as a pass-throughed device. The PCI driver
> works well on physical PC but failed on a full-virtualized PC. The KVM
> confiuration is OK as a pass-throughed network card works. I debuged the
> PCI driver and found that the some registers of this PCI card should be
> filled with the physical address of DMA regions. However, the PCI card can
> only get the virtualized physical address.
>    So for a PCI device, it cannot be passthroughed if it requires some
> physical address to be filled into some registers, right? If it is so, how
> to make it work as a passthroughed device?
>    Appreciated if someone can provide some help.

The IOMMU is responsible for transparently translating guest physical
addresses to host physical addresses.  It's therefore generally not
needed for the device to know any host physical addresses.  Can you
describe a bit more about the device and where it's trying to DMA?
Thanks,

Alex

Re: [Qemu-devel] PCI device assign problems!

[Alex Williamson]

On Tue, 2013-03-05 at 06:46 +0800, GaoYi wrote:
> Hi Alex,
> 
>    Thanks for your prompt response. The driver is as following:
> 
> 
>                virt = (u64)kzalloc(size, GFP_KERNEL);
> 
>                phy_addr = virt_to_bus((void*)virt_addr);
> 
>                iowrite32( phy_addr, (void*)(ptr_DMA_CHANNEL_REG);
> 
>                          ///DMA TRANSACTION code, skipped
> 
>              if(dma_failed)
>              {
>                    status = ioread32( phy_addr, (void*)(ptr_DMA_STATUS_REG);
> 
>              }
> 
>    the DMA status register report uncommon exceptions. But this driver is
> OK for HOST.

I'd expect the code to work, but it ignores things like the DMA mask of
the device, any kind of checking that the physical address is below 4G,
and using virt_to_bus which has been deprecated in favor of the DMA API
for some time.  You probably really want something like:

void *virt;
dma_addr_t phys;

/* Having previous called dma_set_coherent_mask on pdev */
virt = dma_alloc_coherent(pdev, size, &phys, GFP_KERNEL); 

iowrite32(phys, ...)

if (dma_failed) {
   status = ioread32(phys, ...)
}

That's probably only going to fix anything if your physical address
happens to be >32bits, so I'd still like to know why your code doesn't
work.  This is exactly how any driver does DMA; allocate a buffer, give
the device the physical address of the buffer, and execute the DMA.  The
IOMMU is programmed with the guest physical to host physical translation
and transparently redirects the device access to the correct host memory
location.  So is the resulting physical address DMA-able by the device
and what happens between sending the device the address and determining
that the DMA failed?  If there's a completion interrupt then maybe the
problem is with interrupts and not DMA.  Thanks,

Alex

> 2013/3/4 Alex Williamson <alex.williamson@redhat.com>
> 
> > On Mon, 2013-03-04 at 18:13 +0800, GaoYi wrote:
> > > Hi,
> > >
> > >    I am trying to run a PCI card as a pass-throughed device. The PCI
> > driver
> > > works well on physical PC but failed on a full-virtualized PC. The KVM
> > > confiuration is OK as a pass-throughed network card works. I debuged the
> > > PCI driver and found that the some registers of this PCI card should be
> > > filled with the physical address of DMA regions. However, the PCI card
> > can
> > > only get the virtualized physical address.
> > >    So for a PCI device, it cannot be passthroughed if it requires some
> > > physical address to be filled into some registers, right? If it is so,
> > how
> > > to make it work as a passthroughed device?
> > >    Appreciated if someone can provide some help.
> >
> > The IOMMU is responsible for transparently translating guest physical
> > addresses to host physical addresses.  It's therefore generally not
> > needed for the device to know any host physical addresses.  Can you
> > describe a bit more about the device and where it's trying to DMA?
> > Thanks,
> >
> > Alex
> >
> >
> >
> >