Re: [Xen-devel] [Pkg-xen-devel] ioatdma: Boot process hangs then reboots when using Xen + Linux 3.2

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Tue, Mar 06, 2012 at 06:39:12AM -0800, Ian Campbell wrote:
> On Tue, 2012-03-06 at 06:14 -0800, Dan Williams wrote:
> > [    9.276817] ioatdma 0000:00:16.4: desc[0]:
> > (0x300cc7000->0x300cc7040) cookie: 0 flags: 0x2 ctl: 0x29 (op: 0
> > int_en: 1 compl: 1)
> > ...
> > [    9.276832] ioatdma 0000:00:16.4: ioat_get_current_completion:
> > phys_complete: 0xcc7000
> > 
> > Thanks, this clearly shows that our descriptors are above 4GB and that
> > the driver truncates the completion word.
> > 
> > Is this new behavior for xen?
> 
> Xen makes a distinction between physical addresses and DMA addresses and
> the latter can potentially be anywhere in the machine's real address
> space while the former is what GFP_KERNEL etc controls.
> 
> You are using pci_pool_alloc which is the correct API to use for these
> things since it's purpose is to handle cases where PHYS != DMA addr by
> exposing the DMA address to the caller. As part of that you should also
> be using dma_addr_t for DMA addresses since that is the type which is
> defined to handle the appropriate DMA address size on the platform.
> 
> I think this DMA!=PHYS can also be true of some non-x86 architectures

Especially SPARC.
> without Xen too but I guess ioat is quite x86 specific? In any case it
> is wrong, or at least non-portable, to use unsigned long for these
> addresses even though it happens on x86 that physaddr == dma addr
> (usually).

I think with the Intel VT-d that can be different. The bus addresses returned
do seem different.