Re: PCI passthrough issue

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Tue, Feb 01, 2011 at 04:23:09PM +0000, Ian Campbell wrote:
> On Tue, 2011-02-01 at 15:38 +0000, Jean Baptiste Favre wrote:
> > OK, just found it:
> > after domU boot:
> > - log in
> > - ping -s 86 10.0.0.1 (fails)
> > - rmmod sky2
> > - modprobe sky2 copybreak=0 (no packet copied)
> > - ping -s 86 10.0.0.1 (works)
> > 
> > So it's clearly related to that option.
> 
> Awesome!
> 
> > Now the question: what am I supposed to do ?
> 
> I think the next step is to try and reproduce on native 32 bit, with RAM
> artificially limited via the mem= kernel command in option, this will
> let us determine if this is a generic issue or is somehow Xen specific.
> 
> The main difference caused by the copybreak option is that for larger
> frames (i.e. always when copybreak==0) it hits a code path which uses
> pci_map_single and pci_map_page to access the received data. When len <
> copybreak it takes a path which uses pci_dma_sync_single_for_cpu, so it
> seems like the later path is broken somehow. If the issue does turn out

It could also have gotten the direction reverted (the 3c5XX code had it
wrong at some point so..). Might make sense to compile the kernel with
CONFIG_DMA_API_DEBUG  which is good at detecting these issues.

> to be related to Xen then I think that points to the swiotlb code.

> 
> I assume you are not seeing "rx mapping error" in your domU dmesg? Did
> you post a full guest console log at some point? Comparing the logs for
> the 256MB, 398MB and 512MB guest RAM case might be useful.
> 
> Konrad, is there some way to force swiotlb use even for native to make
> the native test cases more relevant? Or is there some other direction we

swiotlb=force will do it.

> should try first?

<shakes his head> I think we have a good lead.

> 
> > It seems that adding this options in /etc/modprobe.d/sky2.conf does not
> > work, neither using /etc/modules

You could just pass as Linux kernel parameter 'sky2.copybreak=0'. But I am
not sure if that stays if the 'sky2' driver is compiled as module.

> 
> That will be a userspace issue in your distro, perhaps with openwrt they
> are not expected to work that way.
> 
> Ian.
>