dma_map_single returns 0 on ARM

dma_map_single returns 0 on ARM

[Zoltan Kiss]

Hi,

I'm trying to fire up the Ethernet port of a Hisilicon D01 board, and it 
fails when it wants to map the buffers to the device:

		phys = dma_map_single(&ndev->dev, priv->rx_buf[i],
				RX_BUF_SIZE, DMA_FROM_DEVICE);

This returns 0, while running on bare metal it works. Does anyone has an 
advice where should I look?

Regards,

Zoltan

Re: dma_map_single returns 0 on ARM

[Stefano Stabellini]

On Fri, 24 Oct 2014, Zoltan Kiss wrote:
> Hi,
> 
> I'm trying to fire up the Ethernet port of a Hisilicon D01 board, and it fails
> when it wants to map the buffers to the device:
> 
> 		phys = dma_map_single(&ndev->dev, priv->rx_buf[i],
> 				RX_BUF_SIZE, DMA_FROM_DEVICE);
> 
> This returns 0, while running on bare metal it works. Does anyone has an
> advice where should I look?

dma_map_single should just be a wrapper around dma_ops->map_page that in
dom0 is implemented by xen_swiotlb_map_page.
>From looking at the code the issue appears to be that dma_capable
returns false for your device.

In any case even if dma_map_single returned a valid address, keep in
mind that at the moment if the device is not dma coherent Linux needs
to be compiled with CONFIG_ARM_LPAE to be able to cache flush the
buffers correctly. But the failure in that case is a memory corruption
after the dma request is completed.

Re: dma_map_single returns 0 on ARM

[Julien Grall]

Hi all,

On 25/10/2014 15:11, Stefano Stabellini wrote:
> On Fri, 24 Oct 2014, Zoltan Kiss wrote:
>> Hi,
>>
>> I'm trying to fire up the Ethernet port of a Hisilicon D01 board, and it fails
>> when it wants to map the buffers to the device:
>>
>> 		phys = dma_map_single(&ndev->dev, priv->rx_buf[i],
>> 				RX_BUF_SIZE, DMA_FROM_DEVICE);
>>
>> This returns 0, while running on bare metal it works. Does anyone has an
>> advice where should I look?
>
> dma_map_single should just be a wrapper around dma_ops->map_page that in
> dom0 is implemented by xen_swiotlb_map_page.
>  From looking at the code the issue appears to be that dma_capable
> returns false for your device.
>
> In any case even if dma_map_single returned a valid address, keep in
> mind that at the moment if the device is not dma coherent Linux needs
> to be compiled with CONFIG_ARM_LPAE to be able to cache flush the
> buffers correctly. But the failure in that case is a memory corruption
> after the dma request is completed.

There is no memory corruption if CONFIG_ARM_LPAE is not enabled. It will 
make crash DOM0 because of a BUG_ON in the swiotlb code (checking 
truncation between DMA and physical address). This would happen on 
platform where there is ram bank above 4G and when a guest is booting.

This option is only necessary for 3.17. Using 3.16 will throw lots of 
warning because netback is mapping multiple time the same grant. Linux 
3.15 and backwards will be "safe".

Regards,

-- 
Julien Grall

Re: dma_map_single returns 0 on ARM

[manish jaggi]

On 27 October 2014 02:09, Julien Grall <julien.grall@linaro.org> wrote:
> Hi all,
>
> On 25/10/2014 15:11, Stefano Stabellini wrote:
>>
>> On Fri, 24 Oct 2014, Zoltan Kiss wrote:
>>>
>>> Hi,
>>>
>>> I'm trying to fire up the Ethernet port of a Hisilicon D01 board, and it
>>> fails
>>> when it wants to map the buffers to the device:
>>>
>>>                 phys = dma_map_single(&ndev->dev, priv->rx_buf[i],
>>>                                 RX_BUF_SIZE, DMA_FROM_DEVICE);
>>>
>>> This returns 0, while running on bare metal it works. Does anyone has an
>>> advice where should I look?
>>
>>
>> dma_map_single should just be a wrapper around dma_ops->map_page that in
>> dom0 is implemented by xen_swiotlb_map_page.
>>  From looking at the code the issue appears to be that dma_capable
>> returns false for your device.
>>
>> In any case even if dma_map_single returned a valid address, keep in
>> mind that at the moment if the device is not dma coherent Linux needs
>> to be compiled with CONFIG_ARM_LPAE to be able to cache flush the
>> buffers correctly. But the failure in that case is a memory corruption
>> after the dma request is completed.
>
>
> There is no memory corruption if CONFIG_ARM_LPAE is not enabled. It will
> make crash DOM0 because of a BUG_ON in the swiotlb code (checking truncation
> between DMA and physical address). This would happen on platform where there
> is ram bank above 4G and when a guest is booting.
>
One basic question, if a system has SMMU (IOMMU) enabled, would xen
still use SWIOTLB for dom0?

> This option is only necessary for 3.17. Using 3.16 will throw lots of
> warning because netback is mapping multiple time the same grant. Linux 3.15
> and backwards will be "safe".
>
> Regards,
>
> --
> Julien Grall
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: dma_map_single returns 0 on ARM

[Stefano Stabellini]

On Mon, 27 Oct 2014, manish jaggi wrote:
> On 27 October 2014 02:09, Julien Grall <julien.grall@linaro.org> wrote:
> > Hi all,
> >
> > On 25/10/2014 15:11, Stefano Stabellini wrote:
> >>
> >> On Fri, 24 Oct 2014, Zoltan Kiss wrote:
> >>>
> >>> Hi,
> >>>
> >>> I'm trying to fire up the Ethernet port of a Hisilicon D01 board, and it
> >>> fails
> >>> when it wants to map the buffers to the device:
> >>>
> >>>                 phys = dma_map_single(&ndev->dev, priv->rx_buf[i],
> >>>                                 RX_BUF_SIZE, DMA_FROM_DEVICE);
> >>>
> >>> This returns 0, while running on bare metal it works. Does anyone has an
> >>> advice where should I look?
> >>
> >>
> >> dma_map_single should just be a wrapper around dma_ops->map_page that in
> >> dom0 is implemented by xen_swiotlb_map_page.
> >>  From looking at the code the issue appears to be that dma_capable
> >> returns false for your device.
> >>
> >> In any case even if dma_map_single returned a valid address, keep in
> >> mind that at the moment if the device is not dma coherent Linux needs
> >> to be compiled with CONFIG_ARM_LPAE to be able to cache flush the
> >> buffers correctly. But the failure in that case is a memory corruption
> >> after the dma request is completed.
> >
> >
> > There is no memory corruption if CONFIG_ARM_LPAE is not enabled. It will
> > make crash DOM0 because of a BUG_ON in the swiotlb code (checking truncation
> > between DMA and physical address). This would happen on platform where there
> > is ram bank above 4G and when a guest is booting.
> >
> One basic question, if a system has SMMU (IOMMU) enabled, would xen
> still use SWIOTLB for dom0?

At the moment yes. Eventually we'll be able to detect that a particular
device is protected by an SMMU and as a consequence we'll just call the
regular dma_ops calls for it.


> > This option is only necessary for 3.17. Using 3.16 will throw lots of
> > warning because netback is mapping multiple time the same grant. Linux 3.15
> > and backwards will be "safe".
> >
> > Regards,
> >
> > --
> > Julien Grall
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
> 

Re: dma_map_single returns 0 on ARM

[Zoltan Kiss]

On 25/10/14 15:11, Stefano Stabellini wrote:
> On Fri, 24 Oct 2014, Zoltan Kiss wrote:
>> Hi,
>>
>> I'm trying to fire up the Ethernet port of a Hisilicon D01 board, and it fails
>> when it wants to map the buffers to the device:
>>
>> 		phys = dma_map_single(&ndev->dev, priv->rx_buf[i],
>> 				RX_BUF_SIZE, DMA_FROM_DEVICE);
>>
>> This returns 0, while running on bare metal it works. Does anyone has an
>> advice where should I look?
>
> dma_map_single should just be a wrapper around dma_ops->map_page that in
> dom0 is implemented by xen_swiotlb_map_page.
>  From looking at the code the issue appears to be that dma_capable
> returns false for your device.
>
> In any case even if dma_map_single returned a valid address, keep in
> mind that at the moment if the device is not dma coherent Linux needs
> to be compiled with CONFIG_ARM_LPAE to be able to cache flush the
> buffers correctly. But the failure in that case is a memory corruption
> after the dma request is completed.
>

Thanks for the advices. I've managed to figure out that dma_map_single 
fails because dev->dma_mask was a NULL pointer. I've set it during probe 
with this:

dma_coerce_mask_and_coherent(&ndev->dev, DMA_BIT_MASK(32));

And the mappings seems to be set up properly (a bit below 512M), but the 
returned buffers are filled up with 0xAA, which I guess is probably some 
default value. Turning on and off CONFIG_ARM_LPAE did not helped.
I'll keep looking why is that, if you have an idea let me know.

Zoli

Re: dma_map_single returns 0 on ARM

[Stefano Stabellini]

On Mon, 27 Oct 2014, Zoltan Kiss wrote:
> On 25/10/14 15:11, Stefano Stabellini wrote:
> > On Fri, 24 Oct 2014, Zoltan Kiss wrote:
> > > Hi,
> > > 
> > > I'm trying to fire up the Ethernet port of a Hisilicon D01 board, and it
> > > fails
> > > when it wants to map the buffers to the device:
> > > 
> > > 		phys = dma_map_single(&ndev->dev, priv->rx_buf[i],
> > > 				RX_BUF_SIZE, DMA_FROM_DEVICE);
> > > 
> > > This returns 0, while running on bare metal it works. Does anyone has an
> > > advice where should I look?
> > 
> > dma_map_single should just be a wrapper around dma_ops->map_page that in
> > dom0 is implemented by xen_swiotlb_map_page.
> >  From looking at the code the issue appears to be that dma_capable
> > returns false for your device.
> > 
> > In any case even if dma_map_single returned a valid address, keep in
> > mind that at the moment if the device is not dma coherent Linux needs
> > to be compiled with CONFIG_ARM_LPAE to be able to cache flush the
> > buffers correctly. But the failure in that case is a memory corruption
> > after the dma request is completed.
> > 
> 
> Thanks for the advices. I've managed to figure out that dma_map_single fails
> because dev->dma_mask was a NULL pointer. I've set it during probe with this:
> 
> dma_coerce_mask_and_coherent(&ndev->dev, DMA_BIT_MASK(32));
> 
> And the mappings seems to be set up properly (a bit below 512M), but the
> returned buffers are filled up with 0xAA, which I guess is probably some
> default value. Turning on and off CONFIG_ARM_LPAE did not helped.
> I'll keep looking why is that, if you have an idea let me know.

Are you trying to do regular DMA in dom0 or are you running other guests?
If you are just doing DMA in dom0, then I don't have any suggestions:
that should work.

Re: dma_map_single returns 0 on ARM

[Zoltan Kiss]

On 27/10/14 15:44, Stefano Stabellini wrote:
> On Mon, 27 Oct 2014, Zoltan Kiss wrote:
>> On 25/10/14 15:11, Stefano Stabellini wrote:
>>> On Fri, 24 Oct 2014, Zoltan Kiss wrote:
>>>> Hi,
>>>>
>>>> I'm trying to fire up the Ethernet port of a Hisilicon D01 board, and it
>>>> fails
>>>> when it wants to map the buffers to the device:
>>>>
>>>> 		phys = dma_map_single(&ndev->dev, priv->rx_buf[i],
>>>> 				RX_BUF_SIZE, DMA_FROM_DEVICE);
>>>>
>>>> This returns 0, while running on bare metal it works. Does anyone has an
>>>> advice where should I look?
>>>
>>> dma_map_single should just be a wrapper around dma_ops->map_page that in
>>> dom0 is implemented by xen_swiotlb_map_page.
>>>   From looking at the code the issue appears to be that dma_capable
>>> returns false for your device.
>>>
>>> In any case even if dma_map_single returned a valid address, keep in
>>> mind that at the moment if the device is not dma coherent Linux needs
>>> to be compiled with CONFIG_ARM_LPAE to be able to cache flush the
>>> buffers correctly. But the failure in that case is a memory corruption
>>> after the dma request is completed.
>>>
>>
>> Thanks for the advices. I've managed to figure out that dma_map_single fails
>> because dev->dma_mask was a NULL pointer. I've set it during probe with this:
>>
>> dma_coerce_mask_and_coherent(&ndev->dev, DMA_BIT_MASK(32));
>>
>> And the mappings seems to be set up properly (a bit below 512M), but the
>> returned buffers are filled up with 0xAA, which I guess is probably some
>> default value. Turning on and off CONFIG_ARM_LPAE did not helped.
>> I'll keep looking why is that, if you have an idea let me know.
>
> Are you trying to do regular DMA in dom0 or are you running other guests?
> If you are just doing DMA in dom0, then I don't have any suggestions:
> that should work.
I have only Dom0 running. And one important thing I forgot to mention: 
due to other restrictions I'm using a 3.14 kernel right now!

Zoli