Re: pvops dom0: no sound after boot; possibly caused by swiotlb

[Ronny.Hegewald@online.de]

>> BTW, what is the name of the driver in the source code? 

Its the driver under sound/pci/hda. In hda_intel.c are all the important calls for DMA and where the coherent_dma_mask is set.

>What is your sound-driver detecting the card as? As 64-bit or 32-bit or
>none of those?

As 64-bit.

>On pvops we can get away from calling dma_alloc_coherent b/c we have
>this piece of logic to determine where the driver can DMA from:
>
>604         if (hwdev != NULL && hwdev->coherent_dma_mask)
>605                 mask = hwdev->coherent_dma_mask;
>606         else
>607                 mask = DMA_BIT_MASK(32);

...

>So the 'xen_swiotlb_alloc_coherent' checks if you have the coherent DMA
>mask and if not, assumes you have a driver that can only access up to
>4GB. While the bare-metal assumes that if the driver doesn't have that
>mask , it checks the gfp_t flag and if it has __GFP_DMA make the mask
>24-bit, otherwise 32-bit.

But thats not quite all whats dma_alloc_coherent does. As it only returns a 32-bit variable all coherent_dma_mask over 32-bit get casted down. This way bare-metal makes sure that the dma-mask is never over 32-bit.

Or are you are saying that when the hardware supports 64 bit and has set the coherent_dma_mask accordingly and dom0 is 32-bit that the allocation of DMA after the 4 GB should work fine? Because thats the assumption i see in the pvops-code.

And from what i understood so far the DMA memory should be allocated preferably in the 24-bit address space or max. 32 bit address space, at least in a 32-bit kernel. 

>The only difference here is that under pvops we behave badly with
>devices that have GFP_DMA set and don't have the coherent_dma_mask
>(which it does not seem to be the case?).

As i understand it the opposite is the case. If coherent_dma_mask is not set xen_swiotlb_alloc_coherent sets it to 32-bit. That should work usually (except the device needs the dma-memory in the 24-bit space). 

The problem-case is that the coherent_dma_mask is set. So pvops-dom0 just uses this value, when bare-metal makes sure that it cant be over 32bit by calling dma_alloc_coherent_mask. 

>So is your sound-driver not detecting the card properly and not setting
>the coherent_dma_mask and/or dma_mask?

>From what i have seen the driver works correct. It checks a register of the soundcard if it supports 64 bit and sets the coherent_dma_mask to 64bit, else to 32bit. 

As my soundcard says that it supports the 64bit the mask is set accordingly.

I added debug-messages in the code to be sure about that, when i researched the issue.

>Can you print out both of those entries when the sound driver
>calls the 'xen_swiotlb_alloc_coherent' (without setting the flags to 32 
>forcefully?)

The value of the coherent_dma_mask in xen_swiotlb_alloc_coherent was 0xFFFFFFFFFFFFFFFF when i didnt set 32bit forcefully. 

Until now i only checked the coherent_dma_mask flag because as i understand it, thats the value that is used when coherent dma memory is requested. And i never saw dma_mask used. 

But i can send the values of dma_mask tomorrow if they are useful in that case.