[PATCH] swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

[PATCH] swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

[Takashi Iwai]

The code refactoring by commit 0176adb00406 ("swiotlb: refactor
coherent buffer allocation") made swiotlb_alloc_buffer() almost always
failing due to a thinko: namely, the function evaluates the
dma_coherent_ok() call incorrectly and dealing as if it's invalid.
This ends up with weird errors like iwlwifi probe failure or amdgpu
screen flickering.

This patch corrects the logic error.

Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088658
Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088902
Fixes: 0176adb00406 ("swiotlb: refactor coherent buffer allocation")
Cc: <stable@vger.kernel.org> # v4.16+
Signed-off-by: Takashi Iwai <tiwai@suse.de>
---
 lib/swiotlb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 47aeb04c1997..de7cc540450f 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -719,7 +719,7 @@ swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle,
 		goto out_warn;
 
 	*dma_handle = __phys_to_dma(dev, phys_addr);
-	if (dma_coherent_ok(dev, *dma_handle, size))
+	if (!dma_coherent_ok(dev, *dma_handle, size))
 		goto out_unmap;
 
 	memset(phys_to_virt(phys_addr), 0, size);
-- 
2.16.3

Re: [PATCH] swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

[Christoph Hellwig]

On Tue, Apr 10, 2018 at 07:05:13PM +0200, Takashi Iwai wrote:
> The code refactoring by commit 0176adb00406 ("swiotlb: refactor
> coherent buffer allocation") made swiotlb_alloc_buffer() almost always
> failing due to a thinko: namely, the function evaluates the
> dma_coherent_ok() call incorrectly and dealing as if it's invalid.
> This ends up with weird errors like iwlwifi probe failure or amdgpu
> screen flickering.
> 
> This patch corrects the logic error.

This looks ok, although even hitting this code is a bad sign.  Do you
know why the previous dma_direct_alloc didn't succeed?

> 
> Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088658
> Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088902
> Fixes: 0176adb00406 ("swiotlb: refactor coherent buffer allocation")
> Cc: <stable@vger.kernel.org> # v4.16+
> Signed-off-by: Takashi Iwai <tiwai@suse.de>
> ---
>  lib/swiotlb.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/lib/swiotlb.c b/lib/swiotlb.c
> index 47aeb04c1997..de7cc540450f 100644
> --- a/lib/swiotlb.c
> +++ b/lib/swiotlb.c
> @@ -719,7 +719,7 @@ swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle,
>  		goto out_warn;
>  
>  	*dma_handle = __phys_to_dma(dev, phys_addr);
> -	if (dma_coherent_ok(dev, *dma_handle, size))
> +	if (!dma_coherent_ok(dev, *dma_handle, size))
>  		goto out_unmap;
>  
>  	memset(phys_to_virt(phys_addr), 0, size);
> -- 
> 2.16.3
---end quoted text---

Re: [PATCH] swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

[Takashi Iwai]

On Tue, 10 Apr 2018 19:06:15 +0200,
Christoph Hellwig wrote:
> 
> On Tue, Apr 10, 2018 at 07:05:13PM +0200, Takashi Iwai wrote:
> > The code refactoring by commit 0176adb00406 ("swiotlb: refactor
> > coherent buffer allocation") made swiotlb_alloc_buffer() almost always
> > failing due to a thinko: namely, the function evaluates the
> > dma_coherent_ok() call incorrectly and dealing as if it's invalid.
> > This ends up with weird errors like iwlwifi probe failure or amdgpu
> > screen flickering.
> > 
> > This patch corrects the logic error.
> 
> This looks ok, although even hitting this code is a bad sign.  Do you
> know why the previous dma_direct_alloc didn't succeed?

That I have no idea, sorry.  I just figured out the regression just
from the dmesg output of 4.16 kernel in bugzilla reports.
Could you take a look at the bugzilla reports there?


thanks,

Takashi

> > Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088658
> > Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088902
> > Fixes: 0176adb00406 ("swiotlb: refactor coherent buffer allocation")
> > Cc: <stable@vger.kernel.org> # v4.16+
> > Signed-off-by: Takashi Iwai <tiwai@suse.de>
> > ---
> >  lib/swiotlb.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/lib/swiotlb.c b/lib/swiotlb.c
> > index 47aeb04c1997..de7cc540450f 100644
> > --- a/lib/swiotlb.c
> > +++ b/lib/swiotlb.c
> > @@ -719,7 +719,7 @@ swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle,
> >  		goto out_warn;
> >  
> >  	*dma_handle = __phys_to_dma(dev, phys_addr);
> > -	if (dma_coherent_ok(dev, *dma_handle, size))
> > +	if (!dma_coherent_ok(dev, *dma_handle, size))
> >  		goto out_unmap;
> >  
> >  	memset(phys_to_virt(phys_addr), 0, size);
> > -- 
> > 2.16.3
> ---end quoted text---
> 

Re: [PATCH] swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

[Robin Murphy]

On 10/04/18 18:07, Takashi Iwai wrote:
> On Tue, 10 Apr 2018 19:06:15 +0200,
> Christoph Hellwig wrote:
>>
>> On Tue, Apr 10, 2018 at 07:05:13PM +0200, Takashi Iwai wrote:
>>> The code refactoring by commit 0176adb00406 ("swiotlb: refactor
>>> coherent buffer allocation") made swiotlb_alloc_buffer() almost always
>>> failing due to a thinko: namely, the function evaluates the
>>> dma_coherent_ok() call incorrectly and dealing as if it's invalid.
>>> This ends up with weird errors like iwlwifi probe failure or amdgpu
>>> screen flickering.
>>>
>>> This patch corrects the logic error.
>>
>> This looks ok, although even hitting this code is a bad sign.  Do you
>> know why the previous dma_direct_alloc didn't succeed?
> 
> That I have no idea, sorry.  I just figured out the regression just
> from the dmesg output of 4.16 kernel in bugzilla reports.
> Could you take a look at the bugzilla reports there?

In the first one, the machine appears to have enough RAM that most of it 
is beyond the device's 36-bit DMA mask, thus it's fairly likely for the 
initial direct alloc to come back with something unsuitable.

Robin.

> 
> 
> thanks,
> 
> Takashi
> 
>>> Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088658
>>> Bugzilla: https://bugzilla.suse.com/show_bug.cgi?id=1088902
>>> Fixes: 0176adb00406 ("swiotlb: refactor coherent buffer allocation")
>>> Cc: <stable@vger.kernel.org> # v4.16+
>>> Signed-off-by: Takashi Iwai <tiwai@suse.de>
>>> ---
>>>   lib/swiotlb.c | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/lib/swiotlb.c b/lib/swiotlb.c
>>> index 47aeb04c1997..de7cc540450f 100644
>>> --- a/lib/swiotlb.c
>>> +++ b/lib/swiotlb.c
>>> @@ -719,7 +719,7 @@ swiotlb_alloc_buffer(struct device *dev, size_t size, dma_addr_t *dma_handle,
>>>   		goto out_warn;
>>>   
>>>   	*dma_handle = __phys_to_dma(dev, phys_addr);
>>> -	if (dma_coherent_ok(dev, *dma_handle, size))
>>> +	if (!dma_coherent_ok(dev, *dma_handle, size))
>>>   		goto out_unmap;
>>>   
>>>   	memset(phys_to_virt(phys_addr), 0, size);
>>> -- 
>>> 2.16.3
>> ---end quoted text---
>>
> _______________________________________________
> iommu mailing list
> iommu@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu
> 

Re: [PATCH] swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

[Christoph Hellwig]

On Tue, Apr 10, 2018 at 06:50:04PM +0100, Robin Murphy wrote:
> In the first one, the machine appears to have enough RAM that most of it is 
> beyond the device's 36-bit DMA mask, thus it's fairly likely for the 
> initial direct alloc to come back with something unsuitable.

But we should try a GFP_DMA32 allocation first, so this is a bit
surprising.

Re: [PATCH] swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

[Takashi Iwai]

On Tue, 10 Apr 2018 20:10:20 +0200,
Christoph Hellwig wrote:
> 
> On Tue, Apr 10, 2018 at 06:50:04PM +0100, Robin Murphy wrote:
> > In the first one, the machine appears to have enough RAM that most of it is 
> > beyond the device's 36-bit DMA mask, thus it's fairly likely for the 
> > initial direct alloc to come back with something unsuitable.
> 
> But we should try a GFP_DMA32 allocation first, so this is a bit
> surprising.

Hm, do we really try that?
Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32
only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi,
it's 36bit, so GFP_DMA isn't set.

We had a fallback allocation with GFP_DMA32 in the past, but this
seems gone long time ago along with cleanups (commit c647c3bb2d16).

But I haven't followed about this topic for long time, so I might have
missed obviously...


thanks,

Takashi

Re: [PATCH] swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

[Christoph Hellwig]

On Wed, Apr 11, 2018 at 09:28:54AM +0200, Takashi Iwai wrote:
> > But we should try a GFP_DMA32 allocation first, so this is a bit
> > surprising.
> 
> Hm, do we really try that?
> Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32
> only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi,
> it's 36bit, so GFP_DMA isn't set.

Oh, yes - it is using an odd dma mask, and amdgpu seems to use an
just as odd 40-bit dma mask.

> We had a fallback allocation with GFP_DMA32 in the past, but this
> seems gone long time ago along with cleanups (commit c647c3bb2d16).
> 
> But I haven't followed about this topic for long time, so I might have
> missed obviously...

I think a fallback would be much better here rather than relying on the
limited swiotlb buffer bool.  dma_direct_alloc (which in 4.17 is also
used for x86) already has a GFP_DMA fallback, so extending this for
GFP_DMA32 as well would seem reasonable.

Any volunteers?

Re: [PATCH] swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

[Takashi Iwai]

[-- Attachment #1: Type: text/plain, Size: 1370 bytes --]

On Thu, 12 Apr 2018 08:02:27 +0200,
Christoph Hellwig wrote:
> 
> On Wed, Apr 11, 2018 at 09:28:54AM +0200, Takashi Iwai wrote:
> > > But we should try a GFP_DMA32 allocation first, so this is a bit
> > > surprising.
> > 
> > Hm, do we really try that?
> > Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32
> > only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi,
> > it's 36bit, so GFP_DMA isn't set.
> 
> Oh, yes - it is using an odd dma mask, and amdgpu seems to use an
> just as odd 40-bit dma mask.
> 
> > We had a fallback allocation with GFP_DMA32 in the past, but this
> > seems gone long time ago along with cleanups (commit c647c3bb2d16).
> > 
> > But I haven't followed about this topic for long time, so I might have
> > missed obviously...
> 
> I think a fallback would be much better here rather than relying on the
> limited swiotlb buffer bool.  dma_direct_alloc (which in 4.17 is also
> used for x86) already has a GFP_DMA fallback, so extending this for
> GFP_DMA32 as well would seem reasonable.
> 
> Any volunteers?

Below is a quick attempt, totally untested.  Actually the retry with
GFP_DMA is superfluous for archs without it, so the first patch
corrects it.  The second patch adds the retry with GFP_DMA32.

I'll resubmit properly if these are OK (and better if anyone could
test them :)


thanks,

Takashi


[-- Attachment #2: 0001-dma-direct-Don-t-repeat-allocation-for-no-op-GFP_DMA.patch --]
[-- Type: application/octet-stream, Size: 1187 bytes --]

>From 490d11cd440d169c8e1a234c1baee3022c2f3464 Mon Sep 17 00:00:00 2001
From: Takashi Iwai <tiwai@suse.de>
Date: Thu, 12 Apr 2018 09:46:12 +0200
Subject: [PATCH 1/2] dma-direct: Don't repeat allocation for no-op GFP_DMA

When an allocation with lower dma_coherent mask fails,
dma_direct_alloc() retries the allocation with GFP_DMA.  But, it's
useless for architectures that has no GFP_DMA, obviously.

This patch fixes it by adding the check of GFP_DMA before retrying the
allocation.

Fixes: 95f183916d4b ("dma-direct: retry allocations using GFP_DMA for small masks")
Signed-off-by: Takashi Iwai <tiwai@suse.de>
---
 lib/dma-direct.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/dma-direct.c b/lib/dma-direct.c
index c0bba30fef0a..191f96453419 100644
--- a/lib/dma-direct.c
+++ b/lib/dma-direct.c
@@ -84,7 +84,7 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
 		__free_pages(page, page_order);
 		page = NULL;
 
-		if (dev->coherent_dma_mask < DMA_BIT_MASK(32) &&
+		if (GFP_DMA && dev->coherent_dma_mask < DMA_BIT_MASK(32) &&
 		    !(gfp & GFP_DMA)) {
 			gfp = (gfp & ~GFP_DMA32) | GFP_DMA;
 			goto again;
-- 
2.16.3


[-- Attachment #3: 0002-dma-direct-Try-reallocation-with-GFP_DMA32-if-possib.patch --]
[-- Type: application/octet-stream, Size: 1154 bytes --]

>From 48ce0bb0a6b916f5615e24861cd9ba78c98ee33a Mon Sep 17 00:00:00 2001
From: Takashi Iwai <tiwai@suse.de>
Date: Thu, 12 Apr 2018 09:53:48 +0200
Subject: [PATCH 2/2] dma-direct: Try reallocation with GFP_DMA32 if possible

This patch adds a similar fallback reallocation with GFP_DMA32 as
we've done with GFP_DMA.  The condition is that the coherent mask is
smaller than 64bit (i.e. some address limitation), and neither GFP_DMA
nor GFP_DMA32 is set beforehand.

Signed-off-by: Takashi Iwai <tiwai@suse.de>
---
 lib/dma-direct.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/lib/dma-direct.c b/lib/dma-direct.c
index 191f96453419..b3cabf06ce90 100644
--- a/lib/dma-direct.c
+++ b/lib/dma-direct.c
@@ -84,6 +84,12 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
 		__free_pages(page, page_order);
 		page = NULL;
 
+		if (GFP_DMA32 && dev->coherent_dma_mask < DMA_BIT_MASK(64) &&
+		    !(gfp & (GFP_DMA32 | GFP_DMA))) {
+			gfp |= GFP_DMA32;
+			goto again;
+		}
+
 		if (GFP_DMA && dev->coherent_dma_mask < DMA_BIT_MASK(32) &&
 		    !(gfp & GFP_DMA)) {
 			gfp = (gfp & ~GFP_DMA32) | GFP_DMA;
-- 
2.16.3

Re: [PATCH] swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

[Takashi Iwai]

On Thu, 12 Apr 2018 10:03:56 +0200,
Takashi Iwai wrote:
> 
> On Thu, 12 Apr 2018 08:02:27 +0200,
> Christoph Hellwig wrote:
> > 
> > On Wed, Apr 11, 2018 at 09:28:54AM +0200, Takashi Iwai wrote:
> > > > But we should try a GFP_DMA32 allocation first, so this is a bit
> > > > surprising.
> > > 
> > > Hm, do we really try that?
> > > Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32
> > > only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi,
> > > it's 36bit, so GFP_DMA isn't set.
> > 
> > Oh, yes - it is using an odd dma mask, and amdgpu seems to use an
> > just as odd 40-bit dma mask.
> > 
> > > We had a fallback allocation with GFP_DMA32 in the past, but this
> > > seems gone long time ago along with cleanups (commit c647c3bb2d16).
> > > 
> > > But I haven't followed about this topic for long time, so I might have
> > > missed obviously...
> > 
> > I think a fallback would be much better here rather than relying on the
> > limited swiotlb buffer bool.  dma_direct_alloc (which in 4.17 is also
> > used for x86) already has a GFP_DMA fallback, so extending this for
> > GFP_DMA32 as well would seem reasonable.
> > 
> > Any volunteers?
> 
> Below is a quick attempt, totally untested.  Actually the retry with
> GFP_DMA is superfluous for archs without it, so the first patch
> corrects it.

Gah, scratch this, it doesn't work.  A different check is needed...


Takashi

Re: [PATCH] swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

[Takashi Iwai]

[-- Attachment #1: Type: text/plain, Size: 1622 bytes --]

On Thu, 12 Apr 2018 10:19:05 +0200,
Takashi Iwai wrote:
> 
> On Thu, 12 Apr 2018 10:03:56 +0200,
> Takashi Iwai wrote:
> > 
> > On Thu, 12 Apr 2018 08:02:27 +0200,
> > Christoph Hellwig wrote:
> > > 
> > > On Wed, Apr 11, 2018 at 09:28:54AM +0200, Takashi Iwai wrote:
> > > > > But we should try a GFP_DMA32 allocation first, so this is a bit
> > > > > surprising.
> > > > 
> > > > Hm, do we really try that?
> > > > Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32
> > > > only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi,
> > > > it's 36bit, so GFP_DMA isn't set.
> > > 
> > > Oh, yes - it is using an odd dma mask, and amdgpu seems to use an
> > > just as odd 40-bit dma mask.
> > > 
> > > > We had a fallback allocation with GFP_DMA32 in the past, but this
> > > > seems gone long time ago along with cleanups (commit c647c3bb2d16).
> > > > 
> > > > But I haven't followed about this topic for long time, so I might have
> > > > missed obviously...
> > > 
> > > I think a fallback would be much better here rather than relying on the
> > > limited swiotlb buffer bool.  dma_direct_alloc (which in 4.17 is also
> > > used for x86) already has a GFP_DMA fallback, so extending this for
> > > GFP_DMA32 as well would seem reasonable.
> > > 
> > > Any volunteers?
> > 
> > Below is a quick attempt, totally untested.  Actually the retry with
> > GFP_DMA is superfluous for archs without it, so the first patch
> > corrects it.
> 
> Gah, scratch this, it doesn't work.  A different check is needed...

The v2 patches are below, replaced with IS_ENABLED(CONFIG_ZONE_DMA).


Takashi


[-- Attachment #2: 0001-dma-direct-Don-t-repeat-allocation-for-no-op-GFP_DMA.patch --]
[-- Type: application/octet-stream, Size: 1228 bytes --]

>From 8d320bed20265ea98087a2b9c4abf2b4bf9cb9ec Mon Sep 17 00:00:00 2001
From: Takashi Iwai <tiwai@suse.de>
Date: Thu, 12 Apr 2018 09:46:12 +0200
Subject: [PATCH v2 1/2] dma-direct: Don't repeat allocation for no-op GFP_DMA

When an allocation with lower dma_coherent mask fails,
dma_direct_alloc() retries the allocation with GFP_DMA.  But, it's
useless for architectures that has no ZONE_DMA, obviously.

This patch fixes it by adding the check of CONFIG_ZONE_DMA before
retrying the allocation.

Fixes: 95f183916d4b ("dma-direct: retry allocations using GFP_DMA for small masks")
Signed-off-by: Takashi Iwai <tiwai@suse.de>
---
 lib/dma-direct.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/dma-direct.c b/lib/dma-direct.c
index c0bba30fef0a..bbfb229aa067 100644
--- a/lib/dma-direct.c
+++ b/lib/dma-direct.c
@@ -84,7 +84,8 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
 		__free_pages(page, page_order);
 		page = NULL;
 
-		if (dev->coherent_dma_mask < DMA_BIT_MASK(32) &&
+		if (IS_ENABLED(CONFIG_ZONE_DMA) &&
+		    dev->coherent_dma_mask < DMA_BIT_MASK(32) &&
 		    !(gfp & GFP_DMA)) {
 			gfp = (gfp & ~GFP_DMA32) | GFP_DMA;
 			goto again;
-- 
2.16.3


[-- Attachment #3: 0002-dma-direct-Try-reallocation-with-GFP_DMA32-if-possib.patch --]
[-- Type: application/octet-stream, Size: 1173 bytes --]

>From 0d3781b8c0c397768c29c0351390f0fa04c9eaa8 Mon Sep 17 00:00:00 2001
From: Takashi Iwai <tiwai@suse.de>
Date: Thu, 12 Apr 2018 09:53:48 +0200
Subject: [PATCH v2 2/2] dma-direct: Try reallocation with GFP_DMA32 if
 possible

This patch adds a similar fallback reallocation with GFP_DMA32 as
we've done with GFP_DMA.  The condition is that the coherent mask is
smaller than 64bit (i.e. some address limitation), and neither GFP_DMA
nor GFP_DMA32 is set beforehand.

Signed-off-by: Takashi Iwai <tiwai@suse.de>
---
 lib/dma-direct.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/lib/dma-direct.c b/lib/dma-direct.c
index bbfb229aa067..970d39155618 100644
--- a/lib/dma-direct.c
+++ b/lib/dma-direct.c
@@ -84,6 +84,13 @@ void *dma_direct_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle,
 		__free_pages(page, page_order);
 		page = NULL;
 
+		if (IS_ENABLED(CONFIG_ZONE_DMA32) &&
+		    dev->coherent_dma_mask < DMA_BIT_MASK(64) &&
+		    !(gfp & (GFP_DMA32 | GFP_DMA))) {
+			gfp |= GFP_DMA32;
+			goto again;
+		}
+
 		if (IS_ENABLED(CONFIG_ZONE_DMA) &&
 		    dev->coherent_dma_mask < DMA_BIT_MASK(32) &&
 		    !(gfp & GFP_DMA)) {
-- 
2.16.3

Re: [PATCH] swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

[Robin Murphy]

On 12/04/18 09:27, Takashi Iwai wrote:
> On Thu, 12 Apr 2018 10:19:05 +0200,
> Takashi Iwai wrote:
>>
>> On Thu, 12 Apr 2018 10:03:56 +0200,
>> Takashi Iwai wrote:
>>>
>>> On Thu, 12 Apr 2018 08:02:27 +0200,
>>> Christoph Hellwig wrote:
>>>>
>>>> On Wed, Apr 11, 2018 at 09:28:54AM +0200, Takashi Iwai wrote:
>>>>>> But we should try a GFP_DMA32 allocation first, so this is a bit
>>>>>> surprising.
>>>>>
>>>>> Hm, do we really try that?
>>>>> Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32
>>>>> only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi,
>>>>> it's 36bit, so GFP_DMA isn't set.
>>>>
>>>> Oh, yes - it is using an odd dma mask, and amdgpu seems to use an
>>>> just as odd 40-bit dma mask.
>>>>
>>>>> We had a fallback allocation with GFP_DMA32 in the past, but this
>>>>> seems gone long time ago along with cleanups (commit c647c3bb2d16).
>>>>>
>>>>> But I haven't followed about this topic for long time, so I might have
>>>>> missed obviously...
>>>>
>>>> I think a fallback would be much better here rather than relying on the
>>>> limited swiotlb buffer bool.  dma_direct_alloc (which in 4.17 is also
>>>> used for x86) already has a GFP_DMA fallback, so extending this for
>>>> GFP_DMA32 as well would seem reasonable.
>>>>
>>>> Any volunteers?
>>>
>>> Below is a quick attempt, totally untested.  Actually the retry with
>>> GFP_DMA is superfluous for archs without it, so the first patch
>>> corrects it.
>>
>> Gah, scratch this, it doesn't work.  A different check is needed...
> 
> The v2 patches are below, replaced with IS_ENABLED(CONFIG_ZONE_DMA).

That looks pretty reasonable to me - I'd be tempted to try a factoring 
out a helper function to avoid the "goto again" logic, but that's hardly 
crucial.

What I'd really love to see is something like the alloc_pages_mask() 
proposal from years ago to come back such that GFP_DMA32 could die 
entirely, but I don't know anywhere near enough about the mm layer to 
consider taking that on myself :(

Robin.

Re: [PATCH] swiotlb: Fix unexpected swiotlb_alloc_coherent() failures

[Takashi Iwai]

On Thu, 12 Apr 2018 12:32:54 +0200,
Robin Murphy wrote:
> 
> On 12/04/18 09:27, Takashi Iwai wrote:
> > On Thu, 12 Apr 2018 10:19:05 +0200,
> > Takashi Iwai wrote:
> >>
> >> On Thu, 12 Apr 2018 10:03:56 +0200,
> >> Takashi Iwai wrote:
> >>>
> >>> On Thu, 12 Apr 2018 08:02:27 +0200,
> >>> Christoph Hellwig wrote:
> >>>>
> >>>> On Wed, Apr 11, 2018 at 09:28:54AM +0200, Takashi Iwai wrote:
> >>>>>> But we should try a GFP_DMA32 allocation first, so this is a bit
> >>>>>> surprising.
> >>>>>
> >>>>> Hm, do we really try that?
> >>>>> Through a quick glance, dma_alloc_coherent_gfp_flags() gives GFP_DMA32
> >>>>> only when coherent mask <= DMA_BIT_MASK(32); in the case of iwlwifi,
> >>>>> it's 36bit, so GFP_DMA isn't set.
> >>>>
> >>>> Oh, yes - it is using an odd dma mask, and amdgpu seems to use an
> >>>> just as odd 40-bit dma mask.
> >>>>
> >>>>> We had a fallback allocation with GFP_DMA32 in the past, but this
> >>>>> seems gone long time ago along with cleanups (commit c647c3bb2d16).
> >>>>>
> >>>>> But I haven't followed about this topic for long time, so I might have
> >>>>> missed obviously...
> >>>>
> >>>> I think a fallback would be much better here rather than relying on the
> >>>> limited swiotlb buffer bool.  dma_direct_alloc (which in 4.17 is also
> >>>> used for x86) already has a GFP_DMA fallback, so extending this for
> >>>> GFP_DMA32 as well would seem reasonable.
> >>>>
> >>>> Any volunteers?
> >>>
> >>> Below is a quick attempt, totally untested.  Actually the retry with
> >>> GFP_DMA is superfluous for archs without it, so the first patch
> >>> corrects it.
> >>
> >> Gah, scratch this, it doesn't work.  A different check is needed...
> >
> > The v2 patches are below, replaced with IS_ENABLED(CONFIG_ZONE_DMA).
> 
> That looks pretty reasonable to me - I'd be tempted to try a factoring
> out a helper function to avoid the "goto again" logic, but that's
> hardly crucial.

Right, this change won't make the code flow more complex than before,
so I guess it's OK as a first step.

I'll submit this second patch as RFC while the first one as a safe
fix (in addition to another regression fix I stumbled on).

> What I'd really love to see is something like the alloc_pages_mask()
> proposal from years ago to come back such that GFP_DMA32 could die
> entirely, but I don't know anywhere near enough about the mm layer to
> consider taking that on myself :(

Yeah, it'd be lovely, but I'm afraid it's hard to fly high...


thanks,

Takashi