Is __dma_direct_alloc_pages broken on s390?

Is __dma_direct_alloc_pages broken on s390?

[Petr Tesarik]

Hi all,

while looking into DMA allocation, I noticed that
__dma_direct_optimal_gfp_mask() in kernel/dma/direct.c can probably be
improved. It uses GFP_DMA if dev->coherent_dma_mask is less than
DMA_BIT_MASK(ARCH_ZONE_DMA_BITS). There is no s390-specific definition
of ARCH_ZONE_DMA_BITS. The default is 24 bits, but the DMA zone on s390
is 31 bits. CCW subchannel devices set sch->dev.coherent_dma_mask to
DMA_BIT_MASK(31), which is greater than DMA_BIT_MASK(24), so buffers
are allocated from the Normal zone first.

Would it make sense to set ARCH_ZONE_BITS to 31 on s390, or did I miss
something?

Petr T

Re: Is __dma_direct_alloc_pages broken on s390?

[Heiko Carstens]

On Thu, Jul 18, 2019 at 09:17:00AM +0200, Petr Tesarik wrote:
> Hi all,
> 
> while looking into DMA allocation, I noticed that
> __dma_direct_optimal_gfp_mask() in kernel/dma/direct.c can probably be
> improved. It uses GFP_DMA if dev->coherent_dma_mask is less than
> DMA_BIT_MASK(ARCH_ZONE_DMA_BITS). There is no s390-specific definition
> of ARCH_ZONE_DMA_BITS. The default is 24 bits, but the DMA zone on s390
> is 31 bits. CCW subchannel devices set sch->dev.coherent_dma_mask to
> DMA_BIT_MASK(31), which is greater than DMA_BIT_MASK(24), so buffers
> are allocated from the Normal zone first.
> 
> Would it make sense to set ARCH_ZONE_BITS to 31 on s390, or did I miss
> something?

No, this seems to be broken. Halil, can you look into this and provide
a patch?

Re: Is __dma_direct_alloc_pages broken on s390?

[Petr Tesarik]

[-- Attachment #1: Type: text/plain, Size: 1229 bytes --]

On Thu, 18 Jul 2019 13:36:33 +0200
Heiko Carstens <heiko.carstens@de.ibm.com> wrote:

> On Thu, Jul 18, 2019 at 09:17:00AM +0200, Petr Tesarik wrote:
> > Hi all,
> > 
> > while looking into DMA allocation, I noticed that
> > __dma_direct_optimal_gfp_mask() in kernel/dma/direct.c can probably be
> > improved. It uses GFP_DMA if dev->coherent_dma_mask is less than
> > DMA_BIT_MASK(ARCH_ZONE_DMA_BITS). There is no s390-specific definition
> > of ARCH_ZONE_DMA_BITS. The default is 24 bits, but the DMA zone on s390
> > is 31 bits. CCW subchannel devices set sch->dev.coherent_dma_mask to
> > DMA_BIT_MASK(31), which is greater than DMA_BIT_MASK(24), so buffers
> > are allocated from the Normal zone first.
> > 
> > Would it make sense to set ARCH_ZONE_BITS to 31 on s390, or did I miss
> > something?  
> 
> No, this seems to be broken. Halil, can you look into this and provide
> a patch?

I wondered why the kernel works OK on my system, and it is in fact not
so bad. If the first allocation fails, the kernel adds GFP_DMA and
retries, so this is not fatal, but with a proper definition of
ARCH_ZONE_DMA_BITS it should be possible to get success in the first
attempt already, let's do it.

Petr T

[-- Attachment #2: =?unknown-8bit?q?Digit=C3=A1ln=C3=AD?= podpis OpenPGP --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Is __dma_direct_alloc_pages broken on s390?

[Halil Pasic]

[-- Attachment #1: Type: text/plain, Size: 1674 bytes --]

On Thu, 18 Jul 2019 13:51:12 +0200
Petr Tesarik <ptesarik@suse.cz> wrote:

> On Thu, 18 Jul 2019 13:36:33 +0200
> Heiko Carstens <heiko.carstens@de.ibm.com> wrote:
> 
> > On Thu, Jul 18, 2019 at 09:17:00AM +0200, Petr Tesarik wrote:
> > > Hi all,
> > > 
> > > while looking into DMA allocation, I noticed that
> > > __dma_direct_optimal_gfp_mask() in kernel/dma/direct.c can probably be
> > > improved. It uses GFP_DMA if dev->coherent_dma_mask is less than
> > > DMA_BIT_MASK(ARCH_ZONE_DMA_BITS). There is no s390-specific definition
> > > of ARCH_ZONE_DMA_BITS. The default is 24 bits, but the DMA zone on s390
> > > is 31 bits. CCW subchannel devices set sch->dev.coherent_dma_mask to
> > > DMA_BIT_MASK(31), which is greater than DMA_BIT_MASK(24), so buffers
> > > are allocated from the Normal zone first.
> > > 
> > > Would it make sense to set ARCH_ZONE_BITS to 31 on s390, or did I miss
> > > something?  
> > 
> > No, this seems to be broken. Halil, can you look into this and provide
> > a patch?
> 
> I wondered why the kernel works OK on my system, and it is in fact not
> so bad. If the first allocation fails, the kernel adds GFP_DMA and
> retries, so this is not fatal, but with a proper definition of
> ARCH_ZONE_DMA_BITS it should be possible to get success in the first
> attempt already, let's do it.
> 
> Petr T

I fully agree! I will post a patch that provides correct
ARCH_ZONE_DMA_BITS for s390.

BTW I wonder if ARCH_ZONE_DMA_BITS can be inferred from MAX_DMA_ADDRESS,
and why do we need both.@Christoph, maybe you can help me understand if
there is a relationship between the two or not, or?

Regards,
Halil

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Is __dma_direct_alloc_pages broken on s390?

[Christoph Hellwig]

On Thu, Jul 18, 2019 at 02:50:44PM +0200, Halil Pasic wrote:
> > I wondered why the kernel works OK on my system, and it is in fact not
> > so bad. If the first allocation fails, the kernel adds GFP_DMA and
> > retries, so this is not fatal, but with a proper definition of
> > ARCH_ZONE_DMA_BITS it should be possible to get success in the first
> > attempt already, let's do it.
> > 
> > Petr T
> 
> I fully agree! I will post a patch that provides correct
> ARCH_ZONE_DMA_BITS for s390.
> 
> BTW I wonder if ARCH_ZONE_DMA_BITS can be inferred from MAX_DMA_ADDRESS,
> and why do we need both.@Christoph, maybe you can help me understand if
> there is a relationship between the two or not, or?

MAX_DMA_ADDRESS is a bit of a weird beast which I honestly do not
understand fully, but most of the uses in common code look a little
bogus, and we should probably get rid of it.

Re: Is __dma_direct_alloc_pages broken on s390?

[Halil Pasic]

On Thu, 18 Jul 2019 06:10:59 -0700
Christoph Hellwig <hch@infradead.org> wrote:

> On Thu, Jul 18, 2019 at 02:50:44PM +0200, Halil Pasic wrote:
> > > I wondered why the kernel works OK on my system, and it is in fact not
> > > so bad. If the first allocation fails, the kernel adds GFP_DMA and
> > > retries, so this is not fatal, but with a proper definition of
> > > ARCH_ZONE_DMA_BITS it should be possible to get success in the first
> > > attempt already, let's do it.
> > > 
> > > Petr T
> > 
> > I fully agree! I will post a patch that provides correct
> > ARCH_ZONE_DMA_BITS for s390.
> > 
> > BTW I wonder if ARCH_ZONE_DMA_BITS can be inferred from MAX_DMA_ADDRESS,
> > and why do we need both.@Christoph, maybe you can help me understand if
> > there is a relationship between the two or not, or?
> 
> MAX_DMA_ADDRESS is a bit of a weird beast which I honestly do not
> understand fully, but most of the uses in common code look a little
> bogus, and we should probably get rid of it.

Thanks!

Regards,
Halil