[PATCH net] octeontx2-af: Fix QMEM struct memory allocation

[PATCH net] octeontx2-af: Fix QMEM struct memory allocation

[Geetha sowjanya]

Currently NIX, NPA queue context memory is being allocated using
GFP_KERNEL flag which inturns allocates from memory reserved for
CMA_DMA. Sizing CMA_DMA memory is getting difficult due to this
dependency, the more number of interfaces enabled the more the
CMA_DMA memory requirement.

To address this issue, GFP_KERNEL flag is replaced with GFP_ATOMIC,
with this memory will be allocated from unreserved memory.

Fixes: 7a37245ef23f ("octeontx2-af: NPA block admin queue init")
Signed-off-by: Sunil Goutham <sgoutham@marvell.com>
Signed-off-by: Geetha sowjanya <gakula@marvell.com>
---
 drivers/net/ethernet/marvell/octeontx2/af/common.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/marvell/octeontx2/af/common.h b/drivers/net/ethernet/marvell/octeontx2/af/common.h
index 8931864ee110..4b4be9ca4d2f 100644
--- a/drivers/net/ethernet/marvell/octeontx2/af/common.h
+++ b/drivers/net/ethernet/marvell/octeontx2/af/common.h
@@ -61,7 +61,7 @@ static inline int qmem_alloc(struct device *dev, struct qmem **q,
 	qmem->entry_sz = entry_sz;
 	qmem->alloc_sz = (qsize * entry_sz) + OTX2_ALIGN;
 	qmem->base = dma_alloc_attrs(dev, qmem->alloc_sz, &qmem->iova,
-				     GFP_KERNEL, DMA_ATTR_FORCE_CONTIGUOUS);
+				     GFP_ATOMIC, DMA_ATTR_FORCE_CONTIGUOUS);
 	if (!qmem->base)
 		return -ENOMEM;
 
-- 
2.25.1

Re: [PATCH net] octeontx2-af: Fix QMEM struct memory allocation

[Pavan Chebbi]

[-- Attachment #1: Type: text/plain, Size: 1762 bytes --]

On Tue, Jan 3, 2023 at 9:39 AM Geetha sowjanya <gakula@marvell.com> wrote:
>
> Currently NIX, NPA queue context memory is being allocated using
> GFP_KERNEL flag which inturns allocates from memory reserved for
> CMA_DMA. Sizing CMA_DMA memory is getting difficult due to this
> dependency, the more number of interfaces enabled the more the
> CMA_DMA memory requirement.
>
> To address this issue, GFP_KERNEL flag is replaced with GFP_ATOMIC,
> with this memory will be allocated from unreserved memory.
>
> Fixes: 7a37245ef23f ("octeontx2-af: NPA block admin queue init")
> Signed-off-by: Sunil Goutham <sgoutham@marvell.com>
> Signed-off-by: Geetha sowjanya <gakula@marvell.com>
> ---
>  drivers/net/ethernet/marvell/octeontx2/af/common.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/net/ethernet/marvell/octeontx2/af/common.h b/drivers/net/ethernet/marvell/octeontx2/af/common.h
> index 8931864ee110..4b4be9ca4d2f 100644
> --- a/drivers/net/ethernet/marvell/octeontx2/af/common.h
> +++ b/drivers/net/ethernet/marvell/octeontx2/af/common.h
> @@ -61,7 +61,7 @@ static inline int qmem_alloc(struct device *dev, struct qmem **q,
>         qmem->entry_sz = entry_sz;
>         qmem->alloc_sz = (qsize * entry_sz) + OTX2_ALIGN;
>         qmem->base = dma_alloc_attrs(dev, qmem->alloc_sz, &qmem->iova,
> -                                    GFP_KERNEL, DMA_ATTR_FORCE_CONTIGUOUS);
> +                                    GFP_ATOMIC, DMA_ATTR_FORCE_CONTIGUOUS);

I am not understanding the problem this change is solving. Can you
describe the issue in some detail?
What do you mean when you say GFP_ATOMIC allocates the memory from
unreserved memory?

>         if (!qmem->base)
>                 return -ENOMEM;
>
> --
> 2.25.1
>

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]

Re: [PATCH net] octeontx2-af: Fix QMEM struct memory allocation

[Leon Romanovsky]

On Tue, Jan 03, 2023 at 09:39:17AM +0530, Geetha sowjanya wrote:
> Currently NIX, NPA queue context memory is being allocated using
> GFP_KERNEL flag which inturns allocates from memory reserved for
> CMA_DMA. Sizing CMA_DMA memory is getting difficult due to this
> dependency, the more number of interfaces enabled the more the
> CMA_DMA memory requirement.
> 
> To address this issue, GFP_KERNEL flag is replaced with GFP_ATOMIC,
> with this memory will be allocated from unreserved memory.

No, GFP_ATOMIC is for memory allocations in atomic context and not for
separation between reserved and unreserved memory.

There is no any explanation to use GFP_ATOMIC except being in atomic
context.

Thanks

Re: [PATCH net] octeontx2-af: Fix QMEM struct memory allocation

[Jakub Kicinski]

On Tue, 3 Jan 2023 12:05:51 +0200 Leon Romanovsky wrote:
> No, GFP_ATOMIC is for memory allocations in atomic context and not for
> separation between reserved and unreserved memory.

Indeed, using ATOMIC to avoid CMA seems like an odd hack.
I haven't encountered this before.