* Re: [PATCH] dma-contiguous: support per-numa CMA for all architectures [not found] <20230512094210.141540-1-yajun.deng@linux.dev> @ 2023-05-15 9:49 ` Christoph Hellwig 2023-05-15 11:23 ` Yajun Deng ` (2 subsequent siblings) 3 siblings, 0 replies; 10+ messages in thread From: Christoph Hellwig @ 2023-05-15 9:49 UTC (permalink / raw) To: Yajun Deng Cc: corbet, catalin.marinas, will, hch, m.szyprowski, robin.murphy, paulmck, bp, akpm, peterz, rdunlap, kim.phillips, rostedt, thunder.leizhen, ardb, bhe, anshuman.khandual, song.bao.hua, linux-doc, linux-kernel, linux-arm-kernel, iommu This looks fine to me. Can you please work with Barry to make sure the slight different place of the initcall doesn't break anything for his setup? I doubt it would, but I'd rather have a Tested-by: tag. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] dma-contiguous: support per-numa CMA for all architectures [not found] <20230512094210.141540-1-yajun.deng@linux.dev> 2023-05-15 9:49 ` [PATCH] dma-contiguous: support per-numa CMA for all architectures Christoph Hellwig @ 2023-05-15 11:23 ` Yajun Deng 2023-05-15 11:38 ` Petr Tesařík 2023-05-16 1:55 ` Yajun Deng 2023-07-31 16:02 ` Christoph Hellwig 3 siblings, 1 reply; 10+ messages in thread From: Yajun Deng @ 2023-05-15 11:23 UTC (permalink / raw) To: Christoph Hellwig Cc: corbet, catalin.marinas, will, m.szyprowski, robin.murphy, paulmck, bp, akpm, peterz, rdunlap, kim.phillips, rostedt, thunder.leizhen, ardb, bhe, anshuman.khandual, song.bao.hua, linux-doc, linux-kernel, linux-arm-kernel, iommu May 15, 2023 5:49 PM, "Christoph Hellwig" <hch@lst.de> wrote: > This looks fine to me. Can you please work with Barry to make sure > the slight different place of the initcall doesn't break anything > for his setup? I doubt it would, but I'd rather have a Tested-by: > tag. Barry's email is no longer in use. I can't reach him. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] dma-contiguous: support per-numa CMA for all architectures 2023-05-15 11:23 ` Yajun Deng @ 2023-05-15 11:38 ` Petr Tesařík 2023-05-16 1:57 ` Yajun Deng ` (2 more replies) 0 siblings, 3 replies; 10+ messages in thread From: Petr Tesařík @ 2023-05-15 11:38 UTC (permalink / raw) To: Yajun Deng Cc: Christoph Hellwig, corbet, catalin.marinas, will, m.szyprowski, robin.murphy, paulmck, bp, akpm, peterz, rdunlap, kim.phillips, rostedt, thunder.leizhen, ardb, bhe, anshuman.khandual, song.bao.hua, linux-doc, linux-kernel, linux-arm-kernel, iommu On Mon, 15 May 2023 11:23:27 +0000 "Yajun Deng" <yajun.deng@linux.dev> wrote: > May 15, 2023 5:49 PM, "Christoph Hellwig" <hch@lst.de> wrote: > > > This looks fine to me. Can you please work with Barry to make sure > > the slight different place of the initcall doesn't break anything > > for his setup? I doubt it would, but I'd rather have a Tested-by: > > tag. > > Barry's email is no longer in use. I can't reach him. Which one? I would hope that his Gmail account is still valid: Barry Song <21cnbao@gmail.com> Petr T _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] dma-contiguous: support per-numa CMA for all architectures 2023-05-15 11:38 ` Petr Tesařík @ 2023-05-16 1:57 ` Yajun Deng 2023-06-24 0:40 ` Andrew Morton 2023-06-25 7:30 ` Yajun Deng 2 siblings, 0 replies; 10+ messages in thread From: Yajun Deng @ 2023-05-16 1:57 UTC (permalink / raw) To: Petr Tesařík Cc: Christoph Hellwig, corbet, catalin.marinas, will, m.szyprowski, robin.murphy, paulmck, bp, akpm, peterz, rdunlap, kim.phillips, rostedt, thunder.leizhen, ardb, bhe, anshuman.khandual, song.bao.hua, linux-doc, linux-kernel, linux-arm-kernel, iommu, 21cnbao On 2023/5/15 19:38, Petr Tesařík wrote: > On Mon, 15 May 2023 11:23:27 +0000 > "Yajun Deng" <yajun.deng@linux.dev> wrote: > >> May 15, 2023 5:49 PM, "Christoph Hellwig" <hch@lst.de> wrote: >> >>> This looks fine to me. Can you please work with Barry to make sure >>> the slight different place of the initcall doesn't break anything >>> for his setup? I doubt it would, but I'd rather have a Tested-by: >>> tag. >> Barry's email is no longer in use. I can't reach him. > Which one? I would hope that his Gmail account is still valid: > > Barry Song <21cnbao@gmail.com> Thanks. > > Petr T _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] dma-contiguous: support per-numa CMA for all architectures 2023-05-15 11:38 ` Petr Tesařík 2023-05-16 1:57 ` Yajun Deng @ 2023-06-24 0:40 ` Andrew Morton 2023-06-25 7:30 ` Yajun Deng 2 siblings, 0 replies; 10+ messages in thread From: Andrew Morton @ 2023-06-24 0:40 UTC (permalink / raw) To: Petr Tesařík Cc: Yajun Deng, Christoph Hellwig, corbet, catalin.marinas, will, m.szyprowski, robin.murphy, paulmck, bp, peterz, rdunlap, kim.phillips, rostedt, thunder.leizhen, ardb, bhe, anshuman.khandual, song.bao.hua, linux-doc, linux-kernel, linux-arm-kernel, iommu, Barry Song On Mon, 15 May 2023 13:38:21 +0200 Petr Tesařík <petr@tesarici.cz> wrote: > On Mon, 15 May 2023 11:23:27 +0000 > "Yajun Deng" <yajun.deng@linux.dev> wrote: > > > May 15, 2023 5:49 PM, "Christoph Hellwig" <hch@lst.de> wrote: > > > > > This looks fine to me. Can you please work with Barry to make sure > > > the slight different place of the initcall doesn't break anything > > > for his setup? I doubt it would, but I'd rather have a Tested-by: > > > tag. > > > > Barry's email is no longer in use. I can't reach him. > > Which one? I would hope that his Gmail account is still valid: > > Barry Song <21cnbao@gmail.com> > Maybe his kernel.org address works... I have this patch stuck in limbo for 6.4. I guess I'll carry it over into the next -rc cycle, see what happens. fwiw, it has been in -next for six weeks, no known issues. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] dma-contiguous: support per-numa CMA for all architectures 2023-05-15 11:38 ` Petr Tesařík 2023-05-16 1:57 ` Yajun Deng 2023-06-24 0:40 ` Andrew Morton @ 2023-06-25 7:30 ` Yajun Deng 2023-06-26 5:32 ` Barry Song 2 siblings, 1 reply; 10+ messages in thread From: Yajun Deng @ 2023-06-25 7:30 UTC (permalink / raw) To: v-songbaohua Cc: Christoph Hellwig, corbet, catalin.marinas, will, m.szyprowski, robin.murphy, paulmck, bp, peterz, rdunlap, kim.phillips, rostedt, thunder.leizhen, ardb, bhe, anshuman.khandual, song.bao.hua, linux-doc, linux-kernel, linux-arm-kernel, iommu, Petr Tesařík, Andrew Morton, Barry Song, 21cnbao June 24, 2023 8:40 AM, "Andrew Morton" <akpm@linux-foundation.org> wrote: > On Mon, 15 May 2023 13:38:21 +0200 Petr Tesařík <petr@tesarici.cz> wrote: > >> On Mon, 15 May 2023 11:23:27 +0000 >> "Yajun Deng" <yajun.deng@linux.dev> wrote: >> >> May 15, 2023 5:49 PM, "Christoph Hellwig" <hch@lst.de> wrote: > > This looks fine to me. Can you please work with Barry to make sure > the slight different place of the initcall doesn't break anything > for his setup? I doubt it would, but I'd rather have a Tested-by: > tag. >> Barry's email is no longer in use. I can't reach him. >> >> Which one? I would hope that his Gmail account is still valid: >> >> Barry Song <21cnbao@gmail.com> > > Maybe his kernel.org address works... > > I have this patch stuck in limbo for 6.4. I guess I'll carry it over > into the next -rc cycle, see what happens. > > fwiw, it has been in -next for six weeks, no known issues. Hi, Barry, The slight different place of the initcall, does break anything? _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] dma-contiguous: support per-numa CMA for all architectures 2023-06-25 7:30 ` Yajun Deng @ 2023-06-26 5:32 ` Barry Song 2023-06-29 1:41 ` Yicong Yang 0 siblings, 1 reply; 10+ messages in thread From: Barry Song @ 2023-06-26 5:32 UTC (permalink / raw) To: Yajun Deng, Yicong Yang, Tian Tao Cc: v-songbaohua, Christoph Hellwig, corbet, catalin.marinas, will, m.szyprowski, robin.murphy, paulmck, bp, peterz, rdunlap, kim.phillips, rostedt, thunder.leizhen, ardb, bhe, anshuman.khandual, song.bao.hua, linux-doc, linux-kernel, linux-arm-kernel, iommu, Petr Tesařík, Andrew Morton On Sun, Jun 25, 2023 at 7:30 PM Yajun Deng <yajun.deng@linux.dev> wrote: > > June 24, 2023 8:40 AM, "Andrew Morton" <akpm@linux-foundation.org> wrote: > > > On Mon, 15 May 2023 13:38:21 +0200 Petr Tesařík <petr@tesarici.cz> wrote: > > > >> On Mon, 15 May 2023 11:23:27 +0000 > >> "Yajun Deng" <yajun.deng@linux.dev> wrote: > >> > >> May 15, 2023 5:49 PM, "Christoph Hellwig" <hch@lst.de> wrote: > > > > This looks fine to me. Can you please work with Barry to make sure > > the slight different place of the initcall doesn't break anything > > for his setup? I doubt it would, but I'd rather have a Tested-by: > > tag. > >> Barry's email is no longer in use. I can't reach him. > >> > >> Which one? I would hope that his Gmail account is still valid: > >> > >> Barry Song <21cnbao@gmail.com> > > > > Maybe his kernel.org address works... > > > > I have this patch stuck in limbo for 6.4. I guess I'll carry it over > > into the next -rc cycle, see what happens. > > > > fwiw, it has been in -next for six weeks, no known issues. > > Hi, Barry, The slight different place of the initcall, does break anything? i don't see a fundamental difference as anyway it is still after arch_numa_init() which is really what we depend on. and i did a test on qemu with the command line: qemu-system-aarch64 -M virt,gic-version=3 -nographic \ -smp cpus=8 \ -numa node,cpus=0-1,nodeid=0 \ -numa node,cpus=2-3,nodeid=1 \ -numa node,cpus=4-5,nodeid=2 \ -numa node,cpus=6-7,nodeid=3 \ -numa dist,src=0,dst=1,val=12 \ -numa dist,src=0,dst=2,val=20 \ -numa dist,src=0,dst=3,val=22 \ -numa dist,src=1,dst=2,val=22 \ -numa dist,src=2,dst=3,val=12 \ -numa dist,src=1,dst=3,val=24 \ -m 4096M -cpu cortex-a57 -kernel arch/arm64/boot/Image \ -nographic -append "cma_pernuma=32M root=/dev/vda2 rw ip=dhcp sched_debug irqchip.gicv3_pseudo_nmi=1" \ -drive if=none,file=extra/ubuntu16.04-arm64.img,id=hd0 -device virtio-blk-device,drive=hd0 \ -net nic -net user,hostfwd=tcp::2222-:22 and in system, i can see all cma areas are correctly reserved: ~# dmesg | grep cma [ 0.000000] cma: cma_declare_contiguous_nid(size 0x0000000002000000, base 0x0000000000000000, limit 0x0000000000000000 alignment 0x0000000000000000) [ 0.000000] cma: Reserved 32 MiB at 0x000000007ce00000 [ 0.000000] cma: dma_pernuma_cma_reserve: reserved 32 MiB on node 0 [ 0.000000] cma: cma_declare_contiguous_nid(size 0x0000000002000000, base 0x0000000000000000, limit 0x0000000000000000 alignment 0x0000000000000000) [ 0.000000] cma: Reserved 32 MiB at 0x00000000bce00000 [ 0.000000] cma: dma_pernuma_cma_reserve: reserved 32 MiB on node 1 [ 0.000000] cma: cma_declare_contiguous_nid(size 0x0000000002000000, base 0x0000000000000000, limit 0x0000000000000000 alignment 0x0000000000000000) [ 0.000000] cma: Reserved 32 MiB at 0x00000000fce00000 [ 0.000000] cma: dma_pernuma_cma_reserve: reserved 32 MiB on node 2 [ 0.000000] cma: cma_declare_contiguous_nid(size 0x0000000002000000, base 0x0000000000000000, limit 0x0000000000000000 alignment 0x0000000000000000) [ 0.000000] cma: Reserved 32 MiB at 0x0000000100000000 [ 0.000000] cma: dma_pernuma_cma_reserve: reserved 32 MiB on node 3 [ 0.000000] cma: dma_contiguous_reserve(limit 100000000) [ 0.000000] cma: dma_contiguous_reserve: reserving 32 MiB for global area [ 0.000000] cma: cma_declare_contiguous_nid(size 0x0000000002000000, base 0x0000000000000000, limit 0x0000000100000000 alignment 0x0000000000000000) [ 0.000000] cma: Reserved 32 MiB at 0x00000000fae00000 [ 0.000000] Kernel command line: cma_pernuma=32M root=/dev/vda2 rw ip=dhcp sched_debug irqchip.gicv3_pseudo_nmi=1 [ 0.000000] Memory: 3848784K/4194304K available (16128K kernel code, 4152K rwdata, 10244K rodata, 8512K init, 612K bss, 181680K reserved, 163840K cma-reserved) [ 0.175309] cma: cma_alloc(cma (____ptrval____), count 128, align 7) [ 0.179264] cma: cma_alloc(): returned (____ptrval____) [ 0.179869] cma: cma_alloc(cma (____ptrval____), count 128, align 7) [ 0.180027] cma: cma_alloc(): returned (____ptrval____) [ 0.180187] cma: cma_alloc(cma (____ptrval____), count 128, align 7) [ 0.180374] cma: cma_alloc(): returned (____ptrval____) so my feeling is that this patch is fine. but I would prefer Yicong and Tiantao who have a real numa machine and we can get some real device drivers to call dma APIs to allocate memory from pernuma cma on arm64 even though it is 99.9% OK. With their testing done, please feel free to add Acked-by: Barry Song <baohua@kernel.org> Thanks Barry _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] dma-contiguous: support per-numa CMA for all architectures 2023-06-26 5:32 ` Barry Song @ 2023-06-29 1:41 ` Yicong Yang 0 siblings, 0 replies; 10+ messages in thread From: Yicong Yang @ 2023-06-29 1:41 UTC (permalink / raw) To: Barry Song, Yajun Deng, Yicong Yang, Tian Tao Cc: v-songbaohua, Christoph Hellwig, corbet, catalin.marinas, will, m.szyprowski, robin.murphy, paulmck, bp, peterz, rdunlap, kim.phillips, rostedt, thunder.leizhen, ardb, bhe, anshuman.khandual, song.bao.hua, linux-doc, linux-kernel, linux-arm-kernel, iommu, Petr Tesařík, Andrew Morton On 2023/6/26 13:32, Barry Song wrote: > On Sun, Jun 25, 2023 at 7:30 PM Yajun Deng <yajun.deng@linux.dev> wrote: >> >> June 24, 2023 8:40 AM, "Andrew Morton" <akpm@linux-foundation.org> wrote: >> >>> On Mon, 15 May 2023 13:38:21 +0200 Petr Tesařík <petr@tesarici.cz> wrote: >>> >>>> On Mon, 15 May 2023 11:23:27 +0000 >>>> "Yajun Deng" <yajun.deng@linux.dev> wrote: >>>> >>>> May 15, 2023 5:49 PM, "Christoph Hellwig" <hch@lst.de> wrote: >>> >>> This looks fine to me. Can you please work with Barry to make sure >>> the slight different place of the initcall doesn't break anything >>> for his setup? I doubt it would, but I'd rather have a Tested-by: >>> tag. >>>> Barry's email is no longer in use. I can't reach him. >>>> >>>> Which one? I would hope that his Gmail account is still valid: >>>> >>>> Barry Song <21cnbao@gmail.com> >>> >>> Maybe his kernel.org address works... >>> >>> I have this patch stuck in limbo for 6.4. I guess I'll carry it over >>> into the next -rc cycle, see what happens. >>> >>> fwiw, it has been in -next for six weeks, no known issues. >> >> Hi, Barry, The slight different place of the initcall, does break anything? > > i don't see a fundamental difference as anyway it is still after > arch_numa_init() > which is really what we depend on. > > and i did a test on qemu with the command line: > qemu-system-aarch64 -M virt,gic-version=3 -nographic \ > -smp cpus=8 \ > -numa node,cpus=0-1,nodeid=0 \ > -numa node,cpus=2-3,nodeid=1 \ > -numa node,cpus=4-5,nodeid=2 \ > -numa node,cpus=6-7,nodeid=3 \ > -numa dist,src=0,dst=1,val=12 \ > -numa dist,src=0,dst=2,val=20 \ > -numa dist,src=0,dst=3,val=22 \ > -numa dist,src=1,dst=2,val=22 \ > -numa dist,src=2,dst=3,val=12 \ > -numa dist,src=1,dst=3,val=24 \ > -m 4096M -cpu cortex-a57 -kernel arch/arm64/boot/Image \ > -nographic -append "cma_pernuma=32M root=/dev/vda2 rw ip=dhcp > sched_debug irqchip.gicv3_pseudo_nmi=1" \ > -drive if=none,file=extra/ubuntu16.04-arm64.img,id=hd0 -device > virtio-blk-device,drive=hd0 \ > -net nic -net user,hostfwd=tcp::2222-:22 > > and in system, i can see all cma areas are correctly reserved: > ~# dmesg | grep cma > [ 0.000000] cma: cma_declare_contiguous_nid(size > 0x0000000002000000, base 0x0000000000000000, limit 0x0000000000000000 > alignment 0x0000000000000000) > [ 0.000000] cma: Reserved 32 MiB at 0x000000007ce00000 > [ 0.000000] cma: dma_pernuma_cma_reserve: reserved 32 MiB on node 0 > [ 0.000000] cma: cma_declare_contiguous_nid(size > 0x0000000002000000, base 0x0000000000000000, limit 0x0000000000000000 > alignment 0x0000000000000000) > [ 0.000000] cma: Reserved 32 MiB at 0x00000000bce00000 > [ 0.000000] cma: dma_pernuma_cma_reserve: reserved 32 MiB on node 1 > [ 0.000000] cma: cma_declare_contiguous_nid(size > 0x0000000002000000, base 0x0000000000000000, limit 0x0000000000000000 > alignment 0x0000000000000000) > [ 0.000000] cma: Reserved 32 MiB at 0x00000000fce00000 > [ 0.000000] cma: dma_pernuma_cma_reserve: reserved 32 MiB on node 2 > [ 0.000000] cma: cma_declare_contiguous_nid(size > 0x0000000002000000, base 0x0000000000000000, limit 0x0000000000000000 > alignment 0x0000000000000000) > [ 0.000000] cma: Reserved 32 MiB at 0x0000000100000000 > [ 0.000000] cma: dma_pernuma_cma_reserve: reserved 32 MiB on node 3 > [ 0.000000] cma: dma_contiguous_reserve(limit 100000000) > [ 0.000000] cma: dma_contiguous_reserve: reserving 32 MiB for global area > [ 0.000000] cma: cma_declare_contiguous_nid(size > 0x0000000002000000, base 0x0000000000000000, limit 0x0000000100000000 > alignment 0x0000000000000000) > [ 0.000000] cma: Reserved 32 MiB at 0x00000000fae00000 > [ 0.000000] Kernel command line: cma_pernuma=32M root=/dev/vda2 rw > ip=dhcp sched_debug irqchip.gicv3_pseudo_nmi=1 > [ 0.000000] Memory: 3848784K/4194304K available (16128K kernel > code, 4152K rwdata, 10244K rodata, 8512K init, 612K bss, 181680K > reserved, 163840K cma-reserved) > [ 0.175309] cma: cma_alloc(cma (____ptrval____), count 128, align 7) > [ 0.179264] cma: cma_alloc(): returned (____ptrval____) > [ 0.179869] cma: cma_alloc(cma (____ptrval____), count 128, align 7) > [ 0.180027] cma: cma_alloc(): returned (____ptrval____) > [ 0.180187] cma: cma_alloc(cma (____ptrval____), count 128, align 7) > [ 0.180374] cma: cma_alloc(): returned (____ptrval____) > > so my feeling is that this patch is fine. but I would prefer Yicong > and Tiantao who have a real numa machine > and we can get some real device drivers to call dma APIs to allocate > memory from pernuma cma on arm64 > even though it is 99.9% OK. > Tested on our 4 NUMA arm64 server based on mainline commit 1ef6663a587b, this patch works well, so: Tested-by: Yicong Yang <yangyicong@hisilicon.com> For pernuma cma reservation: [ 0.000000] cma: cma_declare_contiguous_nid(size 0x0000000040000000, base 0x0000000000000000, limit 0x0000000000000000 alignment 0x0000000000000000) [ 0.000000] cma: Reserved 1024 MiB at 0x0000002081800000 [ 0.000000] cma: dma_pernuma_cma_reserve: reserved 1024 MiB on node 0 [ 0.000000] cma: cma_declare_contiguous_nid(size 0x0000000040000000, base 0x0000000000000000, limit 0x0000000000000000 alignment 0x0000000000000000) [ 0.000000] cma: Reserved 1024 MiB at 0x0000004000000000 [ 0.000000] cma: dma_pernuma_cma_reserve: reserved 1024 MiB on node 1 [ 0.000000] cma: cma_declare_contiguous_nid(size 0x0000000040000000, base 0x0000000000000000, limit 0x0000000000000000 alignment 0x0000000000000000) [ 0.000000] cma: Reserved 1024 MiB at 0x0000202000000000 [ 0.000000] cma: dma_pernuma_cma_reserve: reserved 1024 MiB on node 2 [ 0.000000] cma: cma_declare_contiguous_nid(size 0x0000000040000000, base 0x0000000000000000, limit 0x0000000000000000 alignment 0x0000000000000000) [ 0.000000] cma: Reserved 1024 MiB at 0x0000204000000000 [ 0.000000] cma: dma_pernuma_cma_reserve: reserved 1024 MiB on node 3 [ 0.000000] cma: dma_contiguous_reserve(limit 100000000) [ 0.000000] cma: dma_contiguous_reserve: reserving 384 MiB for global area [ 0.000000] cma: cma_declare_contiguous_nid(size 0x0000000018000000, base 0x0000000000000000, limit 0x0000000100000000 alignment 0x0000000000000000) [ 0.000000] cma: Reserved 384 MiB at 0x0000000068000000 For allocation from pernuma cma, no failure recorded: [root@localhost cma]# pwd /sys/kernel/mm/cma [root@localhost cma]# ls pernuma0 pernuma1 pernuma2 pernuma3 reserved [root@localhost cma]# cat pernuma*/alloc_pages_fail 0 0 0 0 [root@localhost cma]# cat pernuma*/alloc_pages_success 2144 0 2132 0 Thanks. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] dma-contiguous: support per-numa CMA for all architectures [not found] <20230512094210.141540-1-yajun.deng@linux.dev> 2023-05-15 9:49 ` [PATCH] dma-contiguous: support per-numa CMA for all architectures Christoph Hellwig 2023-05-15 11:23 ` Yajun Deng @ 2023-05-16 1:55 ` Yajun Deng 2023-07-31 16:02 ` Christoph Hellwig 3 siblings, 0 replies; 10+ messages in thread From: Yajun Deng @ 2023-05-16 1:55 UTC (permalink / raw) To: 21cnbao Cc: linux-doc, linux-kernel, linux-arm-kernel, iommu, song.bao.hua, anshuman.khandual, corbet, bhe, ardb, thunder.leizhen, rostedt, kim.phillips, rdunlap, peterz, akpm, bp, paulmck, robin.murphy, m.szyprowski, hch, will, catalin.marinas, petr Hello, Barry This patch changed the caller of dma_pernuma_cma_reserve() from bootmem_init() to dma_contiguous_reserve(), do you think would there be something wrong? On 2023/5/12 17:42, Yajun Deng wrote: > In the commit b7176c261cdb ("dma-contiguous: provide the ability to > reserve per-numa CMA"), Barry adds DMA_PERNUMA_CMA for ARM64. > > But this feature is architecture independent, so support per-numa CMA > for all architectures, and enable it by default if NUMA. > > Signed-off-by: Yajun Deng <yajun.deng@linux.dev> > --- > Documentation/admin-guide/kernel-parameters.txt | 2 +- > arch/arm64/mm/init.c | 2 -- > include/linux/dma-map-ops.h | 6 ------ > kernel/dma/Kconfig | 6 +++--- > kernel/dma/contiguous.c | 8 +++++++- > 5 files changed, 11 insertions(+), 13 deletions(-) > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > index 56d9458276a6..ac0002b2e323 100644 > --- a/Documentation/admin-guide/kernel-parameters.txt > +++ b/Documentation/admin-guide/kernel-parameters.txt > @@ -692,7 +692,7 @@ > kernel/dma/contiguous.c > > cma_pernuma=nn[MG] > - [ARM64,KNL,CMA] > + [KNL,CMA] > Sets the size of kernel per-numa memory area for > contiguous memory allocations. A value of 0 disables > per-numa CMA altogether. And If this option is not > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c > index 66e70ca47680..d560aef6aafa 100644 > --- a/arch/arm64/mm/init.c > +++ b/arch/arm64/mm/init.c > @@ -410,8 +410,6 @@ void __init bootmem_init(void) > arm64_hugetlb_cma_reserve(); > #endif > > - dma_pernuma_cma_reserve(); > - > kvm_hyp_reserve(); > > /* > diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h > index 31f114f486c4..7af9949828ff 100644 > --- a/include/linux/dma-map-ops.h > +++ b/include/linux/dma-map-ops.h > @@ -168,12 +168,6 @@ static inline void dma_free_contiguous(struct device *dev, struct page *page, > } > #endif /* CONFIG_DMA_CMA*/ > > -#ifdef CONFIG_DMA_PERNUMA_CMA > -void dma_pernuma_cma_reserve(void); > -#else > -static inline void dma_pernuma_cma_reserve(void) { } > -#endif /* CONFIG_DMA_PERNUMA_CMA */ > - > #ifdef CONFIG_DMA_DECLARE_COHERENT > int dma_declare_coherent_memory(struct device *dev, phys_addr_t phys_addr, > dma_addr_t device_addr, size_t size); > diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig > index 6677d0e64d27..79f83091e3a2 100644 > --- a/kernel/dma/Kconfig > +++ b/kernel/dma/Kconfig > @@ -140,10 +140,10 @@ if DMA_CMA > > config DMA_PERNUMA_CMA > bool "Enable separate DMA Contiguous Memory Area for each NUMA Node" > - default NUMA && ARM64 > + default NUMA > help > - Enable this option to get pernuma CMA areas so that devices like > - ARM64 SMMU can get local memory by DMA coherent APIs. > + Enable this option to get pernuma CMA areas so that NUMA devices > + can get local memory by DMA coherent APIs. > > You can set the size of pernuma CMA by specifying "cma_pernuma=size" > on the kernel's command line. > diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c > index 6ea80ae42622..26a8e5365fcd 100644 > --- a/kernel/dma/contiguous.c > +++ b/kernel/dma/contiguous.c > @@ -128,7 +128,7 @@ static inline __maybe_unused phys_addr_t cma_early_percent_memory(void) > #endif > > #ifdef CONFIG_DMA_PERNUMA_CMA > -void __init dma_pernuma_cma_reserve(void) > +static void __init dma_pernuma_cma_reserve(void) > { > int nid; > > @@ -153,6 +153,10 @@ void __init dma_pernuma_cma_reserve(void) > (unsigned long long)pernuma_size_bytes / SZ_1M, nid); > } > } > +#else > +static inline void __init dma_pernuma_cma_reserve(void) > +{ > +} > #endif > > /** > @@ -171,6 +175,8 @@ void __init dma_contiguous_reserve(phys_addr_t limit) > phys_addr_t selected_limit = limit; > bool fixed = false; > > + dma_pernuma_cma_reserve(); > + > pr_debug("%s(limit %08lx)\n", __func__, (unsigned long)limit); > > if (size_cmdline != -1) { _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] dma-contiguous: support per-numa CMA for all architectures [not found] <20230512094210.141540-1-yajun.deng@linux.dev> ` (2 preceding siblings ...) 2023-05-16 1:55 ` Yajun Deng @ 2023-07-31 16:02 ` Christoph Hellwig 3 siblings, 0 replies; 10+ messages in thread From: Christoph Hellwig @ 2023-07-31 16:02 UTC (permalink / raw) To: Yajun Deng Cc: corbet, catalin.marinas, will, hch, m.szyprowski, robin.murphy, paulmck, bp, akpm, peterz, rdunlap, kim.phillips, rostedt, thunder.leizhen, ardb, bhe, anshuman.khandual, song.bao.hua, linux-doc, linux-kernel, linux-arm-kernel, iommu Thanks, applied to the dma-mapping tree for 6.6. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2023-07-31 16:03 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20230512094210.141540-1-yajun.deng@linux.dev>
2023-05-15 9:49 ` [PATCH] dma-contiguous: support per-numa CMA for all architectures Christoph Hellwig
2023-05-15 11:23 ` Yajun Deng
2023-05-15 11:38 ` Petr Tesařík
2023-05-16 1:57 ` Yajun Deng
2023-06-24 0:40 ` Andrew Morton
2023-06-25 7:30 ` Yajun Deng
2023-06-26 5:32 ` Barry Song
2023-06-29 1:41 ` Yicong Yang
2023-05-16 1:55 ` Yajun Deng
2023-07-31 16:02 ` Christoph Hellwig
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).