* ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 @ 2023-05-16 20:46 Stefan Wahren 2023-05-16 22:17 ` Florian Fainelli 2023-05-17 7:37 ` Arnd Bergmann 0 siblings, 2 replies; 17+ messages in thread From: Stefan Wahren @ 2023-05-16 20:46 UTC (permalink / raw) To: Arnd Bergmann, Alexander Stein, Maxime Ripard Cc: Phil Elwell, Florian Fainelli, Linux ARM Hi, today i wanted to test the new arm/multi_v7_lpae_defconfig on my Raspberry Pi 4 8GB with Linux 6.4-rc2, but it fails everytime to boot properly from SD card: [ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] Linux version 6.4.0-rc2 (stefanw@stefanw-SCHENKER) (arm-linux-gnueabihf-gcc (GCC) 11.3.1 20220604 [releases/gcc-11 revision 591c0f4b92548e3ae2e8173f4f93984b1c7f62bb], GNU ld (Linaro_Binutils-2022.06) 2.37.20220122) #2 SMP Tue May 16 18:57:48 CEST 2023 [ 0.000000] CPU: ARMv7 Processor [410fd083] revision 3 (ARMv7), cr=30c5383d [ 0.000000] CPU: div instructions available: patching division code [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache [ 0.000000] OF: fdt: Machine model: Raspberry Pi 4 Model B Rev 1.4 [ 0.000000] random: crng init done [ 0.000000] Memory policy: Data cache writealloc [ 0.000000] efi: UEFI not found. [ 0.000000] Reserved memory: created CMA memory pool at 0x0000000037000000, size 64 MiB [ 0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool [ 0.000000] OF: reserved mem: 0x0000000037000000..0x000000003affffff (65536 KiB) map reusable linux,cma [ 0.000000] OF: reserved mem: 0x000000003ef625a0..0x000000003ef62681 (0 KiB) nomap non-reusable nvram@0 [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000000000-0x000000003fffefff] [ 0.000000] Normal [mem 0x000000003ffff000-0x000000006fffffff] [ 0.000000] HighMem [mem 0x0000000070000000-0x00000001ffffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000000000-0x000000003b3fffff] [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000fbffffff] [ 0.000000] node 0: [mem 0x0000000100000000-0x00000001ffffffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000001ffffffff] [ 0.000000] On node 0, zone Normal: 1024 pages in unavailable ranges [ 0.000000] percpu: Embedded 16 pages/cpu s35988 r8192 d21356 u65536 [ 0.000000] Kernel command line: dma.dmachans=0x37f5 bcm2709.boardrev=0xd03114 bcm2709.serial=0x2e99bd7a bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=42 bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:91:38:52 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000 console=ttyS1,115200 console=tty1 root=PARTUUID=a5c5156c-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait plymouth.ignore-serial-consoles ... [ 1.422481] pcieport 0000:00:00.0: enabling device (0000 -> 0002) [ 1.422621] pcieport 0000:00:00.0: PME: Signaling with IRQ 35 [ 1.434696] ------------[ cut here ]------------ [ 1.434720] WARNING: CPU: 2 PID: 1 at kernel/dma/swiotlb.c:891 swiotlb_map+0x328/0x330 [ 1.434759] bcm2835-dma fe007000.dma: swiotlb addr 0xffffffffffffffff+4096 overflow (mask ffffffff, bus limit ffffffff). [ 1.434790] Modules linked in: [ 1.434813] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 6.4.0-rc2 #2 [ 1.434838] Hardware name: BCM2711 [ 1.434862] unwind_backtrace from show_stack+0x10/0x14 [ 1.434902] show_stack from dump_stack_lvl+0x40/0x4c [ 1.434938] dump_stack_lvl from __warn+0x7c/0x124 [ 1.434980] __warn from warn_slowpath_fmt+0x118/0x174 [ 1.435024] warn_slowpath_fmt from swiotlb_map+0x328/0x330 [ 1.435062] swiotlb_map from dma_map_page_attrs+0x204/0x328 [ 1.435100] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c [ 1.435141] bcm2835_dma_probe from platform_probe+0x5c/0xbc [ 1.435174] platform_probe from really_probe+0xc8/0x2dc [ 1.435214] really_probe from __driver_probe_device+0x88/0x19c [ 1.435262] __driver_probe_device from driver_probe_device+0x30/0x104 [ 1.435310] driver_probe_device from __driver_attach+0x90/0x174 [ 1.435357] __driver_attach from bus_for_each_dev+0x6c/0xb4 [ 1.435402] bus_for_each_dev from bus_add_driver+0xcc/0x1cc [ 1.435445] bus_add_driver from driver_register+0x7c/0x118 [ 1.435479] driver_register from do_one_initcall+0x40/0x1e0 [ 1.435509] do_one_initcall from kernel_init_freeable+0x1b8/0x220 [ 1.435549] kernel_init_freeable from kernel_init+0x18/0x12c [ 1.435589] kernel_init from ret_from_fork+0x14/0x1c [ 1.435619] Exception stack(0xf081dfb0 to 0xf081dff8) [ 1.435641] dfa0: 00000000 00000000 00000000 00000000 [ 1.435667] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1.435692] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 1.435711] ---[ end trace 0000000000000000 ]--- [ 1.435731] bcm2835-dma fe007000.dma: Failed to map zero page [ 1.435750] bcm2835-dma: probe of fe007000.dma failed with error -12 [ 1.488044] Serial: 8250/16550 driver, 5 ports, IRQ sharing enabled [ 1.490574] printk: console [ttyS1] disabled So i went back to 6.3, 6.2, 6.1 & 5.15 (enabling the config options manually on top of multi_v7_defconfig), but it shows the same broken behavior. If i switch back to VMSPLIT_3G and keep ARM_LPAE, the system boots properly from SD card. Any ideas what's the issue here or how to investigate further? Best regards _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-16 20:46 ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 Stefan Wahren @ 2023-05-16 22:17 ` Florian Fainelli 2023-05-17 6:23 ` Stefan Wahren 2023-05-17 7:37 ` Arnd Bergmann 1 sibling, 1 reply; 17+ messages in thread From: Florian Fainelli @ 2023-05-16 22:17 UTC (permalink / raw) To: Stefan Wahren, Arnd Bergmann, Alexander Stein, Maxime Ripard, Robin Murphy, Christoph Hellwig Cc: Phil Elwell, Linux ARM +Robin, Christoph, Hi Stephan, On 5/16/23 13:46, Stefan Wahren wrote: > Hi, > today i wanted to test the new arm/multi_v7_lpae_defconfig on my > Raspberry Pi 4 8GB with Linux 6.4-rc2, but it fails everytime to boot > properly from SD card: > > [ 0.000000] Booting Linux on physical CPU 0x0 > [ 0.000000] Linux version 6.4.0-rc2 (stefanw@stefanw-SCHENKER) > (arm-linux-gnueabihf-gcc (GCC) 11.3.1 20220604 [releases/gcc-11 revision > 591c0f4b92548e3ae2e8173f4f93984b1c7f62bb], GNU ld > (Linaro_Binutils-2022.06) 2.37.20220122) #2 SMP Tue May 16 18:57:48 CEST > 2023 > [ 0.000000] CPU: ARMv7 Processor [410fd083] revision 3 (ARMv7), > cr=30c5383d > [ 0.000000] CPU: div instructions available: patching division code > [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction > cache > [ 0.000000] OF: fdt: Machine model: Raspberry Pi 4 Model B Rev 1.4 > [ 0.000000] random: crng init done > [ 0.000000] Memory policy: Data cache writealloc > [ 0.000000] efi: UEFI not found. > [ 0.000000] Reserved memory: created CMA memory pool at > 0x0000000037000000, size 64 MiB > [ 0.000000] OF: reserved mem: initialized node linux,cma, compatible > id shared-dma-pool > [ 0.000000] OF: reserved mem: 0x0000000037000000..0x000000003affffff > (65536 KiB) map reusable linux,cma > [ 0.000000] OF: reserved mem: 0x000000003ef625a0..0x000000003ef62681 > (0 KiB) nomap non-reusable nvram@0 > [ 0.000000] Zone ranges: > [ 0.000000] DMA [mem 0x0000000000000000-0x000000003fffefff] > [ 0.000000] Normal [mem 0x000000003ffff000-0x000000006fffffff] > [ 0.000000] HighMem [mem 0x0000000070000000-0x00000001ffffffff] > [ 0.000000] Movable zone start for each node > [ 0.000000] Early memory node ranges > [ 0.000000] node 0: [mem 0x0000000000000000-0x000000003b3fffff] > [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000fbffffff] > [ 0.000000] node 0: [mem 0x0000000100000000-0x00000001ffffffff] > [ 0.000000] Initmem setup node 0 [mem > 0x0000000000000000-0x00000001ffffffff] > [ 0.000000] On node 0, zone Normal: 1024 pages in unavailable ranges > [ 0.000000] percpu: Embedded 16 pages/cpu s35988 r8192 d21356 u65536 > [ 0.000000] Kernel command line: dma.dmachans=0x37f5 > bcm2709.boardrev=0xd03114 bcm2709.serial=0x2e99bd7a > bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=42 > bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:91:38:52 > vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000 > console=ttyS1,115200 console=tty1 root=PARTUUID=a5c5156c-02 > rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait > plymouth.ignore-serial-consoles > > ... > > [ 1.422481] pcieport 0000:00:00.0: enabling device (0000 -> 0002) > [ 1.422621] pcieport 0000:00:00.0: PME: Signaling with IRQ 35 > [ 1.434696] ------------[ cut here ]------------ > [ 1.434720] WARNING: CPU: 2 PID: 1 at kernel/dma/swiotlb.c:891 > swiotlb_map+0x328/0x330 > [ 1.434759] bcm2835-dma fe007000.dma: swiotlb addr > 0xffffffffffffffff+4096 overflow (mask ffffffff, bus limit ffffffff). > [ 1.434790] Modules linked in: > [ 1.434813] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 6.4.0-rc2 #2 > [ 1.434838] Hardware name: BCM2711 > [ 1.434862] unwind_backtrace from show_stack+0x10/0x14 > [ 1.434902] show_stack from dump_stack_lvl+0x40/0x4c > [ 1.434938] dump_stack_lvl from __warn+0x7c/0x124 > [ 1.434980] __warn from warn_slowpath_fmt+0x118/0x174 > [ 1.435024] warn_slowpath_fmt from swiotlb_map+0x328/0x330 > [ 1.435062] swiotlb_map from dma_map_page_attrs+0x204/0x328 > [ 1.435100] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c > [ 1.435141] bcm2835_dma_probe from platform_probe+0x5c/0xbc > [ 1.435174] platform_probe from really_probe+0xc8/0x2dc > [ 1.435214] really_probe from __driver_probe_device+0x88/0x19c > [ 1.435262] __driver_probe_device from driver_probe_device+0x30/0x104 > [ 1.435310] driver_probe_device from __driver_attach+0x90/0x174 > [ 1.435357] __driver_attach from bus_for_each_dev+0x6c/0xb4 > [ 1.435402] bus_for_each_dev from bus_add_driver+0xcc/0x1cc > [ 1.435445] bus_add_driver from driver_register+0x7c/0x118 > [ 1.435479] driver_register from do_one_initcall+0x40/0x1e0 > [ 1.435509] do_one_initcall from kernel_init_freeable+0x1b8/0x220 > [ 1.435549] kernel_init_freeable from kernel_init+0x18/0x12c > [ 1.435589] kernel_init from ret_from_fork+0x14/0x1c > [ 1.435619] Exception stack(0xf081dfb0 to 0xf081dff8) > [ 1.435641] dfa0: 00000000 > 00000000 00000000 00000000 > [ 1.435667] dfc0: 00000000 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 > [ 1.435692] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 > [ 1.435711] ---[ end trace 0000000000000000 ]--- > [ 1.435731] bcm2835-dma fe007000.dma: Failed to map zero page > [ 1.435750] bcm2835-dma: probe of fe007000.dma failed with error -12 > [ 1.488044] Serial: 8250/16550 driver, 5 ports, IRQ sharing enabled > [ 1.490574] printk: console [ttyS1] disabled > > So i went back to 6.3, 6.2, 6.1 & 5.15 (enabling the config options > manually on top of multi_v7_defconfig), but it shows the same broken > behavior. > > If i switch back to VMSPLIT_3G and keep ARM_LPAE, the system boots > properly from SD card. > > Any ideas what's the issue here or how to investigate further? The "brcm,bcm2835-dma" Device Tree node is located within the "soc" bus node which has the following: /* Emulate a contiguous 30-bit address range for DMA */ dma-ranges = <0xc0000000 0x0 0x00000000 0x40000000>; which is necessary in order to use the non-allocating VPU L2 cache aperture. Eventually we should trickle through translate_phys_to_dma and do: >static inline dma_addr_t translate_phys_to_dma(struct device *dev, >> phys_addr_t paddr) { const struct bus_dma_region *m; for (m = dev->dma_range_map; m->size; m++) if (paddr >= m->cpu_start && paddr - m->cpu_start < m->size) >> return (dma_addr_t)paddr - m->offset; /* make sure dma_capable fails when no translation is available */ >> return DMA_MAPPING_ERROR; } and I suspect that we did return DMA_MAPPING_ERROR (~0 = 0xffff_ffff_ffff_ffff) here which would suggest that somehow the dma_range_map function pointer has not been set properly. Could you instrument drivers/of/device.c and see whether that is, or is not the case? Thanks! -- Florian _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-16 22:17 ` Florian Fainelli @ 2023-05-17 6:23 ` Stefan Wahren 2023-05-17 10:06 ` Robin Murphy 0 siblings, 1 reply; 17+ messages in thread From: Stefan Wahren @ 2023-05-17 6:23 UTC (permalink / raw) To: Florian Fainelli, Arnd Bergmann, Maxime Ripard, Robin Murphy, Christoph Hellwig Cc: Phil Elwell, Linux ARM, Alexander Stein Hi, Am 17.05.23 um 00:17 schrieb Florian Fainelli: > +Robin, Christoph, > > Hi Stephan, > > On 5/16/23 13:46, Stefan Wahren wrote: >> Hi, >> today i wanted to test the new arm/multi_v7_lpae_defconfig on my >> Raspberry Pi 4 8GB with Linux 6.4-rc2, but it fails everytime to boot >> properly from SD card: >> >> [ 0.000000] Booting Linux on physical CPU 0x0 >> [ 0.000000] Linux version 6.4.0-rc2 (stefanw@stefanw-SCHENKER) >> (arm-linux-gnueabihf-gcc (GCC) 11.3.1 20220604 [releases/gcc-11 >> revision 591c0f4b92548e3ae2e8173f4f93984b1c7f62bb], GNU ld >> (Linaro_Binutils-2022.06) 2.37.20220122) #2 SMP Tue May 16 18:57:48 >> CEST 2023 >> [ 0.000000] CPU: ARMv7 Processor [410fd083] revision 3 (ARMv7), >> cr=30c5383d >> [ 0.000000] CPU: div instructions available: patching division code >> [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT >> instruction cache >> [ 0.000000] OF: fdt: Machine model: Raspberry Pi 4 Model B Rev 1.4 >> [ 0.000000] random: crng init done >> [ 0.000000] Memory policy: Data cache writealloc >> [ 0.000000] efi: UEFI not found. >> [ 0.000000] Reserved memory: created CMA memory pool at >> 0x0000000037000000, size 64 MiB >> [ 0.000000] OF: reserved mem: initialized node linux,cma, >> compatible id shared-dma-pool >> [ 0.000000] OF: reserved mem: >> 0x0000000037000000..0x000000003affffff (65536 KiB) map reusable linux,cma >> [ 0.000000] OF: reserved mem: >> 0x000000003ef625a0..0x000000003ef62681 (0 KiB) nomap non-reusable nvram@0 >> [ 0.000000] Zone ranges: >> [ 0.000000] DMA [mem 0x0000000000000000-0x000000003fffefff] >> [ 0.000000] Normal [mem 0x000000003ffff000-0x000000006fffffff] >> [ 0.000000] HighMem [mem 0x0000000070000000-0x00000001ffffffff] >> [ 0.000000] Movable zone start for each node >> [ 0.000000] Early memory node ranges >> [ 0.000000] node 0: [mem 0x0000000000000000-0x000000003b3fffff] >> [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000fbffffff] >> [ 0.000000] node 0: [mem 0x0000000100000000-0x00000001ffffffff] >> [ 0.000000] Initmem setup node 0 [mem >> 0x0000000000000000-0x00000001ffffffff] >> [ 0.000000] On node 0, zone Normal: 1024 pages in unavailable ranges >> [ 0.000000] percpu: Embedded 16 pages/cpu s35988 r8192 d21356 u65536 >> [ 0.000000] Kernel command line: dma.dmachans=0x37f5 >> bcm2709.boardrev=0xd03114 bcm2709.serial=0x2e99bd7a >> bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=42 >> bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:91:38:52 >> vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000 >> console=ttyS1,115200 console=tty1 root=PARTUUID=a5c5156c-02 >> rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait >> plymouth.ignore-serial-consoles >> >> ... >> >> [ 1.422481] pcieport 0000:00:00.0: enabling device (0000 -> 0002) >> [ 1.422621] pcieport 0000:00:00.0: PME: Signaling with IRQ 35 >> [ 1.434696] ------------[ cut here ]------------ >> [ 1.434720] WARNING: CPU: 2 PID: 1 at kernel/dma/swiotlb.c:891 >> swiotlb_map+0x328/0x330 >> [ 1.434759] bcm2835-dma fe007000.dma: swiotlb addr >> 0xffffffffffffffff+4096 overflow (mask ffffffff, bus limit ffffffff). >> [ 1.434790] Modules linked in: >> [ 1.434813] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 6.4.0-rc2 #2 >> [ 1.434838] Hardware name: BCM2711 >> [ 1.434862] unwind_backtrace from show_stack+0x10/0x14 >> [ 1.434902] show_stack from dump_stack_lvl+0x40/0x4c >> [ 1.434938] dump_stack_lvl from __warn+0x7c/0x124 >> [ 1.434980] __warn from warn_slowpath_fmt+0x118/0x174 >> [ 1.435024] warn_slowpath_fmt from swiotlb_map+0x328/0x330 >> [ 1.435062] swiotlb_map from dma_map_page_attrs+0x204/0x328 >> [ 1.435100] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c >> [ 1.435141] bcm2835_dma_probe from platform_probe+0x5c/0xbc >> [ 1.435174] platform_probe from really_probe+0xc8/0x2dc >> [ 1.435214] really_probe from __driver_probe_device+0x88/0x19c >> [ 1.435262] __driver_probe_device from driver_probe_device+0x30/0x104 >> [ 1.435310] driver_probe_device from __driver_attach+0x90/0x174 >> [ 1.435357] __driver_attach from bus_for_each_dev+0x6c/0xb4 >> [ 1.435402] bus_for_each_dev from bus_add_driver+0xcc/0x1cc >> [ 1.435445] bus_add_driver from driver_register+0x7c/0x118 >> [ 1.435479] driver_register from do_one_initcall+0x40/0x1e0 >> [ 1.435509] do_one_initcall from kernel_init_freeable+0x1b8/0x220 >> [ 1.435549] kernel_init_freeable from kernel_init+0x18/0x12c >> [ 1.435589] kernel_init from ret_from_fork+0x14/0x1c >> [ 1.435619] Exception stack(0xf081dfb0 to 0xf081dff8) >> [ 1.435641] dfa0: 00000000 >> 00000000 00000000 00000000 >> [ 1.435667] dfc0: 00000000 00000000 00000000 00000000 00000000 >> 00000000 00000000 00000000 >> [ 1.435692] dfe0: 00000000 00000000 00000000 00000000 00000013 >> 00000000 >> [ 1.435711] ---[ end trace 0000000000000000 ]--- >> [ 1.435731] bcm2835-dma fe007000.dma: Failed to map zero page >> [ 1.435750] bcm2835-dma: probe of fe007000.dma failed with error -12 >> [ 1.488044] Serial: 8250/16550 driver, 5 ports, IRQ sharing enabled >> [ 1.490574] printk: console [ttyS1] disabled >> >> So i went back to 6.3, 6.2, 6.1 & 5.15 (enabling the config options >> manually on top of multi_v7_defconfig), but it shows the same broken >> behavior. >> >> If i switch back to VMSPLIT_3G and keep ARM_LPAE, the system boots >> properly from SD card. >> >> Any ideas what's the issue here or how to investigate further? > > The "brcm,bcm2835-dma" Device Tree node is located within the "soc" bus > node which has the following: > > /* Emulate a contiguous 30-bit address range for DMA */ > dma-ranges = <0xc0000000 0x0 0x00000000 0x40000000>; > > which is necessary in order to use the non-allocating VPU L2 cache > aperture. Eventually we should trickle through translate_phys_to_dma and > do: > > >static inline dma_addr_t translate_phys_to_dma(struct device *dev, > >> phys_addr_t paddr) > { > const struct bus_dma_region *m; > > for (m = dev->dma_range_map; m->size; m++) > if (paddr >= m->cpu_start && paddr - m->cpu_start < > m->size) > >> return (dma_addr_t)paddr - m->offset; > > /* make sure dma_capable fails when no translation is > available */ > >> return DMA_MAPPING_ERROR; > } > > and I suspect that we did return DMA_MAPPING_ERROR (~0 = > 0xffff_ffff_ffff_ffff) here which would suggest that somehow the > dma_range_map function pointer has not been set properly. Could you > instrument drivers/of/device.c and see whether that is, or is not the case? i applied this on top of Linux 6.3: diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h index 18aade195884..e337b26a7110 100644 --- a/include/linux/dma-direct.h +++ b/include/linux/dma-direct.h @@ -33,6 +33,8 @@ static inline dma_addr_t translate_phys_to_dma(struct device *dev, if (paddr >= m->cpu_start && paddr - m->cpu_start < m->size) return (dma_addr_t)paddr - m->offset; + WARN_ONCE(1, "Unable to xlate: %pa\n", &paddr); + /* make sure dma_capable fails when no translation is available */ return DMA_MAPPING_ERROR; } and got this on my Raspberry Pi 4 (arm/multi_v7_lpae_defconfig): ... [ 1.419785] ------------[ cut here ]------------ [ 1.419811] WARNING: CPU: 2 PID: 1 at include/linux/dma-direct.h:36 dma_map_page_attrs+0x360/0x368 [ 1.419862] Unable to xlate: 0x000000006ffd9000 [ 1.419879] Modules linked in: [ 1.419902] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 6.3.0-dirty #11 [ 1.419926] Hardware name: BCM2711 [ 1.419950] unwind_backtrace from show_stack+0x10/0x14 [ 1.419990] show_stack from dump_stack_lvl+0x40/0x4c [ 1.420031] dump_stack_lvl from __warn+0x7c/0x124 [ 1.420077] __warn from warn_slowpath_fmt+0x118/0x174 [ 1.420122] warn_slowpath_fmt from dma_map_page_attrs+0x360/0x368 [ 1.420169] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c [ 1.420215] bcm2835_dma_probe from platform_probe+0x5c/0xbc [ 1.420262] platform_probe from really_probe+0xc8/0x2dc [ 1.420300] really_probe from __driver_probe_device+0x84/0xe4 [ 1.420335] __driver_probe_device from driver_probe_device+0x30/0x104 [ 1.420369] driver_probe_device from __driver_attach+0x90/0x174 [ 1.420403] __driver_attach from bus_for_each_dev+0x6c/0xb4 [ 1.420435] bus_for_each_dev from bus_add_driver+0xcc/0x1cc [ 1.420465] bus_add_driver from driver_register+0x7c/0x118 [ 1.420498] driver_register from do_one_initcall+0x40/0x1e0 [ 1.420534] do_one_initcall from kernel_init_freeable+0x1b8/0x220 [ 1.420578] kernel_init_freeable from kernel_init+0x18/0x12c [ 1.420627] kernel_init from ret_from_fork+0x14/0x1c [ 1.420662] Exception stack(0xf081dfb0 to 0xf081dff8) [ 1.420684] dfa0: 00000000 00000000 00000000 00000000 [ 1.420710] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1.420734] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 1.420753] ---[ end trace 0000000000000000 ]--- [ 1.420771] ------------[ cut here ]------------ [ 1.420785] WARNING: CPU: 2 PID: 1 at include/linux/dma-direct.h:36 swiotlb_map+0x1dc/0x3e4 [ 1.420829] Unable to xlate: 0x000000006ffd9000 [ 1.420844] Modules linked in: [ 1.420863] CPU: 2 PID: 1 Comm: swapper/0 Tainted: G W 6.3.0-dirty #11 [ 1.420890] Hardware name: BCM2711 [ 1.420906] unwind_backtrace from show_stack+0x10/0x14 [ 1.420942] show_stack from dump_stack_lvl+0x40/0x4c [ 1.420978] dump_stack_lvl from __warn+0x7c/0x124 [ 1.421021] __warn from warn_slowpath_fmt+0x118/0x174 [ 1.421066] warn_slowpath_fmt from swiotlb_map+0x1dc/0x3e4 [ 1.421114] swiotlb_map from dma_map_page_attrs+0x210/0x368 [ 1.421159] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c [ 1.421201] bcm2835_dma_probe from platform_probe+0x5c/0xbc [ 1.421244] platform_probe from really_probe+0xc8/0x2dc [ 1.421281] really_probe from __driver_probe_device+0x84/0xe4 [ 1.421314] __driver_probe_device from driver_probe_device+0x30/0x104 [ 1.421348] driver_probe_device from __driver_attach+0x90/0x174 [ 1.421382] __driver_attach from bus_for_each_dev+0x6c/0xb4 [ 1.421413] bus_for_each_dev from bus_add_driver+0xcc/0x1cc [ 1.421442] bus_add_driver from driver_register+0x7c/0x118 [ 1.421475] driver_register from do_one_initcall+0x40/0x1e0 [ 1.421509] do_one_initcall from kernel_init_freeable+0x1b8/0x220 [ 1.421551] kernel_init_freeable from kernel_init+0x18/0x12c [ 1.421599] kernel_init from ret_from_fork+0x14/0x1c [ 1.421634] Exception stack(0xf081dfb0 to 0xf081dff8) [ 1.421654] dfa0: 00000000 00000000 00000000 00000000 [ 1.421679] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1.421704] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 1.421722] ---[ end trace 0000000000000000 ]--- [ 1.421747] ------------[ cut here ]------------ > > Thanks! _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-17 6:23 ` Stefan Wahren @ 2023-05-17 10:06 ` Robin Murphy 2023-05-17 10:24 ` Stefan Wahren 0 siblings, 1 reply; 17+ messages in thread From: Robin Murphy @ 2023-05-17 10:06 UTC (permalink / raw) To: Stefan Wahren, Florian Fainelli, Arnd Bergmann, Maxime Ripard, Christoph Hellwig Cc: Phil Elwell, Linux ARM, Alexander Stein On 2023-05-17 07:23, Stefan Wahren wrote: > Hi, > > Am 17.05.23 um 00:17 schrieb Florian Fainelli: >> +Robin, Christoph, >> >> Hi Stephan, >> >> On 5/16/23 13:46, Stefan Wahren wrote: >>> Hi, >>> today i wanted to test the new arm/multi_v7_lpae_defconfig on my >>> Raspberry Pi 4 8GB with Linux 6.4-rc2, but it fails everytime to boot >>> properly from SD card: >>> >>> [ 0.000000] Booting Linux on physical CPU 0x0 >>> [ 0.000000] Linux version 6.4.0-rc2 (stefanw@stefanw-SCHENKER) >>> (arm-linux-gnueabihf-gcc (GCC) 11.3.1 20220604 [releases/gcc-11 >>> revision 591c0f4b92548e3ae2e8173f4f93984b1c7f62bb], GNU ld >>> (Linaro_Binutils-2022.06) 2.37.20220122) #2 SMP Tue May 16 18:57:48 >>> CEST 2023 >>> [ 0.000000] CPU: ARMv7 Processor [410fd083] revision 3 (ARMv7), >>> cr=30c5383d >>> [ 0.000000] CPU: div instructions available: patching division code >>> [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT >>> instruction cache >>> [ 0.000000] OF: fdt: Machine model: Raspberry Pi 4 Model B Rev 1.4 >>> [ 0.000000] random: crng init done >>> [ 0.000000] Memory policy: Data cache writealloc >>> [ 0.000000] efi: UEFI not found. >>> [ 0.000000] Reserved memory: created CMA memory pool at >>> 0x0000000037000000, size 64 MiB >>> [ 0.000000] OF: reserved mem: initialized node linux,cma, >>> compatible id shared-dma-pool >>> [ 0.000000] OF: reserved mem: >>> 0x0000000037000000..0x000000003affffff (65536 KiB) map reusable >>> linux,cma >>> [ 0.000000] OF: reserved mem: >>> 0x000000003ef625a0..0x000000003ef62681 (0 KiB) nomap non-reusable >>> nvram@0 >>> [ 0.000000] Zone ranges: >>> [ 0.000000] DMA [mem 0x0000000000000000-0x000000003fffefff] >>> [ 0.000000] Normal [mem 0x000000003ffff000-0x000000006fffffff] >>> [ 0.000000] HighMem [mem 0x0000000070000000-0x00000001ffffffff] >>> [ 0.000000] Movable zone start for each node >>> [ 0.000000] Early memory node ranges >>> [ 0.000000] node 0: [mem 0x0000000000000000-0x000000003b3fffff] >>> [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000fbffffff] >>> [ 0.000000] node 0: [mem 0x0000000100000000-0x00000001ffffffff] >>> [ 0.000000] Initmem setup node 0 [mem >>> 0x0000000000000000-0x00000001ffffffff] >>> [ 0.000000] On node 0, zone Normal: 1024 pages in unavailable ranges >>> [ 0.000000] percpu: Embedded 16 pages/cpu s35988 r8192 d21356 u65536 >>> [ 0.000000] Kernel command line: dma.dmachans=0x37f5 >>> bcm2709.boardrev=0xd03114 bcm2709.serial=0x2e99bd7a >>> bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=42 >>> bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:91:38:52 >>> vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000 >>> console=ttyS1,115200 console=tty1 root=PARTUUID=a5c5156c-02 >>> rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait >>> plymouth.ignore-serial-consoles >>> >>> ... >>> >>> [ 1.422481] pcieport 0000:00:00.0: enabling device (0000 -> 0002) >>> [ 1.422621] pcieport 0000:00:00.0: PME: Signaling with IRQ 35 >>> [ 1.434696] ------------[ cut here ]------------ >>> [ 1.434720] WARNING: CPU: 2 PID: 1 at kernel/dma/swiotlb.c:891 >>> swiotlb_map+0x328/0x330 >>> [ 1.434759] bcm2835-dma fe007000.dma: swiotlb addr >>> 0xffffffffffffffff+4096 overflow (mask ffffffff, bus limit ffffffff). >>> [ 1.434790] Modules linked in: >>> [ 1.434813] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 6.4.0-rc2 #2 >>> [ 1.434838] Hardware name: BCM2711 >>> [ 1.434862] unwind_backtrace from show_stack+0x10/0x14 >>> [ 1.434902] show_stack from dump_stack_lvl+0x40/0x4c >>> [ 1.434938] dump_stack_lvl from __warn+0x7c/0x124 >>> [ 1.434980] __warn from warn_slowpath_fmt+0x118/0x174 >>> [ 1.435024] warn_slowpath_fmt from swiotlb_map+0x328/0x330 >>> [ 1.435062] swiotlb_map from dma_map_page_attrs+0x204/0x328 >>> [ 1.435100] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c >>> [ 1.435141] bcm2835_dma_probe from platform_probe+0x5c/0xbc >>> [ 1.435174] platform_probe from really_probe+0xc8/0x2dc >>> [ 1.435214] really_probe from __driver_probe_device+0x88/0x19c >>> [ 1.435262] __driver_probe_device from >>> driver_probe_device+0x30/0x104 >>> [ 1.435310] driver_probe_device from __driver_attach+0x90/0x174 >>> [ 1.435357] __driver_attach from bus_for_each_dev+0x6c/0xb4 >>> [ 1.435402] bus_for_each_dev from bus_add_driver+0xcc/0x1cc >>> [ 1.435445] bus_add_driver from driver_register+0x7c/0x118 >>> [ 1.435479] driver_register from do_one_initcall+0x40/0x1e0 >>> [ 1.435509] do_one_initcall from kernel_init_freeable+0x1b8/0x220 >>> [ 1.435549] kernel_init_freeable from kernel_init+0x18/0x12c >>> [ 1.435589] kernel_init from ret_from_fork+0x14/0x1c >>> [ 1.435619] Exception stack(0xf081dfb0 to 0xf081dff8) >>> [ 1.435641] dfa0: 00000000 >>> 00000000 00000000 00000000 >>> [ 1.435667] dfc0: 00000000 00000000 00000000 00000000 00000000 >>> 00000000 00000000 00000000 >>> [ 1.435692] dfe0: 00000000 00000000 00000000 00000000 00000013 >>> 00000000 >>> [ 1.435711] ---[ end trace 0000000000000000 ]--- >>> [ 1.435731] bcm2835-dma fe007000.dma: Failed to map zero page >>> [ 1.435750] bcm2835-dma: probe of fe007000.dma failed with error -12 >>> [ 1.488044] Serial: 8250/16550 driver, 5 ports, IRQ sharing enabled >>> [ 1.490574] printk: console [ttyS1] disabled >>> >>> So i went back to 6.3, 6.2, 6.1 & 5.15 (enabling the config options >>> manually on top of multi_v7_defconfig), but it shows the same broken >>> behavior. >>> >>> If i switch back to VMSPLIT_3G and keep ARM_LPAE, the system boots >>> properly from SD card. >>> >>> Any ideas what's the issue here or how to investigate further? >> >> The "brcm,bcm2835-dma" Device Tree node is located within the "soc" >> bus node which has the following: >> >> /* Emulate a contiguous 30-bit address range for DMA */ >> dma-ranges = <0xc0000000 0x0 0x00000000 0x40000000>; >> >> which is necessary in order to use the non-allocating VPU L2 cache >> aperture. Eventually we should trickle through translate_phys_to_dma >> and do: >> >> >static inline dma_addr_t translate_phys_to_dma(struct device *dev, >> >> phys_addr_t paddr) >> { >> const struct bus_dma_region *m; >> >> for (m = dev->dma_range_map; m->size; m++) >> if (paddr >= m->cpu_start && paddr - m->cpu_start < >> m->size) >> >> return (dma_addr_t)paddr - m->offset; >> >> /* make sure dma_capable fails when no translation is >> available */ >> >> return DMA_MAPPING_ERROR; >> } >> >> and I suspect that we did return DMA_MAPPING_ERROR (~0 = >> 0xffff_ffff_ffff_ffff) here which would suggest that somehow the >> dma_range_map function pointer has not been set properly. Could you >> instrument drivers/of/device.c and see whether that is, or is not the >> case? > > i applied this on top of Linux 6.3: > > diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h > index 18aade195884..e337b26a7110 100644 > --- a/include/linux/dma-direct.h > +++ b/include/linux/dma-direct.h > @@ -33,6 +33,8 @@ static inline dma_addr_t translate_phys_to_dma(struct > device *dev, > if (paddr >= m->cpu_start && paddr - m->cpu_start < m->size) > return (dma_addr_t)paddr - m->offset; > > + WARN_ONCE(1, "Unable to xlate: %pa\n", &paddr); > + > /* make sure dma_capable fails when no translation is available */ > return DMA_MAPPING_ERROR; > } > > and got this on my Raspberry Pi 4 (arm/multi_v7_lpae_defconfig): > > ... > > [ 1.419785] ------------[ cut here ]------------ > [ 1.419811] WARNING: CPU: 2 PID: 1 at include/linux/dma-direct.h:36 > dma_map_page_attrs+0x360/0x368 > [ 1.419862] Unable to xlate: 0x000000006ffd9000 This suggests that the SWIOTLB buffer itself has somehow been allocated from ZONE_NORMAL rather than ZONE_DMA (unfortunately you trimmed the log line from swiotlb_print_info() saying exactly where it is). That can't happen with VMSPLIT_3G where we have less than 1GB of lowmem anyway, which explains that difference. I don't have an idea off-hand why LPAE would affect it. Thanks, Robin. > [ 1.419879] Modules linked in: > [ 1.419902] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 6.3.0-dirty #11 > [ 1.419926] Hardware name: BCM2711 > [ 1.419950] unwind_backtrace from show_stack+0x10/0x14 > [ 1.419990] show_stack from dump_stack_lvl+0x40/0x4c > [ 1.420031] dump_stack_lvl from __warn+0x7c/0x124 > [ 1.420077] __warn from warn_slowpath_fmt+0x118/0x174 > [ 1.420122] warn_slowpath_fmt from dma_map_page_attrs+0x360/0x368 > [ 1.420169] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c > [ 1.420215] bcm2835_dma_probe from platform_probe+0x5c/0xbc > [ 1.420262] platform_probe from really_probe+0xc8/0x2dc > [ 1.420300] really_probe from __driver_probe_device+0x84/0xe4 > [ 1.420335] __driver_probe_device from driver_probe_device+0x30/0x104 > [ 1.420369] driver_probe_device from __driver_attach+0x90/0x174 > [ 1.420403] __driver_attach from bus_for_each_dev+0x6c/0xb4 > [ 1.420435] bus_for_each_dev from bus_add_driver+0xcc/0x1cc > [ 1.420465] bus_add_driver from driver_register+0x7c/0x118 > [ 1.420498] driver_register from do_one_initcall+0x40/0x1e0 > [ 1.420534] do_one_initcall from kernel_init_freeable+0x1b8/0x220 > [ 1.420578] kernel_init_freeable from kernel_init+0x18/0x12c > [ 1.420627] kernel_init from ret_from_fork+0x14/0x1c > [ 1.420662] Exception stack(0xf081dfb0 to 0xf081dff8) > [ 1.420684] dfa0: 00000000 > 00000000 00000000 00000000 > [ 1.420710] dfc0: 00000000 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 > [ 1.420734] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 > [ 1.420753] ---[ end trace 0000000000000000 ]--- > [ 1.420771] ------------[ cut here ]------------ > [ 1.420785] WARNING: CPU: 2 PID: 1 at include/linux/dma-direct.h:36 > swiotlb_map+0x1dc/0x3e4 > [ 1.420829] Unable to xlate: 0x000000006ffd9000 > [ 1.420844] Modules linked in: > [ 1.420863] CPU: 2 PID: 1 Comm: swapper/0 Tainted: G W > 6.3.0-dirty #11 > [ 1.420890] Hardware name: BCM2711 > [ 1.420906] unwind_backtrace from show_stack+0x10/0x14 > [ 1.420942] show_stack from dump_stack_lvl+0x40/0x4c > [ 1.420978] dump_stack_lvl from __warn+0x7c/0x124 > [ 1.421021] __warn from warn_slowpath_fmt+0x118/0x174 > [ 1.421066] warn_slowpath_fmt from swiotlb_map+0x1dc/0x3e4 > [ 1.421114] swiotlb_map from dma_map_page_attrs+0x210/0x368 > [ 1.421159] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c > [ 1.421201] bcm2835_dma_probe from platform_probe+0x5c/0xbc > [ 1.421244] platform_probe from really_probe+0xc8/0x2dc > [ 1.421281] really_probe from __driver_probe_device+0x84/0xe4 > [ 1.421314] __driver_probe_device from driver_probe_device+0x30/0x104 > [ 1.421348] driver_probe_device from __driver_attach+0x90/0x174 > [ 1.421382] __driver_attach from bus_for_each_dev+0x6c/0xb4 > [ 1.421413] bus_for_each_dev from bus_add_driver+0xcc/0x1cc > [ 1.421442] bus_add_driver from driver_register+0x7c/0x118 > [ 1.421475] driver_register from do_one_initcall+0x40/0x1e0 > [ 1.421509] do_one_initcall from kernel_init_freeable+0x1b8/0x220 > [ 1.421551] kernel_init_freeable from kernel_init+0x18/0x12c > [ 1.421599] kernel_init from ret_from_fork+0x14/0x1c > [ 1.421634] Exception stack(0xf081dfb0 to 0xf081dff8) > [ 1.421654] dfa0: 00000000 > 00000000 00000000 00000000 > [ 1.421679] dfc0: 00000000 00000000 00000000 00000000 00000000 > 00000000 00000000 00000000 > [ 1.421704] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 > [ 1.421722] ---[ end trace 0000000000000000 ]--- > [ 1.421747] ------------[ cut here ]------------ > >> >> Thanks! _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-17 10:06 ` Robin Murphy @ 2023-05-17 10:24 ` Stefan Wahren 2023-05-17 10:34 ` Arnd Bergmann 2023-05-17 10:36 ` Robin Murphy 0 siblings, 2 replies; 17+ messages in thread From: Stefan Wahren @ 2023-05-17 10:24 UTC (permalink / raw) To: Robin Murphy, Florian Fainelli, Arnd Bergmann, Maxime Ripard, Christoph Hellwig Cc: Phil Elwell, Linux ARM, Alexander Stein Hi Robin, Am 17.05.23 um 12:06 schrieb Robin Murphy: > On 2023-05-17 07:23, Stefan Wahren wrote: >> Hi, >> >> Am 17.05.23 um 00:17 schrieb Florian Fainelli: ... >> >> [ 1.419785] ------------[ cut here ]------------ >> [ 1.419811] WARNING: CPU: 2 PID: 1 at include/linux/dma-direct.h:36 >> dma_map_page_attrs+0x360/0x368 >> [ 1.419862] Unable to xlate: 0x000000006ffd9000 > > This suggests that the SWIOTLB buffer itself has somehow been allocated > from ZONE_NORMAL rather than ZONE_DMA (unfortunately you trimmed the log > line from swiotlb_print_info() saying exactly where it is). That can't > happen with VMSPLIT_3G where we have less than 1GB of lowmem anyway, > which explains that difference. I don't have an idea off-hand why LPAE > would affect it. This one? [ 0.000000] Dentry cache hash table entries: 262144 (order: 8, 1048576 bytes, linear) [ 0.000000] Inode-cache hash table entries: 131072 (order: 7, 524288 bytes, linear) [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.000000] software IO TLB: area num 4. [ 0.000000] software IO TLB: mapped [mem 0x0000000069e7f000-0x000000006de7f000] (64MB) [ 0.000000] Memory: 3856044K/4050944K available (16384K kernel code, 2481K rwdata, 6284K rodata, 2048K init, 430K bss, 129364K reserved, 65536K cma-reserved, 2293760K highmem) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 > > Thanks, > Robin. > >> [ 1.419879] Modules linked in: >> [ 1.419902] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 6.3.0-dirty #11 >> [ 1.419926] Hardware name: BCM2711 >> [ 1.419950] unwind_backtrace from show_stack+0x10/0x14 >> [ 1.419990] show_stack from dump_stack_lvl+0x40/0x4c >> [ 1.420031] dump_stack_lvl from __warn+0x7c/0x124 >> [ 1.420077] __warn from warn_slowpath_fmt+0x118/0x174 >> [ 1.420122] warn_slowpath_fmt from dma_map_page_attrs+0x360/0x368 >> [ 1.420169] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c >> [ 1.420215] bcm2835_dma_probe from platform_probe+0x5c/0xbc >> [ 1.420262] platform_probe from really_probe+0xc8/0x2dc >> [ 1.420300] really_probe from __driver_probe_device+0x84/0xe4 >> [ 1.420335] __driver_probe_device from driver_probe_device+0x30/0x104 >> [ 1.420369] driver_probe_device from __driver_attach+0x90/0x174 >> [ 1.420403] __driver_attach from bus_for_each_dev+0x6c/0xb4 >> [ 1.420435] bus_for_each_dev from bus_add_driver+0xcc/0x1cc >> [ 1.420465] bus_add_driver from driver_register+0x7c/0x118 >> [ 1.420498] driver_register from do_one_initcall+0x40/0x1e0 >> [ 1.420534] do_one_initcall from kernel_init_freeable+0x1b8/0x220 >> [ 1.420578] kernel_init_freeable from kernel_init+0x18/0x12c >> [ 1.420627] kernel_init from ret_from_fork+0x14/0x1c >> [ 1.420662] Exception stack(0xf081dfb0 to 0xf081dff8) >> [ 1.420684] dfa0: 00000000 >> 00000000 00000000 00000000 >> [ 1.420710] dfc0: 00000000 00000000 00000000 00000000 00000000 >> 00000000 00000000 00000000 >> [ 1.420734] dfe0: 00000000 00000000 00000000 00000000 00000013 >> 00000000 >> [ 1.420753] ---[ end trace 0000000000000000 ]--- >> [ 1.420771] ------------[ cut here ]------------ >> [ 1.420785] WARNING: CPU: 2 PID: 1 at include/linux/dma-direct.h:36 >> swiotlb_map+0x1dc/0x3e4 >> [ 1.420829] Unable to xlate: 0x000000006ffd9000 >> [ 1.420844] Modules linked in: >> [ 1.420863] CPU: 2 PID: 1 Comm: swapper/0 Tainted: G W >> 6.3.0-dirty #11 >> [ 1.420890] Hardware name: BCM2711 >> [ 1.420906] unwind_backtrace from show_stack+0x10/0x14 >> [ 1.420942] show_stack from dump_stack_lvl+0x40/0x4c >> [ 1.420978] dump_stack_lvl from __warn+0x7c/0x124 >> [ 1.421021] __warn from warn_slowpath_fmt+0x118/0x174 >> [ 1.421066] warn_slowpath_fmt from swiotlb_map+0x1dc/0x3e4 >> [ 1.421114] swiotlb_map from dma_map_page_attrs+0x210/0x368 >> [ 1.421159] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c >> [ 1.421201] bcm2835_dma_probe from platform_probe+0x5c/0xbc >> [ 1.421244] platform_probe from really_probe+0xc8/0x2dc >> [ 1.421281] really_probe from __driver_probe_device+0x84/0xe4 >> [ 1.421314] __driver_probe_device from driver_probe_device+0x30/0x104 >> [ 1.421348] driver_probe_device from __driver_attach+0x90/0x174 >> [ 1.421382] __driver_attach from bus_for_each_dev+0x6c/0xb4 >> [ 1.421413] bus_for_each_dev from bus_add_driver+0xcc/0x1cc >> [ 1.421442] bus_add_driver from driver_register+0x7c/0x118 >> [ 1.421475] driver_register from do_one_initcall+0x40/0x1e0 >> [ 1.421509] do_one_initcall from kernel_init_freeable+0x1b8/0x220 >> [ 1.421551] kernel_init_freeable from kernel_init+0x18/0x12c >> [ 1.421599] kernel_init from ret_from_fork+0x14/0x1c >> [ 1.421634] Exception stack(0xf081dfb0 to 0xf081dff8) >> [ 1.421654] dfa0: 00000000 >> 00000000 00000000 00000000 >> [ 1.421679] dfc0: 00000000 00000000 00000000 00000000 00000000 >> 00000000 00000000 00000000 >> [ 1.421704] dfe0: 00000000 00000000 00000000 00000000 00000013 >> 00000000 >> [ 1.421722] ---[ end trace 0000000000000000 ]--- >> [ 1.421747] ------------[ cut here ]------------ >> >>> >>> Thanks! > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-17 10:24 ` Stefan Wahren @ 2023-05-17 10:34 ` Arnd Bergmann 2023-05-17 10:54 ` Arnd Bergmann 2023-05-17 10:36 ` Robin Murphy 1 sibling, 1 reply; 17+ messages in thread From: Arnd Bergmann @ 2023-05-17 10:34 UTC (permalink / raw) To: Stefan Wahren, Robin Murphy, Florian Fainelli, Maxime Ripard, Christoph Hellwig Cc: Phil Elwell, Linux ARM, Alexander Stein On Wed, May 17, 2023, at 12:24, Stefan Wahren wrote: > Am 17.05.23 um 12:06 schrieb Robin Murphy: >> On 2023-05-17 07:23, Stefan Wahren wrote: >>> Am 17.05.23 um 00:17 schrieb Florian Fainelli: >> This suggests that the SWIOTLB buffer itself has somehow been allocated >> from ZONE_NORMAL rather than ZONE_DMA (unfortunately you trimmed the log >> line from swiotlb_print_info() saying exactly where it is). That can't >> happen with VMSPLIT_3G where we have less than 1GB of lowmem anyway, >> which explains that difference. I don't have an idea off-hand why LPAE >> would affect it. I think LPAE doesn't affect it, the configurations that Stefan mentioned were LPAE with VMSPLIT_2G (broken), non-LPAE with VMSPLIT_3G (works) and LPAE with VMSPLIT_3G (works). I would assume that non-LPAE with VMSPLIT_2G or VMSPLIT_3G_OPT is also broken. > This one? > [ 0.000000] software IO TLB: area num 4. > [ 0.000000] software IO TLB: mapped [mem > 0x0000000069e7f000-0x000000006de7f000] (64MB) Indeed, that is clearly above the .dma_zone_size value from the machine descriptor. Arnd _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-17 10:34 ` Arnd Bergmann @ 2023-05-17 10:54 ` Arnd Bergmann 2023-05-17 11:06 ` Russell King (Oracle) ` (2 more replies) 0 siblings, 3 replies; 17+ messages in thread From: Arnd Bergmann @ 2023-05-17 10:54 UTC (permalink / raw) To: Stefan Wahren, Robin Murphy, Florian Fainelli, Maxime Ripard, Christoph Hellwig Cc: Phil Elwell, Linux ARM, Alexander Stein On Wed, May 17, 2023, at 12:34, Arnd Bergmann wrote: > On Wed, May 17, 2023, at 12:24, Stefan Wahren wrote: >> Am 17.05.23 um 12:06 schrieb Robin Murphy: >>> On 2023-05-17 07:23, Stefan Wahren wrote: > > I think LPAE doesn't affect it, the configurations that Stefan > mentioned were LPAE with VMSPLIT_2G (broken), non-LPAE with > VMSPLIT_3G (works) and LPAE with VMSPLIT_3G (works). I would assume > that non-LPAE with VMSPLIT_2G or VMSPLIT_3G_OPT is also broken. > >> This one? >> [ 0.000000] software IO TLB: area num 4. >> [ 0.000000] software IO TLB: mapped [mem >> 0x0000000069e7f000-0x000000006de7f000] (64MB) > > Indeed, that is clearly above the .dma_zone_size value from > the machine descriptor. It looks like Arm does not set the ARCH_LOW_ADDRESS_LIMIT constant for swiotlb, so it gets the default 4GB maximum. I think this would work on arm, though it's probably not the correct fix in general, as I think MAX_DMA_ADDRESS is meant to refer to the ISA_DMA_API limit, not the ZONE_DMA limit.: --- a/include/linux/memblock.h +++ b/include/linux/memblock.h @@ -394,7 +394,7 @@ static inline int memblock_get_region_node(const struct memblock_region *r) #define MEMBLOCK_LOW_LIMIT 0 #ifndef ARCH_LOW_ADDRESS_LIMIT -#define ARCH_LOW_ADDRESS_LIMIT 0xffffffffUL +#define ARCH_LOW_ADDRESS_LIMIT MAX_DMA_ADDRESS #endif phys_addr_t memblock_phys_alloc_range(phys_addr_t size, phys_addr_t align, Can you give this a try? Arnd _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-17 10:54 ` Arnd Bergmann @ 2023-05-17 11:06 ` Russell King (Oracle) 2023-05-17 11:08 ` Russell King (Oracle) 2023-05-17 11:08 ` Robin Murphy 2023-05-17 14:35 ` Stefan Wahren 2 siblings, 1 reply; 17+ messages in thread From: Russell King (Oracle) @ 2023-05-17 11:06 UTC (permalink / raw) To: Arnd Bergmann Cc: Stefan Wahren, Robin Murphy, Florian Fainelli, Maxime Ripard, Christoph Hellwig, Phil Elwell, Linux ARM, Alexander Stein On Wed, May 17, 2023 at 12:54:30PM +0200, Arnd Bergmann wrote: > On Wed, May 17, 2023, at 12:34, Arnd Bergmann wrote: > > On Wed, May 17, 2023, at 12:24, Stefan Wahren wrote: > >> Am 17.05.23 um 12:06 schrieb Robin Murphy: > >>> On 2023-05-17 07:23, Stefan Wahren wrote: > > > > I think LPAE doesn't affect it, the configurations that Stefan > > mentioned were LPAE with VMSPLIT_2G (broken), non-LPAE with > > VMSPLIT_3G (works) and LPAE with VMSPLIT_3G (works). I would assume > > that non-LPAE with VMSPLIT_2G or VMSPLIT_3G_OPT is also broken. > > > >> This one? > >> [ 0.000000] software IO TLB: area num 4. > >> [ 0.000000] software IO TLB: mapped [mem > >> 0x0000000069e7f000-0x000000006de7f000] (64MB) > > > > Indeed, that is clearly above the .dma_zone_size value from > > the machine descriptor. > > It looks like Arm does not set the ARCH_LOW_ADDRESS_LIMIT > constant for swiotlb, so it gets the default 4GB maximum. > > I think this would work on arm, though it's probably not the > correct fix in general, as I think MAX_DMA_ADDRESS is meant > to refer to the ISA_DMA_API limit, not the ZONE_DMA limit.: MAX_DMA_ADDRESS is a _virtual_ address. What is ARCH_LOW_ADDRESS_LIMIT? I can find no reference to this in obvious places (mm/ lib/ kernel/ drivers/base/ drivers/iommu/) -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last! _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-17 11:06 ` Russell King (Oracle) @ 2023-05-17 11:08 ` Russell King (Oracle) 0 siblings, 0 replies; 17+ messages in thread From: Russell King (Oracle) @ 2023-05-17 11:08 UTC (permalink / raw) To: Arnd Bergmann Cc: Stefan Wahren, Robin Murphy, Florian Fainelli, Maxime Ripard, Christoph Hellwig, Phil Elwell, Linux ARM, Alexander Stein On Wed, May 17, 2023 at 12:06:35PM +0100, Russell King (Oracle) wrote: > On Wed, May 17, 2023 at 12:54:30PM +0200, Arnd Bergmann wrote: > > On Wed, May 17, 2023, at 12:34, Arnd Bergmann wrote: > > > On Wed, May 17, 2023, at 12:24, Stefan Wahren wrote: > > >> Am 17.05.23 um 12:06 schrieb Robin Murphy: > > >>> On 2023-05-17 07:23, Stefan Wahren wrote: > > > > > > I think LPAE doesn't affect it, the configurations that Stefan > > > mentioned were LPAE with VMSPLIT_2G (broken), non-LPAE with > > > VMSPLIT_3G (works) and LPAE with VMSPLIT_3G (works). I would assume > > > that non-LPAE with VMSPLIT_2G or VMSPLIT_3G_OPT is also broken. > > > > > >> This one? > > >> [ 0.000000] software IO TLB: area num 4. > > >> [ 0.000000] software IO TLB: mapped [mem > > >> 0x0000000069e7f000-0x000000006de7f000] (64MB) > > > > > > Indeed, that is clearly above the .dma_zone_size value from > > > the machine descriptor. > > > > It looks like Arm does not set the ARCH_LOW_ADDRESS_LIMIT > > constant for swiotlb, so it gets the default 4GB maximum. > > > > I think this would work on arm, though it's probably not the > > correct fix in general, as I think MAX_DMA_ADDRESS is meant > > to refer to the ISA_DMA_API limit, not the ZONE_DMA limit.: > > MAX_DMA_ADDRESS is a _virtual_ address. What is ARCH_LOW_ADDRESS_LIMIT? > > I can find no reference to this in obvious places (mm/ lib/ kernel/ > drivers/base/ drivers/iommu/) Found it... it's a physial address. So your patch is almost certainly wrong, since you're passing a virtual address into a function wanting a physical address. -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last! _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-17 10:54 ` Arnd Bergmann 2023-05-17 11:06 ` Russell King (Oracle) @ 2023-05-17 11:08 ` Robin Murphy 2023-05-17 14:35 ` Stefan Wahren 2 siblings, 0 replies; 17+ messages in thread From: Robin Murphy @ 2023-05-17 11:08 UTC (permalink / raw) To: Arnd Bergmann, Stefan Wahren, Florian Fainelli, Maxime Ripard, Christoph Hellwig Cc: Phil Elwell, Linux ARM, Alexander Stein On 2023-05-17 11:54, Arnd Bergmann wrote: > On Wed, May 17, 2023, at 12:34, Arnd Bergmann wrote: >> On Wed, May 17, 2023, at 12:24, Stefan Wahren wrote: >>> Am 17.05.23 um 12:06 schrieb Robin Murphy: >>>> On 2023-05-17 07:23, Stefan Wahren wrote: >> >> I think LPAE doesn't affect it, the configurations that Stefan >> mentioned were LPAE with VMSPLIT_2G (broken), non-LPAE with >> VMSPLIT_3G (works) and LPAE with VMSPLIT_3G (works). I would assume >> that non-LPAE with VMSPLIT_2G or VMSPLIT_3G_OPT is also broken. >> >>> This one? >>> [ 0.000000] software IO TLB: area num 4. >>> [ 0.000000] software IO TLB: mapped [mem >>> 0x0000000069e7f000-0x000000006de7f000] (64MB) >> >> Indeed, that is clearly above the .dma_zone_size value from >> the machine descriptor. > > It looks like Arm does not set the ARCH_LOW_ADDRESS_LIMIT > constant for swiotlb, so it gets the default 4GB maximum. Ah, so memblock_alloc_low() isn't able to do the right thing, that would explain it indeed. > I think this would work on arm, though it's probably not the > correct fix in general, as I think MAX_DMA_ADDRESS is meant > to refer to the ISA_DMA_API limit, not the ZONE_DMA limit.: > > --- a/include/linux/memblock.h > +++ b/include/linux/memblock.h > @@ -394,7 +394,7 @@ static inline int memblock_get_region_node(const struct memblock_region *r) > #define MEMBLOCK_LOW_LIMIT 0 > > #ifndef ARCH_LOW_ADDRESS_LIMIT > -#define ARCH_LOW_ADDRESS_LIMIT 0xffffffffUL > +#define ARCH_LOW_ADDRESS_LIMIT MAX_DMA_ADDRESS > #endif > > phys_addr_t memblock_phys_alloc_range(phys_addr_t size, phys_addr_t align, > > Can you give this a try? Or perhaps do something similar to arm64 and define it in terms of arm_dma_pfn_limit? Cheers, Robin. (And to clear up my own question from earlier, of course LPAE is significant since it's what selects SWIOTLB at all, derp.) _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-17 10:54 ` Arnd Bergmann 2023-05-17 11:06 ` Russell King (Oracle) 2023-05-17 11:08 ` Robin Murphy @ 2023-05-17 14:35 ` Stefan Wahren 2023-05-17 14:45 ` Russell King (Oracle) 2 siblings, 1 reply; 17+ messages in thread From: Stefan Wahren @ 2023-05-17 14:35 UTC (permalink / raw) To: Arnd Bergmann, Robin Murphy, Florian Fainelli, Maxime Ripard, Christoph Hellwig Cc: Phil Elwell, Linux ARM, Alexander Stein Am 17.05.23 um 12:54 schrieb Arnd Bergmann: > On Wed, May 17, 2023, at 12:34, Arnd Bergmann wrote: >> On Wed, May 17, 2023, at 12:24, Stefan Wahren wrote: >>> Am 17.05.23 um 12:06 schrieb Robin Murphy: >>>> On 2023-05-17 07:23, Stefan Wahren wrote: >> >> I think LPAE doesn't affect it, the configurations that Stefan >> mentioned were LPAE with VMSPLIT_2G (broken), non-LPAE with >> VMSPLIT_3G (works) and LPAE with VMSPLIT_3G (works). I would assume >> that non-LPAE with VMSPLIT_2G or VMSPLIT_3G_OPT is also broken. >> >>> This one? >>> [ 0.000000] software IO TLB: area num 4. >>> [ 0.000000] software IO TLB: mapped [mem >>> 0x0000000069e7f000-0x000000006de7f000] (64MB) >> >> Indeed, that is clearly above the .dma_zone_size value from >> the machine descriptor. > > It looks like Arm does not set the ARCH_LOW_ADDRESS_LIMIT > constant for swiotlb, so it gets the default 4GB maximum. > > I think this would work on arm, though it's probably not the > correct fix in general, as I think MAX_DMA_ADDRESS is meant > to refer to the ISA_DMA_API limit, not the ZONE_DMA limit.: > > --- a/include/linux/memblock.h > +++ b/include/linux/memblock.h > @@ -394,7 +394,7 @@ static inline int memblock_get_region_node(const struct memblock_region *r) > #define MEMBLOCK_LOW_LIMIT 0 > > #ifndef ARCH_LOW_ADDRESS_LIMIT > -#define ARCH_LOW_ADDRESS_LIMIT 0xffffffffUL > +#define ARCH_LOW_ADDRESS_LIMIT MAX_DMA_ADDRESS > #endif > > phys_addr_t memblock_phys_alloc_range(phys_addr_t size, phys_addr_t align, > > Can you give this a try? FWIW: Unfortunately this change didn't had any effect. > > Arnd > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-17 14:35 ` Stefan Wahren @ 2023-05-17 14:45 ` Russell King (Oracle) 2023-05-17 17:40 ` Stefan Wahren 0 siblings, 1 reply; 17+ messages in thread From: Russell King (Oracle) @ 2023-05-17 14:45 UTC (permalink / raw) To: Stefan Wahren Cc: Arnd Bergmann, Robin Murphy, Florian Fainelli, Maxime Ripard, Christoph Hellwig, Phil Elwell, Linux ARM, Alexander Stein On Wed, May 17, 2023 at 04:35:35PM +0200, Stefan Wahren wrote: > Am 17.05.23 um 12:54 schrieb Arnd Bergmann: > > On Wed, May 17, 2023, at 12:34, Arnd Bergmann wrote: > > > On Wed, May 17, 2023, at 12:24, Stefan Wahren wrote: > > > > Am 17.05.23 um 12:06 schrieb Robin Murphy: > > > > > On 2023-05-17 07:23, Stefan Wahren wrote: > > > > > > I think LPAE doesn't affect it, the configurations that Stefan > > > mentioned were LPAE with VMSPLIT_2G (broken), non-LPAE with > > > VMSPLIT_3G (works) and LPAE with VMSPLIT_3G (works). I would assume > > > that non-LPAE with VMSPLIT_2G or VMSPLIT_3G_OPT is also broken. > > > > > > > This one? > > > > [ 0.000000] software IO TLB: area num 4. > > > > [ 0.000000] software IO TLB: mapped [mem > > > > 0x0000000069e7f000-0x000000006de7f000] (64MB) > > > > > > Indeed, that is clearly above the .dma_zone_size value from > > > the machine descriptor. > > > > It looks like Arm does not set the ARCH_LOW_ADDRESS_LIMIT > > constant for swiotlb, so it gets the default 4GB maximum. > > > > I think this would work on arm, though it's probably not the > > correct fix in general, as I think MAX_DMA_ADDRESS is meant > > to refer to the ISA_DMA_API limit, not the ZONE_DMA limit.: > > > > --- a/include/linux/memblock.h > > +++ b/include/linux/memblock.h > > @@ -394,7 +394,7 @@ static inline int memblock_get_region_node(const struct memblock_region *r) > > #define MEMBLOCK_LOW_LIMIT 0 > > #ifndef ARCH_LOW_ADDRESS_LIMIT > > -#define ARCH_LOW_ADDRESS_LIMIT 0xffffffffUL > > +#define ARCH_LOW_ADDRESS_LIMIT MAX_DMA_ADDRESS > > #endif > > phys_addr_t memblock_phys_alloc_range(phys_addr_t size, phys_addr_t align, > > > > Can you give this a try? > > FWIW: Unfortunately this change didn't had any effect. As expected, because MAX_DMA_ADDRESS is an integer virtual address, and ARCH_LOW_ADDRESS_LIMIT wants a physical address. I think we need to define ARCH_LOW_ADDRESS_LIMIT in arch/arm/incude/asm/dma.h as: #ifndef CONFIG_ZONE_DMA ... #else ... extern phys_addr_t arm_dma_limit; #define ARCH_LOW_ADDRESS_LIMIT arm_dma_limit #endif -- RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last! _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-17 14:45 ` Russell King (Oracle) @ 2023-05-17 17:40 ` Stefan Wahren 0 siblings, 0 replies; 17+ messages in thread From: Stefan Wahren @ 2023-05-17 17:40 UTC (permalink / raw) To: Russell King (Oracle) Cc: Arnd Bergmann, Robin Murphy, Florian Fainelli, Maxime Ripard, Christoph Hellwig, Phil Elwell, Linux ARM, Alexander Stein Hi Russel, Am 17.05.23 um 16:45 schrieb Russell King (Oracle): > On Wed, May 17, 2023 at 04:35:35PM +0200, Stefan Wahren wrote: >> Am 17.05.23 um 12:54 schrieb Arnd Bergmann: >>> On Wed, May 17, 2023, at 12:34, Arnd Bergmann wrote: >>>> On Wed, May 17, 2023, at 12:24, Stefan Wahren wrote: >>>>> Am 17.05.23 um 12:06 schrieb Robin Murphy: >>>>>> On 2023-05-17 07:23, Stefan Wahren wrote: >>>> >>>> I think LPAE doesn't affect it, the configurations that Stefan >>>> mentioned were LPAE with VMSPLIT_2G (broken), non-LPAE with >>>> VMSPLIT_3G (works) and LPAE with VMSPLIT_3G (works). I would assume >>>> that non-LPAE with VMSPLIT_2G or VMSPLIT_3G_OPT is also broken. >>>> >>>>> This one? >>>>> [ 0.000000] software IO TLB: area num 4. >>>>> [ 0.000000] software IO TLB: mapped [mem >>>>> 0x0000000069e7f000-0x000000006de7f000] (64MB) >>>> >>>> Indeed, that is clearly above the .dma_zone_size value from >>>> the machine descriptor. >>> >>> It looks like Arm does not set the ARCH_LOW_ADDRESS_LIMIT >>> constant for swiotlb, so it gets the default 4GB maximum. >>> >>> I think this would work on arm, though it's probably not the >>> correct fix in general, as I think MAX_DMA_ADDRESS is meant >>> to refer to the ISA_DMA_API limit, not the ZONE_DMA limit.: >>> >>> --- a/include/linux/memblock.h >>> +++ b/include/linux/memblock.h >>> @@ -394,7 +394,7 @@ static inline int memblock_get_region_node(const struct memblock_region *r) >>> #define MEMBLOCK_LOW_LIMIT 0 >>> #ifndef ARCH_LOW_ADDRESS_LIMIT >>> -#define ARCH_LOW_ADDRESS_LIMIT 0xffffffffUL >>> +#define ARCH_LOW_ADDRESS_LIMIT MAX_DMA_ADDRESS >>> #endif >>> phys_addr_t memblock_phys_alloc_range(phys_addr_t size, phys_addr_t align, >>> >>> Can you give this a try? >> >> FWIW: Unfortunately this change didn't had any effect. > > As expected, because MAX_DMA_ADDRESS is an integer virtual address, and > ARCH_LOW_ADDRESS_LIMIT wants a physical address. > > I think we need to define ARCH_LOW_ADDRESS_LIMIT in > arch/arm/incude/asm/dma.h as: > > #ifndef CONFIG_ZONE_DMA > ... > #else > ... > extern phys_addr_t arm_dma_limit; > #define ARCH_LOW_ADDRESS_LIMIT arm_dma_limit > #endif > Yes, the follow patch worked: diff --git a/arch/arm/include/asm/dma.h b/arch/arm/include/asm/dma.h index c6aded1b069c..e2a1916013e7 100644 --- a/arch/arm/include/asm/dma.h +++ b/arch/arm/include/asm/dma.h @@ -12,6 +12,9 @@ extern phys_addr_t arm_dma_zone_size; \ arm_dma_zone_size && arm_dma_zone_size < (0x100000000ULL - PAGE_OFFSET) ? \ (PAGE_OFFSET + arm_dma_zone_size) : 0xffffffffUL; }) + +extern phys_addr_t arm_dma_limit; +#define ARCH_LOW_ADDRESS_LIMIT arm_dma_limit #endif #ifdef CONFIG_ISA_DMA_API I successfully tested the following combinations: Raspberry Pi 4 (8 GB RAM) VMSPLIT_3G + LPAE VMSPLIT_2G + LPAE VMSPLIT_1G + LPAE Raspberry Pi 3 (1 GB RAM) VMSPLIT_3G VMSPLIT_2G Thanks to all for the fast feedback. Unfortunately i don't feel comfortable to prepare a patch with the proper wording. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-17 10:24 ` Stefan Wahren 2023-05-17 10:34 ` Arnd Bergmann @ 2023-05-17 10:36 ` Robin Murphy 1 sibling, 0 replies; 17+ messages in thread From: Robin Murphy @ 2023-05-17 10:36 UTC (permalink / raw) To: Stefan Wahren, Florian Fainelli, Arnd Bergmann, Maxime Ripard, Christoph Hellwig Cc: Phil Elwell, Linux ARM, Alexander Stein On 2023-05-17 11:24, Stefan Wahren wrote: > Hi Robin, > > Am 17.05.23 um 12:06 schrieb Robin Murphy: >> On 2023-05-17 07:23, Stefan Wahren wrote: >>> Hi, >>> >>> Am 17.05.23 um 00:17 schrieb Florian Fainelli: > > ... > >>> >>> [ 1.419785] ------------[ cut here ]------------ >>> [ 1.419811] WARNING: CPU: 2 PID: 1 at >>> include/linux/dma-direct.h:36 dma_map_page_attrs+0x360/0x368 >>> [ 1.419862] Unable to xlate: 0x000000006ffd9000 >> >> This suggests that the SWIOTLB buffer itself has somehow been >> allocated from ZONE_NORMAL rather than ZONE_DMA (unfortunately you >> trimmed the log line from swiotlb_print_info() saying exactly where it >> is). That can't happen with VMSPLIT_3G where we have less than 1GB of >> lowmem anyway, which explains that difference. I don't have an idea >> off-hand why LPAE would affect it. > > This one? > > [ 0.000000] Dentry cache hash table entries: 262144 (order: 8, > 1048576 bytes, linear) > [ 0.000000] Inode-cache hash table entries: 131072 (order: 7, 524288 > bytes, linear) > [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off > [ 0.000000] software IO TLB: area num 4. > [ 0.000000] software IO TLB: mapped [mem > 0x0000000069e7f000-0x000000006de7f000] (64MB) Yup, there you go - I may have slightly misinterpreted the stack trace, but the underlying symptom still stood out. SWIOTLB is unable to help because it can only bounce to somewhere that's still outside the DMA range map. From the initial log, your zones *do* look OK: [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000000000-0x000000003fffefff] [ 0.000000] Normal [mem 0x000000003ffff000-0x000000006fffffff] [ 0.000000] HighMem [mem 0x0000000070000000-0x00000001ffffffff] So apparently something's gone screwy in the area of the swiotlb_init() call. Thanks, Robin. > [ 0.000000] Memory: 3856044K/4050944K available (16384K kernel code, > 2481K rwdata, 6284K rodata, 2048K init, 430K bss, 129364K reserved, > 65536K cma-reserved, 2293760K highmem) > [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 > >> >> Thanks, >> Robin. >> >>> [ 1.419879] Modules linked in: >>> [ 1.419902] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 6.3.0-dirty #11 >>> [ 1.419926] Hardware name: BCM2711 >>> [ 1.419950] unwind_backtrace from show_stack+0x10/0x14 >>> [ 1.419990] show_stack from dump_stack_lvl+0x40/0x4c >>> [ 1.420031] dump_stack_lvl from __warn+0x7c/0x124 >>> [ 1.420077] __warn from warn_slowpath_fmt+0x118/0x174 >>> [ 1.420122] warn_slowpath_fmt from dma_map_page_attrs+0x360/0x368 >>> [ 1.420169] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c >>> [ 1.420215] bcm2835_dma_probe from platform_probe+0x5c/0xbc >>> [ 1.420262] platform_probe from really_probe+0xc8/0x2dc >>> [ 1.420300] really_probe from __driver_probe_device+0x84/0xe4 >>> [ 1.420335] __driver_probe_device from >>> driver_probe_device+0x30/0x104 >>> [ 1.420369] driver_probe_device from __driver_attach+0x90/0x174 >>> [ 1.420403] __driver_attach from bus_for_each_dev+0x6c/0xb4 >>> [ 1.420435] bus_for_each_dev from bus_add_driver+0xcc/0x1cc >>> [ 1.420465] bus_add_driver from driver_register+0x7c/0x118 >>> [ 1.420498] driver_register from do_one_initcall+0x40/0x1e0 >>> [ 1.420534] do_one_initcall from kernel_init_freeable+0x1b8/0x220 >>> [ 1.420578] kernel_init_freeable from kernel_init+0x18/0x12c >>> [ 1.420627] kernel_init from ret_from_fork+0x14/0x1c >>> [ 1.420662] Exception stack(0xf081dfb0 to 0xf081dff8) >>> [ 1.420684] dfa0: 00000000 >>> 00000000 00000000 00000000 >>> [ 1.420710] dfc0: 00000000 00000000 00000000 00000000 00000000 >>> 00000000 00000000 00000000 >>> [ 1.420734] dfe0: 00000000 00000000 00000000 00000000 00000013 >>> 00000000 >>> [ 1.420753] ---[ end trace 0000000000000000 ]--- >>> [ 1.420771] ------------[ cut here ]------------ >>> [ 1.420785] WARNING: CPU: 2 PID: 1 at >>> include/linux/dma-direct.h:36 swiotlb_map+0x1dc/0x3e4 >>> [ 1.420829] Unable to xlate: 0x000000006ffd9000 >>> [ 1.420844] Modules linked in: >>> [ 1.420863] CPU: 2 PID: 1 Comm: swapper/0 Tainted: G W >>> 6.3.0-dirty #11 >>> [ 1.420890] Hardware name: BCM2711 >>> [ 1.420906] unwind_backtrace from show_stack+0x10/0x14 >>> [ 1.420942] show_stack from dump_stack_lvl+0x40/0x4c >>> [ 1.420978] dump_stack_lvl from __warn+0x7c/0x124 >>> [ 1.421021] __warn from warn_slowpath_fmt+0x118/0x174 >>> [ 1.421066] warn_slowpath_fmt from swiotlb_map+0x1dc/0x3e4 >>> [ 1.421114] swiotlb_map from dma_map_page_attrs+0x210/0x368 >>> [ 1.421159] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c >>> [ 1.421201] bcm2835_dma_probe from platform_probe+0x5c/0xbc >>> [ 1.421244] platform_probe from really_probe+0xc8/0x2dc >>> [ 1.421281] really_probe from __driver_probe_device+0x84/0xe4 >>> [ 1.421314] __driver_probe_device from >>> driver_probe_device+0x30/0x104 >>> [ 1.421348] driver_probe_device from __driver_attach+0x90/0x174 >>> [ 1.421382] __driver_attach from bus_for_each_dev+0x6c/0xb4 >>> [ 1.421413] bus_for_each_dev from bus_add_driver+0xcc/0x1cc >>> [ 1.421442] bus_add_driver from driver_register+0x7c/0x118 >>> [ 1.421475] driver_register from do_one_initcall+0x40/0x1e0 >>> [ 1.421509] do_one_initcall from kernel_init_freeable+0x1b8/0x220 >>> [ 1.421551] kernel_init_freeable from kernel_init+0x18/0x12c >>> [ 1.421599] kernel_init from ret_from_fork+0x14/0x1c >>> [ 1.421634] Exception stack(0xf081dfb0 to 0xf081dff8) >>> [ 1.421654] dfa0: 00000000 >>> 00000000 00000000 00000000 >>> [ 1.421679] dfc0: 00000000 00000000 00000000 00000000 00000000 >>> 00000000 00000000 00000000 >>> [ 1.421704] dfe0: 00000000 00000000 00000000 00000000 00000013 >>> 00000000 >>> [ 1.421722] ---[ end trace 0000000000000000 ]--- >>> [ 1.421747] ------------[ cut here ]------------ >>> >>>> >>>> Thanks! >> >> _______________________________________________ >> linux-arm-kernel mailing list >> linux-arm-kernel@lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-16 20:46 ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 Stefan Wahren 2023-05-16 22:17 ` Florian Fainelli @ 2023-05-17 7:37 ` Arnd Bergmann 2023-05-17 10:40 ` Stefan Wahren 1 sibling, 1 reply; 17+ messages in thread From: Arnd Bergmann @ 2023-05-17 7:37 UTC (permalink / raw) To: Stefan Wahren, Alexander Stein, Maxime Ripard Cc: Phil Elwell, Florian Fainelli, Linux ARM On Tue, May 16, 2023, at 22:46, Stefan Wahren wrote: > Hi, > today i wanted to test the new arm/multi_v7_lpae_defconfig on my > Raspberry Pi 4 8GB with Linux 6.4-rc2, but it fails everytime to boot > properly from SD card: > [ 1.435024] warn_slowpath_fmt from swiotlb_map+0x328/0x330 > [ 1.435062] swiotlb_map from dma_map_page_attrs+0x204/0x328 > [ 1.435100] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c > [ 1.435141] bcm2835_dma_probe from platform_probe+0x5c/0xbc ... > So i went back to 6.3, 6.2, 6.1 & 5.15 (enabling the config options > manually on top of multi_v7_defconfig), but it shows the same broken > behavior. > > If i switch back to VMSPLIT_3G and keep ARM_LPAE, the system boots > properly from SD card. > > Any ideas what's the issue here or how to investigate further? I haven't found the exact problem, but I looked at this and have some observations that may help in narrowing it down further: - the bcm2835_dma_probe() maps the zero page only to detect and optimize DMA from zero page, failing to map this would not need to be a fatal error here if we can just skip the optimization - apparently the zero page here is above the 1GB boundary, which leads to the swiotlb-map call instead of just returning the direct-mapped DMA address. If the zero page is swiotlb translated, the optimization in the dma driver becomes pointless, and other bits are likely also slowed down - I suspect that allocating the zero page from low memory would avoid the problem and generally help here, but we still need to understand the rest of the problem. Does this patch avoid the problem? --- a/arch/arm/mm/mmu.c +++ b/arch/arm/mm/mmu.c @@ -1783,7 +1783,7 @@ void __init paging_init(const struct machine_desc *mdesc) top_pmd = pmd_off_k(0xffff0000); /* allocate the zero page. */ - zero_page = early_alloc(PAGE_SIZE); + zero_page = memblock_alloc_low(PAGE_SIZE, PAGE_SIZE); bootmem_init(); - The swiotlb_map function allocates a bounce buffer page and then does a DMA translation. From your log output, it looks like the allocation was successful but the map failed, which would indicate that the bounce buffer page is outside of the addressable range for this device, but I don't see how that can happen - The swiotlb bounce buffers should get allocated from ZONE_DMA if that is enabled, and ZONE_DMA is supposed to be reachable from all DMA master devices in a system - The bcm2835 platform sets the DMA limit to SZ_1G and selects CONFIG_ZONE_DMA, so the swiotlb buffers are meant to work here. - Can you printk() the swiotlb_addr in swiotlb_map() and the "start" address in swiotlb_init_io_tlb_mem() to see if they are in range? Arnd _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-17 7:37 ` Arnd Bergmann @ 2023-05-17 10:40 ` Stefan Wahren 2023-05-17 11:00 ` Arnd Bergmann 0 siblings, 1 reply; 17+ messages in thread From: Stefan Wahren @ 2023-05-17 10:40 UTC (permalink / raw) To: Arnd Bergmann, Alexander Stein, Maxime Ripard Cc: Phil Elwell, Florian Fainelli, Linux ARM Hi Arnd, Am 17.05.23 um 09:37 schrieb Arnd Bergmann: > On Tue, May 16, 2023, at 22:46, Stefan Wahren wrote: >> Hi, >> today i wanted to test the new arm/multi_v7_lpae_defconfig on my >> Raspberry Pi 4 8GB with Linux 6.4-rc2, but it fails everytime to boot >> properly from SD card: > >> [ 1.435024] warn_slowpath_fmt from swiotlb_map+0x328/0x330 >> [ 1.435062] swiotlb_map from dma_map_page_attrs+0x204/0x328 >> [ 1.435100] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c >> [ 1.435141] bcm2835_dma_probe from platform_probe+0x5c/0xbc > ... >> So i went back to 6.3, 6.2, 6.1 & 5.15 (enabling the config options >> manually on top of multi_v7_defconfig), but it shows the same broken >> behavior. >> >> If i switch back to VMSPLIT_3G and keep ARM_LPAE, the system boots >> properly from SD card. >> >> Any ideas what's the issue here or how to investigate further? > > I haven't found the exact problem, but I looked at this and have some > observations that may help in narrowing it down further: > > - the bcm2835_dma_probe() maps the zero page only to detect and > optimize DMA from zero page, failing to map this would not need > to be a fatal error here if we can just skip the optimization > > - apparently the zero page here is above the 1GB boundary, which > leads to the swiotlb-map call instead of just returning the > direct-mapped DMA address. If the zero page is swiotlb translated, > the optimization in the dma driver becomes pointless, and > other bits are likely also slowed down > > - I suspect that allocating the zero page from low memory would > avoid the problem and generally help here, but we still need to > understand the rest of the problem. Does this patch avoid the > problem? > --- a/arch/arm/mm/mmu.c > +++ b/arch/arm/mm/mmu.c > @@ -1783,7 +1783,7 @@ void __init paging_init(const struct machine_desc *mdesc) > top_pmd = pmd_off_k(0xffff0000); > > /* allocate the zero page. */ > - zero_page = early_alloc(PAGE_SIZE); > + zero_page = memblock_alloc_low(PAGE_SIZE, PAGE_SIZE); > > bootmem_init(); > i tried this with ARM_LPAE + VMSPLIT_2G, but i don't see any changes. The warnings are still there and the DMA driver still fails: [ 1.420969] ------------[ cut here ]------------ [ 1.420994] WARNING: CPU: 2 PID: 1 at include/linux/dma-direct.h:36 dma_map_page_attrs+0x360/0x368 [ 1.421044] Unable to xlate: 0x000000006ffd9000 [ 1.421061] Modules linked in: [ 1.421083] CPU: 2 PID: 1 Comm: swapper/0 Not tainted 6.3.0-dirty #12 [ 1.421108] Hardware name: BCM2711 [ 1.421132] unwind_backtrace from show_stack+0x10/0x14 [ 1.421172] show_stack from dump_stack_lvl+0x40/0x4c [ 1.421213] dump_stack_lvl from __warn+0x7c/0x124 [ 1.421259] __warn from warn_slowpath_fmt+0x118/0x174 [ 1.421304] warn_slowpath_fmt from dma_map_page_attrs+0x360/0x368 [ 1.421351] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c [ 1.421397] bcm2835_dma_probe from platform_probe+0x5c/0xbc [ 1.421443] platform_probe from really_probe+0xc8/0x2dc [ 1.421480] really_probe from __driver_probe_device+0x84/0xe4 [ 1.421514] __driver_probe_device from driver_probe_device+0x30/0x104 [ 1.421549] driver_probe_device from __driver_attach+0x90/0x174 [ 1.421582] __driver_attach from bus_for_each_dev+0x6c/0xb4 [ 1.421614] bus_for_each_dev from bus_add_driver+0xcc/0x1cc [ 1.421644] bus_add_driver from driver_register+0x7c/0x118 [ 1.421677] driver_register from do_one_initcall+0x40/0x1e0 [ 1.421713] do_one_initcall from kernel_init_freeable+0x1b8/0x220 [ 1.421758] kernel_init_freeable from kernel_init+0x18/0x12c [ 1.421806] kernel_init from ret_from_fork+0x14/0x1c [ 1.421841] Exception stack(0xf081dfb0 to 0xf081dff8) [ 1.421862] dfa0: 00000000 00000000 00000000 00000000 [ 1.421887] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1.421911] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 1.421929] ---[ end trace 0000000000000000 ]--- [ 1.421948] ------------[ cut here ]------------ [ 1.421963] WARNING: CPU: 2 PID: 1 at include/linux/dma-direct.h:36 swiotlb_map+0x1dc/0x3e4 [ 1.422006] Unable to xlate: 0x000000006ffd9000 [ 1.422022] Modules linked in: [ 1.422041] CPU: 2 PID: 1 Comm: swapper/0 Tainted: G W 6.3.0-dirty #12 [ 1.422068] Hardware name: BCM2711 [ 1.422085] unwind_backtrace from show_stack+0x10/0x14 [ 1.422121] show_stack from dump_stack_lvl+0x40/0x4c [ 1.422156] dump_stack_lvl from __warn+0x7c/0x124 [ 1.422199] __warn from warn_slowpath_fmt+0x118/0x174 [ 1.422244] warn_slowpath_fmt from swiotlb_map+0x1dc/0x3e4 [ 1.422292] swiotlb_map from dma_map_page_attrs+0x210/0x368 [ 1.422336] dma_map_page_attrs from bcm2835_dma_probe+0x1d8/0x41c [ 1.422379] bcm2835_dma_probe from platform_probe+0x5c/0xbc [ 1.422421] platform_probe from really_probe+0xc8/0x2dc [ 1.422458] really_probe from __driver_probe_device+0x84/0xe4 [ 1.422492] __driver_probe_device from driver_probe_device+0x30/0x104 [ 1.422526] driver_probe_device from __driver_attach+0x90/0x174 [ 1.422559] __driver_attach from bus_for_each_dev+0x6c/0xb4 [ 1.422590] bus_for_each_dev from bus_add_driver+0xcc/0x1cc [ 1.422619] bus_add_driver from driver_register+0x7c/0x118 [ 1.422652] driver_register from do_one_initcall+0x40/0x1e0 [ 1.422687] do_one_initcall from kernel_init_freeable+0x1b8/0x220 [ 1.422729] kernel_init_freeable from kernel_init+0x18/0x12c [ 1.422777] kernel_init from ret_from_fork+0x14/0x1c [ 1.422812] Exception stack(0xf081dfb0 to 0xf081dff8) [ 1.422832] dfa0: 00000000 00000000 00000000 00000000 [ 1.422857] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1.422881] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 1.422899] ---[ end trace 0000000000000000 ]--- [ 1.422924] ------------[ cut here ]------------ [ 1.422939] WARNING: CPU: 2 PID: 1 at kernel/dma/swiotlb.c:896 swiotlb_map+0x3d0/0x3e4 [ 1.422981] bcm2835-dma fe007000.dma: swiotlb addr 0xffffffffffffffff+4096 overflow (mask ffffffff, bus limit ffffffff). [ 1.423011] Modules linked in: [ 1.423029] CPU: 2 PID: 1 Comm: swapper/0 Tainted: G W 6.3.0-dirty #12 [ 1.423055] Hardware name: BCM2711 [ 1.423071] unwind_backtrace from show_stack+0x10/0x14 [ 1.423106] show_stack from dump_stack_lvl+0x40/0x4c [ 1.423141] dump_stack_lvl from __warn+0x7c/0x124 [ 1.423184] __warn from warn_slowpath_fmt+0x118/0x174 [ 1.423228] warn_slowpath_fmt from swiot0x18/0x12c [ 1.423758] kernel_init from ret_from_fork+0x14/0x1c [ 1.423793] Exception stack(0xf081dfb0 to 0xf081dff8) [ 1.423812] dfa0: 00000000 00000000 00000000 00000000 [ 1.423837] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1.423861] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 1.423879] ---[ end trace 0000000000000000 ]--- [ 1.423898] bcm2835-dma fe007000.dma: Failed to map zero page [ 1.423916] bcm2835-dma: probe of fe007000.dma failed with error -12 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 2023-05-17 10:40 ` Stefan Wahren @ 2023-05-17 11:00 ` Arnd Bergmann 0 siblings, 0 replies; 17+ messages in thread From: Arnd Bergmann @ 2023-05-17 11:00 UTC (permalink / raw) To: Stefan Wahren, Alexander Stein, Maxime Ripard Cc: Phil Elwell, Florian Fainelli, Linux ARM On Wed, May 17, 2023, at 12:40, Stefan Wahren wrote: > Am 17.05.23 um 09:37 schrieb Arnd Bergmann: >> On Tue, May 16, 2023, at 22:46, Stefan Wahren wrote: >> --- a/arch/arm/mm/mmu.c >> +++ b/arch/arm/mm/mmu.c >> @@ -1783,7 +1783,7 @@ void __init paging_init(const struct machine_desc *mdesc) >> top_pmd = pmd_off_k(0xffff0000); >> >> /* allocate the zero page. */ >> - zero_page = early_alloc(PAGE_SIZE); >> + zero_page = memblock_alloc_low(PAGE_SIZE, PAGE_SIZE); >> >> bootmem_init(); >> > > i tried this with ARM_LPAE + VMSPLIT_2G, but i don't see any changes. > The warnings are still there and the DMA driver still fails: Ok. As I just found and wrote in my latest reply, memblock_alloc_low() is also what gets used in the swiotlb code for the bounce buffer, and the problem is the limit used in there. Setting ARCH_LOW_ADDRESS_LIMIT to an appropriate value should fix the swiotlb, and in combination with the patch above would also mean that the swiotlb bounce is not needed for the zero page. Arnd _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2023-05-17 17:41 UTC | newest] Thread overview: 17+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2023-05-16 20:46 ARM_LPAE + VMSPLIT_2G cause swiotlb warning on Raspberry Pi 4 Stefan Wahren 2023-05-16 22:17 ` Florian Fainelli 2023-05-17 6:23 ` Stefan Wahren 2023-05-17 10:06 ` Robin Murphy 2023-05-17 10:24 ` Stefan Wahren 2023-05-17 10:34 ` Arnd Bergmann 2023-05-17 10:54 ` Arnd Bergmann 2023-05-17 11:06 ` Russell King (Oracle) 2023-05-17 11:08 ` Russell King (Oracle) 2023-05-17 11:08 ` Robin Murphy 2023-05-17 14:35 ` Stefan Wahren 2023-05-17 14:45 ` Russell King (Oracle) 2023-05-17 17:40 ` Stefan Wahren 2023-05-17 10:36 ` Robin Murphy 2023-05-17 7:37 ` Arnd Bergmann 2023-05-17 10:40 ` Stefan Wahren 2023-05-17 11:00 ` Arnd Bergmann
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.