Re: [REGRESSION] Commit 447cbe95ebb9 causes IOMMU DMA faults on macvlan/vlan with bnxt_en

[Eric Dumazet <edumazet@kernel.org>]

On 10/4/26 12:26, Stefan Fleischmann wrote:
> Hi,
> 
> I have bisected a regression introduced in kernel 6.18.51 (and 7.2.3).
> The issue is still present in 6.18.55 and 7.2.9. Verified by reverting
> 1517d1996b52 (the equivalent of 447cbe95ebb9 in the 6.18 branch) on top
> of 6.18.51 and 6.18.55.
> 
> The hardware is a Dell R640 server with Intel Xeon Silver 4210 and
> Broadcom BCM57412 NetXtreme-E NIC.
> 
> The network interface is configured as an untagged interface and has a
> couple of tagged VLAN interfaces configured on top. All of these are
> used by LXC containers with Macvlan.
> 
> The interface comes up fine after boot, but after some time (anywhere
> between 2 to 40 minutes) I see a DMA error in the kernel log (attached),
> and the driver ends up tearing down the interface. It can be brought up
> again after that, but it will hit a DMA error again after a while and
> the interface goes down again.
> 
> I'm happy to provide more debug info if needed, or test patches.

Hi Stefan,

Thanks for the bisect and detailed report.

Notice that 0xfc499000 is on an exact 4KB page boundary. This points to 
a DMA read overrun where the Broadcom DMA engine reads past the end of a 
buffer mapped in page 0xfc498xxx into the adjacent unmapped page 0xfc499000.

Have you tried a recent net kernel ?

Could you please test whether disabling TSO or hardware VLAN offload 
prevents the crash? (maybe adding one option at a time)

   # ethtool -K eno1np0 tso off

   # ethtool -K eno1np0 tx-vlan-offload off

Adding Joe to this thread, because some parts in
tso_dma_map_init() or tso_start() might have bugs.

# Disable USO on the physical interface
ethtool -K eno1np0 tx-udp-segmentation off

This reminds me on a prior attempt I made months ago to sanitize tso_start()

Note that my email address has changed to edumazet@kernel.org

Thanks.

> 
> I've already updated to the latest firmware released by Dell, the problem
> persist. Here is some more info on the network device:
> 
> 18:00.0 Ethernet controller [0200]: Broadcom Inc. and subsidiaries BCM57412 NetXtreme-E 10Gb RDMA Ethernet Controller [14e4:16d6] (rev 01)
> 	Subsystem: Broadcom Inc. and subsidiaries NetXtreme E-Series Advanced Dual-port 10Gb SFP+ Ethernet Network Daughter Card [14e4:4120]
> 
> # ethtool -i eno1np0
> driver: bnxt_en
> version: 7.2.9-1-generic
> firmware-version: 238.1.168.0/pkg 38.11.70.00
> expansion-rom-version:
> bus-info: 0000:18:00.0
> supports-statistics: yes
> supports-test: yes
> supports-eeprom-access: yes
> supports-register-dump: yes
> supports-priv-flags: no
> 
> Best,
> Stefan