* netfilter regression on aarch64 16k page size
@ 2024-12-20 12:15 Amby @ Hyperbeam
2024-12-20 13:35 ` Pablo Neira Ayuso
0 siblings, 1 reply; 6+ messages in thread
From: Amby @ Hyperbeam @ 2024-12-20 12:15 UTC (permalink / raw)
To: stable; +Cc: regressions, pablo
Greetings,
We are seeing a regression in 6.6.66 which I have narrowed down to this commit:
stable: 86c27603514cb8ead29857365cdd145404ee9706
upstream: 7ffc7481153bbabf3332c6a19b289730c7e1edf5
Kernel version: 6.6.66 on aarch64 with 16k page size
Last known good version: 6.6.65
Steps to repro:
- run a 16k page size kernel (check with getconf PAGESIZE)
- try to load an nftables config file on the problem
Expected:
- no errors
Actual:
- system enters a broken state, with the following trace in dmesg:
[ 40.939230] Unable to handle kernel paging request at virtual
address ffff00015ad7e4cc
[ 40.939841] Mem abort info:
[ 40.940079] ESR = 0x0000000096000021
[ 40.940389] EC = 0x25: DABT (current EL), IL = 32 bits
[ 40.940820] SET = 0, FnV = 0
[ 40.941042] EA = 0, S1PTW = 0
[ 40.941289] FSC = 0x21: alignment fault
[ 40.941570] Data abort info:
[ 40.941805] ISV = 0, ISS = 0x00000021, ISS2 = 0x00000000
[ 40.942229] CM = 0, WnR = 0, TnD = 0, TagAccess = 0
[ 40.942857] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[ 40.943313] swapper pgtable: 16k pages, 48-bit VAs, pgdp=00000000474f0000
[ 40.943865] [ffff00015ad7e4cc] pgd=180000043f7e8003,
p4d=180000043f7e8003, pud=180000043f7e4003, pmd=180000043f52c003,
pte=006800019ad7cf07
[ 40.945664] Internal error: Oops: 0000000096000021 [#1] SMP
[ 40.946055] Modules linked in: zstd zram zsmalloc nf_log_syslog
nft_log nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject
nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables
crct10dif_ce polyval_ce polyval_generic ghash_ce tcp_bbr sch_fq fuse
nfnetlink vsock_loopback vmw_vsock_virtio_transport_common
vmw_vsock_vmci_transport vmw_vmci vsock bpf_preload qemu_fw_cfg
ip_tables squashfs virtio_net net_failover virtio_blk gpio_keys
failover virtio_mmio virtio_scsi virtio_console virtio_balloon
virtio_gpu virtio_dma_buf megaraid_sas
[ 40.950262] CPU: 7 PID: 116 Comm: kworker/7:1 Not tainted 6.6.67-1-lts #1
[ 40.951542] Hardware name: netcup KVM Server, BIOS VPS 2000 ARM G11
08/07/2024
[ 40.952287] Workqueue: events_power_efficient nft_rhash_gc [nf_tables]
[ 40.952798] pstate: 20401005 (nzCv daif +PAN -UAO -TCO -DIT +SSBS BTYPE=--)
[ 40.953401] pc : nft_rhash_gc+0x208/0x2c0 [nf_tables]
[ 40.954054] lr : nft_rhash_gc+0x134/0x2c0 [nf_tables]
[ 40.954806] sp : ffff800081fb3cf0
[ 40.955138] x29: ffff800081fb3d50 x28: 0000000000000000 x27: 0000000000000000
[ 40.955974] x26: ffff0000cc760ef0 x25: ffff0000ccba9c80 x24: ffff0000cc758f78
[ 40.956750] x23: 0000000000000010 x22: ffff0000ca789000 x21: ffff0000cc760f78
[ 40.957455] x20: ffffd62d3ac3be40 x19: ffff00015ad7e4c0 x18: 0000000000000000
[ 40.959005] x17: 0000000000000000 x16: ffffd62d37926880 x15: 000000400002fef8
[ 40.959614] x14: ffffffffffffffff x13: 0000000000000030 x12: ffff0000cc760ef0
[ 40.960197] x11: 0000000000000000 x10: ffff800081fb3d08 x9 : ffff0000cba89fe0
[ 40.960739] x8 : 0000000000000003 x7 : 0000000000000000 x6 : 0000000000000000
[ 40.961287] x5 : 0000000000000040 x4 : ffff0000cba89ff0 x3 : ffff00015ad7e4c0
[ 40.961814] x2 : ffff00015ad7e4cc x1 : ffff00015ad7e4cc x0 : 0000000000000004
[ 40.962339] Call trace:
[ 40.962531] nft_rhash_gc+0x208/0x2c0 [nf_tables]
[ 40.962911] process_one_work+0x178/0x3e0
[ 40.963710] worker_thread+0x2ac/0x3e0
[ 40.964530] kthread+0xf0/0x108
[ 40.966259] ret_from_fork+0x10/0x20
[ 40.967287] Code: 54fff9a4 d503201f d2800080 91003261 (f820303f)
[ 40.968245] ---[ end trace 0000000000000000 ]---
faddr2line gave me the following:
nft_rhash_gc+0x208/0x2c0:
__lse_atomic64_or at
/root/gg/setup/linux-lts/src/linux-6.6.67/./arch/arm64/include/asm/atomic_lse.h:132
(inlined by) arch_atomic64_or at
/root/gg/setup/linux-lts/src/linux-6.6.67/./arch/arm64/include/asm/atomic.h:65
(inlined by) raw_atomic64_or at
/root/gg/setup/linux-lts/src/linux-6.6.67/./include/linux/atomic/atomic-arch-fallback.h:3771
(inlined by) raw_atomic_long_or at
/root/gg/setup/linux-lts/src/linux-6.6.67/./include/linux/atomic/atomic-long.h:1069
(inlined by) arch_set_bit at
/root/gg/setup/linux-lts/src/linux-6.6.67/./include/asm-generic/bitops/atomic.h:18
(inlined by) set_bit at
/root/gg/setup/linux-lts/src/linux-6.6.67/./include/asm-generic/bitops/instrumented-atomic.h:29
(inlined by) nft_set_elem_dead at
/root/gg/setup/linux-lts/src/linux-6.6.67/./include/net/netfilter/nf_tables.h:1576
(inlined by) nft_rhash_gc at
/root/gg/setup/linux-lts/src/linux-6.6.67/net/netfilter/nft_set_hash.c:375
By looking at the diff between 6.6.65 and 6.6.66 I was able to narrow
it down to the above commit and I can confirm that reverting it fixes
the issue.
Best
--
Amby Balaji
Co-founder & CTO
Hyperbeam, Inc.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: netfilter regression on aarch64 16k page size
2024-12-20 12:15 netfilter regression on aarch64 16k page size Amby @ Hyperbeam
@ 2024-12-20 13:35 ` Pablo Neira Ayuso
2024-12-20 13:56 ` Pablo Neira Ayuso
0 siblings, 1 reply; 6+ messages in thread
From: Pablo Neira Ayuso @ 2024-12-20 13:35 UTC (permalink / raw)
To: Amby @ Hyperbeam; +Cc: stable, regressions
Hi,
Thanks for your report, it is an unalign atomic that results from
this.
I will post a patch asap to address this.
On Fri, Dec 20, 2024 at 12:15:36PM +0000, Amby @ Hyperbeam wrote:
> Greetings,
> We are seeing a regression in 6.6.66 which I have narrowed down to this commit:
> stable: 86c27603514cb8ead29857365cdd145404ee9706
> upstream: 7ffc7481153bbabf3332c6a19b289730c7e1edf5
>
> Kernel version: 6.6.66 on aarch64 with 16k page size
> Last known good version: 6.6.65
>
> Steps to repro:
> - run a 16k page size kernel (check with getconf PAGESIZE)
> - try to load an nftables config file on the problem
>
> Expected:
> - no errors
>
> Actual:
> - system enters a broken state, with the following trace in dmesg:
> [ 40.939230] Unable to handle kernel paging request at virtual
> address ffff00015ad7e4cc
> [ 40.939841] Mem abort info:
> [ 40.940079] ESR = 0x0000000096000021
> [ 40.940389] EC = 0x25: DABT (current EL), IL = 32 bits
> [ 40.940820] SET = 0, FnV = 0
> [ 40.941042] EA = 0, S1PTW = 0
> [ 40.941289] FSC = 0x21: alignment fault
> [ 40.941570] Data abort info:
> [ 40.941805] ISV = 0, ISS = 0x00000021, ISS2 = 0x00000000
> [ 40.942229] CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> [ 40.942857] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
> [ 40.943313] swapper pgtable: 16k pages, 48-bit VAs, pgdp=00000000474f0000
> [ 40.943865] [ffff00015ad7e4cc] pgd=180000043f7e8003,
> p4d=180000043f7e8003, pud=180000043f7e4003, pmd=180000043f52c003,
> pte=006800019ad7cf07
> [ 40.945664] Internal error: Oops: 0000000096000021 [#1] SMP
> [ 40.946055] Modules linked in: zstd zram zsmalloc nf_log_syslog
> nft_log nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject
> nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables
> crct10dif_ce polyval_ce polyval_generic ghash_ce tcp_bbr sch_fq fuse
> nfnetlink vsock_loopback vmw_vsock_virtio_transport_common
> vmw_vsock_vmci_transport vmw_vmci vsock bpf_preload qemu_fw_cfg
> ip_tables squashfs virtio_net net_failover virtio_blk gpio_keys
> failover virtio_mmio virtio_scsi virtio_console virtio_balloon
> virtio_gpu virtio_dma_buf megaraid_sas
> [ 40.950262] CPU: 7 PID: 116 Comm: kworker/7:1 Not tainted 6.6.67-1-lts #1
> [ 40.951542] Hardware name: netcup KVM Server, BIOS VPS 2000 ARM G11
> 08/07/2024
> [ 40.952287] Workqueue: events_power_efficient nft_rhash_gc [nf_tables]
> [ 40.952798] pstate: 20401005 (nzCv daif +PAN -UAO -TCO -DIT +SSBS BTYPE=--)
> [ 40.953401] pc : nft_rhash_gc+0x208/0x2c0 [nf_tables]
> [ 40.954054] lr : nft_rhash_gc+0x134/0x2c0 [nf_tables]
> [ 40.954806] sp : ffff800081fb3cf0
> [ 40.955138] x29: ffff800081fb3d50 x28: 0000000000000000 x27: 0000000000000000
> [ 40.955974] x26: ffff0000cc760ef0 x25: ffff0000ccba9c80 x24: ffff0000cc758f78
> [ 40.956750] x23: 0000000000000010 x22: ffff0000ca789000 x21: ffff0000cc760f78
> [ 40.957455] x20: ffffd62d3ac3be40 x19: ffff00015ad7e4c0 x18: 0000000000000000
> [ 40.959005] x17: 0000000000000000 x16: ffffd62d37926880 x15: 000000400002fef8
> [ 40.959614] x14: ffffffffffffffff x13: 0000000000000030 x12: ffff0000cc760ef0
> [ 40.960197] x11: 0000000000000000 x10: ffff800081fb3d08 x9 : ffff0000cba89fe0
> [ 40.960739] x8 : 0000000000000003 x7 : 0000000000000000 x6 : 0000000000000000
> [ 40.961287] x5 : 0000000000000040 x4 : ffff0000cba89ff0 x3 : ffff00015ad7e4c0
> [ 40.961814] x2 : ffff00015ad7e4cc x1 : ffff00015ad7e4cc x0 : 0000000000000004
> [ 40.962339] Call trace:
> [ 40.962531] nft_rhash_gc+0x208/0x2c0 [nf_tables]
> [ 40.962911] process_one_work+0x178/0x3e0
> [ 40.963710] worker_thread+0x2ac/0x3e0
> [ 40.964530] kthread+0xf0/0x108
> [ 40.966259] ret_from_fork+0x10/0x20
> [ 40.967287] Code: 54fff9a4 d503201f d2800080 91003261 (f820303f)
> [ 40.968245] ---[ end trace 0000000000000000 ]---
>
> faddr2line gave me the following:
> nft_rhash_gc+0x208/0x2c0:
> __lse_atomic64_or at
> /root/gg/setup/linux-lts/src/linux-6.6.67/./arch/arm64/include/asm/atomic_lse.h:132
> (inlined by) arch_atomic64_or at
> /root/gg/setup/linux-lts/src/linux-6.6.67/./arch/arm64/include/asm/atomic.h:65
> (inlined by) raw_atomic64_or at
> /root/gg/setup/linux-lts/src/linux-6.6.67/./include/linux/atomic/atomic-arch-fallback.h:3771
> (inlined by) raw_atomic_long_or at
> /root/gg/setup/linux-lts/src/linux-6.6.67/./include/linux/atomic/atomic-long.h:1069
> (inlined by) arch_set_bit at
> /root/gg/setup/linux-lts/src/linux-6.6.67/./include/asm-generic/bitops/atomic.h:18
> (inlined by) set_bit at
> /root/gg/setup/linux-lts/src/linux-6.6.67/./include/asm-generic/bitops/instrumented-atomic.h:29
> (inlined by) nft_set_elem_dead at
> /root/gg/setup/linux-lts/src/linux-6.6.67/./include/net/netfilter/nf_tables.h:1576
> (inlined by) nft_rhash_gc at
> /root/gg/setup/linux-lts/src/linux-6.6.67/net/netfilter/nft_set_hash.c:375
>
> By looking at the diff between 6.6.65 and 6.6.66 I was able to narrow
> it down to the above commit and I can confirm that reverting it fixes
> the issue.
>
> Best
> --
> Amby Balaji
> Co-founder & CTO
> Hyperbeam, Inc.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: netfilter regression on aarch64 16k page size
2024-12-20 13:35 ` Pablo Neira Ayuso
@ 2024-12-20 13:56 ` Pablo Neira Ayuso
2024-12-20 20:00 ` Amby @ Hyperbeam
0 siblings, 1 reply; 6+ messages in thread
From: Pablo Neira Ayuso @ 2024-12-20 13:56 UTC (permalink / raw)
To: Amby @ Hyperbeam; +Cc: stable, regressions
[-- Attachment #1: Type: text/plain, Size: 5555 bytes --]
Hi,
Could you give a try to this quick patch?
I will have to add BUILD_BUG_ON() as well to make sure struct
nft_set_ext is aligned so this atomic operation does not break again.
Thanks.
On Fri, Dec 20, 2024 at 02:35:06PM +0100, Pablo Neira Ayuso wrote:
> Hi,
>
> Thanks for your report, it is an unalign atomic that results from
> this.
>
> I will post a patch asap to address this.
>
> On Fri, Dec 20, 2024 at 12:15:36PM +0000, Amby @ Hyperbeam wrote:
> > Greetings,
> > We are seeing a regression in 6.6.66 which I have narrowed down to this commit:
> > stable: 86c27603514cb8ead29857365cdd145404ee9706
> > upstream: 7ffc7481153bbabf3332c6a19b289730c7e1edf5
> >
> > Kernel version: 6.6.66 on aarch64 with 16k page size
> > Last known good version: 6.6.65
> >
> > Steps to repro:
> > - run a 16k page size kernel (check with getconf PAGESIZE)
> > - try to load an nftables config file on the problem
> >
> > Expected:
> > - no errors
> >
> > Actual:
> > - system enters a broken state, with the following trace in dmesg:
> > [ 40.939230] Unable to handle kernel paging request at virtual
> > address ffff00015ad7e4cc
> > [ 40.939841] Mem abort info:
> > [ 40.940079] ESR = 0x0000000096000021
> > [ 40.940389] EC = 0x25: DABT (current EL), IL = 32 bits
> > [ 40.940820] SET = 0, FnV = 0
> > [ 40.941042] EA = 0, S1PTW = 0
> > [ 40.941289] FSC = 0x21: alignment fault
> > [ 40.941570] Data abort info:
> > [ 40.941805] ISV = 0, ISS = 0x00000021, ISS2 = 0x00000000
> > [ 40.942229] CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> > [ 40.942857] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
> > [ 40.943313] swapper pgtable: 16k pages, 48-bit VAs, pgdp=00000000474f0000
> > [ 40.943865] [ffff00015ad7e4cc] pgd=180000043f7e8003,
> > p4d=180000043f7e8003, pud=180000043f7e4003, pmd=180000043f52c003,
> > pte=006800019ad7cf07
> > [ 40.945664] Internal error: Oops: 0000000096000021 [#1] SMP
> > [ 40.946055] Modules linked in: zstd zram zsmalloc nf_log_syslog
> > nft_log nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject
> > nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables
> > crct10dif_ce polyval_ce polyval_generic ghash_ce tcp_bbr sch_fq fuse
> > nfnetlink vsock_loopback vmw_vsock_virtio_transport_common
> > vmw_vsock_vmci_transport vmw_vmci vsock bpf_preload qemu_fw_cfg
> > ip_tables squashfs virtio_net net_failover virtio_blk gpio_keys
> > failover virtio_mmio virtio_scsi virtio_console virtio_balloon
> > virtio_gpu virtio_dma_buf megaraid_sas
> > [ 40.950262] CPU: 7 PID: 116 Comm: kworker/7:1 Not tainted 6.6.67-1-lts #1
> > [ 40.951542] Hardware name: netcup KVM Server, BIOS VPS 2000 ARM G11
> > 08/07/2024
> > [ 40.952287] Workqueue: events_power_efficient nft_rhash_gc [nf_tables]
> > [ 40.952798] pstate: 20401005 (nzCv daif +PAN -UAO -TCO -DIT +SSBS BTYPE=--)
> > [ 40.953401] pc : nft_rhash_gc+0x208/0x2c0 [nf_tables]
> > [ 40.954054] lr : nft_rhash_gc+0x134/0x2c0 [nf_tables]
> > [ 40.954806] sp : ffff800081fb3cf0
> > [ 40.955138] x29: ffff800081fb3d50 x28: 0000000000000000 x27: 0000000000000000
> > [ 40.955974] x26: ffff0000cc760ef0 x25: ffff0000ccba9c80 x24: ffff0000cc758f78
> > [ 40.956750] x23: 0000000000000010 x22: ffff0000ca789000 x21: ffff0000cc760f78
> > [ 40.957455] x20: ffffd62d3ac3be40 x19: ffff00015ad7e4c0 x18: 0000000000000000
> > [ 40.959005] x17: 0000000000000000 x16: ffffd62d37926880 x15: 000000400002fef8
> > [ 40.959614] x14: ffffffffffffffff x13: 0000000000000030 x12: ffff0000cc760ef0
> > [ 40.960197] x11: 0000000000000000 x10: ffff800081fb3d08 x9 : ffff0000cba89fe0
> > [ 40.960739] x8 : 0000000000000003 x7 : 0000000000000000 x6 : 0000000000000000
> > [ 40.961287] x5 : 0000000000000040 x4 : ffff0000cba89ff0 x3 : ffff00015ad7e4c0
> > [ 40.961814] x2 : ffff00015ad7e4cc x1 : ffff00015ad7e4cc x0 : 0000000000000004
> > [ 40.962339] Call trace:
> > [ 40.962531] nft_rhash_gc+0x208/0x2c0 [nf_tables]
> > [ 40.962911] process_one_work+0x178/0x3e0
> > [ 40.963710] worker_thread+0x2ac/0x3e0
> > [ 40.964530] kthread+0xf0/0x108
> > [ 40.966259] ret_from_fork+0x10/0x20
> > [ 40.967287] Code: 54fff9a4 d503201f d2800080 91003261 (f820303f)
> > [ 40.968245] ---[ end trace 0000000000000000 ]---
> >
> > faddr2line gave me the following:
> > nft_rhash_gc+0x208/0x2c0:
> > __lse_atomic64_or at
> > /root/gg/setup/linux-lts/src/linux-6.6.67/./arch/arm64/include/asm/atomic_lse.h:132
> > (inlined by) arch_atomic64_or at
> > /root/gg/setup/linux-lts/src/linux-6.6.67/./arch/arm64/include/asm/atomic.h:65
> > (inlined by) raw_atomic64_or at
> > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/linux/atomic/atomic-arch-fallback.h:3771
> > (inlined by) raw_atomic_long_or at
> > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/linux/atomic/atomic-long.h:1069
> > (inlined by) arch_set_bit at
> > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/asm-generic/bitops/atomic.h:18
> > (inlined by) set_bit at
> > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/asm-generic/bitops/instrumented-atomic.h:29
> > (inlined by) nft_set_elem_dead at
> > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/net/netfilter/nf_tables.h:1576
> > (inlined by) nft_rhash_gc at
> > /root/gg/setup/linux-lts/src/linux-6.6.67/net/netfilter/nft_set_hash.c:375
> >
> > By looking at the diff between 6.6.65 and 6.6.66 I was able to narrow
> > it down to the above commit and I can confirm that reverting it fixes
> > the issue.
> >
> > Best
> > --
> > Amby Balaji
> > Co-founder & CTO
> > Hyperbeam, Inc.
[-- Attachment #2: x.patch --]
[-- Type: text/x-diff, Size: 417 bytes --]
diff --git a/net/netfilter/nft_set_hash.c b/net/netfilter/nft_set_hash.c
index 8bfac4185ac7..614caac77d74 100644
--- a/net/netfilter/nft_set_hash.c
+++ b/net/netfilter/nft_set_hash.c
@@ -31,7 +31,7 @@ struct nft_rhash_elem {
struct nft_elem_priv priv;
struct rhash_head node;
u32 wq_gc_seq;
- struct nft_set_ext ext;
+ struct nft_set_ext ext __aligned(BITS_PER_LONG/8);
};
struct nft_rhash_cmp_arg {
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: netfilter regression on aarch64 16k page size
2024-12-20 13:56 ` Pablo Neira Ayuso
@ 2024-12-20 20:00 ` Amby @ Hyperbeam
2024-12-22 10:04 ` Pablo Neira Ayuso
0 siblings, 1 reply; 6+ messages in thread
From: Amby @ Hyperbeam @ 2024-12-20 20:00 UTC (permalink / raw)
To: Pablo Neira Ayuso; +Cc: stable, regressions
The patch worked, thank you!
On Fri, 20 Dec 2024 at 13:56, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
>
> Hi,
>
> Could you give a try to this quick patch?
>
> I will have to add BUILD_BUG_ON() as well to make sure struct
> nft_set_ext is aligned so this atomic operation does not break again.
>
> Thanks.
>
> On Fri, Dec 20, 2024 at 02:35:06PM +0100, Pablo Neira Ayuso wrote:
> > Hi,
> >
> > Thanks for your report, it is an unalign atomic that results from
> > this.
> >
> > I will post a patch asap to address this.
> >
> > On Fri, Dec 20, 2024 at 12:15:36PM +0000, Amby @ Hyperbeam wrote:
> > > Greetings,
> > > We are seeing a regression in 6.6.66 which I have narrowed down to this commit:
> > > stable: 86c27603514cb8ead29857365cdd145404ee9706
> > > upstream: 7ffc7481153bbabf3332c6a19b289730c7e1edf5
> > >
> > > Kernel version: 6.6.66 on aarch64 with 16k page size
> > > Last known good version: 6.6.65
> > >
> > > Steps to repro:
> > > - run a 16k page size kernel (check with getconf PAGESIZE)
> > > - try to load an nftables config file on the problem
> > >
> > > Expected:
> > > - no errors
> > >
> > > Actual:
> > > - system enters a broken state, with the following trace in dmesg:
> > > [ 40.939230] Unable to handle kernel paging request at virtual
> > > address ffff00015ad7e4cc
> > > [ 40.939841] Mem abort info:
> > > [ 40.940079] ESR = 0x0000000096000021
> > > [ 40.940389] EC = 0x25: DABT (current EL), IL = 32 bits
> > > [ 40.940820] SET = 0, FnV = 0
> > > [ 40.941042] EA = 0, S1PTW = 0
> > > [ 40.941289] FSC = 0x21: alignment fault
> > > [ 40.941570] Data abort info:
> > > [ 40.941805] ISV = 0, ISS = 0x00000021, ISS2 = 0x00000000
> > > [ 40.942229] CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> > > [ 40.942857] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
> > > [ 40.943313] swapper pgtable: 16k pages, 48-bit VAs, pgdp=00000000474f0000
> > > [ 40.943865] [ffff00015ad7e4cc] pgd=180000043f7e8003,
> > > p4d=180000043f7e8003, pud=180000043f7e4003, pmd=180000043f52c003,
> > > pte=006800019ad7cf07
> > > [ 40.945664] Internal error: Oops: 0000000096000021 [#1] SMP
> > > [ 40.946055] Modules linked in: zstd zram zsmalloc nf_log_syslog
> > > nft_log nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject
> > > nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables
> > > crct10dif_ce polyval_ce polyval_generic ghash_ce tcp_bbr sch_fq fuse
> > > nfnetlink vsock_loopback vmw_vsock_virtio_transport_common
> > > vmw_vsock_vmci_transport vmw_vmci vsock bpf_preload qemu_fw_cfg
> > > ip_tables squashfs virtio_net net_failover virtio_blk gpio_keys
> > > failover virtio_mmio virtio_scsi virtio_console virtio_balloon
> > > virtio_gpu virtio_dma_buf megaraid_sas
> > > [ 40.950262] CPU: 7 PID: 116 Comm: kworker/7:1 Not tainted 6.6.67-1-lts #1
> > > [ 40.951542] Hardware name: netcup KVM Server, BIOS VPS 2000 ARM G11
> > > 08/07/2024
> > > [ 40.952287] Workqueue: events_power_efficient nft_rhash_gc [nf_tables]
> > > [ 40.952798] pstate: 20401005 (nzCv daif +PAN -UAO -TCO -DIT +SSBS BTYPE=--)
> > > [ 40.953401] pc : nft_rhash_gc+0x208/0x2c0 [nf_tables]
> > > [ 40.954054] lr : nft_rhash_gc+0x134/0x2c0 [nf_tables]
> > > [ 40.954806] sp : ffff800081fb3cf0
> > > [ 40.955138] x29: ffff800081fb3d50 x28: 0000000000000000 x27: 0000000000000000
> > > [ 40.955974] x26: ffff0000cc760ef0 x25: ffff0000ccba9c80 x24: ffff0000cc758f78
> > > [ 40.956750] x23: 0000000000000010 x22: ffff0000ca789000 x21: ffff0000cc760f78
> > > [ 40.957455] x20: ffffd62d3ac3be40 x19: ffff00015ad7e4c0 x18: 0000000000000000
> > > [ 40.959005] x17: 0000000000000000 x16: ffffd62d37926880 x15: 000000400002fef8
> > > [ 40.959614] x14: ffffffffffffffff x13: 0000000000000030 x12: ffff0000cc760ef0
> > > [ 40.960197] x11: 0000000000000000 x10: ffff800081fb3d08 x9 : ffff0000cba89fe0
> > > [ 40.960739] x8 : 0000000000000003 x7 : 0000000000000000 x6 : 0000000000000000
> > > [ 40.961287] x5 : 0000000000000040 x4 : ffff0000cba89ff0 x3 : ffff00015ad7e4c0
> > > [ 40.961814] x2 : ffff00015ad7e4cc x1 : ffff00015ad7e4cc x0 : 0000000000000004
> > > [ 40.962339] Call trace:
> > > [ 40.962531] nft_rhash_gc+0x208/0x2c0 [nf_tables]
> > > [ 40.962911] process_one_work+0x178/0x3e0
> > > [ 40.963710] worker_thread+0x2ac/0x3e0
> > > [ 40.964530] kthread+0xf0/0x108
> > > [ 40.966259] ret_from_fork+0x10/0x20
> > > [ 40.967287] Code: 54fff9a4 d503201f d2800080 91003261 (f820303f)
> > > [ 40.968245] ---[ end trace 0000000000000000 ]---
> > >
> > > faddr2line gave me the following:
> > > nft_rhash_gc+0x208/0x2c0:
> > > __lse_atomic64_or at
> > > /root/gg/setup/linux-lts/src/linux-6.6.67/./arch/arm64/include/asm/atomic_lse.h:132
> > > (inlined by) arch_atomic64_or at
> > > /root/gg/setup/linux-lts/src/linux-6.6.67/./arch/arm64/include/asm/atomic.h:65
> > > (inlined by) raw_atomic64_or at
> > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/linux/atomic/atomic-arch-fallback.h:3771
> > > (inlined by) raw_atomic_long_or at
> > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/linux/atomic/atomic-long.h:1069
> > > (inlined by) arch_set_bit at
> > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/asm-generic/bitops/atomic.h:18
> > > (inlined by) set_bit at
> > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/asm-generic/bitops/instrumented-atomic.h:29
> > > (inlined by) nft_set_elem_dead at
> > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/net/netfilter/nf_tables.h:1576
> > > (inlined by) nft_rhash_gc at
> > > /root/gg/setup/linux-lts/src/linux-6.6.67/net/netfilter/nft_set_hash.c:375
> > >
> > > By looking at the diff between 6.6.65 and 6.6.66 I was able to narrow
> > > it down to the above commit and I can confirm that reverting it fixes
> > > the issue.
> > >
> > > Best
> > > --
> > > Amby Balaji
> > > Co-founder & CTO
> > > Hyperbeam, Inc.
--
Amby Balaji
Co-founder & CTO
Hyperbeam, Inc.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: netfilter regression on aarch64 16k page size
2024-12-20 20:00 ` Amby @ Hyperbeam
@ 2024-12-22 10:04 ` Pablo Neira Ayuso
2024-12-22 21:38 ` Amby @ Hyperbeam
0 siblings, 1 reply; 6+ messages in thread
From: Pablo Neira Ayuso @ 2024-12-22 10:04 UTC (permalink / raw)
To: Amby @ Hyperbeam; +Cc: stable, regressions
Hi again,
I posted this alternative fix:
https://patchwork.ozlabs.org/project/netfilter-devel/patch/20241222100239.336289-1-pablo@netfilter.org/
in case you have a chance to test it too.
Thanks.
On Fri, Dec 20, 2024 at 08:00:43PM +0000, Amby @ Hyperbeam wrote:
> The patch worked, thank you!
>
> On Fri, 20 Dec 2024 at 13:56, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> >
> > Hi,
> >
> > Could you give a try to this quick patch?
> >
> > I will have to add BUILD_BUG_ON() as well to make sure struct
> > nft_set_ext is aligned so this atomic operation does not break again.
> >
> > Thanks.
> >
> > On Fri, Dec 20, 2024 at 02:35:06PM +0100, Pablo Neira Ayuso wrote:
> > > Hi,
> > >
> > > Thanks for your report, it is an unalign atomic that results from
> > > this.
> > >
> > > I will post a patch asap to address this.
> > >
> > > On Fri, Dec 20, 2024 at 12:15:36PM +0000, Amby @ Hyperbeam wrote:
> > > > Greetings,
> > > > We are seeing a regression in 6.6.66 which I have narrowed down to this commit:
> > > > stable: 86c27603514cb8ead29857365cdd145404ee9706
> > > > upstream: 7ffc7481153bbabf3332c6a19b289730c7e1edf5
> > > >
> > > > Kernel version: 6.6.66 on aarch64 with 16k page size
> > > > Last known good version: 6.6.65
> > > >
> > > > Steps to repro:
> > > > - run a 16k page size kernel (check with getconf PAGESIZE)
> > > > - try to load an nftables config file on the problem
> > > >
> > > > Expected:
> > > > - no errors
> > > >
> > > > Actual:
> > > > - system enters a broken state, with the following trace in dmesg:
> > > > [ 40.939230] Unable to handle kernel paging request at virtual
> > > > address ffff00015ad7e4cc
> > > > [ 40.939841] Mem abort info:
> > > > [ 40.940079] ESR = 0x0000000096000021
> > > > [ 40.940389] EC = 0x25: DABT (current EL), IL = 32 bits
> > > > [ 40.940820] SET = 0, FnV = 0
> > > > [ 40.941042] EA = 0, S1PTW = 0
> > > > [ 40.941289] FSC = 0x21: alignment fault
> > > > [ 40.941570] Data abort info:
> > > > [ 40.941805] ISV = 0, ISS = 0x00000021, ISS2 = 0x00000000
> > > > [ 40.942229] CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> > > > [ 40.942857] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
> > > > [ 40.943313] swapper pgtable: 16k pages, 48-bit VAs, pgdp=00000000474f0000
> > > > [ 40.943865] [ffff00015ad7e4cc] pgd=180000043f7e8003,
> > > > p4d=180000043f7e8003, pud=180000043f7e4003, pmd=180000043f52c003,
> > > > pte=006800019ad7cf07
> > > > [ 40.945664] Internal error: Oops: 0000000096000021 [#1] SMP
> > > > [ 40.946055] Modules linked in: zstd zram zsmalloc nf_log_syslog
> > > > nft_log nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject
> > > > nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables
> > > > crct10dif_ce polyval_ce polyval_generic ghash_ce tcp_bbr sch_fq fuse
> > > > nfnetlink vsock_loopback vmw_vsock_virtio_transport_common
> > > > vmw_vsock_vmci_transport vmw_vmci vsock bpf_preload qemu_fw_cfg
> > > > ip_tables squashfs virtio_net net_failover virtio_blk gpio_keys
> > > > failover virtio_mmio virtio_scsi virtio_console virtio_balloon
> > > > virtio_gpu virtio_dma_buf megaraid_sas
> > > > [ 40.950262] CPU: 7 PID: 116 Comm: kworker/7:1 Not tainted 6.6.67-1-lts #1
> > > > [ 40.951542] Hardware name: netcup KVM Server, BIOS VPS 2000 ARM G11
> > > > 08/07/2024
> > > > [ 40.952287] Workqueue: events_power_efficient nft_rhash_gc [nf_tables]
> > > > [ 40.952798] pstate: 20401005 (nzCv daif +PAN -UAO -TCO -DIT +SSBS BTYPE=--)
> > > > [ 40.953401] pc : nft_rhash_gc+0x208/0x2c0 [nf_tables]
> > > > [ 40.954054] lr : nft_rhash_gc+0x134/0x2c0 [nf_tables]
> > > > [ 40.954806] sp : ffff800081fb3cf0
> > > > [ 40.955138] x29: ffff800081fb3d50 x28: 0000000000000000 x27: 0000000000000000
> > > > [ 40.955974] x26: ffff0000cc760ef0 x25: ffff0000ccba9c80 x24: ffff0000cc758f78
> > > > [ 40.956750] x23: 0000000000000010 x22: ffff0000ca789000 x21: ffff0000cc760f78
> > > > [ 40.957455] x20: ffffd62d3ac3be40 x19: ffff00015ad7e4c0 x18: 0000000000000000
> > > > [ 40.959005] x17: 0000000000000000 x16: ffffd62d37926880 x15: 000000400002fef8
> > > > [ 40.959614] x14: ffffffffffffffff x13: 0000000000000030 x12: ffff0000cc760ef0
> > > > [ 40.960197] x11: 0000000000000000 x10: ffff800081fb3d08 x9 : ffff0000cba89fe0
> > > > [ 40.960739] x8 : 0000000000000003 x7 : 0000000000000000 x6 : 0000000000000000
> > > > [ 40.961287] x5 : 0000000000000040 x4 : ffff0000cba89ff0 x3 : ffff00015ad7e4c0
> > > > [ 40.961814] x2 : ffff00015ad7e4cc x1 : ffff00015ad7e4cc x0 : 0000000000000004
> > > > [ 40.962339] Call trace:
> > > > [ 40.962531] nft_rhash_gc+0x208/0x2c0 [nf_tables]
> > > > [ 40.962911] process_one_work+0x178/0x3e0
> > > > [ 40.963710] worker_thread+0x2ac/0x3e0
> > > > [ 40.964530] kthread+0xf0/0x108
> > > > [ 40.966259] ret_from_fork+0x10/0x20
> > > > [ 40.967287] Code: 54fff9a4 d503201f d2800080 91003261 (f820303f)
> > > > [ 40.968245] ---[ end trace 0000000000000000 ]---
> > > >
> > > > faddr2line gave me the following:
> > > > nft_rhash_gc+0x208/0x2c0:
> > > > __lse_atomic64_or at
> > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./arch/arm64/include/asm/atomic_lse.h:132
> > > > (inlined by) arch_atomic64_or at
> > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./arch/arm64/include/asm/atomic.h:65
> > > > (inlined by) raw_atomic64_or at
> > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/linux/atomic/atomic-arch-fallback.h:3771
> > > > (inlined by) raw_atomic_long_or at
> > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/linux/atomic/atomic-long.h:1069
> > > > (inlined by) arch_set_bit at
> > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/asm-generic/bitops/atomic.h:18
> > > > (inlined by) set_bit at
> > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/asm-generic/bitops/instrumented-atomic.h:29
> > > > (inlined by) nft_set_elem_dead at
> > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/net/netfilter/nf_tables.h:1576
> > > > (inlined by) nft_rhash_gc at
> > > > /root/gg/setup/linux-lts/src/linux-6.6.67/net/netfilter/nft_set_hash.c:375
> > > >
> > > > By looking at the diff between 6.6.65 and 6.6.66 I was able to narrow
> > > > it down to the above commit and I can confirm that reverting it fixes
> > > > the issue.
> > > >
> > > > Best
> > > > --
> > > > Amby Balaji
> > > > Co-founder & CTO
> > > > Hyperbeam, Inc.
>
>
>
> --
> Amby Balaji
> Co-founder & CTO
> Hyperbeam, Inc.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: netfilter regression on aarch64 16k page size
2024-12-22 10:04 ` Pablo Neira Ayuso
@ 2024-12-22 21:38 ` Amby @ Hyperbeam
0 siblings, 0 replies; 6+ messages in thread
From: Amby @ Hyperbeam @ 2024-12-22 21:38 UTC (permalink / raw)
To: Pablo Neira Ayuso; +Cc: stable, regressions
This also works
On Sun, 22 Dec 2024 at 10:05, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
>
> Hi again,
>
> I posted this alternative fix:
>
> https://patchwork.ozlabs.org/project/netfilter-devel/patch/20241222100239.336289-1-pablo@netfilter.org/
>
> in case you have a chance to test it too.
>
> Thanks.
>
> On Fri, Dec 20, 2024 at 08:00:43PM +0000, Amby @ Hyperbeam wrote:
> > The patch worked, thank you!
> >
> > On Fri, 20 Dec 2024 at 13:56, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> > >
> > > Hi,
> > >
> > > Could you give a try to this quick patch?
> > >
> > > I will have to add BUILD_BUG_ON() as well to make sure struct
> > > nft_set_ext is aligned so this atomic operation does not break again.
> > >
> > > Thanks.
> > >
> > > On Fri, Dec 20, 2024 at 02:35:06PM +0100, Pablo Neira Ayuso wrote:
> > > > Hi,
> > > >
> > > > Thanks for your report, it is an unalign atomic that results from
> > > > this.
> > > >
> > > > I will post a patch asap to address this.
> > > >
> > > > On Fri, Dec 20, 2024 at 12:15:36PM +0000, Amby @ Hyperbeam wrote:
> > > > > Greetings,
> > > > > We are seeing a regression in 6.6.66 which I have narrowed down to this commit:
> > > > > stable: 86c27603514cb8ead29857365cdd145404ee9706
> > > > > upstream: 7ffc7481153bbabf3332c6a19b289730c7e1edf5
> > > > >
> > > > > Kernel version: 6.6.66 on aarch64 with 16k page size
> > > > > Last known good version: 6.6.65
> > > > >
> > > > > Steps to repro:
> > > > > - run a 16k page size kernel (check with getconf PAGESIZE)
> > > > > - try to load an nftables config file on the problem
> > > > >
> > > > > Expected:
> > > > > - no errors
> > > > >
> > > > > Actual:
> > > > > - system enters a broken state, with the following trace in dmesg:
> > > > > [ 40.939230] Unable to handle kernel paging request at virtual
> > > > > address ffff00015ad7e4cc
> > > > > [ 40.939841] Mem abort info:
> > > > > [ 40.940079] ESR = 0x0000000096000021
> > > > > [ 40.940389] EC = 0x25: DABT (current EL), IL = 32 bits
> > > > > [ 40.940820] SET = 0, FnV = 0
> > > > > [ 40.941042] EA = 0, S1PTW = 0
> > > > > [ 40.941289] FSC = 0x21: alignment fault
> > > > > [ 40.941570] Data abort info:
> > > > > [ 40.941805] ISV = 0, ISS = 0x00000021, ISS2 = 0x00000000
> > > > > [ 40.942229] CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> > > > > [ 40.942857] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
> > > > > [ 40.943313] swapper pgtable: 16k pages, 48-bit VAs, pgdp=00000000474f0000
> > > > > [ 40.943865] [ffff00015ad7e4cc] pgd=180000043f7e8003,
> > > > > p4d=180000043f7e8003, pud=180000043f7e4003, pmd=180000043f52c003,
> > > > > pte=006800019ad7cf07
> > > > > [ 40.945664] Internal error: Oops: 0000000096000021 [#1] SMP
> > > > > [ 40.946055] Modules linked in: zstd zram zsmalloc nf_log_syslog
> > > > > nft_log nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject
> > > > > nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables
> > > > > crct10dif_ce polyval_ce polyval_generic ghash_ce tcp_bbr sch_fq fuse
> > > > > nfnetlink vsock_loopback vmw_vsock_virtio_transport_common
> > > > > vmw_vsock_vmci_transport vmw_vmci vsock bpf_preload qemu_fw_cfg
> > > > > ip_tables squashfs virtio_net net_failover virtio_blk gpio_keys
> > > > > failover virtio_mmio virtio_scsi virtio_console virtio_balloon
> > > > > virtio_gpu virtio_dma_buf megaraid_sas
> > > > > [ 40.950262] CPU: 7 PID: 116 Comm: kworker/7:1 Not tainted 6.6.67-1-lts #1
> > > > > [ 40.951542] Hardware name: netcup KVM Server, BIOS VPS 2000 ARM G11
> > > > > 08/07/2024
> > > > > [ 40.952287] Workqueue: events_power_efficient nft_rhash_gc [nf_tables]
> > > > > [ 40.952798] pstate: 20401005 (nzCv daif +PAN -UAO -TCO -DIT +SSBS BTYPE=--)
> > > > > [ 40.953401] pc : nft_rhash_gc+0x208/0x2c0 [nf_tables]
> > > > > [ 40.954054] lr : nft_rhash_gc+0x134/0x2c0 [nf_tables]
> > > > > [ 40.954806] sp : ffff800081fb3cf0
> > > > > [ 40.955138] x29: ffff800081fb3d50 x28: 0000000000000000 x27: 0000000000000000
> > > > > [ 40.955974] x26: ffff0000cc760ef0 x25: ffff0000ccba9c80 x24: ffff0000cc758f78
> > > > > [ 40.956750] x23: 0000000000000010 x22: ffff0000ca789000 x21: ffff0000cc760f78
> > > > > [ 40.957455] x20: ffffd62d3ac3be40 x19: ffff00015ad7e4c0 x18: 0000000000000000
> > > > > [ 40.959005] x17: 0000000000000000 x16: ffffd62d37926880 x15: 000000400002fef8
> > > > > [ 40.959614] x14: ffffffffffffffff x13: 0000000000000030 x12: ffff0000cc760ef0
> > > > > [ 40.960197] x11: 0000000000000000 x10: ffff800081fb3d08 x9 : ffff0000cba89fe0
> > > > > [ 40.960739] x8 : 0000000000000003 x7 : 0000000000000000 x6 : 0000000000000000
> > > > > [ 40.961287] x5 : 0000000000000040 x4 : ffff0000cba89ff0 x3 : ffff00015ad7e4c0
> > > > > [ 40.961814] x2 : ffff00015ad7e4cc x1 : ffff00015ad7e4cc x0 : 0000000000000004
> > > > > [ 40.962339] Call trace:
> > > > > [ 40.962531] nft_rhash_gc+0x208/0x2c0 [nf_tables]
> > > > > [ 40.962911] process_one_work+0x178/0x3e0
> > > > > [ 40.963710] worker_thread+0x2ac/0x3e0
> > > > > [ 40.964530] kthread+0xf0/0x108
> > > > > [ 40.966259] ret_from_fork+0x10/0x20
> > > > > [ 40.967287] Code: 54fff9a4 d503201f d2800080 91003261 (f820303f)
> > > > > [ 40.968245] ---[ end trace 0000000000000000 ]---
> > > > >
> > > > > faddr2line gave me the following:
> > > > > nft_rhash_gc+0x208/0x2c0:
> > > > > __lse_atomic64_or at
> > > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./arch/arm64/include/asm/atomic_lse.h:132
> > > > > (inlined by) arch_atomic64_or at
> > > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./arch/arm64/include/asm/atomic.h:65
> > > > > (inlined by) raw_atomic64_or at
> > > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/linux/atomic/atomic-arch-fallback.h:3771
> > > > > (inlined by) raw_atomic_long_or at
> > > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/linux/atomic/atomic-long.h:1069
> > > > > (inlined by) arch_set_bit at
> > > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/asm-generic/bitops/atomic.h:18
> > > > > (inlined by) set_bit at
> > > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/asm-generic/bitops/instrumented-atomic.h:29
> > > > > (inlined by) nft_set_elem_dead at
> > > > > /root/gg/setup/linux-lts/src/linux-6.6.67/./include/net/netfilter/nf_tables.h:1576
> > > > > (inlined by) nft_rhash_gc at
> > > > > /root/gg/setup/linux-lts/src/linux-6.6.67/net/netfilter/nft_set_hash.c:375
> > > > >
> > > > > By looking at the diff between 6.6.65 and 6.6.66 I was able to narrow
> > > > > it down to the above commit and I can confirm that reverting it fixes
> > > > > the issue.
> > > > >
> > > > > Best
> > > > > --
> > > > > Amby Balaji
> > > > > Co-founder & CTO
> > > > > Hyperbeam, Inc.
> >
> >
> >
> > --
> > Amby Balaji
> > Co-founder & CTO
> > Hyperbeam, Inc.
--
Amby Balaji
Co-founder & CTO
Hyperbeam, Inc.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-12-22 21:39 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-12-20 12:15 netfilter regression on aarch64 16k page size Amby @ Hyperbeam
2024-12-20 13:35 ` Pablo Neira Ayuso
2024-12-20 13:56 ` Pablo Neira Ayuso
2024-12-20 20:00 ` Amby @ Hyperbeam
2024-12-22 10:04 ` Pablo Neira Ayuso
2024-12-22 21:38 ` Amby @ Hyperbeam
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox