* [PATCH net 1/4] net/mlx5e: Don't delay release of hardware objects
2023-06-05 8:09 [PATCH net 0/4] Fix mixing atomic/non-atomic contexts in mlx5 IPsec code Leon Romanovsky
@ 2023-06-05 8:09 ` Leon Romanovsky
2023-06-06 9:07 ` Simon Horman
2023-06-05 8:09 ` [PATCH net 2/4] net/mlx5e: Fix ESN update kernel panic Leon Romanovsky
` (2 subsequent siblings)
3 siblings, 1 reply; 9+ messages in thread
From: Leon Romanovsky @ 2023-06-05 8:09 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Leon Romanovsky, Eric Dumazet, netdev, Paolo Abeni,
Patrisious Haddad, Raed Salem, Saeed Mahameed, Steffen Klassert
From: Leon Romanovsky <leonro@nvidia.com>
XFRM core provides two callbacks to release resources, one is .xdo_dev_policy_delete()
and another is .xdo_dev_policy_free(). This separation allows delayed release so
"ip xfrm policy free" commands won't starve. Unfortunately, mlx5 command interface
can't run in .xdo_dev_policy_free() callbacks as the latter runs in ATOMIC context.
BUG: scheduling while atomic: swapper/7/0/0x00000100
Modules linked in: act_mirred act_tunnel_key cls_flower sch_ingress vxlan mlx5_vdpa vringh vhost_iotlb vdpa rpcrdma rdma_ucm ib_iser libiscsi ib_umad scsi_transport_iscsi rdma_cm ib_ipoib iw_cm ib_cm mlx5_ib ib_uverbs ib_core xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat br_netfilter rpcsec_gss_krb5 auth_rpcgss oid_registry overlay mlx5_core zram zsmalloc fuse
CPU: 7 PID: 0 Comm: swapper/7 Not tainted 6.3.0+ #1
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
Call Trace:
<IRQ>
dump_stack_lvl+0x33/0x50
__schedule_bug+0x4e/0x60
__schedule+0x5d5/0x780
? __mod_timer+0x286/0x3d0
schedule+0x50/0x90
schedule_timeout+0x7c/0xf0
? __bpf_trace_tick_stop+0x10/0x10
__wait_for_common+0x88/0x190
? usleep_range_state+0x90/0x90
cmd_exec+0x42e/0xb40 [mlx5_core]
mlx5_cmd_do+0x1e/0x40 [mlx5_core]
mlx5_cmd_exec+0x18/0x30 [mlx5_core]
mlx5_cmd_delete_fte+0xa8/0xd0 [mlx5_core]
del_hw_fte+0x60/0x120 [mlx5_core]
mlx5_del_flow_rules+0xec/0x270 [mlx5_core]
? default_send_IPI_single_phys+0x26/0x30
mlx5e_accel_ipsec_fs_del_pol+0x1a/0x60 [mlx5_core]
mlx5e_xfrm_free_policy+0x15/0x20 [mlx5_core]
xfrm_policy_destroy+0x5a/0xb0
xfrm4_dst_destroy+0x7b/0x100
dst_destroy+0x37/0x120
rcu_core+0x2d6/0x540
__do_softirq+0xcd/0x273
irq_exit_rcu+0x82/0xb0
sysvec_apic_timer_interrupt+0x72/0x90
</IRQ>
<TASK>
asm_sysvec_apic_timer_interrupt+0x16/0x20
RIP: 0010:default_idle+0x13/0x20
Code: c0 08 00 00 00 4d 29 c8 4c 01 c7 4c 29 c2 e9 72 ff ff ff cc cc cc cc 8b 05 7a 4d ee 00 85 c0 7e 07 0f 00 2d 2f 98 2e 00 fb f4 <fa> c3 66 66 2e 0f 1f 84 00 00 00 00 00 65 48 8b 04 25 40 b4 02 00
RSP: 0018:ffff888100843ee0 EFLAGS: 00000242
RAX: 0000000000000001 RBX: ffff888100812b00 RCX: 4000000000000000
RDX: 0000000000000001 RSI: 0000000000000083 RDI: 000000000002d2ec
RBP: 0000000000000007 R08: 00000021daeded59 R09: 0000000000000001
R10: 0000000000000000 R11: 000000000000000f R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
default_idle_call+0x30/0xb0
do_idle+0x1c1/0x1d0
cpu_startup_entry+0x19/0x20
start_secondary+0xfe/0x120
secondary_startup_64_no_verify+0xf3/0xfb
</TASK>
bad: scheduling from the idle thread!
Fixes: a5b8ca9471d3 ("net/mlx5e: Add XFRM policy offload logic")
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
index 55b38544422f..d1c801723d35 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
@@ -1040,11 +1040,17 @@ static int mlx5e_xfrm_add_policy(struct xfrm_policy *x,
return err;
}
-static void mlx5e_xfrm_free_policy(struct xfrm_policy *x)
+static void mlx5e_xfrm_del_policy(struct xfrm_policy *x)
{
struct mlx5e_ipsec_pol_entry *pol_entry = to_ipsec_pol_entry(x);
mlx5e_accel_ipsec_fs_del_pol(pol_entry);
+}
+
+static void mlx5e_xfrm_free_policy(struct xfrm_policy *x)
+{
+ struct mlx5e_ipsec_pol_entry *pol_entry = to_ipsec_pol_entry(x);
+
kfree(pol_entry);
}
@@ -1065,6 +1071,7 @@ static const struct xfrmdev_ops mlx5e_ipsec_packet_xfrmdev_ops = {
.xdo_dev_state_update_curlft = mlx5e_xfrm_update_curlft,
.xdo_dev_policy_add = mlx5e_xfrm_add_policy,
+ .xdo_dev_policy_delete = mlx5e_xfrm_del_policy,
.xdo_dev_policy_free = mlx5e_xfrm_free_policy,
};
--
2.40.1
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH net 1/4] net/mlx5e: Don't delay release of hardware objects
2023-06-05 8:09 ` [PATCH net 1/4] net/mlx5e: Don't delay release of hardware objects Leon Romanovsky
@ 2023-06-06 9:07 ` Simon Horman
0 siblings, 0 replies; 9+ messages in thread
From: Simon Horman @ 2023-06-06 9:07 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Jakub Kicinski, Leon Romanovsky, Eric Dumazet, netdev,
Paolo Abeni, Patrisious Haddad, Raed Salem, Saeed Mahameed,
Steffen Klassert
On Mon, Jun 05, 2023 at 11:09:49AM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@nvidia.com>
>
> XFRM core provides two callbacks to release resources, one is .xdo_dev_policy_delete()
> and another is .xdo_dev_policy_free(). This separation allows delayed release so
> "ip xfrm policy free" commands won't starve. Unfortunately, mlx5 command interface
> can't run in .xdo_dev_policy_free() callbacks as the latter runs in ATOMIC context.
>
> BUG: scheduling while atomic: swapper/7/0/0x00000100
> Modules linked in: act_mirred act_tunnel_key cls_flower sch_ingress vxlan mlx5_vdpa vringh vhost_iotlb vdpa rpcrdma rdma_ucm ib_iser libiscsi ib_umad scsi_transport_iscsi rdma_cm ib_ipoib iw_cm ib_cm mlx5_ib ib_uverbs ib_core xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat br_netfilter rpcsec_gss_krb5 auth_rpcgss oid_registry overlay mlx5_core zram zsmalloc fuse
> CPU: 7 PID: 0 Comm: swapper/7 Not tainted 6.3.0+ #1
> Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
> Call Trace:
> <IRQ>
> dump_stack_lvl+0x33/0x50
> __schedule_bug+0x4e/0x60
> __schedule+0x5d5/0x780
> ? __mod_timer+0x286/0x3d0
> schedule+0x50/0x90
> schedule_timeout+0x7c/0xf0
> ? __bpf_trace_tick_stop+0x10/0x10
> __wait_for_common+0x88/0x190
> ? usleep_range_state+0x90/0x90
> cmd_exec+0x42e/0xb40 [mlx5_core]
> mlx5_cmd_do+0x1e/0x40 [mlx5_core]
> mlx5_cmd_exec+0x18/0x30 [mlx5_core]
> mlx5_cmd_delete_fte+0xa8/0xd0 [mlx5_core]
> del_hw_fte+0x60/0x120 [mlx5_core]
> mlx5_del_flow_rules+0xec/0x270 [mlx5_core]
> ? default_send_IPI_single_phys+0x26/0x30
> mlx5e_accel_ipsec_fs_del_pol+0x1a/0x60 [mlx5_core]
> mlx5e_xfrm_free_policy+0x15/0x20 [mlx5_core]
> xfrm_policy_destroy+0x5a/0xb0
> xfrm4_dst_destroy+0x7b/0x100
> dst_destroy+0x37/0x120
> rcu_core+0x2d6/0x540
> __do_softirq+0xcd/0x273
> irq_exit_rcu+0x82/0xb0
> sysvec_apic_timer_interrupt+0x72/0x90
> </IRQ>
> <TASK>
> asm_sysvec_apic_timer_interrupt+0x16/0x20
> RIP: 0010:default_idle+0x13/0x20
> Code: c0 08 00 00 00 4d 29 c8 4c 01 c7 4c 29 c2 e9 72 ff ff ff cc cc cc cc 8b 05 7a 4d ee 00 85 c0 7e 07 0f 00 2d 2f 98 2e 00 fb f4 <fa> c3 66 66 2e 0f 1f 84 00 00 00 00 00 65 48 8b 04 25 40 b4 02 00
> RSP: 0018:ffff888100843ee0 EFLAGS: 00000242
> RAX: 0000000000000001 RBX: ffff888100812b00 RCX: 4000000000000000
> RDX: 0000000000000001 RSI: 0000000000000083 RDI: 000000000002d2ec
> RBP: 0000000000000007 R08: 00000021daeded59 R09: 0000000000000001
> R10: 0000000000000000 R11: 000000000000000f R12: 0000000000000000
> R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> default_idle_call+0x30/0xb0
> do_idle+0x1c1/0x1d0
> cpu_startup_entry+0x19/0x20
> start_secondary+0xfe/0x120
> secondary_startup_64_no_verify+0xf3/0xfb
> </TASK>
> bad: scheduling from the idle thread!
>
> Fixes: a5b8ca9471d3 ("net/mlx5e: Add XFRM policy offload logic")
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH net 2/4] net/mlx5e: Fix ESN update kernel panic
2023-06-05 8:09 [PATCH net 0/4] Fix mixing atomic/non-atomic contexts in mlx5 IPsec code Leon Romanovsky
2023-06-05 8:09 ` [PATCH net 1/4] net/mlx5e: Don't delay release of hardware objects Leon Romanovsky
@ 2023-06-05 8:09 ` Leon Romanovsky
2023-06-06 9:08 ` Simon Horman
2023-06-05 8:09 ` [PATCH net 3/4] net/mlx5e: Drop XFRM state lock when modifying flow steering Leon Romanovsky
2023-06-05 8:09 ` [PATCH net 4/4] net/mlx5e: Fix scheduling of IPsec ASO query while in atomic Leon Romanovsky
3 siblings, 1 reply; 9+ messages in thread
From: Leon Romanovsky @ 2023-06-05 8:09 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Patrisious Haddad, Eric Dumazet, netdev, Paolo Abeni, Raed Salem,
Saeed Mahameed, Steffen Klassert
From: Patrisious Haddad <phaddad@nvidia.com>
Previously during mlx5e_ipsec_handle_event the driver tried to execute
an operation that could sleep, while holding a spinlock, which caused
the kernel panic mentioned below.
Move the function call that can sleep outside of the spinlock context.
Call Trace:
<TASK>
dump_stack_lvl+0x49/0x6c
__schedule_bug.cold+0x42/0x4e
schedule_debug.constprop.0+0xe0/0x118
__schedule+0x59/0x58a
? __mod_timer+0x2a1/0x3ef
schedule+0x5e/0xd4
schedule_timeout+0x99/0x164
? __pfx_process_timeout+0x10/0x10
__wait_for_common+0x90/0x1da
? __pfx_schedule_timeout+0x10/0x10
wait_func+0x34/0x142 [mlx5_core]
mlx5_cmd_invoke+0x1f3/0x313 [mlx5_core]
cmd_exec+0x1fe/0x325 [mlx5_core]
mlx5_cmd_do+0x22/0x50 [mlx5_core]
mlx5_cmd_exec+0x1c/0x40 [mlx5_core]
mlx5_modify_ipsec_obj+0xb2/0x17f [mlx5_core]
mlx5e_ipsec_update_esn_state+0x69/0xf0 [mlx5_core]
? wake_affine+0x62/0x1f8
mlx5e_ipsec_handle_event+0xb1/0xc0 [mlx5_core]
process_one_work+0x1e2/0x3e6
? __pfx_worker_thread+0x10/0x10
worker_thread+0x54/0x3ad
? __pfx_worker_thread+0x10/0x10
kthread+0xda/0x101
? __pfx_kthread+0x10/0x10
ret_from_fork+0x29/0x37
</TASK>
BUG: workqueue leaked lock or atomic: kworker/u256:4/0x7fffffff/189754#012 last function: mlx5e_ipsec_handle_event [mlx5_core]
CPU: 66 PID: 189754 Comm: kworker/u256:4 Kdump: loaded Tainted: G W 6.2.0-2596.20230309201517_5.el8uek.rc1.x86_64 #2
Hardware name: Oracle Corporation ORACLE SERVER X9-2/ASMMBX9-2, BIOS 61070300 08/17/2022
Workqueue: mlx5e_ipsec: eth%d mlx5e_ipsec_handle_event [mlx5_core]
Call Trace:
<TASK>
dump_stack_lvl+0x49/0x6c
process_one_work.cold+0x2b/0x3c
? __pfx_worker_thread+0x10/0x10
worker_thread+0x54/0x3ad
? __pfx_worker_thread+0x10/0x10
kthread+0xda/0x101
? __pfx_kthread+0x10/0x10
ret_from_fork+0x29/0x37
</TASK>
BUG: scheduling while atomic: kworker/u256:4/189754/0x00000000
Fixes: cee137a63431 ("net/mlx5e: Handle ESN update events")
Signed-off-by: Patrisious Haddad <phaddad@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
.../mellanox/mlx5/core/en_accel/ipsec_offload.c | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c
index df90e19066bc..ca16cb9807ea 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c
@@ -305,7 +305,17 @@ static void mlx5e_ipsec_update_esn_state(struct mlx5e_ipsec_sa_entry *sa_entry,
}
mlx5e_ipsec_build_accel_xfrm_attrs(sa_entry, &attrs);
+
+ /* It is safe to execute the modify below unlocked since the only flows
+ * that could affect this HW object, are create, destroy and this work.
+ *
+ * Creation flow can't co-exist with this modify work, the destruction
+ * flow would cancel this work, and this work is a single entity that
+ * can't conflict with it self.
+ */
+ spin_unlock_bh(&sa_entry->x->lock);
mlx5_accel_esp_modify_xfrm(sa_entry, &attrs);
+ spin_lock_bh(&sa_entry->x->lock);
data.data_offset_condition_operand =
MLX5_IPSEC_ASO_REMOVE_FLOW_PKT_CNT_OFFSET;
@@ -431,7 +441,7 @@ static void mlx5e_ipsec_handle_event(struct work_struct *_work)
aso = sa_entry->ipsec->aso;
attrs = &sa_entry->attrs;
- spin_lock(&sa_entry->x->lock);
+ spin_lock_bh(&sa_entry->x->lock);
ret = mlx5e_ipsec_aso_query(sa_entry, NULL);
if (ret)
goto unlock;
@@ -447,7 +457,7 @@ static void mlx5e_ipsec_handle_event(struct work_struct *_work)
mlx5e_ipsec_handle_limits(sa_entry);
unlock:
- spin_unlock(&sa_entry->x->lock);
+ spin_unlock_bh(&sa_entry->x->lock);
kfree(work);
}
--
2.40.1
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH net 2/4] net/mlx5e: Fix ESN update kernel panic
2023-06-05 8:09 ` [PATCH net 2/4] net/mlx5e: Fix ESN update kernel panic Leon Romanovsky
@ 2023-06-06 9:08 ` Simon Horman
0 siblings, 0 replies; 9+ messages in thread
From: Simon Horman @ 2023-06-06 9:08 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Jakub Kicinski, Patrisious Haddad, Eric Dumazet, netdev,
Paolo Abeni, Raed Salem, Saeed Mahameed, Steffen Klassert
On Mon, Jun 05, 2023 at 11:09:50AM +0300, Leon Romanovsky wrote:
> From: Patrisious Haddad <phaddad@nvidia.com>
>
> Previously during mlx5e_ipsec_handle_event the driver tried to execute
> an operation that could sleep, while holding a spinlock, which caused
> the kernel panic mentioned below.
>
> Move the function call that can sleep outside of the spinlock context.
>
> Call Trace:
> <TASK>
> dump_stack_lvl+0x49/0x6c
> __schedule_bug.cold+0x42/0x4e
> schedule_debug.constprop.0+0xe0/0x118
> __schedule+0x59/0x58a
> ? __mod_timer+0x2a1/0x3ef
> schedule+0x5e/0xd4
> schedule_timeout+0x99/0x164
> ? __pfx_process_timeout+0x10/0x10
> __wait_for_common+0x90/0x1da
> ? __pfx_schedule_timeout+0x10/0x10
> wait_func+0x34/0x142 [mlx5_core]
> mlx5_cmd_invoke+0x1f3/0x313 [mlx5_core]
> cmd_exec+0x1fe/0x325 [mlx5_core]
> mlx5_cmd_do+0x22/0x50 [mlx5_core]
> mlx5_cmd_exec+0x1c/0x40 [mlx5_core]
> mlx5_modify_ipsec_obj+0xb2/0x17f [mlx5_core]
> mlx5e_ipsec_update_esn_state+0x69/0xf0 [mlx5_core]
> ? wake_affine+0x62/0x1f8
> mlx5e_ipsec_handle_event+0xb1/0xc0 [mlx5_core]
> process_one_work+0x1e2/0x3e6
> ? __pfx_worker_thread+0x10/0x10
> worker_thread+0x54/0x3ad
> ? __pfx_worker_thread+0x10/0x10
> kthread+0xda/0x101
> ? __pfx_kthread+0x10/0x10
> ret_from_fork+0x29/0x37
> </TASK>
> BUG: workqueue leaked lock or atomic: kworker/u256:4/0x7fffffff/189754#012 last function: mlx5e_ipsec_handle_event [mlx5_core]
> CPU: 66 PID: 189754 Comm: kworker/u256:4 Kdump: loaded Tainted: G W 6.2.0-2596.20230309201517_5.el8uek.rc1.x86_64 #2
> Hardware name: Oracle Corporation ORACLE SERVER X9-2/ASMMBX9-2, BIOS 61070300 08/17/2022
> Workqueue: mlx5e_ipsec: eth%d mlx5e_ipsec_handle_event [mlx5_core]
> Call Trace:
> <TASK>
> dump_stack_lvl+0x49/0x6c
> process_one_work.cold+0x2b/0x3c
> ? __pfx_worker_thread+0x10/0x10
> worker_thread+0x54/0x3ad
> ? __pfx_worker_thread+0x10/0x10
> kthread+0xda/0x101
> ? __pfx_kthread+0x10/0x10
> ret_from_fork+0x29/0x37
> </TASK>
> BUG: scheduling while atomic: kworker/u256:4/189754/0x00000000
>
> Fixes: cee137a63431 ("net/mlx5e: Handle ESN update events")
> Signed-off-by: Patrisious Haddad <phaddad@nvidia.com>
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH net 3/4] net/mlx5e: Drop XFRM state lock when modifying flow steering
2023-06-05 8:09 [PATCH net 0/4] Fix mixing atomic/non-atomic contexts in mlx5 IPsec code Leon Romanovsky
2023-06-05 8:09 ` [PATCH net 1/4] net/mlx5e: Don't delay release of hardware objects Leon Romanovsky
2023-06-05 8:09 ` [PATCH net 2/4] net/mlx5e: Fix ESN update kernel panic Leon Romanovsky
@ 2023-06-05 8:09 ` Leon Romanovsky
2023-06-06 9:12 ` Simon Horman
2023-06-05 8:09 ` [PATCH net 4/4] net/mlx5e: Fix scheduling of IPsec ASO query while in atomic Leon Romanovsky
3 siblings, 1 reply; 9+ messages in thread
From: Leon Romanovsky @ 2023-06-05 8:09 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Leon Romanovsky, Eric Dumazet, netdev, Paolo Abeni,
Patrisious Haddad, Raed Salem, Saeed Mahameed, Steffen Klassert
From: Leon Romanovsky <leonro@nvidia.com>
XFRM state which is changed to be XFRM_STATE_EXPIRED doesn't really
need to hold lock while modifying flow steering rules to drop traffic.
That state can be deleted only and as such mlx5e_ipsec_handle_tx_limit()
work will be canceled anyway and won't run in parallel.
Fixes: b2f7b01d36a9 ("net/mlx5e: Simulate missing IPsec TX limits hardware functionality")
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
.../ethernet/mellanox/mlx5/core/en_accel/ipsec.c | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
index d1c801723d35..891d39b4bfd4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.c
@@ -61,16 +61,19 @@ static void mlx5e_ipsec_handle_tx_limit(struct work_struct *_work)
struct mlx5e_ipsec_sa_entry *sa_entry = dwork->sa_entry;
struct xfrm_state *x = sa_entry->x;
- spin_lock(&x->lock);
+ if (sa_entry->attrs.drop)
+ return;
+
+ spin_lock_bh(&x->lock);
xfrm_state_check_expire(x);
if (x->km.state == XFRM_STATE_EXPIRED) {
sa_entry->attrs.drop = true;
- mlx5e_accel_ipsec_fs_modify(sa_entry);
- }
- spin_unlock(&x->lock);
+ spin_unlock_bh(&x->lock);
- if (sa_entry->attrs.drop)
+ mlx5e_accel_ipsec_fs_modify(sa_entry);
return;
+ }
+ spin_unlock_bh(&x->lock);
queue_delayed_work(sa_entry->ipsec->wq, &dwork->dwork,
MLX5_IPSEC_RESCHED);
--
2.40.1
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH net 3/4] net/mlx5e: Drop XFRM state lock when modifying flow steering
2023-06-05 8:09 ` [PATCH net 3/4] net/mlx5e: Drop XFRM state lock when modifying flow steering Leon Romanovsky
@ 2023-06-06 9:12 ` Simon Horman
0 siblings, 0 replies; 9+ messages in thread
From: Simon Horman @ 2023-06-06 9:12 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Jakub Kicinski, Leon Romanovsky, Eric Dumazet, netdev,
Paolo Abeni, Patrisious Haddad, Raed Salem, Saeed Mahameed,
Steffen Klassert
On Mon, Jun 05, 2023 at 11:09:51AM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@nvidia.com>
>
> XFRM state which is changed to be XFRM_STATE_EXPIRED doesn't really
> need to hold lock while modifying flow steering rules to drop traffic.
>
> That state can be deleted only and as such mlx5e_ipsec_handle_tx_limit()
> work will be canceled anyway and won't run in parallel.
>
> Fixes: b2f7b01d36a9 ("net/mlx5e: Simulate missing IPsec TX limits hardware functionality")
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH net 4/4] net/mlx5e: Fix scheduling of IPsec ASO query while in atomic
2023-06-05 8:09 [PATCH net 0/4] Fix mixing atomic/non-atomic contexts in mlx5 IPsec code Leon Romanovsky
` (2 preceding siblings ...)
2023-06-05 8:09 ` [PATCH net 3/4] net/mlx5e: Drop XFRM state lock when modifying flow steering Leon Romanovsky
@ 2023-06-05 8:09 ` Leon Romanovsky
2023-06-06 9:13 ` Simon Horman
3 siblings, 1 reply; 9+ messages in thread
From: Leon Romanovsky @ 2023-06-05 8:09 UTC (permalink / raw)
To: Jakub Kicinski
Cc: Leon Romanovsky, Eric Dumazet, netdev, Paolo Abeni,
Patrisious Haddad, Raed Salem, Saeed Mahameed, Steffen Klassert
From: Leon Romanovsky <leonro@nvidia.com>
ASO query can be scheduled in atomic context as such it can't use usleep.
Use udelay as recommended in Documentation/timers/timers-howto.rst.
Fixes: 76e463f6508b ("net/mlx5e: Overcome slow response for first IPsec ASO WQE")
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
---
.../net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c
index ca16cb9807ea..a3554bde3e07 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_offload.c
@@ -606,7 +606,8 @@ int mlx5e_ipsec_aso_query(struct mlx5e_ipsec_sa_entry *sa_entry,
do {
ret = mlx5_aso_poll_cq(aso->aso, false);
if (ret)
- usleep_range(2, 10);
+ /* We are in atomic context */
+ udelay(10);
} while (ret && time_is_after_jiffies(expires));
spin_unlock_bh(&aso->lock);
return ret;
--
2.40.1
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [PATCH net 4/4] net/mlx5e: Fix scheduling of IPsec ASO query while in atomic
2023-06-05 8:09 ` [PATCH net 4/4] net/mlx5e: Fix scheduling of IPsec ASO query while in atomic Leon Romanovsky
@ 2023-06-06 9:13 ` Simon Horman
0 siblings, 0 replies; 9+ messages in thread
From: Simon Horman @ 2023-06-06 9:13 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Jakub Kicinski, Leon Romanovsky, Eric Dumazet, netdev,
Paolo Abeni, Patrisious Haddad, Raed Salem, Saeed Mahameed,
Steffen Klassert
On Mon, Jun 05, 2023 at 11:09:52AM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@nvidia.com>
>
> ASO query can be scheduled in atomic context as such it can't use usleep.
> Use udelay as recommended in Documentation/timers/timers-howto.rst.
>
> Fixes: 76e463f6508b ("net/mlx5e: Overcome slow response for first IPsec ASO WQE")
> Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
^ permalink raw reply [flat|nested] 9+ messages in thread