[PATCH net] xfrm: retry inexact policy lookup after node reinsertion

[Chengfeng Ye <nicoyip.dev@gmail.com>]

When a newly inserted inexact policy covers multiple existing nodes,
xfrm_policy_inexact_node_merge() removes each policy from the old node with
hlist_del_rcu(). xfrm_policy_inexact_list_reinsert() then immediately adds
the same bydst node to another hlist. hlist_del_rcu() leaves ->next intact
for readers already at the removed entry, but the add overwrites it before
an RCU grace period.

This permits the following interleaving:

  CPU 0                                  CPU 1
  -----                                  -----
  reach policy P in the old hlist
                                         delete P from the old hlist
                                         add P to the new hlist
  follow P->next into the new hlist

The lookup can then skip a matching policy and make an incorrect IPsec
decision.

During concurrent policy insertion and route lookup, KCSAN reported:

  BUG: KCSAN: data-race in __xfrm_policy_link / xfrm_lookup_with_ifid
  write to ... by task 92 on cpu 0:
    __xfrm_policy_link
    xfrm_policy_insert
    xfrm_add_policy
  read to ... by task 91 on cpu 1:
    xfrm_lookup_with_ifid
    xfrm_lookup_route
    ip_route_output_flow

The per-bin sequence counter already brackets inexact tree changes,
including node merges. Snapshot it before candidate discovery and retry
after evaluating all candidate lists if it changed. This rejects results
from any traversal overlapping reinsertion while leaving stable lookup
order and locking unchanged.

Fixes: 9cf545ebd591 ("xfrm: policy: store inexact policies in a tree ordered by destination address")
Cc: stable@vger.kernel.org
Signed-off-by: Chengfeng Ye <nicoyip.dev@gmail.com>
---
 net/xfrm/xfrm_policy.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
index 932a313b9460..5d4e863863df 100644
--- a/net/xfrm/xfrm_policy.c
+++ b/net/xfrm/xfrm_policy.c
@@ -2157,6 +2157,7 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type,
 	struct xfrm_pol_inexact_bin *bin;
 	struct xfrm_policy *pol, *ret;
 	struct hlist_head *chain;
+	unsigned int inexact_sequence;
 	unsigned int sequence;
 	int err;
 
@@ -2191,12 +2192,18 @@ static struct xfrm_policy *xfrm_policy_lookup_bytype(struct net *net, u8 type,
 		goto skip_inexact;
 
 	bin = xfrm_policy_inexact_lookup_rcu(net, type, family, dir, if_id);
-	if (!bin || !xfrm_policy_find_inexact_candidates(&cand, bin, saddr,
-							 daddr))
+	if (!bin)
+		goto skip_inexact;
+
+	inexact_sequence = read_seqcount_begin(&bin->count);
+	if (!xfrm_policy_find_inexact_candidates(&cand, bin, saddr, daddr))
 		goto skip_inexact;
 
 	pol = xfrm_policy_eval_candidates(&cand, ret, fl, type,
 					  family, if_id);
+	if (read_seqcount_retry(&bin->count, inexact_sequence))
+		goto retry;
+
 	if (pol) {
 		ret = pol;
 		if (IS_ERR(pol))
-- 
2.43.0