[PATCH net-next v4 0/4] net: openvswitch: Decouple flow operations from RTNL

[Adrian Moreno <amorenoz@redhat.com>]

When RTNL is contended, network-related control-plane operations can be
delayed.

In such scenario, if OVS control-plane operations (such as vport
creation) take a bit longer, it's acceptable. However, flow installation
operations happen as part of upcall processing, executed in the context
of of handler threads. If they get delayed, it affects the data-plane
and can even result in packet drops.

Because flow operations also use ovs_mutex for concurrency protection and
given RTNL can nest under ovs_mutex, contention can be easily transferred
from RTNL to ovs_mutex, causing delay in flow operations.

In order to protect flow operations from RTNL delays, this series
decouples them from ovs_mutex. First, the flow_table is converted into an
rcu-protected pointer. Then two locking mechanisms are introduced: a
per-table mutex and a refcount.

The mutex protects the flow_table against concurrent modifications,
while the refcount is used to extend the lifetime of the flow_table
beyond the rcu read-protected region used to dereference it.

The following is an example of how concurrent datapath deletion and
flow modifcation would work.
Datapath deletion:

  ovs_lock();
  table = rcu_dereference_protected(dp->table, ...);
  rcu_assign_pointer(dp->table, NULL);
  ovs_flow_tbl_put(table);
  ovs_unlock();

Flow modification:

  rcu_read_lock();
  dp = get_dp(...);
  table = rcu_dereference(dp->table);
  ovs_flow_tbl_get(table);
  rcu_read_unlock();

  mutex_lock(&table->lock);
  /* Perform modifications on the flow_table */
  mutex_unlock(&table->lock);
  ovs_flow_tbl_put(table);

v4:
- Further split patches: move lockdep checks to an independent patch
  (Paolo)
- Add an additional patch to reduce the nested rcu callback scheduling
  (Paolo)
- consolidate flow_table_put after if/else (Paolo)

v3:
- Split in 2 patches (Paolo)
- Improve locking in get_dp_stats (Paolo and Sashiko)
- Use __always_unused in lockdep stubs (Paolo)
- Use READ_ONCE/WRITE_ONCE for table->count (Aaron)
- Take a reference in ovs_dp_masks_rebalance (Aaron) 

v2: Fix argument in ovs_flow_tbl_put (sparse)
    Remove rcu checks in ovs_dp_masks_rebalance


Adrian Moreno (4):
  net: openvswitch: make flow_table an rcu pointer
  net: openvswitch: add per-flow_table lockdep checks
  net: openvswitch: decouple flow_table from ovs_mutex
  net: openvswitch: avoid double-rcu wait period

 net/openvswitch/datapath.c   | 294 ++++++++++++++++++++++++-----------
 net/openvswitch/datapath.h   |   2 +-
 net/openvswitch/flow.c       |  13 +-
 net/openvswitch/flow.h       |   9 +-
 net/openvswitch/flow_table.c | 211 ++++++++++++++++---------
 net/openvswitch/flow_table.h |  56 ++++++-
 6 files changed, 406 insertions(+), 179 deletions(-)

-- 
2.54.0