[RFC 0/6] Fix pthread mutexes for multi-process support

[RFC 0/6] Fix pthread mutexes for multi-process support

[Stephen Hemminger]

While look at the back catalog of bugs, started to use AI
to analyze Bugzilla ID 662.

POSIX mutexes are by default private to the process creating them.
Several places in DPDK use pthread_mutex to protect resources in
shared memory that are accessed by multiple processes (primary and
secondary). These mutexes must be initialized with the
PTHREAD_PROCESS_SHARED attribute for correct synchronization.

Without this attribute, synchronization between processes is
undefined behavior according to POSIX, and may silently fail
on some implementations.

This problem was originally reported against the failsafe driver,
but the issue exists in multiple locations. Every place calling
pthread_mutex_init() in DPDK was suspected.

The following table summarizes the analysis:

  | Component                          | Mutex(es)                      | In Shared Mem | Multi-process | Needs Fix |
  |------------------------------------|--------------------------------|---------------|---------------|-----------|
  | lib/ethdev/ethdev_driver.c         | flow_ops_mutex                 | Yes           | Yes           | YES       |
  | drivers/net/failsafe/failsafe.c    | hotplug_mutex                  | Yes           | Yes           | YES       |
  | drivers/net/atlantic/atl_ethdev.c  | mbox_mutex                     | Yes           | Yes           | YES       |
  | drivers/net/axgbe/axgbe_ethdev.c   | xpcs_mutex, i2c_mutex,         | Yes           | Yes           | YES       |
  |                                    | an_mutex, phy_mutex            |               |               |           |
  | drivers/net/bnxt/bnxt_ethdev.c     | flow_lock, def_cp_lock,        | Yes           | Yes           | YES       |
  |                                    | health_check_lock,             |               |               |           |
  |                                    | err_recovery_lock,             |               |               |           |
  |                                    | vfr_start_lock                 |               |               |           |
  | drivers/net/bnxt/bnxt_txq.c        | txq_lock                       | Yes           | Yes           | YES       |
  | drivers/net/bnxt/tf_ulp/bnxt_ulp.c | bnxt_ulp_mutex                 | Yes           | Yes           | YES       |
  | drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c | flow_db_lock              | Yes           | Yes           | YES       |
  | drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c | flow_db_lock             | Yes           | Yes           | YES       |
  | drivers/net/hinic/base/hinic_compat.h | hinic_mutex_init wrapper  | Yes           | Yes           | YES       |
  | lib/vhost/socket.c                 | conn_mutex, mutex              | No (heap)     | No            | OK        |
  | lib/vhost/fd_man.c                 | fd_mutex                       | No (heap)     | No            | OK        |
  | drivers/common/cnxk/roc_bphy_cgx.c | lock                           | TBD           | Unlikely      | TBD       |
  | drivers/common/cnxk/roc_dev.c      | sync.mutex                     | No            | No            | OK        |
  | drivers/net/cnxk/cnxk_rep.c        | repte_msg_proc.mutex           | No            | No            | OK        |
  | drivers/net/intel/iavf/iavf_vchnl.c| event_handler.lock             | No (static)   | No            | OK        |
  | drivers/net/intel/ixgbe/base/ixgbe_osdep.h | ixgbe_lock macro      | Yes           | Likely        | Maintainer|

This RFC since obviously don't have the hardware to retest all
these devices. Some of the bugs are in the base code, but
a bug is a bug and should be fixed.

Note: There may be additional drivers with similar issues that
were not analyzed in this series. A grep for pthread_mutex_init
in the source tree shows many potential locations that should
be audited.

Stephen Hemminger (6):
  ethdev: fix flow_ops_mutex for multi-process
  net/failsafe: fix hotplug_mutex for multi-process
  net/atlantic: fix mbox_mutex for multi-process
  net/axgbe: fix mutexes for multi-process
  net/bnxt: fix mutexes for multi-process
  net/hinic: fix mutexes for multi-process

 drivers/net/atlantic/atl_ethdev.c      | 14 +++++++++++++-
 drivers/net/axgbe/axgbe_ethdev.c       | 19 +++++++++++++++----
 drivers/net/bnxt/bnxt_ethdev.c         | 11 ++++++-----
 drivers/net/bnxt/bnxt_txq.c            |  3 ++-
 drivers/net/bnxt/bnxt_util.c           | 13 +++++++++++++
 drivers/net/bnxt/bnxt_util.h           |  2 ++
 drivers/net/bnxt/tf_ulp/bnxt_ulp.c     |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c  |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c |  2 +-
 drivers/net/failsafe/failsafe.c        | 15 ++++++++++++---
 drivers/net/hinic/base/hinic_compat.h  | 13 ++++++++++++-
 lib/ethdev/ethdev_driver.c             | 18 +++++++++++++++++-
 12 files changed, 95 insertions(+), 19 deletions(-)

-- 
2.51.0

[RFC 1/6] ethdev: fix flow_ops_mutex for multi-process

[Stephen Hemminger]

The flow_ops_mutex in rte_eth_dev_data is located in shared memory
that is accessed by both primary and secondary processes. However,
the mutex is initialized without PTHREAD_PROCESS_SHARED attribute,
which means synchronization between processes is undefined behavior.

POSIX mutexes are by default private to the process creating them.
When a mutex protects data structures in shared memory that are
accessed by multiple processes, pthread_mutexattr_setpshared() must
be called with PTHREAD_PROCESS_SHARED.

This bug has existed since the flow_ops_mutex was added and affects
any multi-process DPDK application using rte_flow APIs.

Bugzilla ID: 662
Fixes: 80d1a9aff7f6 ("ethdev: make flow API thread safe")
Cc: stable@dpdk.org

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 lib/ethdev/ethdev_driver.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c
index 2ec665be0f..d12395efe0 100644
--- a/lib/ethdev/ethdev_driver.c
+++ b/lib/ethdev/ethdev_driver.c
@@ -89,6 +89,22 @@ eth_dev_set_dummy_fops(struct rte_eth_dev *eth_dev)
 	eth_dev->recycle_rx_descriptors_refill = rte_eth_recycle_rx_descriptors_refill_dummy;
 }
 
+/*
+ * Initialize mutex for process-shared access.
+ * The rte_eth_dev_data structure is in shared memory, so any mutex
+ * protecting it must use PTHREAD_PROCESS_SHARED.
+ */
+static void
+eth_dev_flow_ops_mutex_init(pthread_mutex_t *mutex)
+{
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+}
+
 RTE_EXPORT_INTERNAL_SYMBOL(rte_eth_dev_allocate)
 struct rte_eth_dev *
 rte_eth_dev_allocate(const char *name)
@@ -135,7 +151,7 @@ rte_eth_dev_allocate(const char *name)
 	eth_dev->data->port_id = port_id;
 	eth_dev->data->backer_port_id = RTE_MAX_ETHPORTS;
 	eth_dev->data->mtu = RTE_ETHER_MTU;
-	pthread_mutex_init(&eth_dev->data->flow_ops_mutex, NULL);
+	eth_dev_flow_ops_mutex_init(&eth_dev->data->flow_ops_mutex);
 	RTE_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY);
 	eth_dev_shared_data->allocated_ports++;
 
-- 
2.51.0

[RFC 2/6] net/failsafe: fix hotplug_mutex for multi-process

[Stephen Hemminger]

The failsafe driver supports secondary process attachment via
rte_eth_dev_attach_secondary() in rte_pmd_failsafe_probe(). The
hotplug_mutex in fs_priv protects shared state but is initialized
without PTHREAD_PROCESS_SHARED attribute.

POSIX mutexes are by default private to the process creating them.
When a mutex protects data structures accessed by multiple processes,
pthread_mutexattr_setpshared() must be called with PTHREAD_PROCESS_SHARED.

Also add missing pthread_mutexattr_destroy() call to avoid resource leak.

Bugzilla ID: 662
Fixes: 655fcd68c7d2 ("net/failsafe: fix hotplug races")
Cc: stable@dpdk.org

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 drivers/net/failsafe/failsafe.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/drivers/net/failsafe/failsafe.c b/drivers/net/failsafe/failsafe.c
index 3e590d38f7..aa3fe53b22 100644
--- a/drivers/net/failsafe/failsafe.c
+++ b/drivers/net/failsafe/failsafe.c
@@ -144,11 +144,20 @@ fs_mutex_init(struct fs_priv *priv)
 	/* Allow mutex relocks for the thread holding the mutex. */
 	ret = pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE);
 	if (ret) {
-		ERROR("Cannot set mutex type - %s", strerror(ret));
-		return ret;
+		ERROR("Cannot set mutex recursive - %s", strerror(ret));
+		goto out;
+	}
+	/* Allow mutex to be shared between processes. */
+	ret = pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	if (ret) {
+		ERROR("Cannot set mutex pshared - %s", strerror(ret));
+		goto out;
 	}
+	pthread_mutex_init(&priv->hotplug_mutex, &attr);
 
-	return pthread_mutex_init(&priv->hotplug_mutex, &attr);
+out:
+	pthread_mutexattr_destroy(&attr);
+	return ret;
 }
 
 static int
-- 
2.51.0

[RFC 3/6] net/atlantic: fix mbox_mutex for multi-process

[Stephen Hemminger]

The Atlantic driver supports secondary processes, as shown by the
explicit check in eth_atl_dev_init(). The mbox_mutex in aq_hw_s
is located in dev_private which is in shared memory accessible
by both primary and secondary processes.

However, the mutex is initialized without PTHREAD_PROCESS_SHARED
attribute, which means synchronization between processes is
undefined behavior.

POSIX mutexes are by default private to the process creating them.
When a mutex protects data structures in shared memory that are
accessed by multiple processes, pthread_mutexattr_setpshared() must
be called with PTHREAD_PROCESS_SHARED.

Bugzilla ID: 662
Fixes: e9924638f5c9 ("net/atlantic: add FW mailbox guard mutex")
Cc: stable@dpdk.org

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 drivers/net/atlantic/atl_ethdev.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/drivers/net/atlantic/atl_ethdev.c b/drivers/net/atlantic/atl_ethdev.c
index 2925dc2478..7282f9b691 100644
--- a/drivers/net/atlantic/atl_ethdev.c
+++ b/drivers/net/atlantic/atl_ethdev.c
@@ -3,6 +3,7 @@
  */
 
 #include <rte_string_fns.h>
+#include <pthread.h>
 #include <ethdev_pci.h>
 #include <rte_alarm.h>
 
@@ -355,6 +356,17 @@ atl_disable_intr(struct aq_hw_s *hw)
 	hw_atl_itr_irq_msk_clearlsw_set(hw, 0xffffffff);
 }
 
+static void
+atl_init_mutex(pthread_mutex_t *mutex)
+{
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+}
+
 static int
 eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 {
@@ -405,7 +417,7 @@ eth_atl_dev_init(struct rte_eth_dev *eth_dev)
 
 	hw->aq_nic_cfg = &adapter->hw_cfg;
 
-	pthread_mutex_init(&hw->mbox_mutex, NULL);
+	atl_init_mutex(&hw->mbox_mutex);
 
 	/* disable interrupt */
 	atl_disable_intr(hw);
-- 
2.51.0

[RFC 4/6] net/axgbe: fix mutexes for multi-process

[Stephen Hemminger]

The AXGBE driver supports secondary processes, as shown by the
explicit check in eth_axgbe_dev_init(). The xpcs_mutex, i2c_mutex,
an_mutex, and phy_mutex in axgbe_port are located in dev_private
which is in shared memory accessible by both primary and secondary
processes.

However, the mutexes are initialized without PTHREAD_PROCESS_SHARED
attribute, which means synchronization between processes is
undefined behavior.

POSIX mutexes are by default private to the process creating them.
When a mutex protects data structures in shared memory that are
accessed by multiple processes, pthread_mutexattr_setpshared() must
be called with PTHREAD_PROCESS_SHARED.

Bugzilla ID: 662
Fixes: 572890ef6625 ("net/axgbe: add structs for MAC init and reset")
Cc: stable@dpdk.org

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 drivers/net/axgbe/axgbe_ethdev.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/drivers/net/axgbe/axgbe_ethdev.c b/drivers/net/axgbe/axgbe_ethdev.c
index cf3b0d9ef5..0fedac72ab 100644
--- a/drivers/net/axgbe/axgbe_ethdev.c
+++ b/drivers/net/axgbe/axgbe_ethdev.c
@@ -193,6 +193,17 @@ static const struct axgbe_xstats axgbe_xstats_strings[] = {
 #define AMD_PCI_AXGBE_DEVICE_V2A 0x1458
 #define AMD_PCI_AXGBE_DEVICE_V2B 0x1459
 
+static void
+axgbe_init_mutex(pthread_mutex_t *mutex)
+{
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+}
+
 static const struct rte_pci_id pci_id_axgbe_map[] = {
 	{RTE_PCI_DEVICE(AMD_PCI_VENDOR_ID, AMD_PCI_AXGBE_DEVICE_V2A)},
 	{RTE_PCI_DEVICE(AMD_PCI_VENDOR_ID, AMD_PCI_AXGBE_DEVICE_V2B)},
@@ -2403,10 +2414,10 @@ eth_axgbe_dev_init(struct rte_eth_dev *eth_dev)
 
 	pdata->tx_desc_count = AXGBE_MAX_RING_DESC;
 	pdata->rx_desc_count = AXGBE_MAX_RING_DESC;
-	pthread_mutex_init(&pdata->xpcs_mutex, NULL);
-	pthread_mutex_init(&pdata->i2c_mutex, NULL);
-	pthread_mutex_init(&pdata->an_mutex, NULL);
-	pthread_mutex_init(&pdata->phy_mutex, NULL);
+	axgbe_init_mutex(&pdata->xpcs_mutex);
+	axgbe_init_mutex(&pdata->i2c_mutex);
+	axgbe_init_mutex(&pdata->an_mutex);
+	axgbe_init_mutex(&pdata->phy_mutex);
 
 	ret = pdata->phy_if.phy_init(pdata);
 	if (ret) {
-- 
2.51.0

[RFC 5/6] net/bnxt: fix mutexes for multi-process

[Stephen Hemminger]

The BNXT driver supports secondary processes, as shown by the
explicit check in bnxt_dev_init(). Multiple mutexes are located
in structures allocated in shared memory (dev_private and
rte_zmalloc'd structures) that are accessible by both primary
and secondary processes.

However, the mutexes are initialized without PTHREAD_PROCESS_SHARED
attribute, which means synchronization between processes is
undefined behavior.

POSIX mutexes are by default private to the process creating them.
When a mutex protects data structures in shared memory that are
accessed by multiple processes, pthread_mutexattr_setpshared() must
be called with PTHREAD_PROCESS_SHARED.

This patch adds a helper function and updates all mutex
initializations in the BNXT driver:
  - flow_lock, def_cp_lock, health_check_lock, err_recovery_lock
    in struct bnxt (bnxt_ethdev.c)
  - vfr_start_lock in rep_info (bnxt_ethdev.c)
  - txq_lock in struct bnxt_tx_queue (bnxt_txq.c)
  - bnxt_ulp_mutex in session state (bnxt_ulp.c)
  - flow_db_lock in cfg_data (bnxt_ulp_tf.c, bnxt_ulp_tfc.c)

Bugzilla ID: 662
Fixes: 1cb3d39a48f7 ("net/bnxt: synchronize between flow related functions")
Fixes: 5526c8025d4d ("net/bnxt: fix race between interrupt handler and dev config")
Fixes: b59e4be2b6a7 ("net/bnxt: fix VF representor port add")
Cc: stable@dpdk.org

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 drivers/net/bnxt/bnxt_ethdev.c         | 11 ++++++-----
 drivers/net/bnxt/bnxt_txq.c            |  3 ++-
 drivers/net/bnxt/bnxt_util.c           | 13 +++++++++++++
 drivers/net/bnxt/bnxt_util.h           |  2 ++
 drivers/net/bnxt/tf_ulp/bnxt_ulp.c     |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c  |  2 +-
 drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c |  2 +-
 7 files changed, 26 insertions(+), 9 deletions(-)

diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c
index 3c618c6e82..a954514014 100644
--- a/drivers/net/bnxt/bnxt_ethdev.c
+++ b/drivers/net/bnxt/bnxt_ethdev.c
@@ -5858,10 +5858,10 @@ static int bnxt_get_config(struct bnxt *bp)
 static int
 bnxt_init_locks(struct bnxt *bp)
 {
-	pthread_mutex_init(&bp->flow_lock, NULL);
-	pthread_mutex_init(&bp->def_cp_lock, NULL);
-	pthread_mutex_init(&bp->health_check_lock, NULL);
-	pthread_mutex_init(&bp->err_recovery_lock, NULL);
+	bnxt_init_mutex(&bp->flow_lock);
+	bnxt_init_mutex(&bp->def_cp_lock);
+	bnxt_init_mutex(&bp->health_check_lock);
+	bnxt_init_mutex(&bp->err_recovery_lock);
 
 	return 0;
 }
@@ -6879,7 +6879,8 @@ static int bnxt_init_rep_info(struct bnxt *bp)
 	for (i = 0; i < BNXT_MAX_CFA_CODE; i++)
 		bp->cfa_code_map[i] = BNXT_VF_IDX_INVALID;
 
-	return pthread_mutex_init(&bp->rep_info->vfr_start_lock, NULL);
+	bnxt_init_mutex(&bp->rep_info->vfr_start_lock);
+	return 0;
 }
 
 static int bnxt_rep_port_probe(struct rte_pci_device *pci_dev,
diff --git a/drivers/net/bnxt/bnxt_txq.c b/drivers/net/bnxt/bnxt_txq.c
index 3938ebc709..6deed08ad7 100644
--- a/drivers/net/bnxt/bnxt_txq.c
+++ b/drivers/net/bnxt/bnxt_txq.c
@@ -198,7 +198,8 @@ int bnxt_tx_queue_setup_op(struct rte_eth_dev *eth_dev,
 		goto err;
 	}
 
-	return pthread_mutex_init(&txq->txq_lock, NULL);
+	bnxt_init_mutex(&txq->txq_lock);
+	return 0;
 err:
 	bnxt_tx_queue_release_op(eth_dev, queue_idx);
 	return rc;
diff --git a/drivers/net/bnxt/bnxt_util.c b/drivers/net/bnxt/bnxt_util.c
index aa184496c2..c3f0a03f25 100644
--- a/drivers/net/bnxt/bnxt_util.c
+++ b/drivers/net/bnxt/bnxt_util.c
@@ -3,6 +3,7 @@
  * All rights reserved.
  */
 
+#include <pthread.h>
 #include <inttypes.h>
 #include <rte_ether.h>
 
@@ -37,3 +38,15 @@ uint8_t hweight32(uint32_t word32)
 	res = res + (res >> 8);
 	return (res + (res >> 16)) & 0x000000FF;
 }
+
+/* Initialize mutex so that it can be shared between processes */
+void
+bnxt_init_mutex(pthread_mutex_t *mutex)
+{
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(mutex, &attr);
+	pthread_mutexattr_destroy(&attr);
+}
diff --git a/drivers/net/bnxt/bnxt_util.h b/drivers/net/bnxt/bnxt_util.h
index 416a6a2609..e1b45d1bb5 100644
--- a/drivers/net/bnxt/bnxt_util.h
+++ b/drivers/net/bnxt/bnxt_util.h
@@ -16,4 +16,6 @@
 int bnxt_check_zero_bytes(const uint8_t *bytes, int len);
 void bnxt_eth_hw_addr_random(uint8_t *mac_addr);
 uint8_t hweight32(uint32_t word32);
+void bnxt_init_mutex(pthread_mutex_t *mutex);
+
 #endif /* _BNXT_UTIL_H_ */
diff --git a/drivers/net/bnxt/tf_ulp/bnxt_ulp.c b/drivers/net/bnxt/tf_ulp/bnxt_ulp.c
index 0c03ae7a83..c2c93859ff 100644
--- a/drivers/net/bnxt/tf_ulp/bnxt_ulp.c
+++ b/drivers/net/bnxt/tf_ulp/bnxt_ulp.c
@@ -214,7 +214,7 @@ ulp_session_init(struct bnxt *bp,
 			session->pci_info.domain = pci_addr->domain;
 			session->pci_info.bus = pci_addr->bus;
 			memcpy(session->dsn, bp->dsn, sizeof(session->dsn));
-			pthread_mutex_init(&session->bnxt_ulp_mutex, NULL);
+			bnxt_init_mutex(&session->bnxt_ulp_mutex);
 			STAILQ_INSERT_TAIL(&bnxt_ulp_session_list,
 					   session, next);
 		}
diff --git a/drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c b/drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c
index 38cb5c8b31..5cb79a03e4 100644
--- a/drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c
+++ b/drivers/net/bnxt/tf_ulp/bnxt_ulp_tf.c
@@ -1469,7 +1469,7 @@ ulp_tf_init(struct bnxt *bp,
 		goto jump_to_error;
 	}
 
-	pthread_mutex_init(&bp->ulp_ctx->cfg_data->flow_db_lock, NULL);
+	bnxt_init_mutex(&bp->ulp_ctx->cfg_data->flow_db_lock);
 
 	/* Initialize ulp dparms with values devargs passed */
 	rc = ulp_tf_dparms_init(bp, bp->ulp_ctx);
diff --git a/drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c b/drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c
index ad44ec93ca..0c3d1f7dae 100644
--- a/drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c
+++ b/drivers/net/bnxt/tf_ulp/bnxt_ulp_tfc.c
@@ -1038,7 +1038,7 @@ ulp_tfc_init(struct bnxt *bp,
 		goto jump_to_error;
 	}
 
-	pthread_mutex_init(&bp->ulp_ctx->cfg_data->flow_db_lock, NULL);
+	bnxt_init_mutex(&bp->ulp_ctx->cfg_data->flow_db_lock);
 
 	rc = ulp_tfc_dparms_init(bp, bp->ulp_ctx, ulp_dev_id);
 	if (rc) {
-- 
2.51.0

[RFC 6/6] net/hinic: fix mutexes for multi-process

[Stephen Hemminger]

The hinic driver supports secondary processes, as shown by the
explicit check in hinic_dev_init(). Multiple mutexes are located
in structures allocated in shared memory (dev_private and
rte_zmalloc'd structures) that are accessible by both primary
and secondary processes.

However, the mutexes are initialized without PTHREAD_PROCESS_SHARED
attribute, which means synchronization between processes is
undefined behavior.

POSIX mutexes are by default private to the process creating them.
When a mutex protects data structures in shared memory that are
accessed by multiple processes, pthread_mutexattr_setpshared() must
be called with PTHREAD_PROCESS_SHARED.

Fix the hinic_mutex_init() wrapper function to set the
PTHREAD_PROCESS_SHARED attribute on all mutexes.

Bugzilla ID: 662
Fixes: ae865766b334 ("net/hinic: replace spinlock with mutex")
Cc: stable@dpdk.org

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 drivers/net/hinic/base/hinic_compat.h | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/net/hinic/base/hinic_compat.h b/drivers/net/hinic/base/hinic_compat.h
index d3994c50e9..18148a119e 100644
--- a/drivers/net/hinic/base/hinic_compat.h
+++ b/drivers/net/hinic/base/hinic_compat.h
@@ -197,10 +197,21 @@ static inline u16 ilog2(u32 n)
 	return res;
 }
 
+/*
+ * Initialize mutex for process-shared access.
+ * Structures may be in shared memory accessible by multiple processes,
+ * so mutexes must use PTHREAD_PROCESS_SHARED.
+ */
 static inline int hinic_mutex_init(pthread_mutex_t *pthreadmutex,
 					const pthread_mutexattr_t *mattr)
 {
-	return pthread_mutex_init(pthreadmutex, mattr);
+	pthread_mutexattr_t attr;
+
+	pthread_mutexattr_init(&attr);
+	pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
+	pthread_mutex_init(pthreadmutex, mattr ? mattr : &attr);
+	pthread_mutexattr_destroy(&attr);
+	return 0;
 }
 
 static inline int hinic_mutex_destroy(pthread_mutex_t *pthreadmutex)
-- 
2.51.0