* [PATCH] opensm/osm_lid_mgr: revert 'coming_out_of_standby' replacement patch
From: Sasha Khapyorsky @ 2010-05-22 13:16 UTC (permalink / raw)
To: linux-rdma; +Cc: Eli Dorfman, Alex Netes
In-Reply-To: <20100522130941.GP28549@me>
This reverts the patch where 'coming_out_of_standby' flag was replaced by
'first_time_master_sweep'. It is probably a good idea in general to
merge those, but right now it will change the default lid manager
behavior, since it will ignore guid2lid file in initial sweep.
Signed-off-by: Sasha Khapyorsky <sashak-smomgflXvOZWk0Htik3J/w@public.gmane.org>
---
opensm/opensm/osm_lid_mgr.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/opensm/opensm/osm_lid_mgr.c b/opensm/opensm/osm_lid_mgr.c
index 7da4eb0..97b1de6 100644
--- a/opensm/opensm/osm_lid_mgr.c
+++ b/opensm/opensm/osm_lid_mgr.c
@@ -304,7 +304,7 @@ static int lid_mgr_init_sweep(IN osm_lid_mgr_t * p_mgr)
info we might have.
Do this only if the honor_guid2lid_file option is FALSE. If not, then
need to honor this file. */
- if (p_mgr->p_subn->first_time_master_sweep == TRUE) {
+ if (p_mgr->p_subn->coming_out_of_standby == TRUE) {
osm_db_clear(p_mgr->p_g2l);
memset(p_mgr->used_lids, 0, sizeof(p_mgr->used_lids));
if (p_mgr->p_subn->opt.honor_guid2lid_file == FALSE)
--
1.7.0.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: [PATCH] opensm/osm_lid_mgr: use 'first_time_master_sweep' flag
From: Sasha Khapyorsky @ 2010-05-22 13:09 UTC (permalink / raw)
To: linux-rdma; +Cc: Eli Dorfman, Alex Netes
In-Reply-To: <20100426104838.GS23994@me>
On 13:48 Mon 26 Apr , Sasha Khapyorsky wrote:
>
> Use 'first_time_master_sweep' as initial sweep indication instead of
> 'coming_out_of_standby'. The last flag will be set only when OpenSM
> previously was in standby state, this doesn't cover some other cases
> such as temporary SM port disconnection.
>
> Signed-off-by: Sasha Khapyorsky <sashak-smomgflXvOZWk0Htik3J/w@public.gmane.org>
> ---
> opensm/opensm/osm_lid_mgr.c | 2 +-
> 1 files changed, 1 insertions(+), 1 deletions(-)
>
> diff --git a/opensm/opensm/osm_lid_mgr.c b/opensm/opensm/osm_lid_mgr.c
> index 0fd3b20..84f3508 100644
> --- a/opensm/opensm/osm_lid_mgr.c
> +++ b/opensm/opensm/osm_lid_mgr.c
> @@ -304,7 +304,7 @@ static int lid_mgr_init_sweep(IN osm_lid_mgr_t * p_mgr)
> info we might have.
> Do this only if the honor_guid2lid_file option is FALSE. If not, then
> need to honor this file. */
> - if (p_mgr->p_subn->coming_out_of_standby == TRUE) {
> + if (p_mgr->p_subn->first_time_master_sweep == TRUE) {
Actually this changes the current behavior. Flag 'coming_out_of_standby'
is not set on first initial sweep and 'first_time_master_sweep' is. So
I'm reverting this patch yet up to more cleanup will be performed.
Sasha
> osm_db_clear(p_mgr->p_g2l);
> if (p_mgr->p_subn->opt.honor_guid2lid_file == FALSE) {
> OSM_LOG(p_mgr->p_log, OSM_LOG_DEBUG,
> --
> 1.7.0.4
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH] opensm/osm_log.c: add OSM_LOG_SYS to default flags
From: Sasha Khapyorsky @ 2010-05-22 10:35 UTC (permalink / raw)
To: Yevgeny Kliteynik; +Cc: Linux RDMA
In-Reply-To: <4BF0EE9C.7030607-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
On 10:22 Mon 17 May , Yevgeny Kliteynik wrote:
> As suggested by Sasha, instead of checking OSM_LOG_SYS in
> osm_log_is_active(), just adding OSM_LOG_SYS to default flags.
>
> Signed-off-by: Yevgeny Kliteynik <kliteyn-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
Applied. Thanks.
Sasha
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 10/27] drivers/infiniband/core: Use memdup_user
From: walter harms @ 2010-05-22 9:39 UTC (permalink / raw)
To: Julia Lawall
Cc: Roland Dreier, Sean Hefty, Hal Rosenstock,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
kernel-janitors-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <Pine.LNX.4.64.1005221021030.13021-QfmoRoYWmW9knbxzx/v8hQ@public.gmane.org>
Julia Lawall schrieb:
> From: Julia Lawall <julia-dAYI7NvHqcQ@public.gmane.org>
>
> Use memdup_user when user data is immediately copied into the
> allocated region.
>
> The semantic patch that makes this change is as follows:
> (http://coccinelle.lip6.fr/)
>
> // <smpl>
> @@
> expression from,to,size,flag;
> position p;
> identifier l1,l2;
> @@
>
> - to = \(kmalloc@p\|kzalloc@p\)(size,flag);
> + to = memdup_user(from,size);
> if (
> - to==NULL
> + IS_ERR(to)
> || ...) {
> <+... when != goto l1;
> - -ENOMEM
> + PTR_ERR(to)
> ...+>
> }
> - if (copy_from_user(to, from, size) != 0) {
> - <+... when != goto l2;
> - -EFAULT
> - ...+>
> - }
> // </smpl>
>
> Signed-off-by: Julia Lawall <julia-dAYI7NvHqcQ@public.gmane.org>
>
> ---
> drivers/infiniband/core/ucm.c | 11 +++--------
> 1 file changed, 3 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/infiniband/core/ucm.c b/drivers/infiniband/core/ucm.c
> index 4647484..08f948d 100644
> --- a/drivers/infiniband/core/ucm.c
> +++ b/drivers/infiniband/core/ucm.c
> @@ -706,14 +706,9 @@ static int ib_ucm_alloc_data(const void **dest, u64 src, u32 len)
> if (!len)
> return 0;
>
> - data = kmalloc(len, GFP_KERNEL);
> - if (!data)
> - return -ENOMEM;
> -
> - if (copy_from_user(data, (void __user *)(unsigned long)src, len)) {
> - kfree(data);
> - return -EFAULT;
> - }
> + data = memdup_user((void __user *)(unsigned long)src, len);
> + if (IS_ERR(data))
> + return PTR_ERR(data);
>
> *dest = data;
> return 0;
> --
This cast look strange, can it happen that (unsigned long)<(u64) ?
(is there a 32bit infiniband) ?
just my 2 cents,
wh
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH 10/27] drivers/infiniband/core: Use memdup_user
From: Julia Lawall @ 2010-05-22 8:21 UTC (permalink / raw)
To: Roland Dreier, Sean Hefty, Hal Rosenstock, linux-rdma,
linux-kernel, kernel-jani
From: Julia Lawall <julia@diku.dk>
Use memdup_user when user data is immediately copied into the
allocated region.
The semantic patch that makes this change is as follows:
(http://coccinelle.lip6.fr/)
// <smpl>
@@
expression from,to,size,flag;
position p;
identifier l1,l2;
@@
- to = \(kmalloc@p\|kzalloc@p\)(size,flag);
+ to = memdup_user(from,size);
if (
- to==NULL
+ IS_ERR(to)
|| ...) {
<+... when != goto l1;
- -ENOMEM
+ PTR_ERR(to)
...+>
}
- if (copy_from_user(to, from, size) != 0) {
- <+... when != goto l2;
- -EFAULT
- ...+>
- }
// </smpl>
Signed-off-by: Julia Lawall <julia@diku.dk>
---
drivers/infiniband/core/ucm.c | 11 +++--------
1 file changed, 3 insertions(+), 8 deletions(-)
diff --git a/drivers/infiniband/core/ucm.c b/drivers/infiniband/core/ucm.c
index 4647484..08f948d 100644
--- a/drivers/infiniband/core/ucm.c
+++ b/drivers/infiniband/core/ucm.c
@@ -706,14 +706,9 @@ static int ib_ucm_alloc_data(const void **dest, u64 src, u32 len)
if (!len)
return 0;
- data = kmalloc(len, GFP_KERNEL);
- if (!data)
- return -ENOMEM;
-
- if (copy_from_user(data, (void __user *)(unsigned long)src, len)) {
- kfree(data);
- return -EFAULT;
- }
+ data = memdup_user((void __user *)(unsigned long)src, len);
+ if (IS_ERR(data))
+ return PTR_ERR(data);
*dest = data;
return 0;
^ permalink raw reply related
* RE: [ANNOUNCE] librdmacm 1.0.12
From: Sean Hefty @ 2010-05-21 23:11 UTC (permalink / raw)
To: Hefty, Sean, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
'Vladimir Sokolovsky'
In-Reply-To: <CE35DBA4CC524BD3BDA61C7DE2BD39CD-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org>
>I've pushed out release 1.0.12 of librdmacm.
Uhm... the release is at:
http://www.openfabrics.org/downloads/rdmacm/librdmacm-1.0.12.tar.gz
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [ANNOUNCE] ib_acm 1.0.0
From: Sean Hefty @ 2010-05-21 23:10 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA
I've packaged an initial release of IB ACM for inclusion into OFED 1.5.2. The
IB ACM service provides user space path record caching for users of the
librdmacm. The download is available from:
http://www.openfabrics.org/downloads/rdmacm/
I will submit release notes for the ib_acm next week.
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [ANNOUNCE] librdmacm 1.0.12
From: Sean Hefty @ 2010-05-21 23:06 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA, 'Vladimir Sokolovsky'
I've pushed out release 1.0.12 of librdmacm. This incorporates several new APIs
that should make connecting and transferring data over RDMA devices simpler for
user applications, plus adds support for user space path record caching.
New APIs include simple wrappers for common verbs operations, such as memory
registration, posting of work requests, and retrieving completions. Support for
synchronous connection establishment was added, along with newer calls that
combine the work done using multiple existing calls. To help address MPI
scaling issues, this release provides support for IB ACM path record caching
capabilities, enabled through the use of a compile time option. New client and
server test applications were also added to demonstrate the use of the new APIs.
(Note that the source code includes changes for AF_IB support, but AF_IB has
been disabled in the code to avoid potential ABI compatibility issues until the
kernel changes have been accepted upstream.) This version wants the soon to be
released version of libibverbs, but will work with older versions installed.
Please pull this release into OFED 1.5.2. I will provide a set of release notes
next week.
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH] RDMA/nes: AE for closed QP causes crash
From: Faisal Latif @ 2010-05-21 21:55 UTC (permalink / raw)
To: Roland Dreier; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
Under abnormal termination, modify_qp() closes qp and AE handling also
attempts to close qp causing system to crash.
Now we check the state of QP before processing the AE.
Signed-off-by: Faisal Latif <faisal.latif-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/nes/nes_hw.c | 9 +++++++++
1 files changed, 9 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/hw/nes/nes_hw.c b/drivers/infiniband/hw/nes/nes_hw.c
index c36a3f5..bfc9969 100644
--- a/drivers/infiniband/hw/nes/nes_hw.c
+++ b/drivers/infiniband/hw/nes/nes_hw.c
@@ -3424,6 +3424,7 @@ static void nes_process_iwarp_aeqe(struct nes_device *nesdev,
struct nes_adapter *nesadapter = nesdev->nesadapter;
u32 aeq_info;
u32 next_iwarp_state = 0;
+ u32 aeqe_cq_id;
u16 async_event_id;
u8 tcp_state;
u8 iwarp_state;
@@ -3451,6 +3452,14 @@ static void nes_process_iwarp_aeqe(struct nes_device *nesdev,
le32_to_cpu(aeqe->aeqe_words[NES_AEQE_COMP_QP_CQ_ID_IDX]), aeqe,
nes_tcp_state_str[tcp_state], nes_iwarp_state_str[iwarp_state]);
+ aeqe_cq_id = le32_to_cpu(aeqe->aeqe_words[NES_AEQE_COMP_QP_CQ_ID_IDX]);
+ if (aeq_info & NES_AEQE_QP) {
+ if ((!nes_is_resource_allocated(nesadapter, nesadapter->allocated_qps,
+ aeqe_cq_id)) ||
+ (atomic_read(&nesqp->close_timer_started)))
+ return;
+ }
+
switch (async_event_id) {
case NES_AEQE_AEID_LLP_FIN_RECEIVED:
if (nesqp->term_flags)
--
1.6.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH] RDMA/nes: ethtool read hardware registers for rx/tx stats
From: Faisal Latif @ 2010-05-21 21:54 UTC (permalink / raw)
To: Roland Dreier; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
Enhance ethtool to read hardware registers for rcv/tx error stats. Also
added support for free pbl resources and remove cq depth stats which are not
used.
Signed-off-by: Faisal Latif <faisal.latif-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/nes/nes_nic.c | 72 ++++++++++++++++++++++++++--------
1 files changed, 55 insertions(+), 17 deletions(-)
diff --git a/drivers/infiniband/hw/nes/nes_nic.c b/drivers/infiniband/hw/nes/nes_nic.c
index 3d550dc..812a448 100644
--- a/drivers/infiniband/hw/nes/nes_nic.c
+++ b/drivers/infiniband/hw/nes/nes_nic.c
@@ -988,6 +988,7 @@ static int nes_netdev_change_mtu(struct net_device *netdev, int new_mtu)
return ret;
}
+
static const char nes_ethtool_stringset[][ETH_GSTRING_LEN] = {
"Link Change Interrupts",
"Linearized SKBs",
@@ -1002,11 +1003,15 @@ static const char nes_ethtool_stringset[][ETH_GSTRING_LEN] = {
"Rx Jabber Errors",
"Rx Oversized Frames",
"Rx Short Frames",
+ "Rx Length Errors",
+ "Rx CRC Errors",
+ "Rx Port Discard",
"Endnode Rx Discards",
"Endnode Rx Octets",
"Endnode Rx Frames",
"Endnode Tx Octets",
"Endnode Tx Frames",
+ "Tx Errors",
"mh detected",
"mh pauses",
"Retransmission Count",
@@ -1035,19 +1040,13 @@ static const char nes_ethtool_stringset[][ETH_GSTRING_LEN] = {
"CM Nodes Destroyed",
"CM Accel Drops",
"CM Resets Received",
+ "Free 4Kpbls",
+ "Free 256pbls",
"Timer Inits",
- "CQ Depth 1",
- "CQ Depth 4",
- "CQ Depth 16",
- "CQ Depth 24",
- "CQ Depth 32",
- "CQ Depth 128",
- "CQ Depth 256",
"LRO aggregated",
"LRO flushed",
"LRO no_desc",
};
-
#define NES_ETHTOOL_STAT_COUNT ARRAY_SIZE(nes_ethtool_stringset)
/**
@@ -1107,12 +1106,14 @@ static void nes_netdev_get_strings(struct net_device *netdev, u32 stringset,
/**
* nes_netdev_get_ethtool_stats
*/
+
static void nes_netdev_get_ethtool_stats(struct net_device *netdev,
struct ethtool_stats *target_ethtool_stats, u64 *target_stat_values)
{
u64 u64temp;
struct nes_vnic *nesvnic = netdev_priv(netdev);
struct nes_device *nesdev = nesvnic->nesdev;
+ struct nes_adapter *nesadapter = nesdev->nesadapter;
u32 nic_count;
u32 u32temp;
u32 index = 0;
@@ -1141,6 +1142,46 @@ static void nes_netdev_get_ethtool_stats(struct net_device *netdev,
nesvnic->nesdev->port_tx_discards += u32temp;
nesvnic->netstats.tx_dropped += u32temp;
+ u32temp = nes_read_indexed(nesdev,
+ NES_IDX_MAC_RX_SHORT_FRAMES + (nesvnic->nesdev->mac_index*0x200));
+ nesvnic->netstats.rx_dropped += u32temp;
+ nesvnic->nesdev->mac_rx_errors += u32temp;
+ nesvnic->nesdev->mac_rx_short_frames += u32temp;
+
+ u32temp = nes_read_indexed(nesdev,
+ NES_IDX_MAC_RX_OVERSIZED_FRAMES + (nesvnic->nesdev->mac_index*0x200));
+ nesvnic->netstats.rx_dropped += u32temp;
+ nesvnic->nesdev->mac_rx_errors += u32temp;
+ nesvnic->nesdev->mac_rx_oversized_frames += u32temp;
+
+ u32temp = nes_read_indexed(nesdev,
+ NES_IDX_MAC_RX_JABBER_FRAMES + (nesvnic->nesdev->mac_index*0x200));
+ nesvnic->netstats.rx_dropped += u32temp;
+ nesvnic->nesdev->mac_rx_errors += u32temp;
+ nesvnic->nesdev->mac_rx_jabber_frames += u32temp;
+
+ u32temp = nes_read_indexed(nesdev,
+ NES_IDX_MAC_RX_SYMBOL_ERR_FRAMES + (nesvnic->nesdev->mac_index*0x200));
+ nesvnic->netstats.rx_dropped += u32temp;
+ nesvnic->nesdev->mac_rx_errors += u32temp;
+ nesvnic->nesdev->mac_rx_symbol_err_frames += u32temp;
+
+ u32temp = nes_read_indexed(nesdev,
+ NES_IDX_MAC_RX_LENGTH_ERR_FRAMES + (nesvnic->nesdev->mac_index*0x200));
+ nesvnic->netstats.rx_length_errors += u32temp;
+ nesvnic->nesdev->mac_rx_errors += u32temp;
+
+ u32temp = nes_read_indexed(nesdev,
+ NES_IDX_MAC_RX_CRC_ERR_FRAMES + (nesvnic->nesdev->mac_index*0x200));
+ nesvnic->nesdev->mac_rx_errors += u32temp;
+ nesvnic->nesdev->mac_rx_crc_errors += u32temp;
+ nesvnic->netstats.rx_crc_errors += u32temp;
+
+ u32temp = nes_read_indexed(nesdev,
+ NES_IDX_MAC_TX_ERRORS + (nesvnic->nesdev->mac_index*0x200));
+ nesvnic->nesdev->mac_tx_errors += u32temp;
+ nesvnic->netstats.tx_errors += u32temp;
+
for (nic_count = 0; nic_count < NES_MAX_PORT_COUNT; nic_count++) {
if (nesvnic->qp_nic_index[nic_count] == 0xf)
break;
@@ -1205,11 +1246,15 @@ static void nes_netdev_get_ethtool_stats(struct net_device *netdev,
target_stat_values[++index] = nesvnic->nesdev->mac_rx_jabber_frames;
target_stat_values[++index] = nesvnic->nesdev->mac_rx_oversized_frames;
target_stat_values[++index] = nesvnic->nesdev->mac_rx_short_frames;
+ target_stat_values[++index] = nesvnic->netstats.rx_length_errors;
+ target_stat_values[++index] = nesvnic->nesdev->mac_rx_crc_errors;
+ target_stat_values[++index] = nesvnic->nesdev->port_rx_discards;
target_stat_values[++index] = nesvnic->endnode_nstat_rx_discard;
target_stat_values[++index] = nesvnic->endnode_nstat_rx_octets;
target_stat_values[++index] = nesvnic->endnode_nstat_rx_frames;
target_stat_values[++index] = nesvnic->endnode_nstat_tx_octets;
target_stat_values[++index] = nesvnic->endnode_nstat_tx_frames;
+ target_stat_values[++index] = nesvnic->nesdev->mac_tx_errors;
target_stat_values[++index] = mh_detected;
target_stat_values[++index] = mh_pauses_sent;
target_stat_values[++index] = nesvnic->endnode_ipv4_tcp_retransmits;
@@ -1238,21 +1283,14 @@ static void nes_netdev_get_ethtool_stats(struct net_device *netdev,
target_stat_values[++index] = atomic_read(&cm_nodes_destroyed);
target_stat_values[++index] = atomic_read(&cm_accel_dropped_pkts);
target_stat_values[++index] = atomic_read(&cm_resets_recvd);
+ target_stat_values[++index] = nesadapter->free_4kpbl;
+ target_stat_values[++index] = nesadapter->free_256pbl;
target_stat_values[++index] = int_mod_timer_init;
- target_stat_values[++index] = int_mod_cq_depth_1;
- target_stat_values[++index] = int_mod_cq_depth_4;
- target_stat_values[++index] = int_mod_cq_depth_16;
- target_stat_values[++index] = int_mod_cq_depth_24;
- target_stat_values[++index] = int_mod_cq_depth_32;
- target_stat_values[++index] = int_mod_cq_depth_128;
- target_stat_values[++index] = int_mod_cq_depth_256;
target_stat_values[++index] = nesvnic->lro_mgr.stats.aggregated;
target_stat_values[++index] = nesvnic->lro_mgr.stats.flushed;
target_stat_values[++index] = nesvnic->lro_mgr.stats.no_desc;
-
}
-
/**
* nes_netdev_get_drvinfo
*/
--
1.6.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH] dapl-2.0: update dat.conf manpage for static configuration options
From: Davis, Arlin R @ 2010-05-21 21:31 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, ofw_list
Update dat.conf manpage to explain options for different providers and devices
Signed-off-by: Arlin Davis <arlin.r.davis-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
diff --git a/man/dat.conf.5 b/man/dat.conf.5
index 6dee668..8360e92 100644
--- a/man/dat.conf.5
+++ b/man/dat.conf.5
@@ -45,18 +45,63 @@ entry is termed Static Registration.
- ia params, IA specific parameters - device name and port
- platform params, (not used)
.PP
-.SH Example netdev entries for OpenFabrics rdma_cm providers, both v1.2 and v2.0
+.SH OpenFabrics RDMA providers:
\br
+ Provider options for both 1.2 and 2.0, each using different CM services
+
+ 1. cma - OpenFabrics rdma_cm - uses rdma_cm services for connections
+ - requires IPoIB and SA/SM services for IB
+ - netdev used for device name, without port designation (ia_params)
+ - Supports any transport rdma_cm supports including IB, iWARP, RoCEE
+ - libdaplcma.so (1.2), libdaplofa (2.0)
+
+ 2. scm - uDAPL socket based CM - exchanges CM information over sockets
+ - eliminates the need for rdma_cm, IPoIB, and SA for IB
+ - verbs device used for device name with port designation (ia_param)
+ - Supports IB, RoCEE. Doesn't support iWARP
+ - libdaplscm.so (1.2), libdaploscm (2.0)
+
+ 3. ucm - uDAPL unreliable IB CM - exchanges CM information via IB UD QP's
+ - eliminates the need for sockets or rdma_cm
+ - verbs device used for device name with port designation (ia_param)
+ - Supports IB only, no name service.
+ - libdaplucm.so (1.2), libdaploucm (2.0)
+.PP
+.SH Example entries for each OpenFabrics provider
+\br
+
+ 1. cma - OpenFarbrics rdma_cm (v1.2 and v2.0 examples)
+
OpenIB-cma u1.2 nonthreadsafe default libdaplcma.so.1 dapl.1.2 "ib0 0" ""
ofa-v2-ib0 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "ib0 0" ""
+ ofa-v2-iwarp u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "eth2 0" ""
+ ofa-v2-cma-roe-eth2 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "eth2 0" ""
+ ofa-v2-cma-roe-eth3 u2.0 nonthreadsafe default libdaplofa.so.2 dapl.2.0 "eth3 0" ""
- NOTE: The OpenFabrics providers use <ia_params> to specify the device with one of the following:
+ NOTE: The OpenFabrics CMA providers use <ia_params> to specify the device with one of the following:
network address, network hostname, or netdev name; along with port number.
- The OpenIB- and ofa-v2- IA names are unique mappings. Reserved for OpenFabrics providers.
+ 2. scm - uDAPL socket based CM (v1.2 and v2.0 examples)
+
+ OpenIB-mlx4_0-1 u1.2 nonthreadsafe default libdaplscm.so.1 dapl.1.2 "mlx4_0 1" ""
+ OpenIB-ipath0-1 u1.2 nonthreadsafe default libdaplscm.so.1 dapl.1.2 "ipath0 1" ""
+ ofa-v2-mlx4_0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 "mlx4_0 1" ""
+ ofa-v2-mlx4_0-2 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 "mlx4_0 2" ""
+ ofa-v2-mlx4_1-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 "mlx4_1 1" ""
+ ofa-v2-ehca0-1 u2.0 nonthreadsafe default libdaploscm.so.2 dapl.2.0 "ehca0 1" ""
+
+ 3. ucm - uDAPL unreliable IB CM (not supported in 1.2, v2.0 examples)
+
+ ofa-v2-mlx4_0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 "mlx4_0 1" ""
+ ofa-v2-mlx4_0-2u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 "mlx4_0 2" ""
+ ofa-v2-ipath0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 "ipath0 1" ""
+ ofa-v2-ehca0-1u u2.0 nonthreadsafe default libdaploucm.so.2 dapl.2.0 "ehca0 1" ""
+
+ Note: OpenIB- and ofa-v2- IA names are unique mappings, reserved for OpenFabrics providers.
+.PP
+The default location for this configuration file is /etc/dat.conf.
.PP
-The default location for this configuration file is /etc/dat.conf.
The file location may be overridden with the environment variable DAT_OVERRIDE=/your_own_directory/your_dat.conf.
.PP
-.SH "SEE ALSO"
+.SH "SEE ALSO" rdma_cm verbs socket
.PP
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: [PATCH] opensm/qos.c: Revert port ranges for calls to sl2vl_update_table().
From: Jim Schutt @ 2010-05-21 20:29 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Cc: sashak-smomgflXvOZWk0Htik3J/w@public.gmane.org
In-Reply-To: <1851_1274473140_o4LKDZ3n001852_1274473115-3153-1-git-send-email-jaschut-4OHPYypu0djtX7QSmKvirg@public.gmane.org>
Sorry, I somehow got this wrong.
Corrected patch below.
-- Jim
On Fri, 2010-05-21 at 14:18 -0600, Jim Schutt wrote:
> Before commit 051a1dd5 (opensm/osm_qos.c: split switch external and end
> ports setup), osm_qos_setup() would end up calling sl2vl_update_table()
> for output ports 1-N, and inport ports 0-N.
>
> Commit 051a1dd5 changed this around to be output ports 0-N, and input
> ports 1-N, and an InfiniScale IV based fabric would log lots of errors
> like these:
>
> log_rcv_cb_error: ERR 3111: Received MAD with error status = 0x1C
> SubnGetResp(SLtoVLMappingTable), attr_mod 0x2300, TID 0xad069
> Initial path: 0,1,1,4,13 Return path: 0,25,1,7,10
>
> The attr_mod in every such message has 0x00 in the least significant
> byte.
>
> With the port ranges restored to their old values, the above log messages
> stop.
>
> Signed-off-by: Jim Schutt <jaschut-4OHPYypu0djtX7QSmKvirg@public.gmane.org>
> ---
> opensm/opensm/osm_qos.c | 8 ++++----
> 1 files changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/opensm/opensm/osm_qos.c b/opensm/opensm/osm_qos.c
> index 6bbbfa2..b8c3111 100644
> --- a/opensm/opensm/osm_qos.c
> +++ b/opensm/opensm/osm_qos.c
> @@ -230,12 +230,12 @@ static int qos_extports_setup(osm_sm_t * sm, osm_node_t *node,
> &qcfg->sl2vl);
> }
>
> - for (i = 1; i < num_ports; i++) {
> + for (i = 0; i < num_ports; i++) {
> p = osm_node_get_physp_ptr(node, i);
> force_update = p->need_update || sm->p_subn->need_update;
> - for (j = 0; j < num_ports; j++)
> - if (sl2vl_update_table(sm, p, i, i << 8 | j,
> - force_update, &qcfg->sl2vl))
> + for (j = 1; j < num_ports; j++)
> + if (sl2vl_update_table(sm, p, i, j, force_update,
> + &qcfg->sl2vl))
> ret = -1;
> }
>
diff --git a/opensm/opensm/osm_qos.c b/opensm/opensm/osm_qos.c
index 6bbbfa2..7d76c75 100644
@@ -230,10 +230,10 @@ static int qos_extports_setup(osm_sm_t * sm, osm_node_t *node,
&qcfg->sl2vl);
}
- for (i = 1; i < num_ports; i++) {
+ for (i = 0; i < num_ports; i++) {
p = osm_node_get_physp_ptr(node, i);
force_update = p->need_update || sm->p_subn->need_update;
- for (j = 0; j < num_ports; j++)
+ for (j = 1; j < num_ports; j++)
if (sl2vl_update_table(sm, p, i, i << 8 | j,
force_update, &qcfg->sl2vl))
ret = -1;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH] opensm/qos.c: Revert port ranges for calls to sl2vl_update_table().
From: Jim Schutt @ 2010-05-21 20:18 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA
Cc: sashak-smomgflXvOZWk0Htik3J/w, Jim Schutt
Before commit 051a1dd5 (opensm/osm_qos.c: split switch external and end
ports setup), osm_qos_setup() would end up calling sl2vl_update_table()
for output ports 1-N, and inport ports 0-N.
Commit 051a1dd5 changed this around to be output ports 0-N, and input
ports 1-N, and an InfiniScale IV based fabric would log lots of errors
like these:
log_rcv_cb_error: ERR 3111: Received MAD with error status = 0x1C
SubnGetResp(SLtoVLMappingTable), attr_mod 0x2300, TID 0xad069
Initial path: 0,1,1,4,13 Return path: 0,25,1,7,10
The attr_mod in every such message has 0x00 in the least significant
byte.
With the port ranges restored to their old values, the above log messages
stop.
Signed-off-by: Jim Schutt <jaschut-4OHPYypu0djtX7QSmKvirg@public.gmane.org>
---
opensm/opensm/osm_qos.c | 8 ++++----
1 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/opensm/opensm/osm_qos.c b/opensm/opensm/osm_qos.c
index 6bbbfa2..b8c3111 100644
--- a/opensm/opensm/osm_qos.c
+++ b/opensm/opensm/osm_qos.c
@@ -230,12 +230,12 @@ static int qos_extports_setup(osm_sm_t * sm, osm_node_t *node,
&qcfg->sl2vl);
}
- for (i = 1; i < num_ports; i++) {
+ for (i = 0; i < num_ports; i++) {
p = osm_node_get_physp_ptr(node, i);
force_update = p->need_update || sm->p_subn->need_update;
- for (j = 0; j < num_ports; j++)
- if (sl2vl_update_table(sm, p, i, i << 8 | j,
- force_update, &qcfg->sl2vl))
+ for (j = 1; j < num_ports; j++)
+ if (sl2vl_update_table(sm, p, i, j, force_update,
+ &qcfg->sl2vl))
ret = -1;
}
--
1.6.2.2
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: Proposal for adding ib_usa to the Linux Infiniband Subsystem
From: Jason Gunthorpe @ 2010-05-21 19:45 UTC (permalink / raw)
To: Sean Hefty; +Cc: 'Mike Heinz', linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <9EAF4B21EF0A4B0AB50AF7790DE2C046-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org>
On Fri, May 21, 2010 at 10:57:17AM -0700, Sean Hefty wrote:
> IMO, someone needs to propose a specific user space API that provides this
> capability. Personally, I don't think MADs provide the desired interface.
My thinking was sort of:
fd = open("/dev/..umad..");
ioctl(fd, SUBSCRIBE_TRAP, &trap_description)
read(fd..); // Events arrive
close(fd); // Event is unsubscribed
The reason this fits nicely with umad is that you really actually want
the GMP trap, not some processed version.. So this is like a multicast
subscribe in IP.
So.. The main question is how to do formulate a SUBSCRIBE_TRAP
operation that:
1) Can describe GSI traps for multiple managers, not just SA
2) Specifies how to filter incoming GMPs
3) Lets the kernel send and refcount the trap subscribe message
4) Lets the kernel send an unsubscribe message.
5) Lets the kernel send a single repress message if necessary
The first two seem easy, use the trap ID, class ID, and possibly LID
as well.
I think the latter three are not too bad, the subscribes/unsubscribe
messages are pretty uniform. Tricky bit might be the ref counting?
Mostly just reading the spec to see..
> I suggested looking at the libibverbs async events, but I didn't
> take the time to look at the details of how well that API would fit.
> It's used to report client_reregister events, so using it to report
> notifications doesn't seem too out of line.
This seems reasonable to me for some high level notices that are
commonly used and shouldn't require privilege to access. But if you
want to access raw trap/notices then umad seems a better bet.
To me, if it is OK for the kernel to always subscribe to the
trap/notice then async events seem to be a good choice. I'm not sure
the kernel should unconditionally subscribe to things like gid in/out
of service, etc..
> Isn't access to the umad interface usually privileged? If so, is that
> acceptable for the anticipated usage?
Subscribing to an arbitary trap should be privileged.. IHMO.
But I agree, is there a single specific trap you are trying to access
with ib_usa??
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Problem with RDMA device removal architecture
From: Roland Dreier @ 2010-05-21 18:30 UTC (permalink / raw)
To: Steve Wise; +Cc: Sean Hefty, linux-rdma, Wen Xiong
In-Reply-To: <4BF6CBAA.5020906-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
> > I'll ask the experts on LKML when I get a chance to write up the problem description.
> Did you ever get any feedback from the LKML folks?
No, I forgot about writing it up. Thanks for the reminder.
--
Roland Dreier <rolandd-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org> || For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: Proposal for adding ib_usa to the Linux Infiniband Subsystem
From: Mike Heinz @ 2010-05-21 18:19 UTC (permalink / raw)
To: Sean Hefty, 'Jason Gunthorpe'
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <9EAF4B21EF0A4B0AB50AF7790DE2C046-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org>
> IMO, someone needs to propose a specific user space API that provides this
> capability. Personally, I don't think MADs provide the desired interface.
Agreed. I also agree with Jason that the proliferation of interfaces and modules is an impediment to IB adoption.
And if we need to get the API nailed down first, so be it, that's why I made the proposal the way I did - to get feedback on what was needed.
Let me spend a few days reviewing libibumad and libibverbs, see if can get an idea of what would be the better fit.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Problem with RDMA device removal architecture
From: Steve Wise @ 2010-05-21 18:06 UTC (permalink / raw)
To: Roland Dreier; +Cc: Sean Hefty, linux-rdma, Wen Xiong
In-Reply-To: <adaociaykn1.fsf-BjVyx320WGW9gfZ95n9DRSW4+XlvGpQz@public.gmane.org>
Roland Dreier wrote:
> I'll ask the experts on LKML when I get a chance to write up the problem description.
>
Hey Roland,
Did you ever get any feedback from the LKML folks?
Thanks,
Steve.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: Proposal for adding ib_usa to the Linux Infiniband Subsystem
From: Sean Hefty @ 2010-05-21 17:57 UTC (permalink / raw)
To: 'Jason Gunthorpe', 'Mike Heinz'
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20100521172026.GW15969-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
>verbs? Why verbs?
>
>umad. The only thing that is needed is to subscribe to a GSI
>notification, replicate incoming notifications and route them as
>normal mads over the umad interface to all subscribers..
IMO, someone needs to propose a specific user space API that provides this
capability. Personally, I don't think MADs provide the desired interface.
I suggested looking at the libibverbs async events, but I didn't take the time
to look at the details of how well that API would fit. It's used to report
client_reregister events, so using it to report notifications doesn't seem too
out of line.
What's missing is a mechanism to register for the event, which does map better
to umad. Maybe the kernel user_mad module snoops MADs and reacts accordingly.
Isn't access to the umad interface usually privileged? If so, is that
acceptable for the anticipated usage?
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH] ib_types.h add debug assert
From: Smith, Stan @ 2010-05-21 17:49 UTC (permalink / raw)
To: Sasha Khapyorsky; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Add a debug assert to catch incorrect MAD attr offset size.
This proved to be useful in catching incorrect struct sizes as MAD attrs need to be a multiple of 8 bytes.
signed-off-by: stan smith <stan.smith-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
diff --git a/opensm/include/iba/ib_types.h b/opensm/include/iba/ib_types.h
index e1bc102..203c319 100644
--- a/opensm/include/iba/ib_types.h
+++ b/opensm/include/iba/ib_types.h
@@ -4395,6 +4395,7 @@ static inline uint32_t OSM_API ib_get_attr_size(IN const ib_net16_t attr_offset)
static inline ib_net16_t OSM_API ib_get_attr_offset(IN const uint32_t attr_size)
{
+ CL_ASSERT((attr_size & 0x07) == 0);
return (cl_hton16((uint16_t) (attr_size >> 3)));
}
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: Proposal for adding ib_usa to the Linux Infiniband Subsystem
From: Jason Gunthorpe @ 2010-05-21 17:43 UTC (permalink / raw)
To: Mike Heinz; +Cc: Sean Hefty, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <4C2744E8AD2982428C5BFE523DF8CDCB49A488D186-amwN6d8PyQWXx9kJd3VG2h2eb7JE58TQ@public.gmane.org>
On Fri, May 21, 2010 at 12:29:31PM -0500, Mike Heinz wrote:
>> umad. The only thing that is needed is to subscribe to a GSI
>> notification, replicate incoming notifications and route them as
>> normal mads over the umad interface to all subscribers..
> However - you're wrong about "the only thing needed" being to
> subscribe to a GSI notification; notifications are sent to ports,
> not to individual queue pairs or processes - that's why the ib_sa
> module has a mechanism for for managing notifications. Like ib_umad,
> ib_usa is the mechanism for extending ib_sa into user space.
To be fair I had two things in that statement..
umad instances are per-port. You do an ioctl or something on the umad
FD to register for the notification, and the umad module replicates an
incoming notification GMP to all registrants. This seems totally
straightforward to me..
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH] opensm/libvendor Reduce stack consumption
From: Smith, Stan @ 2010-05-21 17:37 UTC (permalink / raw)
To: Sasha Khapyorsky; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Reduce stack consumption by using a union of structs instead of individual struct allocations.
Code borrowed from osm_vendor_ibumad_sa.c
signed-off-by: stan smith <stan.smith-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
diff --git a/opensm/libvendor/osm_vendor_mlx_sa.c b/opensm/libvendor/osm_vendor_mlx_sa.c
index 91f82fa..0cc3f19 100644
--- a/opensm/libvendor/osm_vendor_mlx_sa.c
+++ b/opensm/libvendor/osm_vendor_mlx_sa.c
@@ -576,14 +576,19 @@ ib_api_status_t
osmv_query_sa(IN osm_bind_handle_t h_bind,
IN const osmv_query_req_t * const p_query_req)
{
- osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *) h_bind;
+ union {
+ ib_service_record_t svc_rec;
+ ib_node_record_t node_rec;
+ ib_portinfo_record_t port_info;
+ ib_path_rec_t path_rec;
+#ifdef DUAL_SIDED_RMPP
+ ib_multipath_rec_t multipath_rec;
+#endif
+ ib_class_port_info_t class_port_info;
+ } u;
osmv_sa_mad_data_t sa_mad_data;
+ osmv_sa_bind_info_t *p_bind = (osmv_sa_bind_info_t *) h_bind;
osmv_user_query_t *p_user_query;
- ib_service_record_t svc_rec;
- ib_node_record_t node_rec;
- ib_portinfo_record_t port_info;
- ib_path_rec_t path_rec;
- ib_class_port_info_t class_port_info;
osm_log_t *p_log = p_bind->p_log;
ib_api_status_t status;
@@ -617,7 +622,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind,
sa_mad_data.attr_offset =
ib_get_attr_offset(sizeof(ib_service_record_t));
sa_mad_data.comp_mask = 0;
- sa_mad_data.p_attr = &svc_rec;
+ sa_mad_data.p_attr = &u.svc_rec;
break;
case OSMV_QUERY_SVC_REC_BY_NAME:
@@ -628,8 +633,8 @@ osmv_query_sa(IN osm_bind_handle_t h_bind,
sa_mad_data.comp_mask = IB_SR_COMPMASK_SNAME;
sa_mad_data.attr_offset =
ib_get_attr_offset(sizeof(ib_service_record_t));
- sa_mad_data.p_attr = &svc_rec;
- memcpy(svc_rec.service_name, p_query_req->p_query_input,
+ sa_mad_data.p_attr = &u.svc_rec;
+ memcpy(u.svc_rec.service_name, p_query_req->p_query_input,
sizeof(ib_svc_name_t));
break;
@@ -640,8 +645,8 @@ osmv_query_sa(IN osm_bind_handle_t h_bind,
sa_mad_data.comp_mask = IB_SR_COMPMASK_SID;
sa_mad_data.attr_offset =
ib_get_attr_offset(sizeof(ib_service_record_t));
- sa_mad_data.p_attr = &svc_rec;
- svc_rec.service_id =
+ sa_mad_data.p_attr = &u.svc_rec;
+ u.svc_rec.service_id =
*(ib_net64_t *) (p_query_req->p_query_input);
break;
@@ -653,7 +658,7 @@ osmv_query_sa(IN osm_bind_handle_t h_bind,
sa_mad_data.attr_offset =
ib_get_attr_offset(sizeof(ib_class_port_info_t));
sa_mad_data.comp_mask = 0;
- sa_mad_data.p_attr = &class_port_info;
+ sa_mad_data.p_attr = &u.class_port_info;
break;
@@ -665,8 +670,8 @@ osmv_query_sa(IN osm_bind_handle_t h_bind,
sa_mad_data.attr_offset =
ib_get_attr_offset(sizeof(ib_node_record_t));
sa_mad_data.comp_mask = IB_NR_COMPMASK_NODEGUID;
- sa_mad_data.p_attr = &node_rec;
- node_rec.node_info.node_guid =
+ sa_mad_data.p_attr = &u.node_rec;
+ u.node_rec.node_info.node_guid =
*(ib_net64_t *) (p_query_req->p_query_input);
break;
@@ -678,8 +683,8 @@ osmv_query_sa(IN osm_bind_handle_t h_bind,
sa_mad_data.attr_offset =
ib_get_attr_offset(sizeof(ib_portinfo_record_t));
sa_mad_data.comp_mask = IB_PIR_COMPMASK_LID;
- sa_mad_data.p_attr = &port_info;
- port_info.lid = *(ib_net16_t *) (p_query_req->p_query_input);
+ sa_mad_data.p_attr = &u.port_info;
+ u.port_info.lid = *(ib_net16_t *) (p_query_req->p_query_input);
break;
case OSMV_QUERY_PORT_REC_BY_LID_AND_NUM:
@@ -729,19 +734,19 @@ osmv_query_sa(IN osm_bind_handle_t h_bind,
case OSMV_QUERY_PATH_REC_BY_PORT_GUIDS:
osm_log(p_log, OSM_LOG_DEBUG,
"osmv_query_sa DBG:001 %s", "PATH_REC_BY_PORT_GUIDS\n");
- memset(&path_rec, 0, sizeof(ib_path_rec_t));
+ memset(&u.path_rec, 0, sizeof(ib_path_rec_t));
sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD;
sa_mad_data.attr_offset =
ib_get_attr_offset(sizeof(ib_path_rec_t));
sa_mad_data.comp_mask =
(IB_PR_COMPMASK_DGID | IB_PR_COMPMASK_SGID | IB_PR_COMPMASK_NUMBPATH);
- path_rec.num_path = 0x7f;
- sa_mad_data.p_attr = &path_rec;
- ib_gid_set_default(&path_rec.dgid,
+ u.path_rec.num_path = 0x7f;
+ sa_mad_data.p_attr = &u.path_rec;
+ ib_gid_set_default(&u.path_rec.dgid,
((osmv_guid_pair_t *) (p_query_req->
p_query_input))->
dest_guid);
- ib_gid_set_default(&path_rec.sgid,
+ ib_gid_set_default(&u.path_rec.sgid,
((osmv_guid_pair_t *) (p_query_req->
p_query_input))->
src_guid);
@@ -750,18 +755,18 @@ osmv_query_sa(IN osm_bind_handle_t h_bind,
case OSMV_QUERY_PATH_REC_BY_GIDS:
osm_log(p_log, OSM_LOG_DEBUG,
"osmv_query_sa DBG:001 %s", "PATH_REC_BY_GIDS\n");
- memset(&path_rec, 0, sizeof(ib_path_rec_t));
+ memset(&u.path_rec, 0, sizeof(ib_path_rec_t));
sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD;
sa_mad_data.attr_offset =
ib_get_attr_offset(sizeof(ib_path_rec_t));
sa_mad_data.comp_mask =
- (IB_PR_COMPMASK_DGID | IB_PR_COMPMASK_SGID | IB_PR_COMPMASK_NUMBPATH);
- path_rec.num_path = 0x7f;
- sa_mad_data.p_attr = &path_rec;
- memcpy(&path_rec.dgid,
+ (IB_PR_COMPMASK_DGID | IB_PR_COMPMASK_SGID | IB_PR_COMPMASK_NUMBPATH);
+ u.path_rec.num_path = 0x7f;
+ sa_mad_data.p_attr = &u.path_rec;
+ memcpy(&u.path_rec.dgid,
&((osmv_gid_pair_t *) (p_query_req->p_query_input))->
dest_gid, sizeof(ib_gid_t));
- memcpy(&path_rec.sgid,
+ memcpy(&u.path_rec.sgid,
&((osmv_gid_pair_t *) (p_query_req->p_query_input))->
src_gid, sizeof(ib_gid_t));
break;
@@ -769,18 +774,18 @@ osmv_query_sa(IN osm_bind_handle_t h_bind,
case OSMV_QUERY_PATH_REC_BY_LIDS:
osm_log(p_log, OSM_LOG_DEBUG,
"osmv_query_sa DBG:001 %s", "PATH_REC_BY_LIDS\n");
- memset(&path_rec, 0, sizeof(ib_path_rec_t));
+ memset(&u.path_rec, 0, sizeof(ib_path_rec_t));
sa_mad_data.method = IB_MAD_METHOD_GET;
sa_mad_data.attr_id = IB_MAD_ATTR_PATH_RECORD;
sa_mad_data.attr_offset =
ib_get_attr_offset(sizeof(ib_path_rec_t));
sa_mad_data.comp_mask =
(IB_PR_COMPMASK_DLID | IB_PR_COMPMASK_SLID);
- sa_mad_data.p_attr = &path_rec;
- path_rec.dlid =
+ sa_mad_data.p_attr = &u.path_rec;
+ u.path_rec.dlid =
((osmv_lid_pair_t *) (p_query_req->p_query_input))->
dest_lid;
- path_rec.slid =
+ u.path_rec.slid =
((osmv_lid_pair_t *) (p_query_req->p_query_input))->src_lid;
break;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* RE: Proposal for adding ib_usa to the Linux Infiniband Subsystem
From: Mike Heinz @ 2010-05-21 17:29 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Sean Hefty, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <20100521172026.GW15969-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
Jason,
I mentioned verbs because Sean had suggested it and, after all, he's the original author of ib_usa.
However - you're wrong about "the only thing needed" being to subscribe to a GSI notification; notifications are sent to ports, not to individual queue pairs or processes - that's why the ib_sa module has a mechanism for for managing notifications. Like ib_umad, ib_usa is the mechanism for extending ib_sa into user space.
-----Original Message-----
From: Jason Gunthorpe [mailto:jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org]
Sent: Friday, May 21, 2010 1:20 PM
To: Mike Heinz
Cc: Sean Hefty; linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Re: Proposal for adding ib_usa to the Linux Infiniband Subsystem
On Fri, May 21, 2010 at 12:04:17PM -0500, Mike Heinz wrote:
> I have to agree that in some ways this seems a good fit with
> ibverbs. Also, without the multicast support, the whole thing is
> well under 1 kloc in the kernel, making it a good candidate for
> being combined with another module.
verbs? Why verbs?
umad. The only thing that is needed is to subscribe to a GSI
notification, replicate incoming notifications and route them as
normal mads over the umad interface to all subscribers..
With some cleverness you could probably make the subscribe/unsubscribe
fairly general..
Remember SA notifications are not the only unsolicited events that can
occure in the GSI framework.
> 1. If it's been working for a couple of years now, do we really need
> to change it?
We need to live with it forever once it is accepted..
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH] osmtest - use helper function
From: Smith, Stan @ 2010-05-21 17:23 UTC (permalink / raw)
To: Sasha Khapyorsky; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Use defined inline helper function instead of actual implementation.
Separate implementation from use.
signed-off-by: stan smith <stan.smith-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
diff --git a/opensm/osmtest/osmtest.c b/opensm/osmtest/osmtest.c
index 50f94db..5d9c54c 100644
--- a/opensm/osmtest/osmtest.c
+++ b/opensm/osmtest/osmtest.c
@@ -719,7 +719,7 @@ osmtest_get_node_rec(IN osmtest_t * const p_osmt,
p_context->p_osmt = p_osmt;
user.comp_mask = IB_NR_COMPMASK_NODEGUID;
user.attr_id = IB_MAD_ATTR_NODE_RECORD;
- user.attr_offset = cl_ntoh16((uint16_t) (sizeof(record) >> 3));
+ user.attr_offset = ib_get_attr_offset((uint16_t) (sizeof(record)));
user.p_attr = &record;
req.query_type = OSMV_QUERY_USER_DEFINED;
@@ -793,7 +793,7 @@ osmtest_get_node_rec_by_lid(IN osmtest_t * const p_osmt,
p_context->p_osmt = p_osmt;
user.comp_mask = IB_NR_COMPMASK_LID;
user.attr_id = IB_MAD_ATTR_NODE_RECORD;
- user.attr_offset = cl_ntoh16((uint16_t) (sizeof(record) >> 3));
+ user.attr_offset = ib_get_attr_offset((uint16_t) (sizeof(record)));
user.p_attr = &record;
req.query_type = OSMV_QUERY_USER_DEFINED;
@@ -1057,7 +1057,7 @@ osmtest_get_port_rec(IN osmtest_t * const p_osmt,
p_context->p_osmt = p_osmt;
user.comp_mask = IB_PIR_COMPMASK_LID;
user.attr_id = IB_MAD_ATTR_PORTINFO_RECORD;
- user.attr_offset = cl_ntoh16((uint16_t) (sizeof(record) >> 3));
+ user.attr_offset = ib_get_attr_offset((uint16_t) (sizeof(record)));
user.p_attr = &record;
req.query_type = OSMV_QUERY_USER_DEFINED;
@@ -4171,7 +4171,7 @@ osmtest_get_link_rec_by_lid(IN osmtest_t * const p_osmt,
if (to_lid)
user.comp_mask |= IB_LR_COMPMASK_TO_LID;
user.attr_id = IB_MAD_ATTR_LINK_RECORD;
- user.attr_offset = cl_ntoh16((uint16_t) (sizeof(record) >> 3));
+ user.attr_offset = ib_get_attr_offset((uint16_t) (sizeof(record)));
user.p_attr = &record;
req.query_type = OSMV_QUERY_USER_DEFINED;
@@ -4249,7 +4249,7 @@ osmtest_get_guidinfo_rec_by_lid(IN osmtest_t * const p_osmt,
p_context->p_osmt = p_osmt;
user.comp_mask = IB_GIR_COMPMASK_LID;
user.attr_id = IB_MAD_ATTR_GUIDINFO_RECORD;
- user.attr_offset = cl_ntoh16((uint16_t) (sizeof(record) >> 3));
+ user.attr_offset = ib_get_attr_offset((uint16_t) (sizeof(record)));
user.p_attr = &record;
req.query_type = OSMV_QUERY_USER_DEFINED;
@@ -4328,7 +4328,7 @@ osmtest_get_pkeytbl_rec_by_lid(IN osmtest_t * const p_osmt,
p_context->p_osmt = p_osmt;
user.comp_mask = IB_PKEY_COMPMASK_LID;
user.attr_id = IB_MAD_ATTR_PKEY_TBL_RECORD;
- user.attr_offset = cl_ntoh16((uint16_t) (sizeof(record) >> 3));
+ user.attr_offset = ib_get_attr_offset((uint16_t) (sizeof(record)));
user.p_attr = &record;
req.query_type = OSMV_QUERY_USER_DEFINED;
@@ -4407,7 +4407,7 @@ osmtest_get_sw_info_rec_by_lid(IN osmtest_t * const p_osmt,
if (lid)
user.comp_mask = IB_SWIR_COMPMASK_LID;
user.attr_id = IB_MAD_ATTR_SWITCH_INFO_RECORD;
- user.attr_offset = cl_ntoh16((uint16_t) (sizeof(record) >> 3));
+ user.attr_offset = ib_get_attr_offset((uint16_t) (sizeof(record)));
user.p_attr = &record;
req.query_type = OSMV_QUERY_USER_DEFINED;
@@ -4486,7 +4486,7 @@ osmtest_get_lft_rec_by_lid(IN osmtest_t * const p_osmt,
if (lid)
user.comp_mask = IB_LFTR_COMPMASK_LID;
user.attr_id = IB_MAD_ATTR_LFT_RECORD;
- user.attr_offset = cl_ntoh16((uint16_t) (sizeof(record) >> 3));
+ user.attr_offset = ib_get_attr_offset((uint16_t) (sizeof(record)));
user.p_attr = &record;
req.query_type = OSMV_QUERY_USER_DEFINED;
@@ -4565,7 +4565,7 @@ osmtest_get_mft_rec_by_lid(IN osmtest_t * const p_osmt,
if (lid)
user.comp_mask = IB_MFTR_COMPMASK_LID;
user.attr_id = IB_MAD_ATTR_MFT_RECORD;
- user.attr_offset = cl_ntoh16((uint16_t) (sizeof(record) >> 3));
+ user.attr_offset = ib_get_attr_offset((uint16_t) (sizeof(record)));
user.p_attr = &record;
req.query_type = OSMV_QUERY_USER_DEFINED;
@@ -4637,7 +4637,7 @@ osmtest_sminfo_record_request(IN osmtest_t * const p_osmt,
p_context->p_osmt = p_osmt;
user.attr_id = IB_MAD_ATTR_SMINFO_RECORD;
- user.attr_offset = cl_ntoh16((uint16_t) (sizeof(record) >> 3));
+ user.attr_offset = ib_get_attr_offset((uint16_t) (sizeof(record)));
p_sm_info_opt = p_options;
if (p_sm_info_opt->sm_guid != 0) {
record.sm_info.guid = p_sm_info_opt->sm_guid;
@@ -4737,7 +4737,7 @@ osmtest_informinfo_request(IN osmtest_t * const p_osmt,
p_context->p_osmt = p_osmt;
user.attr_id = attr_id;
if (attr_id == IB_MAD_ATTR_INFORM_INFO_RECORD) {
- user.attr_offset = cl_ntoh16((uint16_t) (sizeof(record) >> 3));
+ user.attr_offset = ib_get_attr_offset((uint16_t) (sizeof(record)));
p_inform_info_rec_opt = p_options;
if (p_inform_info_rec_opt->subscriber_gid.unicast.prefix != 0 &&
p_inform_info_rec_opt->subscriber_gid.unicast.
@@ -4751,7 +4751,7 @@ osmtest_informinfo_request(IN osmtest_t * const p_osmt,
user.comp_mask |= IB_IIR_COMPMASK_ENUM;
user.p_attr = &record;
} else {
- user.attr_offset = cl_ntoh16((uint16_t) (sizeof(rec) >> 3));
+ user.attr_offset = ib_get_attr_offset((uint16_t) (sizeof(record)));
/* comp mask bits below are for InformInfoRecord rather than InformInfo */
/* as currently no comp mask bits defined for InformInfo!!! */
user.comp_mask = IB_IIR_COMPMASK_SUBSCRIBE;
@@ -4900,7 +4900,7 @@ osmtest_validate_single_node_rec_lid(IN osmtest_t * const p_osmt,
context.p_osmt = p_osmt;
user.comp_mask = IB_NR_COMPMASK_LID;
user.attr_id = IB_MAD_ATTR_NODE_RECORD;
- user.attr_offset = cl_ntoh16((uint16_t) (sizeof(record) >> 3));
+ user.attr_offset = ib_get_attr_offset((uint16_t) sizeof(record));
user.p_attr = &record;
req.query_type = OSMV_QUERY_USER_DEFINED;
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: Proposal for adding ib_usa to the Linux Infiniband Subsystem
From: Jason Gunthorpe @ 2010-05-21 17:20 UTC (permalink / raw)
To: Mike Heinz; +Cc: Sean Hefty, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <4C2744E8AD2982428C5BFE523DF8CDCB49A488D181-amwN6d8PyQWXx9kJd3VG2h2eb7JE58TQ@public.gmane.org>
On Fri, May 21, 2010 at 12:04:17PM -0500, Mike Heinz wrote:
> I have to agree that in some ways this seems a good fit with
> ibverbs. Also, without the multicast support, the whole thing is
> well under 1 kloc in the kernel, making it a good candidate for
> being combined with another module.
verbs? Why verbs?
umad. The only thing that is needed is to subscribe to a GSI
notification, replicate incoming notifications and route them as
normal mads over the umad interface to all subscribers..
With some cleverness you could probably make the subscribe/unsubscribe
fairly general..
Remember SA notifications are not the only unsolicited events that can
occure in the GSI framework.
> 1. If it's been working for a couple of years now, do we really need
> to change it?
We need to live with it forever once it is accepted..
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: Proposal for adding ib_usa to the Linux Infiniband Subsystem
From: Mike Heinz @ 2010-05-21 17:04 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Sean Hefty, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <20100521165018.GA8856-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
I have to agree that in some ways this seems a good fit with ibverbs. Also, without the multicast support, the whole thing is well under 1 kloc in the kernel, making it a good candidate for being combined with another module.
My biggest objections, though, would be:
1. If it's been working for a couple of years now, do we really need to change it?
2. The async events in ibverbs are all local events, are we really keeping things simple by adding remote notifications to mix, or making a simple interface much more complicated?
-----Original Message-----
From: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org [mailto:linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org] On Behalf Of Jason Gunthorpe
Sent: Friday, May 21, 2010 12:50 PM
To: Mike Heinz
Cc: Sean Hefty; linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Re: Proposal for adding ib_usa to the Linux Infiniband Subsystem
I've said this before, but again..
There are too many orthogonal interfaces here. We really don't need
more libraries or kernel modules or /dev/ devices. Really.
Subscribing and multiplexing notifications from a GSI service should
just be a general function in the mad code..
On Fri, May 21, 2010 at 10:57:48AM -0500, Mike Heinz wrote:
> Would it be better to omit the multicast support from ib_usa and simply add it as a way to handle notifications?
>
> From: Sean Hefty [mailto:sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org]
> Sent: Friday, May 21, 2010 11:30 AM
> To: Mike Heinz; linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> Subject: RE: Proposal for adding ib_usa to the Linux Infiniband Subsystem
>
> >Why is this needed?
> >The SA protocol allows notices and multicast membership to be managed at a CA
> >port level.? As a result, if multiple processes want to receive notices or join
> >multicast groups, that registration must be coordinated per CA port.? Existing
> >code in the Linux kernel exists for this purpose, coordinating the needs of
> >various Infiniband kernel modules and presenting a single "per server CA port"
> >perspective to the SA.? The IB user space SA module provides a mechanism for
> >user space applications to take advantage of this existing kernel code for
> >managing SA registrations.
>
> The AF_IB support that is being requested for the rdma_cm would provide greater
> support for IB multicast join/leave membership. Some additional work would be
> needed to allow the user to specify the full MCMemberRecord, but the basic
> infrastructure should be there. Just mentioning this as a possibility. (The IB
> ACM code joins multicast groups using libibumad and sends MADs directly to the
> SA.)
>
> - Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox