* [PATCH] opensm: Add the precreation of multicast groups
@ 2011-10-07 0:13 Ira Weiny
[not found] ` <20111006171326.3b143b03.weiny2-i2BcT+NCU+M@public.gmane.org>
0 siblings, 1 reply; 4+ messages in thread
From: Ira Weiny @ 2011-10-07 0:13 UTC (permalink / raw)
To: Alex Netes; +Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Allow for the pre-creation of these groups on a partition by partition basis.
P_Key is taken from the partition specification. Q_Key, TClass, rate, and mtu
can be specified.
For IP groups, rate and mtu are verified to match the broadcast groups
parameters. P_Key can be specified in the mgid itself and is verified to match
the P_Key of the partition if != 0x0000. If pkey == 0x0000 then the P_Key is
taken from the partition specification.
The syntax extends the existing syntax by allowing MC groups to be specified
one per line, intermixed with the port specifications.
Signed-off-by: Ira Weiny <weiny2-i2BcT+NCU+M@public.gmane.org>
---
include/opensm/osm_base.h | 5 +
include/opensm/osm_partition.h | 12 +-
man/opensm.8.in | 121 +++++++++++------
opensm/osm_prtn.c | 111 ++++++++++++----
opensm/osm_prtn_config.c | 278 ++++++++++++++++++++++++++++++++++++----
opensm/osm_qos_policy.c | 48 ++++----
opensm/osm_subnet.c | 2 +-
7 files changed, 454 insertions(+), 123 deletions(-)
diff --git a/include/opensm/osm_base.h b/include/opensm/osm_base.h
index e558c55..869ba9c 100644
--- a/include/opensm/osm_base.h
+++ b/include/opensm/osm_base.h
@@ -53,6 +53,7 @@
#endif
#include <complib/cl_types.h>
+#include <iba/ib_types.h>
#ifdef __cplusplus
# define BEGIN_C_DECLS extern "C" {
@@ -954,6 +955,10 @@ typedef enum _osm_sm_signal {
#define OSM_VENDOR_ID_HP4 0x00237D
#define OSM_VENDOR_ID_OPENIB 0x001405
+/* IPoIB Broadcast Defaults */
+#define OSM_IPOIB_BROADCAST_MGRP_QKEY 0x0b1b
+extern const ib_gid_t osm_ipoib_broadcast_mgid;
+
/**********/
END_C_DECLS
diff --git a/include/opensm/osm_partition.h b/include/opensm/osm_partition.h
index fdb34b9..7277eac 100644
--- a/include/opensm/osm_partition.h
+++ b/include/opensm/osm_partition.h
@@ -94,10 +94,11 @@ typedef struct osm_prtn {
cl_map_item_t map_item;
ib_net16_t pkey;
uint8_t sl;
- osm_mgrp_t *mgrp;
cl_map_t full_guid_tbl;
cl_map_t part_guid_tbl;
char name[32];
+ osm_mgrp_t **mgrps;
+ int nmgrps;
} osm_prtn_t;
/*
* FIELDS
@@ -110,9 +111,10 @@ typedef struct osm_prtn {
* sl
* The Service Level (SL) associated with this Partiton.
*
-* mgrp
-* The pointer to the well known Multicast Group
-* that was created for this partition (when configured).
+* mgrps
+* List of well known Multicast Groups
+* that were created for this partition (when configured).
+* This includes the IPoIB broadcast group.
*
* full_guid_tbl
* Container of pointers to all Port objects in the Partition
@@ -139,7 +141,7 @@ typedef struct osm_prtn {
*
* SYNOPSIS
*/
-void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn);
+void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn, osm_subn_t * p_subn);
/*
* PARAMETERS
* pp_prtn
diff --git a/man/opensm.8.in b/man/opensm.8.in
index 042bee3..da0c247 100644
--- a/man/opensm.8.in
+++ b/man/opensm.8.in
@@ -523,45 +523,76 @@ parser.
General file format:
-<Partition Definition>:<PortGUIDs list> ;
-
-Partition Definition:
-
-[PartitionName][=PKey][,flag[=value]][,defmember=full|limited]
-
- PartitionName - string, will be used with logging. When omitted
- empty string will be used.
- PKey - P_Key value for this partition. Only low 15 bits will
- be used. When omitted will be autogenerated.
- flag - used to indicate IPoIB capability of this partition.
- defmember=full|limited - specifies default membership for port guid
- list. Default is limited.
-
-Currently recognized flags are:
-
- ipoib - indicates that this partition may be used for IPoIB, as
- result IPoIB capable MC group will be created.
- rate=<val> - specifies rate for this IPoIB MC group
- (default is 3 (10GBps))
- mtu=<val> - specifies MTU for this IPoIB MC group
- (default is 4 (2048))
- sl=<val> - specifies SL for this IPoIB MC group
- (default is 0)
- scope=<val> - specifies scope for this IPoIB MC group
- (default is 2 (link local)). Multiple scope settings
- are permitted for a partition.
-
-Note that values for rate, mtu, and scope should be specified as
-defined in the IBTA specification (for example, mtu=4 for 2048).
-
-PortGUIDs list:
-
- PortGUID - GUID of partition member EndPort. Hexadecimal
- numbers should start from 0x, decimal numbers
- are accepted too.
- full or limited - indicates full or limited membership for this
- port. When omitted (or unrecognized) limited
- membership is assumed.
+<Partition Definition>:[<newline>]<Partition Properties>;
+
+ Partition Definition:
+ [PartitionName][=PKey][,part_flag[=value]][,defmember=full|limited]
+
+ PartitionName - string, will be used with logging. When omitted
+ empty string will be used.
+ PKey - P_Key value for this partition. Only low 15 bits will
+ be used. When omitted will be autogenerated.
+ part_flags - used to indicate/specify IPoIB capability of this partition.
+ defmember=full|limited - specifies default membership for port guid
+ list. Default is limited.
+
+ part_flag:
+ ipoib - indicates that this partition may be used for IPoIB, as
+ result IPoIB capable MC group will be created.
+ rate=<val> - specifies rate for this IPoIB MC group
+ (default is 3 (10GBps))
+ mtu=<val> - specifies MTU for this IPoIB MC group
+ (default is 4 (2048))
+ sl=<val> - specifies SL for this IPoIB MC group
+ (default is 0)
+ scope=<val> - specifies scope for this IPoIB MC group
+ (default is 2 (link local)). Multiple scope settings
+ are permitted for a partition.
+
+ Partition Properties:
+ [<Port list>|<MCast Group>]* | <Port list>
+
+ Port list:
+ <Port Specifier>[,<Port Specifier>]
+
+ Port Specifier:
+ <PortGUID>[=[full|limited]]
+
+ PortGUID - GUID of partition member EndPort. Hexadecimal
+ numbers should start from 0x, decimal numbers
+ are accepted too.
+ full or limited - indicates full or limited membership for this
+ port. When omitted (or unrecognized) limited
+ membership is assumed.
+
+ MCast Group:
+ mgid=gid[,mgroup_flag=val]*<newline>
+
+ mgroup_flag:
+ rate=<val> - specifies rate for this MC group
+ (default is 3 (10GBps))
+ mtu=<val> - specifies MTU for this MC group
+ (default is 4 (2048))
+ sl=<val> - specifies SL for this MC group
+ (default is 0)
+ scope=<val> - specifies scope for this MC group
+ (default is 2 (link local)). Multiple scope settings
+ are permitted for a partition.
+ NOTE: This overwrites the scope nibble of the specified
+ mgid. Furthermore specifying multiple scope
+ settings will result in multiple MC groups
+ being created.
+ qkey=<val> - specifies the Q_Key for this MC group
+ (default: 0x0b1b for IP groups, 0 for other groups)
+ tclass=<val> - specifies tclass for this MC group
+ (default is 0)
+
+ newline: '\n'
+
+
+Note that values for rate, mtu, and scope, for both partitions and multicast
+groups, should be specified as defined in the IBTA specification (for example,
+mtu=4 for 2048).
There are several useful keywords for PortGUID definition:
@@ -577,9 +608,6 @@ Notes:
White space is permitted between delimiters ('=', ',',':',';').
-The line can be wrapped after ':' followed after Partition Definition and
-between.
-
PartitionName does not need to be unique, PKey does need to be unique.
If PKey is repeated then those partition configurations will be merged
and first PartitionName will be used (see also next note).
@@ -606,6 +634,15 @@ Examples:
ShareIO = 0x80 , defmember=full : 0x123459, 0x12345a;
ShareIO = 0x80 , defmember=full : 0x12345b, 0x12345c=limited, 0x12345d;
+ # multicast groups added to default
+ Default=0x7fff,ipoib:
+ mgid=ff12:401b::0707,sl=1 # random IPv4 group
+ mgid=ff12:601b::16 # MLDv2-capable routers
+ mgid=ff12:401b::16 # IGMP
+ mgid=ff12:601b::2 # All routers
+ mgid=ff12::1,sl=1,Q_Key=0xDEADBEEF,rate=3,mtu=2 # random group
+ ALL=full;
+
Note:
diff --git a/opensm/osm_prtn.c b/opensm/osm_prtn.c
index 3fd4fc0..2e2837a 100644
--- a/opensm/osm_prtn.c
+++ b/opensm/osm_prtn.c
@@ -53,6 +53,8 @@
#include <opensm/osm_node.h>
#include <opensm/osm_sa.h>
#include <opensm/osm_multicast.h>
+#include <arpa/inet.h>
+#include <errno.h>
extern int osm_prtn_config_parse_file(osm_log_t * p_log, osm_subn_t * p_subn,
const char *file_name);
@@ -68,6 +70,8 @@ osm_prtn_t *osm_prtn_new(IN const char *name, IN uint16_t pkey)
memset(p, 0, sizeof(*p));
p->pkey = pkey;
p->sl = OSM_DEFAULT_SL;
+ p->mgrps = NULL;
+ p->nmgrps = 0;
cl_map_construct(&p->full_guid_tbl);
cl_map_init(&p->full_guid_tbl, 32);
cl_map_construct(&p->part_guid_tbl);
@@ -81,14 +85,34 @@ osm_prtn_t *osm_prtn_new(IN const char *name, IN uint16_t pkey)
return p;
}
-void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn)
+void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn, osm_subn_t * p_subn)
{
+ char gid_str[INET6_ADDRSTRLEN];
+ int i = 0;
osm_prtn_t *p = *pp_prtn;
cl_map_remove_all(&p->full_guid_tbl);
cl_map_destroy(&p->full_guid_tbl);
cl_map_remove_all(&p->part_guid_tbl);
cl_map_destroy(&p->part_guid_tbl);
+
+ if (p->mgrps) {
+ /* Clean up mgrps */
+ for (i = 0; i < p->nmgrps; i++) {
+ /* osm_mgrp_cleanup will not delete
+ * "well_known" groups */
+ p->mgrps[i]->well_known = FALSE;
+ osm_mgrp_cleanup(p_subn, p->mgrps[i]);
+ OSM_LOG(&p_subn->p_osm->log, OSM_LOG_ERROR,
+ "removing mgroup %s from partition (0x%x)\n",
+ inet_ntop(AF_INET6, p->mgrps[i]->mcmember_rec.mgid.raw,
+ gid_str, sizeof gid_str),
+ cl_hton16(p->pkey));
+ }
+
+ free(p->mgrps);
+ }
+
free(p);
*pp_prtn = NULL;
}
@@ -156,21 +180,47 @@ _err:
return status;
}
-static const ib_gid_t osm_ipoib_mgid = {
- {
- 0xff, /* multicast field */
- 0x12, /* non-permanent bit, link local scope */
- 0x40, 0x1b, /* IPv4 signature */
- 0xff, 0xff, /* 16 bits of P_Key (to be filled in) */
- 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 48 bits of zeros */
- 0xff, 0xff, 0xff, 0xff, /* 32 bit IPv4 broadcast address */
- },
-};
+static ib_api_status_t
+track_mgrp_w_partition(osm_log_t *p_log, osm_prtn_t *p, osm_mgrp_t *mgrp,
+ osm_subn_t *p_subn, const ib_gid_t *mgid,
+ ib_net16_t pkey)
+{
+ char gid_str[INET6_ADDRSTRLEN];
+ osm_mgrp_t **tmp;
+ int i = 0;
+
+ /* check if we are already tracking this group */
+ for (i = 0; i< p->nmgrps; i++)
+ if (p->mgrps[i] == mgrp)
+ return (IB_SUCCESS);
+
+ /* otherwise add it to our list */
+ tmp = realloc(p->mgrps, (p->nmgrps +1) * sizeof(*p->mgrps));
+ if (tmp) {
+ p->mgrps = tmp;
+ p->mgrps[p->nmgrps] = mgrp;
+ p->nmgrps++;
+ } else {
+ OSM_LOG(p_log, OSM_LOG_ERROR,
+ "ERR: realloc error to create MC group (%s) in "
+ "partition (pkey 0x%04x)\n",
+ inet_ntop(AF_INET6, mgid->raw,
+ gid_str, sizeof gid_str),
+ cl_ntoh16(pkey));
+ mgrp->well_known = FALSE;
+ osm_mgrp_cleanup(p_subn, mgrp);
+ return (IB_ERROR);
+ }
+ mgrp->well_known = TRUE;
+ return (IB_SUCCESS);
+}
ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, osm_subn_t * p_subn,
osm_prtn_t * p, uint8_t rate, uint8_t mtu,
- uint8_t scope)
+ uint8_t sl, uint8_t scope, uint32_t Q_Key,
+ uint8_t tclass, const ib_gid_t *mgid)
{
+ char gid_str[INET6_ADDRSTRLEN];
ib_member_rec_t mc_rec;
ib_net64_t comp_mask;
ib_net16_t pkey;
@@ -185,16 +235,15 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, osm_subn_t * p_subn,
memset(&mc_rec, 0, sizeof(mc_rec));
- mc_rec.mgid = osm_ipoib_mgid; /* ipv4 broadcast group */
- memcpy(&mc_rec.mgid.raw[4], &pkey, sizeof(pkey));
+ mc_rec.mgid = *mgid;
- mc_rec.qkey = CL_HTON32(0x0b1b);
- mc_rec.mtu = (mtu ? mtu : OSM_DEFAULT_MGRP_MTU) | (2 << 6); /* 2048 Bytes */
- mc_rec.tclass = 0;
+ mc_rec.qkey = CL_HTON32(Q_Key);
+ mc_rec.mtu = mtu | (2 << 6); /* 2048 Bytes */
+ mc_rec.tclass = tclass;
mc_rec.pkey = pkey;
- mc_rec.rate = (rate ? rate : OSM_DEFAULT_MGRP_RATE) | (2 << 6); /* 10Gb/sec */
+ mc_rec.rate = rate | (2 << 6); /* 10Gb/sec */
mc_rec.pkt_life = p_subn->opt.subnet_timeout;
- mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(p->sl, 0, hop_limit);
+ mc_rec.sl_flow_hop = ib_member_set_sl_flow_hop(sl, 0, hop_limit);
/* Scope in MCMemberRecord (if present) needs to be consistent with MGID */
mc_rec.scope_state =
ib_member_set_scope_state(scope, IB_MC_REC_STATE_FULL_MEMBER);
@@ -206,15 +255,13 @@ ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, osm_subn_t * p_subn,
mgrp = osm_mcmr_rcv_find_or_create_new_mgrp(p_sa, comp_mask, &mc_rec);
if (!mgrp) {
OSM_LOG(p_log, OSM_LOG_ERROR,
- "Failed to create MC group with pkey 0x%04x\n",
+ "Failed to create MC group (%s) with pkey 0x%04x\n",
+ inet_ntop(AF_INET6, mgid->raw, gid_str, sizeof gid_str),
cl_ntoh16(pkey));
return IB_ERROR;
}
- mgrp->well_known = TRUE;
- p->mgrp = mgrp;
-
- return IB_SUCCESS;
+ return (track_mgrp_w_partition(p_log, p, mgrp, p_subn, mgid, pkey));
}
static uint16_t generate_pkey(osm_subn_t * p_subn)
@@ -274,7 +321,7 @@ osm_prtn_t *osm_prtn_make_new(osm_log_t * p_log, osm_subn_t * p_subn,
" definition: \'%s\' (0x%04x) prev name \'%s\'"
". Will use it\n",
name, cl_ntoh16(pkey), p_check->name);
- osm_prtn_delete(&p);
+ osm_prtn_delete(&p, p_subn);
p = p_check;
}
@@ -298,8 +345,12 @@ static ib_api_status_t prtn_make_default(osm_log_t * p_log, osm_subn_t * p_subn,
status =
osm_prtn_add_port(p_log, p_subn, p, p_subn->sm_port_guid, TRUE);
+ /* ipv4 broadcast group */
if (no_config)
- osm_prtn_add_mcgroup(p_log, p_subn, p, 0, 0, 0);
+ osm_prtn_add_mcgroup(p_log, p_subn, p, OSM_DEFAULT_MGRP_RATE,
+ OSM_DEFAULT_MGRP_MTU, OSM_DEFAULT_SL,
+ 0, OSM_IPOIB_BROADCAST_MGRP_QKEY, 0,
+ &osm_ipoib_broadcast_mgid);
_err:
return status;
@@ -316,8 +367,12 @@ ib_api_status_t osm_prtn_make_partitions(osm_log_t * p_log, osm_subn_t * p_subn)
file_name = p_subn->opt.partition_config_file ?
p_subn->opt.partition_config_file : OSM_DEFAULT_PARTITION_CONFIG_FILE;
- if (stat(file_name, &statbuf))
+ if (stat(file_name, &statbuf)) {
+ OSM_LOG(p_log, OSM_LOG_ERROR, "Partition configuration "
+ "%s is not accesible (%s)\n", file_name,
+ strerror(errno));
is_config = FALSE;
+ }
/* clean up current port maps */
p_next = cl_qmap_head(&p_subn->prtn_pkey_tbl);
@@ -347,7 +402,7 @@ ib_api_status_t osm_prtn_make_partitions(osm_log_t * p_log, osm_subn_t * p_subn)
cl_map_count(&p->full_guid_tbl) == 0) {
cl_qmap_remove_item(&p_subn->prtn_pkey_tbl,
(cl_map_item_t *) p);
- osm_prtn_delete(&p);
+ osm_prtn_delete(&p, p_subn);
}
}
diff --git a/opensm/osm_prtn_config.c b/opensm/osm_prtn_config.c
index 0d02597..0954c8d 100644
--- a/opensm/osm_prtn_config.c
+++ b/opensm/osm_prtn_config.c
@@ -51,7 +51,25 @@
#include <opensm/osm_partition.h>
#include <opensm/osm_subnet.h>
#include <opensm/osm_log.h>
+#include <arpa/inet.h>
+
+const ib_gid_t osm_ipoib_broadcast_mgid = {
+ {
+ 0xff, /* multicast field */
+ 0x12, /* non-permanent bit, link local scope */
+ 0x40, 0x1b, /* IPv4 signature */
+ 0xff, 0xff, /* 16 bits of P_Key (to be filled in) */
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 48 bits of zeros */
+ 0xff, 0xff, 0xff, 0xff, /* 32 bit IPv4 broadcast address */
+ },
+};
+struct precreate_mgroup {
+ ib_gid_t mgid;
+ unsigned mtu, rate, sl, scope_mask;
+ uint32_t Q_Key;
+ uint8_t TClass;
+};
struct part_conf {
osm_log_t *p_log;
osm_subn_t *p_subn;
@@ -68,16 +86,104 @@ extern ib_api_status_t osm_prtn_add_all(osm_log_t * p_log, osm_subn_t * p_subn,
extern ib_api_status_t osm_prtn_add_port(osm_log_t * p_log,
osm_subn_t * p_subn, osm_prtn_t * p,
ib_net64_t guid, boolean_t full);
-extern ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log,
- osm_subn_t * p_subn, osm_prtn_t * p,
- uint8_t rate,
- uint8_t mtu, uint8_t scope);
+
+ib_api_status_t osm_prtn_add_mcgroup(osm_log_t * p_log, osm_subn_t * p_subn,
+ osm_prtn_t * p, uint8_t rate, uint8_t mtu,
+ uint8_t sl, uint8_t scope, uint32_t Q_Key,
+ uint8_t TClass, const ib_gid_t *mgid);
+
+
+static inline int mgid_is_broadcast(const ib_gid_t *mgid)
+{
+ return (memcmp(mgid, &osm_ipoib_broadcast_mgid,
+ sizeof(osm_ipoib_broadcast_mgid)) == 0);
+}
+
+static inline int mgid_is_ip(const ib_gid_t *mgid)
+{
+ ib_net16_t ipsig = *(ib_net16_t *)&mgid->raw[2];
+ return (ipsig == cl_hton16(0x401b) || ipsig == cl_hton16(0x601b));
+}
+
+static inline int ip_mgroup_pkey_ok(struct part_conf *conf,
+ struct precreate_mgroup *group)
+{
+ ib_net16_t mpkey = *(ib_net16_t *)&group->mgid.raw[4];
+ char gid_str[INET6_ADDRSTRLEN];
+
+ if (mgid_is_broadcast(&group->mgid)
+ || mpkey == 0x0000 /* user requested "wild card" of pkey */
+ || mpkey == conf->p_prtn->pkey) /* user was smart enough to match */
+ return (1);
+
+ OSM_LOG(conf->p_log, OSM_LOG_ERROR,
+ "ERR: IP MC group (%s) specified with invalid pkey 0x%04x "
+ "for partition pkey = 0x%04x (%s)\n",
+ inet_ntop(AF_INET6, group->mgid.raw, gid_str, sizeof gid_str),
+ cl_ntoh16(mpkey), cl_ntoh16(conf->p_prtn->pkey), conf->p_prtn->name);
+ return (0);
+}
+
+static inline int ip_mgroup_rate_ok(struct part_conf *conf,
+ struct precreate_mgroup *group)
+{
+ char gid_str[INET6_ADDRSTRLEN];
+
+ if (group->rate == conf->rate)
+ return (1);
+
+ OSM_LOG(conf->p_log, OSM_LOG_ERROR,
+ "ERR: IP MC group (%s) specified with invalid rate (%d): "
+ "partition pkey = 0x%04x (%s) "
+ "[Partition broadcast group rate = %d]\n",
+ inet_ntop(AF_INET6, group->mgid.raw, gid_str, sizeof gid_str),
+ group->rate, cl_ntoh16(conf->p_prtn->pkey), conf->p_prtn->name, conf->rate);
+ return (0);
+}
+
+static inline int ip_mgroup_mtu_ok(struct part_conf *conf,
+ struct precreate_mgroup *group)
+{
+ char gid_str[INET6_ADDRSTRLEN];
+
+ if (group->mtu == conf->mtu)
+ return (1);
+
+ OSM_LOG(conf->p_log, OSM_LOG_ERROR,
+ "ERR: IP MC group (%s) specified with invalid mtu (%d): "
+ "partition pkey = 0x%04x (%s) "
+ "[Partition broadcast group mtu = %d]\n",
+ inet_ntop(AF_INET6, group->mgid.raw, gid_str, sizeof gid_str),
+ group->mtu, cl_ntoh16(conf->p_prtn->pkey), conf->p_prtn->name, conf->mtu);
+ return (0);
+}
+
+static void __create_mgrp(struct part_conf *conf, struct precreate_mgroup *group)
+{
+ unsigned int scope;
+
+ if (!group->scope_mask) {
+ osm_prtn_add_mcgroup(conf->p_log, conf->p_subn, conf->p_prtn,
+ (uint8_t) group->rate, (uint8_t) group->mtu,
+ group->sl, 0, group->Q_Key,
+ group->TClass, &group->mgid);
+ } else {
+ for (scope = 0; scope < 16; scope++) {
+ if (((1<<scope) & group->scope_mask) == 0)
+ continue;
+
+ osm_prtn_add_mcgroup(conf->p_log, conf->p_subn, conf->p_prtn,
+ (uint8_t)group->rate, (uint8_t)group->mtu,
+ (uint8_t)group->sl, (uint8_t)scope, group->Q_Key,
+ group->TClass, &group->mgid);
+ }
+ }
+}
static int partition_create(unsigned lineno, struct part_conf *conf,
char *name, char *id, char *flag, char *flag_val)
{
uint16_t pkey;
- unsigned int scope;
if (!id && name && isdigit(*name)) {
id = name;
@@ -107,26 +213,18 @@ static int partition_create(unsigned lineno, struct part_conf *conf,
}
conf->p_prtn->sl = (uint8_t) conf->sl;
- if (!conf->is_ipoib)
- return 0;
-
- if (!conf->scope_mask) {
- osm_prtn_add_mcgroup(conf->p_log, conf->p_subn, conf->p_prtn,
- (uint8_t) conf->rate,
- (uint8_t) conf->mtu,
- 0);
- return 0;
+ if (conf->is_ipoib) {
+ struct precreate_mgroup broadcast_mgroup;
+ memset(&broadcast_mgroup, 0, sizeof(broadcast_mgroup));
+ broadcast_mgroup.mgid = osm_ipoib_broadcast_mgid;
+ broadcast_mgroup.Q_Key = OSM_IPOIB_BROADCAST_MGRP_QKEY;
+ broadcast_mgroup.mtu = conf->mtu;
+ broadcast_mgroup.rate = conf->rate;
+ broadcast_mgroup.sl = conf->sl;
+ broadcast_mgroup.scope_mask = conf->scope_mask;
+ __create_mgrp(conf, &broadcast_mgroup);
}
- for (scope = 0; scope < 16; scope++) {
- if (((1<<scope) & conf->scope_mask) == 0)
- continue;
-
- osm_prtn_add_mcgroup(conf->p_log, conf->p_subn, conf->p_prtn,
- (uint8_t) conf->rate,
- (uint8_t) conf->mtu,
- (uint8_t) scope);
- }
return 0;
}
@@ -288,6 +386,129 @@ static int parse_name_token(char *str, char **name, char **val)
return len;
}
+static int parse_mgroup_flags(osm_log_t * p_log,
+ struct precreate_mgroup *mgroup,
+ char *p, unsigned lineno)
+{
+ int ret, len = 0;
+ char *flag, *val, *q;
+ do {
+ flag = val = NULL;
+ q = strchr(p, ',');
+ if (q)
+ *q++ = '\0';
+
+ ret = parse_name_token(p, &flag, &val);
+
+ if (!strncmp(flag, "mtu", strlen(flag))) {
+ if (!val || (mgroup->mtu = strtoul(val, NULL, 0)) == 0)
+ OSM_LOG(p_log, OSM_LOG_VERBOSE,
+ "PARSE WARN: line %d: "
+ "flag \'mtu\' requires valid value"
+ " - using default\n", lineno);
+ } else if (!strncmp(flag, "rate", strlen(flag))) {
+ if (!val || (mgroup->rate = strtoul(val, NULL, 0)) == 0)
+ OSM_LOG(p_log, OSM_LOG_VERBOSE,
+ "PARSE WARN: line %d: "
+ "flag \'rate\' requires valid value"
+ " - using default\n", lineno);
+ } else if (!strncmp(flag, "scope", strlen(flag))) {
+ unsigned int scope;
+ if (!val || (scope = strtoul(val, NULL, 0)) == 0 || scope > 0xF)
+ OSM_LOG(p_log, OSM_LOG_VERBOSE,
+ "PARSE WARN: line %d: "
+ "flag \'scope\' requires valid value"
+ " - using default\n", lineno);
+ else
+ mgroup->scope_mask |= (1<<scope);
+ } else if (!strncmp(flag, "Q_Key", strlen(flag))) {
+ if (!val || (mgroup->Q_Key = strtoul(val, NULL, 0)) == 0)
+ OSM_LOG(p_log, OSM_LOG_VERBOSE,
+ "PARSE WARN: line %d: "
+ "flag \'Q_Key\' requires valid value"
+ " - using '0'\n", lineno);
+ } else if (!strncmp(flag, "TClass", strlen(flag))) {
+ if (!val || (mgroup->TClass = strtoul(val, NULL, 0)) == 0)
+ OSM_LOG(p_log, OSM_LOG_VERBOSE,
+ "PARSE WARN: line %d: "
+ "flag \'TClass\' requires valid value"
+ " - using '0'\n", lineno);
+ } else if (!strncmp(flag, "sl", strlen(flag))) {
+ unsigned sl;
+ char *end;
+
+ if (!val || !*val || (sl = strtoul(val, &end, 0)) > 15 ||
+ (*end && !isspace(*end)))
+ OSM_LOG(p_log, OSM_LOG_VERBOSE,
+ "PARSE WARN: line %d: "
+ "flag \'sl\' requires valid value"
+ " - using '0'\n", lineno);
+ else
+ mgroup->sl = sl;
+ } else {
+ OSM_LOG(p_log, OSM_LOG_VERBOSE,
+ "PARSE WARN: line %d: "
+ "unrecognized mgroup flag \'%s\'"
+ " - ignored\n", lineno, flag);
+ }
+ p += ret;
+ len += ret;
+ } while (q);
+
+ return (len);
+}
+
+static int mgroup_create(char *p, char *mgid, unsigned lineno, struct part_conf *conf)
+{
+ int ret = 0;
+ struct precreate_mgroup mgroup;
+
+ memset(&mgroup, 0, sizeof(mgroup));
+
+ if (inet_pton(AF_INET6, mgid, &mgroup.mgid) != 1
+ || mgroup.mgid.raw[0] != 0xff) {
+ OSM_LOG(conf->p_log, OSM_LOG_ERROR,
+ "PARSE ERROR partition conf file line %d: mgid \"%s\": format "
+ "is invalid\n", lineno, mgid);
+ return 0;
+ }
+
+ /* inherit partition flags */
+ mgroup.mtu = conf->mtu;
+ mgroup.rate = conf->rate;
+ mgroup.sl = conf->sl;
+ mgroup.scope_mask = conf->scope_mask;
+
+ /* override with user specified flags */
+ ret = parse_mgroup_flags(conf->p_log, &mgroup, p, lineno);
+
+ /* check/verify special IP group parameters */
+ if (mgid_is_ip(&mgroup.mgid)) {
+ ib_net16_t pkey = conf->p_prtn->pkey | cl_hton16(0x8000);
+
+ if (!ip_mgroup_pkey_ok(conf, &mgroup)
+ || !ip_mgroup_rate_ok(conf, &mgroup)
+ || !ip_mgroup_mtu_ok(conf, &mgroup))
+ goto error;
+
+ /* set special IP settings */
+ memcpy(&mgroup.mgid.raw[4], &pkey, sizeof(pkey));
+
+ if (mgroup.Q_Key != OSM_IPOIB_BROADCAST_MGRP_QKEY)
+ mgroup.Q_Key = OSM_IPOIB_BROADCAST_MGRP_QKEY;
+ }
+
+ /* don't create multiple copies of the group */
+ if (osm_get_mgrp_by_mgid(conf->p_subn, &mgroup.mgid))
+ goto error;
+
+ /* create the group */
+ __create_mgrp(conf, &mgroup);
+
+error:
+ return ret;
+}
+
static struct part_conf *new_part_conf(osm_log_t * p_log, osm_subn_t * p_subn)
{
static struct part_conf part;
@@ -299,6 +520,8 @@ static struct part_conf *new_part_conf(osm_log_t * p_log, osm_subn_t * p_subn)
conf->p_prtn = NULL;
conf->is_ipoib = 0;
conf->sl = OSM_DEFAULT_SL;
+ conf->rate = OSM_DEFAULT_MGRP_RATE;
+ conf->mtu = OSM_DEFAULT_MGRP_MTU;
conf->full = FALSE;
return conf;
}
@@ -382,6 +605,12 @@ skip_header:
if (q)
*q++ = '\0';
ret = parse_name_token(p, &name, &flag);
+
+ if (strcmp(name, "mgid") == 0) {
+ /* parse an mgid line if specified. */
+ len += mgroup_create(p+ret, flag, lineno, conf);
+ goto done; /* We're done: this consumes the line */
+ }
if (partition_add_port(lineno, conf, name, flag) < 0) {
OSM_LOG(conf->p_log, OSM_LOG_ERROR,
"PARSE ERROR: line %d: "
@@ -394,13 +623,14 @@ skip_header:
len += ret;
} while (q);
+done:
return len;
}
int osm_prtn_config_parse_file(osm_log_t * p_log, osm_subn_t * p_subn,
const char *file_name)
{
- char line[1024];
+ char line[4096];
struct part_conf *conf = NULL;
FILE *file;
int lineno;
diff --git a/opensm/osm_qos_policy.c b/opensm/osm_qos_policy.c
index 3780968..9f8697e 100644
--- a/opensm/osm_qos_policy.c
+++ b/opensm/osm_qos_policy.c
@@ -770,9 +770,7 @@ static void __qos_policy_validate_pkey(
osm_qos_match_rule_t * p_qos_match_rule,
osm_prtn_t * p_prtn)
{
- uint8_t sl;
- uint32_t flow;
- uint8_t hop;
+ int i = 0;
if (!p_qos_policy || !p_qos_match_rule || !p_prtn)
return;
@@ -790,26 +788,30 @@ static void __qos_policy_validate_pkey(
p_prtn->sl = p_qos_match_rule->p_qos_level->sl;
- /* If this partition is an IPoIB partition, there should
- be a matching MCast group. Fix this group's SL too */
- if (!p_prtn->mgrp)
- return;
-
- CL_ASSERT((cl_ntoh16(p_prtn->mgrp->mcmember_rec.pkey) & 0x7fff) ==
- (cl_ntoh16(p_prtn->pkey) & 0x7fff));
-
- ib_member_get_sl_flow_hop(p_prtn->mgrp->mcmember_rec.sl_flow_hop,
- &sl, &flow, &hop);
- if (sl != p_prtn->sl) {
- char gid_str[INET6_ADDRSTRLEN];
- OSM_LOG(&p_qos_policy->p_subn->p_osm->log, OSM_LOG_DEBUG,
- "Updating MCGroup (MGID %s) SL to "
- "match partition SL (%u)\n",
- inet_ntop(AF_INET6, p_prtn->mgrp->mcmember_rec.mgid.raw,
- gid_str, sizeof gid_str),
- p_prtn->sl);
- p_prtn->mgrp->mcmember_rec.sl_flow_hop =
- ib_member_set_sl_flow_hop(p_prtn->sl, flow, hop);
+ /* If this partition has default MCast groups.
+ * Fix those group's SL too */
+ for (i = 0; i < p_prtn->nmgrps; i++)
+ {
+ uint8_t sl;
+ uint32_t flow;
+ uint8_t hop;
+
+ CL_ASSERT((cl_ntoh16(p_prtn->mgrps[i]->mcmember_rec.pkey) & 0x7fff) ==
+ (cl_ntoh16(p_prtn->pkey) & 0x7fff));
+
+ ib_member_get_sl_flow_hop(p_prtn->mgrps[i]->mcmember_rec.sl_flow_hop,
+ &sl, &flow, &hop);
+ if (sl != p_prtn->sl) {
+ char gid_str[INET6_ADDRSTRLEN];
+ OSM_LOG(&p_qos_policy->p_subn->p_osm->log, OSM_LOG_DEBUG,
+ "Updating MCGroup (MGID %s) SL to "
+ "match partition SL (%u)\n",
+ inet_ntop(AF_INET6, p_prtn->mgrps[i]->mcmember_rec.mgid.raw,
+ gid_str, sizeof gid_str),
+ p_prtn->sl);
+ p_prtn->mgrps[i]->mcmember_rec.sl_flow_hop =
+ ib_member_set_sl_flow_hop(p_prtn->sl, flow, hop);
+ }
}
}
diff --git a/opensm/osm_subnet.c b/opensm/osm_subnet.c
index 8d636bc..aaee7bf 100644
--- a/opensm/osm_subnet.c
+++ b/opensm/osm_subnet.c
@@ -492,7 +492,7 @@ void osm_subn_destroy(IN osm_subn_t * p_subn)
(osm_prtn_t *) cl_qmap_end(&p_subn->prtn_pkey_tbl)) {
p_prtn = p_next_prtn;
p_next_prtn = (osm_prtn_t *) cl_qmap_next(&p_prtn->map_item);
- osm_prtn_delete(&p_prtn);
+ osm_prtn_delete(&p_prtn, p_subn);
}
cl_fmap_remove_all(&p_subn->mgrp_mgid_tbl);
--
1.7.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] opensm: Add the precreation of multicast groups
[not found] ` <20111006171326.3b143b03.weiny2-i2BcT+NCU+M@public.gmane.org>
@ 2011-10-27 19:41 ` Hal Rosenstock
[not found] ` <4EA9B401.5030101-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
0 siblings, 1 reply; 4+ messages in thread
From: Hal Rosenstock @ 2011-10-27 19:41 UTC (permalink / raw)
To: Ira Weiny; +Cc: Alex Netes, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
On 10/6/2011 8:13 PM, Ira Weiny wrote:
>
> Allow for the pre-creation of these groups on a partition by partition basis.
This looks good to me and has been needed for some time now; Thanks for
doing this!
Just some minor comments below from scanning through the patch. I
haven't tried it yet...
> P_Key is taken from the partition specification. Q_Key, TClass, rate, and mtu
> can be specified.
If TClass is added, should FlowLabel also be added ?
> For IP groups, rate and mtu are verified to match the broadcast groups
> parameters. P_Key can be specified in the mgid itself and is verified to match
> the P_Key of the partition if != 0x0000. If pkey == 0x0000 then the P_Key is
> taken from the partition specification.
Do you mean the pkey in the IPoIB MGID here by "partition specification" ?
> The syntax extends the existing syntax by allowing MC groups to be specified
> one per line, intermixed with the port specifications.
>
> Signed-off-by: Ira Weiny <weiny2-i2BcT+NCU+M@public.gmane.org>
> ---
> include/opensm/osm_base.h | 5 +
> include/opensm/osm_partition.h | 12 +-
> man/opensm.8.in | 121 +++++++++++------
Similar changes should be made to doc/partition-config.txt which is
where this part of the man page came from originally.
> opensm/osm_prtn.c | 111 ++++++++++++----
> opensm/osm_prtn_config.c | 278 ++++++++++++++++++++++++++++++++++++----
> opensm/osm_qos_policy.c | 48 ++++----
> opensm/osm_subnet.c | 2 +-
> 7 files changed, 454 insertions(+), 123 deletions(-)
>
> diff --git a/include/opensm/osm_base.h b/include/opensm/osm_base.h
> index e558c55..869ba9c 100644
> --- a/include/opensm/osm_base.h
> +++ b/include/opensm/osm_base.h
> @@ -53,6 +53,7 @@
> #endif
>
> #include <complib/cl_types.h>
> +#include <iba/ib_types.h>
>
> #ifdef __cplusplus
> # define BEGIN_C_DECLS extern "C" {
> @@ -954,6 +955,10 @@ typedef enum _osm_sm_signal {
> #define OSM_VENDOR_ID_HP4 0x00237D
> #define OSM_VENDOR_ID_OPENIB 0x001405
>
> +/* IPoIB Broadcast Defaults */
> +#define OSM_IPOIB_BROADCAST_MGRP_QKEY 0x0b1b
> +extern const ib_gid_t osm_ipoib_broadcast_mgid;
> +
> /**********/
>
> END_C_DECLS
> diff --git a/include/opensm/osm_partition.h b/include/opensm/osm_partition.h
> index fdb34b9..7277eac 100644
> --- a/include/opensm/osm_partition.h
> +++ b/include/opensm/osm_partition.h
> @@ -94,10 +94,11 @@ typedef struct osm_prtn {
> cl_map_item_t map_item;
> ib_net16_t pkey;
> uint8_t sl;
> - osm_mgrp_t *mgrp;
> cl_map_t full_guid_tbl;
> cl_map_t part_guid_tbl;
> char name[32];
> + osm_mgrp_t **mgrps;
> + int nmgrps;
> } osm_prtn_t;
> /*
> * FIELDS
> @@ -110,9 +111,10 @@ typedef struct osm_prtn {
> * sl
> * The Service Level (SL) associated with this Partiton.
> *
> -* mgrp
> -* The pointer to the well known Multicast Group
> -* that was created for this partition (when configured).
> +* mgrps
> +* List of well known Multicast Groups
> +* that were created for this partition (when configured).
> +* This includes the IPoIB broadcast group.
> *
> * full_guid_tbl
> * Container of pointers to all Port objects in the Partition
> @@ -139,7 +141,7 @@ typedef struct osm_prtn {
> *
> * SYNOPSIS
> */
> -void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn);
> +void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn, osm_subn_t * p_subn);
IN osm_subn_t * p_subn
Also, reverse the parameters to be consistent with the coding convention.
> /*
> * PARAMETERS
> * pp_prtn
> diff --git a/man/opensm.8.in b/man/opensm.8.in
> index 042bee3..da0c247 100644
> --- a/man/opensm.8.in
> +++ b/man/opensm.8.in
> @@ -523,45 +523,76 @@ parser.
>
> General file format:
>
> -<Partition Definition>:<PortGUIDs list> ;
> -
> -Partition Definition:
> -
> -[PartitionName][=PKey][,flag[=value]][,defmember=full|limited]
> -
> - PartitionName - string, will be used with logging. When omitted
> - empty string will be used.
> - PKey - P_Key value for this partition. Only low 15 bits will
> - be used. When omitted will be autogenerated.
> - flag - used to indicate IPoIB capability of this partition.
> - defmember=full|limited - specifies default membership for port guid
> - list. Default is limited.
> -
> -Currently recognized flags are:
> -
> - ipoib - indicates that this partition may be used for IPoIB, as
> - result IPoIB capable MC group will be created.
> - rate=<val> - specifies rate for this IPoIB MC group
> - (default is 3 (10GBps))
> - mtu=<val> - specifies MTU for this IPoIB MC group
> - (default is 4 (2048))
> - sl=<val> - specifies SL for this IPoIB MC group
> - (default is 0)
> - scope=<val> - specifies scope for this IPoIB MC group
> - (default is 2 (link local)). Multiple scope settings
> - are permitted for a partition.
> -
> -Note that values for rate, mtu, and scope should be specified as
> -defined in the IBTA specification (for example, mtu=4 for 2048).
> -
> -PortGUIDs list:
> -
> - PortGUID - GUID of partition member EndPort. Hexadecimal
> - numbers should start from 0x, decimal numbers
> - are accepted too.
> - full or limited - indicates full or limited membership for this
> - port. When omitted (or unrecognized) limited
> - membership is assumed.
> +<Partition Definition>:[<newline>]<Partition Properties>;
> +
> + Partition Definition:
> + [PartitionName][=PKey][,part_flag[=value]][,defmember=full|limited]
> +
> + PartitionName - string, will be used with logging. When omitted
> + empty string will be used.
> + PKey - P_Key value for this partition. Only low 15 bits will
> + be used. When omitted will be autogenerated.
> + part_flags - used to indicate/specify IPoIB capability of this partition.
> + defmember=full|limited - specifies default membership for port guid
> + list. Default is limited.
> +
> + part_flag:
> + ipoib - indicates that this partition may be used for IPoIB, as
> + result IPoIB capable MC group will be created.
> + rate=<val> - specifies rate for this IPoIB MC group
> + (default is 3 (10GBps))
> + mtu=<val> - specifies MTU for this IPoIB MC group
> + (default is 4 (2048))
> + sl=<val> - specifies SL for this IPoIB MC group
> + (default is 0)
> + scope=<val> - specifies scope for this IPoIB MC group
> + (default is 2 (link local)). Multiple scope settings
> + are permitted for a partition.
> +
> + Partition Properties:
> + [<Port list>|<MCast Group>]* | <Port list>
> +
> + Port list:
> + <Port Specifier>[,<Port Specifier>]
> +
> + Port Specifier:
> + <PortGUID>[=[full|limited]]
> +
> + PortGUID - GUID of partition member EndPort. Hexadecimal
> + numbers should start from 0x, decimal numbers
> + are accepted too.
> + full or limited - indicates full or limited membership for this
> + port. When omitted (or unrecognized) limited
> + membership is assumed.
> +
> + MCast Group:
> + mgid=gid[,mgroup_flag=val]*<newline>
> +
> + mgroup_flag:
> + rate=<val> - specifies rate for this MC group
> + (default is 3 (10GBps))
> + mtu=<val> - specifies MTU for this MC group
> + (default is 4 (2048))
> + sl=<val> - specifies SL for this MC group
> + (default is 0)
> + scope=<val> - specifies scope for this MC group
> + (default is 2 (link local)). Multiple scope settings
> + are permitted for a partition.
> + NOTE: This overwrites the scope nibble of the specified
> + mgid. Furthermore specifying multiple scope
> + settings will result in multiple MC groups
> + being created.
> + qkey=<val> - specifies the Q_Key for this MC group
> + (default: 0x0b1b for IP groups, 0 for other groups)
> + tclass=<val> - specifies tclass for this MC group
> + (default is 0)
> +
> + newline: '\n'
> +
> +
> +Note that values for rate, mtu, and scope, for both partitions and multicast
> +groups, should be specified as defined in the IBTA specification (for example,
> +mtu=4 for 2048).
>
> There are several useful keywords for PortGUID definition:
>
> @@ -577,9 +608,6 @@ Notes:
>
> White space is permitted between delimiters ('=', ',',':',';').
>
> -The line can be wrapped after ':' followed after Partition Definition and
> -between.
> -
> PartitionName does not need to be unique, PKey does need to be unique.
> If PKey is repeated then those partition configurations will be merged
> and first PartitionName will be used (see also next note).
> @@ -606,6 +634,15 @@ Examples:
> ShareIO = 0x80 , defmember=full : 0x123459, 0x12345a;
> ShareIO = 0x80 , defmember=full : 0x12345b, 0x12345c=limited, 0x12345d;
>
> + # multicast groups added to default
> + Default=0x7fff,ipoib:
> + mgid=ff12:401b::0707,sl=1 # random IPv4 group
> + mgid=ff12:601b::16 # MLDv2-capable routers
> + mgid=ff12:401b::16 # IGMP
> + mgid=ff12:601b::2 # All routers
> + mgid=ff12::1,sl=1,Q_Key=0xDEADBEEF,rate=3,mtu=2 # random group
> + ALL=full;
> +
>
> Note:
>
> diff --git a/opensm/osm_prtn.c b/opensm/osm_prtn.c
> index 3fd4fc0..2e2837a 100644
> --- a/opensm/osm_prtn.c
> +++ b/opensm/osm_prtn.c
> @@ -53,6 +53,8 @@
> #include <opensm/osm_node.h>
> #include <opensm/osm_sa.h>
> #include <opensm/osm_multicast.h>
> +#include <arpa/inet.h>
> +#include <errno.h>
>
> extern int osm_prtn_config_parse_file(osm_log_t * p_log, osm_subn_t * p_subn,
> const char *file_name);
> @@ -68,6 +70,8 @@ osm_prtn_t *osm_prtn_new(IN const char *name, IN uint16_t pkey)
> memset(p, 0, sizeof(*p));
> p->pkey = pkey;
> p->sl = OSM_DEFAULT_SL;
> + p->mgrps = NULL;
> + p->nmgrps = 0;
> cl_map_construct(&p->full_guid_tbl);
> cl_map_init(&p->full_guid_tbl, 32);
> cl_map_construct(&p->part_guid_tbl);
> @@ -81,14 +85,34 @@ osm_prtn_t *osm_prtn_new(IN const char *name, IN uint16_t pkey)
> return p;
> }
>
> -void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn)
> +void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn, osm_subn_t * p_subn)
> {
> + char gid_str[INET6_ADDRSTRLEN];
> + int i = 0;
> osm_prtn_t *p = *pp_prtn;
>
> cl_map_remove_all(&p->full_guid_tbl);
> cl_map_destroy(&p->full_guid_tbl);
> cl_map_remove_all(&p->part_guid_tbl);
> cl_map_destroy(&p->part_guid_tbl);
> +
> + if (p->mgrps) {
> + /* Clean up mgrps */
> + for (i = 0; i < p->nmgrps; i++) {
> + /* osm_mgrp_cleanup will not delete
> + * "well_known" groups */
> + p->mgrps[i]->well_known = FALSE;
> + osm_mgrp_cleanup(p_subn, p->mgrps[i]);
> + OSM_LOG(&p_subn->p_osm->log, OSM_LOG_ERROR,
Should this some other log level like VERBOSE or DEBUG ?
> + "removing mgroup %s from partition (0x%x)\n",
> + inet_ntop(AF_INET6, p->mgrps[i]->mcmember_rec.mgid.raw,
> + gid_str, sizeof gid_str),
> + cl_hton16(p->pkey));
> + }
> +
> + free(p->mgrps);
> + }
> +
> free(p);
> *pp_prtn = NULL;
> }
> @@ -156,21 +180,47 @@ _err:
> return status;
> }
>
> -static const ib_gid_t osm_ipoib_mgid = {
> - {
> - 0xff, /* multicast field */
> - 0x12, /* non-permanent bit, link local scope */
> - 0x40, 0x1b, /* IPv4 signature */
> - 0xff, 0xff, /* 16 bits of P_Key (to be filled in) */
> - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 48 bits of zeros */
> - 0xff, 0xff, 0xff, 0xff, /* 32 bit IPv4 broadcast address */
> - },
> -};
> +static ib_api_status_t
> +track_mgrp_w_partition(osm_log_t *p_log, osm_prtn_t *p, osm_mgrp_t *mgrp,
> + osm_subn_t *p_subn, const ib_gid_t *mgid,
> + ib_net16_t pkey)
> +{
> + char gid_str[INET6_ADDRSTRLEN];
> + osm_mgrp_t **tmp;
> + int i = 0;
> +
> + /* check if we are already tracking this group */
> + for (i = 0; i< p->nmgrps; i++)
> + if (p->mgrps[i] == mgrp)
> + return (IB_SUCCESS);
> +
> + /* otherwise add it to our list */
> + tmp = realloc(p->mgrps, (p->nmgrps +1) * sizeof(*p->mgrps));
> + if (tmp) {
> + p->mgrps = tmp;
> + p->mgrps[p->nmgrps] = mgrp;
> + p->nmgrps++;
> + } else {
> + OSM_LOG(p_log, OSM_LOG_ERROR,
> + "ERR: realloc error to create MC group (%s) in "
Other similar errors in this file don't have the "ERR: " part. There are
also other instances below but I snipped the rest.
-- Hal
<snip...>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] opensm: Add the precreation of multicast groups
[not found] ` <4EA9B401.5030101-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
@ 2011-10-31 20:12 ` Ira Weiny
[not found] ` <20111031131218.92b27d9f.weiny2-i2BcT+NCU+M@public.gmane.org>
0 siblings, 1 reply; 4+ messages in thread
From: Ira Weiny @ 2011-10-31 20:12 UTC (permalink / raw)
To: Hal Rosenstock
Cc: Alex Netes, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
On Thu, 27 Oct 2011 12:41:53 -0700
Hal Rosenstock <hal-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
> On 10/6/2011 8:13 PM, Ira Weiny wrote:
> >
> > Allow for the pre-creation of these groups on a partition by partition basis.
>
> This looks good to me and has been needed for some time now; Thanks for
> doing this!
>
> Just some minor comments below from scanning through the patch. I
> haven't tried it yet...
>
> > P_Key is taken from the partition specification. Q_Key, TClass, rate, and mtu
> > can be specified.
>
> If TClass is added, should FlowLabel also be added ?
Yes, added in next version. Also added Q_Key, tclass, and FlowLabel to BC
group config which uses common parsing code.
>
> > For IP groups, rate and mtu are verified to match the broadcast groups
> > parameters. P_Key can be specified in the mgid itself and is verified to match
> > the P_Key of the partition if != 0x0000. If pkey == 0x0000 then the P_Key is
> > taken from the partition specification.
>
> Do you mean the pkey in the IPoIB MGID here by "partition specification" ?
Yes, if the MGID has 0x0000 in the P_Key bits the partition manager fills in
those bits with the P_Key specified for that partition.
For example:
Default=0x7fff,ipoib:
mgid=ff12:401b::0707,sl=1 # random IPv4 group
ALL=full;
results in an MGID of ff12:401b:ffff::0707
Default=0x7fff,ipoib:
mgid=ff12:401b:f123::0707,sl=1 # random IPv4 group
ALL=full;
results in an error 0xf123 != 0xffff
>
> > The syntax extends the existing syntax by allowing MC groups to be specified
> > one per line, intermixed with the port specifications.
> >
> > Signed-off-by: Ira Weiny <weiny2-i2BcT+NCU+M@public.gmane.org>
> > ---
> > include/opensm/osm_base.h | 5 +
> > include/opensm/osm_partition.h | 12 +-
> > man/opensm.8.in | 121 +++++++++++------
>
> Similar changes should be made to doc/partition-config.txt which is
> where this part of the man page came from originally.
Alex, would you accept a patch which removes that file? It seems the man page
is more appropriate for this documentation. Even having a separate referenced
man page for partitions would be better.
For now I have modified doc/partition-config.txt to match the man page in the
patch.
>
> > opensm/osm_prtn.c | 111 ++++++++++++----
> > opensm/osm_prtn_config.c | 278 ++++++++++++++++++++++++++++++++++++----
> > opensm/osm_qos_policy.c | 48 ++++----
> > opensm/osm_subnet.c | 2 +-
> > 7 files changed, 454 insertions(+), 123 deletions(-)
> >
> > diff --git a/include/opensm/osm_base.h b/include/opensm/osm_base.h
> > index e558c55..869ba9c 100644
> > --- a/include/opensm/osm_base.h
> > +++ b/include/opensm/osm_base.h
> > @@ -53,6 +53,7 @@
> > #endif
> >
> > #include <complib/cl_types.h>
> > +#include <iba/ib_types.h>
> >
> > #ifdef __cplusplus
> > # define BEGIN_C_DECLS extern "C" {
> > @@ -954,6 +955,10 @@ typedef enum _osm_sm_signal {
> > #define OSM_VENDOR_ID_HP4 0x00237D
> > #define OSM_VENDOR_ID_OPENIB 0x001405
> >
> > +/* IPoIB Broadcast Defaults */
> > +#define OSM_IPOIB_BROADCAST_MGRP_QKEY 0x0b1b
> > +extern const ib_gid_t osm_ipoib_broadcast_mgid;
> > +
> > /**********/
> >
> > END_C_DECLS
> > diff --git a/include/opensm/osm_partition.h b/include/opensm/osm_partition.h
> > index fdb34b9..7277eac 100644
> > --- a/include/opensm/osm_partition.h
> > +++ b/include/opensm/osm_partition.h
> > @@ -94,10 +94,11 @@ typedef struct osm_prtn {
> > cl_map_item_t map_item;
> > ib_net16_t pkey;
> > uint8_t sl;
> > - osm_mgrp_t *mgrp;
> > cl_map_t full_guid_tbl;
> > cl_map_t part_guid_tbl;
> > char name[32];
> > + osm_mgrp_t **mgrps;
> > + int nmgrps;
> > } osm_prtn_t;
> > /*
> > * FIELDS
> > @@ -110,9 +111,10 @@ typedef struct osm_prtn {
> > * sl
> > * The Service Level (SL) associated with this Partiton.
> > *
> > -* mgrp
> > -* The pointer to the well known Multicast Group
> > -* that was created for this partition (when configured).
> > +* mgrps
> > +* List of well known Multicast Groups
> > +* that were created for this partition (when configured).
> > +* This includes the IPoIB broadcast group.
> > *
> > * full_guid_tbl
> > * Container of pointers to all Port objects in the Partition
> > @@ -139,7 +141,7 @@ typedef struct osm_prtn {
> > *
> > * SYNOPSIS
> > */
> > -void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn);
> > +void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn, osm_subn_t * p_subn);
>
> IN osm_subn_t * p_subn
>
> Also, reverse the parameters to be consistent with the coding convention.
done
>
> > /*
> > * PARAMETERS
> > * pp_prtn
> > diff --git a/man/opensm.8.in b/man/opensm.8.in
> > index 042bee3..da0c247 100644
> > --- a/man/opensm.8.in
> > +++ b/man/opensm.8.in
> > @@ -523,45 +523,76 @@ parser.
> >
> > General file format:
> >
> > -<Partition Definition>:<PortGUIDs list> ;
> > -
> > -Partition Definition:
> > -
> > -[PartitionName][=PKey][,flag[=value]][,defmember=full|limited]
> > -
> > - PartitionName - string, will be used with logging. When omitted
> > - empty string will be used.
> > - PKey - P_Key value for this partition. Only low 15 bits will
> > - be used. When omitted will be autogenerated.
> > - flag - used to indicate IPoIB capability of this partition.
> > - defmember=full|limited - specifies default membership for port guid
> > - list. Default is limited.
> > -
> > -Currently recognized flags are:
> > -
> > - ipoib - indicates that this partition may be used for IPoIB, as
> > - result IPoIB capable MC group will be created.
> > - rate=<val> - specifies rate for this IPoIB MC group
> > - (default is 3 (10GBps))
> > - mtu=<val> - specifies MTU for this IPoIB MC group
> > - (default is 4 (2048))
> > - sl=<val> - specifies SL for this IPoIB MC group
> > - (default is 0)
> > - scope=<val> - specifies scope for this IPoIB MC group
> > - (default is 2 (link local)). Multiple scope settings
> > - are permitted for a partition.
> > -
> > -Note that values for rate, mtu, and scope should be specified as
> > -defined in the IBTA specification (for example, mtu=4 for 2048).
> > -
> > -PortGUIDs list:
> > -
> > - PortGUID - GUID of partition member EndPort. Hexadecimal
> > - numbers should start from 0x, decimal numbers
> > - are accepted too.
> > - full or limited - indicates full or limited membership for this
> > - port. When omitted (or unrecognized) limited
> > - membership is assumed.
> > +<Partition Definition>:[<newline>]<Partition Properties>;
> > +
> > + Partition Definition:
> > + [PartitionName][=PKey][,part_flag[=value]][,defmember=full|limited]
> > +
> > + PartitionName - string, will be used with logging. When omitted
> > + empty string will be used.
> > + PKey - P_Key value for this partition. Only low 15 bits will
> > + be used. When omitted will be autogenerated.
> > + part_flags - used to indicate/specify IPoIB capability of this partition.
> > + defmember=full|limited - specifies default membership for port guid
> > + list. Default is limited.
> > +
> > + part_flag:
> > + ipoib - indicates that this partition may be used for IPoIB, as
> > + result IPoIB capable MC group will be created.
> > + rate=<val> - specifies rate for this IPoIB MC group
> > + (default is 3 (10GBps))
> > + mtu=<val> - specifies MTU for this IPoIB MC group
> > + (default is 4 (2048))
> > + sl=<val> - specifies SL for this IPoIB MC group
> > + (default is 0)
> > + scope=<val> - specifies scope for this IPoIB MC group
> > + (default is 2 (link local)). Multiple scope settings
> > + are permitted for a partition.
> > +
> > + Partition Properties:
> > + [<Port list>|<MCast Group>]* | <Port list>
> > +
> > + Port list:
> > + <Port Specifier>[,<Port Specifier>]
> > +
> > + Port Specifier:
> > + <PortGUID>[=[full|limited]]
> > +
> > + PortGUID - GUID of partition member EndPort. Hexadecimal
> > + numbers should start from 0x, decimal numbers
> > + are accepted too.
> > + full or limited - indicates full or limited membership for this
> > + port. When omitted (or unrecognized) limited
> > + membership is assumed.
> > +
> > + MCast Group:
> > + mgid=gid[,mgroup_flag=val]*<newline>
> > +
> > + mgroup_flag:
> > + rate=<val> - specifies rate for this MC group
> > + (default is 3 (10GBps))
> > + mtu=<val> - specifies MTU for this MC group
> > + (default is 4 (2048))
> > + sl=<val> - specifies SL for this MC group
> > + (default is 0)
> > + scope=<val> - specifies scope for this MC group
> > + (default is 2 (link local)). Multiple scope settings
> > + are permitted for a partition.
> > + NOTE: This overwrites the scope nibble of the specified
> > + mgid. Furthermore specifying multiple scope
> > + settings will result in multiple MC groups
> > + being created.
> > + qkey=<val> - specifies the Q_Key for this MC group
> > + (default: 0x0b1b for IP groups, 0 for other groups)
> > + tclass=<val> - specifies tclass for this MC group
> > + (default is 0)
> > +
> > + newline: '\n'
> > +
> > +
> > +Note that values for rate, mtu, and scope, for both partitions and multicast
> > +groups, should be specified as defined in the IBTA specification (for example,
> > +mtu=4 for 2048).
> >
> > There are several useful keywords for PortGUID definition:
> >
> > @@ -577,9 +608,6 @@ Notes:
> >
> > White space is permitted between delimiters ('=', ',',':',';').
> >
> > -The line can be wrapped after ':' followed after Partition Definition and
> > -between.
> > -
> > PartitionName does not need to be unique, PKey does need to be unique.
> > If PKey is repeated then those partition configurations will be merged
> > and first PartitionName will be used (see also next note).
> > @@ -606,6 +634,15 @@ Examples:
> > ShareIO = 0x80 , defmember=full : 0x123459, 0x12345a;
> > ShareIO = 0x80 , defmember=full : 0x12345b, 0x12345c=limited, 0x12345d;
> >
> > + # multicast groups added to default
> > + Default=0x7fff,ipoib:
> > + mgid=ff12:401b::0707,sl=1 # random IPv4 group
> > + mgid=ff12:601b::16 # MLDv2-capable routers
> > + mgid=ff12:401b::16 # IGMP
> > + mgid=ff12:601b::2 # All routers
> > + mgid=ff12::1,sl=1,Q_Key=0xDEADBEEF,rate=3,mtu=2 # random group
> > + ALL=full;
> > +
> >
> > Note:
> >
> > diff --git a/opensm/osm_prtn.c b/opensm/osm_prtn.c
> > index 3fd4fc0..2e2837a 100644
> > --- a/opensm/osm_prtn.c
> > +++ b/opensm/osm_prtn.c
> > @@ -53,6 +53,8 @@
> > #include <opensm/osm_node.h>
> > #include <opensm/osm_sa.h>
> > #include <opensm/osm_multicast.h>
> > +#include <arpa/inet.h>
> > +#include <errno.h>
> >
> > extern int osm_prtn_config_parse_file(osm_log_t * p_log, osm_subn_t * p_subn,
> > const char *file_name);
> > @@ -68,6 +70,8 @@ osm_prtn_t *osm_prtn_new(IN const char *name, IN uint16_t pkey)
> > memset(p, 0, sizeof(*p));
> > p->pkey = pkey;
> > p->sl = OSM_DEFAULT_SL;
> > + p->mgrps = NULL;
> > + p->nmgrps = 0;
> > cl_map_construct(&p->full_guid_tbl);
> > cl_map_init(&p->full_guid_tbl, 32);
> > cl_map_construct(&p->part_guid_tbl);
> > @@ -81,14 +85,34 @@ osm_prtn_t *osm_prtn_new(IN const char *name, IN uint16_t pkey)
> > return p;
> > }
> >
> > -void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn)
> > +void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn, osm_subn_t * p_subn)
> > {
> > + char gid_str[INET6_ADDRSTRLEN];
> > + int i = 0;
> > osm_prtn_t *p = *pp_prtn;
> >
> > cl_map_remove_all(&p->full_guid_tbl);
> > cl_map_destroy(&p->full_guid_tbl);
> > cl_map_remove_all(&p->part_guid_tbl);
> > cl_map_destroy(&p->part_guid_tbl);
> > +
> > + if (p->mgrps) {
> > + /* Clean up mgrps */
> > + for (i = 0; i < p->nmgrps; i++) {
> > + /* osm_mgrp_cleanup will not delete
> > + * "well_known" groups */
> > + p->mgrps[i]->well_known = FALSE;
> > + osm_mgrp_cleanup(p_subn, p->mgrps[i]);
> > + OSM_LOG(&p_subn->p_osm->log, OSM_LOG_ERROR,
>
> Should this some other log level like VERBOSE or DEBUG ?
Ooops, yep, DEBUG, that was left over from debugging. ;-)
Also cleaned up indent of the parameters.
>
> > + "removing mgroup %s from partition (0x%x)\n",
> > + inet_ntop(AF_INET6, p->mgrps[i]->mcmember_rec.mgid.raw,
> > + gid_str, sizeof gid_str),
> > + cl_hton16(p->pkey));
> > + }
> > +
> > + free(p->mgrps);
> > + }
> > +
> > free(p);
> > *pp_prtn = NULL;
> > }
> > @@ -156,21 +180,47 @@ _err:
> > return status;
> > }
> >
> > -static const ib_gid_t osm_ipoib_mgid = {
> > - {
> > - 0xff, /* multicast field */
> > - 0x12, /* non-permanent bit, link local scope */
> > - 0x40, 0x1b, /* IPv4 signature */
> > - 0xff, 0xff, /* 16 bits of P_Key (to be filled in) */
> > - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 48 bits of zeros */
> > - 0xff, 0xff, 0xff, 0xff, /* 32 bit IPv4 broadcast address */
> > - },
> > -};
> > +static ib_api_status_t
> > +track_mgrp_w_partition(osm_log_t *p_log, osm_prtn_t *p, osm_mgrp_t *mgrp,
> > + osm_subn_t *p_subn, const ib_gid_t *mgid,
> > + ib_net16_t pkey)
> > +{
> > + char gid_str[INET6_ADDRSTRLEN];
> > + osm_mgrp_t **tmp;
> > + int i = 0;
> > +
> > + /* check if we are already tracking this group */
> > + for (i = 0; i< p->nmgrps; i++)
> > + if (p->mgrps[i] == mgrp)
> > + return (IB_SUCCESS);
> > +
> > + /* otherwise add it to our list */
> > + tmp = realloc(p->mgrps, (p->nmgrps +1) * sizeof(*p->mgrps));
> > + if (tmp) {
> > + p->mgrps = tmp;
> > + p->mgrps[p->nmgrps] = mgrp;
> > + p->nmgrps++;
> > + } else {
> > + OSM_LOG(p_log, OSM_LOG_ERROR,
> > + "ERR: realloc error to create MC group (%s) in "
>
> Other similar errors in this file don't have the "ERR: " part. There are
> also other instances below but I snipped the rest.
Do you feel "ERR:" should be removed? Or the other messages have "ERR:"
added?
I just followed the convention with "ERR:" printed but did not have any
numbers to follow...
Alex, your call.
Ira
>
> -- Hal
>
> <snip...>
--
Ira Weiny
Math Programmer/Computer Scientist
Lawrence Livermore National Lab
925-423-8008
weiny2-i2BcT+NCU+M@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] opensm: Add the precreation of multicast groups
[not found] ` <20111031131218.92b27d9f.weiny2-i2BcT+NCU+M@public.gmane.org>
@ 2011-11-01 13:46 ` Hal Rosenstock
0 siblings, 0 replies; 4+ messages in thread
From: Hal Rosenstock @ 2011-11-01 13:46 UTC (permalink / raw)
To: Ira Weiny; +Cc: Alex Netes, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
On 10/31/2011 4:12 PM, Ira Weiny wrote:
> On Thu, 27 Oct 2011 12:41:53 -0700
> Hal Rosenstock <hal-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
>
>> On 10/6/2011 8:13 PM, Ira Weiny wrote:
>>>
>>> Allow for the pre-creation of these groups on a partition by partition basis.
>>
>> This looks good to me and has been needed for some time now; Thanks for
>> doing this!
>>
>> Just some minor comments below from scanning through the patch. I
>> haven't tried it yet...
>>
>>> P_Key is taken from the partition specification. Q_Key, TClass, rate, and mtu
>>> can be specified.
>>
>> If TClass is added, should FlowLabel also be added ?
>
> Yes, added in next version. Also added Q_Key, tclass, and FlowLabel to BC
> group config which uses common parsing code.
>
>>
>>> For IP groups, rate and mtu are verified to match the broadcast groups
>>> parameters. P_Key can be specified in the mgid itself and is verified to match
>>> the P_Key of the partition if != 0x0000. If pkey == 0x0000 then the P_Key is
>>> taken from the partition specification.
>>
>> Do you mean the pkey in the IPoIB MGID here by "partition specification" ?
>
> Yes, if the MGID has 0x0000 in the P_Key bits the partition manager fills in
> those bits with the P_Key specified for that partition.
It would be better to be more explicit than just say abstractly
"partition specification".
> For example:
>
> Default=0x7fff,ipoib:
> mgid=ff12:401b::0707,sl=1 # random IPv4 group
> ALL=full;
>
> results in an MGID of ff12:401b:ffff::0707
>
> Default=0x7fff,ipoib:
> mgid=ff12:401b:f123::0707,sl=1 # random IPv4 group
> ALL=full;
>
> results in an error 0xf123 != 0xffff
>
>>
>>> The syntax extends the existing syntax by allowing MC groups to be specified
>>> one per line, intermixed with the port specifications.
>>>
>>> Signed-off-by: Ira Weiny <weiny2-i2BcT+NCU+M@public.gmane.org>
>>> ---
>>> include/opensm/osm_base.h | 5 +
>>> include/opensm/osm_partition.h | 12 +-
>>> man/opensm.8.in | 121 +++++++++++------
>>
>> Similar changes should be made to doc/partition-config.txt which is
>> where this part of the man page came from originally.
>
> Alex, would you accept a patch which removes that file?
I, for one, find the separate current-routing doc useful and don't want
to see it eliminated. I'd also note there are other similar files there.
> It seems the man page is more appropriate for this documentation.
Why more appropriate ?
> Even having a separate referenced man page for partitions would be better.
This is a separate issue. Jim did some opensm man page separation for
torus2qos. Much more could be done as that man page is too large.
> For now I have modified doc/partition-config.txt to match the man page in the
> patch.
Thanks.
>>
>>> opensm/osm_prtn.c | 111 ++++++++++++----
>>> opensm/osm_prtn_config.c | 278 ++++++++++++++++++++++++++++++++++++----
>>> opensm/osm_qos_policy.c | 48 ++++----
>>> opensm/osm_subnet.c | 2 +-
>>> 7 files changed, 454 insertions(+), 123 deletions(-)
>>>
>>> diff --git a/include/opensm/osm_base.h b/include/opensm/osm_base.h
>>> index e558c55..869ba9c 100644
>>> --- a/include/opensm/osm_base.h
>>> +++ b/include/opensm/osm_base.h
>>> @@ -53,6 +53,7 @@
>>> #endif
>>>
>>> #include <complib/cl_types.h>
>>> +#include <iba/ib_types.h>
>>>
>>> #ifdef __cplusplus
>>> # define BEGIN_C_DECLS extern "C" {
>>> @@ -954,6 +955,10 @@ typedef enum _osm_sm_signal {
>>> #define OSM_VENDOR_ID_HP4 0x00237D
>>> #define OSM_VENDOR_ID_OPENIB 0x001405
>>>
>>> +/* IPoIB Broadcast Defaults */
>>> +#define OSM_IPOIB_BROADCAST_MGRP_QKEY 0x0b1b
>>> +extern const ib_gid_t osm_ipoib_broadcast_mgid;
>>> +
>>> /**********/
>>>
>>> END_C_DECLS
>>> diff --git a/include/opensm/osm_partition.h b/include/opensm/osm_partition.h
>>> index fdb34b9..7277eac 100644
>>> --- a/include/opensm/osm_partition.h
>>> +++ b/include/opensm/osm_partition.h
>>> @@ -94,10 +94,11 @@ typedef struct osm_prtn {
>>> cl_map_item_t map_item;
>>> ib_net16_t pkey;
>>> uint8_t sl;
>>> - osm_mgrp_t *mgrp;
>>> cl_map_t full_guid_tbl;
>>> cl_map_t part_guid_tbl;
>>> char name[32];
>>> + osm_mgrp_t **mgrps;
>>> + int nmgrps;
>>> } osm_prtn_t;
>>> /*
>>> * FIELDS
>>> @@ -110,9 +111,10 @@ typedef struct osm_prtn {
>>> * sl
>>> * The Service Level (SL) associated with this Partiton.
>>> *
>>> -* mgrp
>>> -* The pointer to the well known Multicast Group
>>> -* that was created for this partition (when configured).
>>> +* mgrps
>>> +* List of well known Multicast Groups
>>> +* that were created for this partition (when configured).
>>> +* This includes the IPoIB broadcast group.
>>> *
>>> * full_guid_tbl
>>> * Container of pointers to all Port objects in the Partition
>>> @@ -139,7 +141,7 @@ typedef struct osm_prtn {
>>> *
>>> * SYNOPSIS
>>> */
>>> -void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn);
>>> +void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn, osm_subn_t * p_subn);
>>
>> IN osm_subn_t * p_subn
>>
>> Also, reverse the parameters to be consistent with the coding convention.
>
> done
>
>>
>>> /*
>>> * PARAMETERS
>>> * pp_prtn
>>> diff --git a/man/opensm.8.in b/man/opensm.8.in
>>> index 042bee3..da0c247 100644
>>> --- a/man/opensm.8.in
>>> +++ b/man/opensm.8.in
>>> @@ -523,45 +523,76 @@ parser.
>>>
>>> General file format:
>>>
>>> -<Partition Definition>:<PortGUIDs list> ;
>>> -
>>> -Partition Definition:
>>> -
>>> -[PartitionName][=PKey][,flag[=value]][,defmember=full|limited]
>>> -
>>> - PartitionName - string, will be used with logging. When omitted
>>> - empty string will be used.
>>> - PKey - P_Key value for this partition. Only low 15 bits will
>>> - be used. When omitted will be autogenerated.
>>> - flag - used to indicate IPoIB capability of this partition.
>>> - defmember=full|limited - specifies default membership for port guid
>>> - list. Default is limited.
>>> -
>>> -Currently recognized flags are:
>>> -
>>> - ipoib - indicates that this partition may be used for IPoIB, as
>>> - result IPoIB capable MC group will be created.
>>> - rate=<val> - specifies rate for this IPoIB MC group
>>> - (default is 3 (10GBps))
>>> - mtu=<val> - specifies MTU for this IPoIB MC group
>>> - (default is 4 (2048))
>>> - sl=<val> - specifies SL for this IPoIB MC group
>>> - (default is 0)
>>> - scope=<val> - specifies scope for this IPoIB MC group
>>> - (default is 2 (link local)). Multiple scope settings
>>> - are permitted for a partition.
>>> -
>>> -Note that values for rate, mtu, and scope should be specified as
>>> -defined in the IBTA specification (for example, mtu=4 for 2048).
>>> -
>>> -PortGUIDs list:
>>> -
>>> - PortGUID - GUID of partition member EndPort. Hexadecimal
>>> - numbers should start from 0x, decimal numbers
>>> - are accepted too.
>>> - full or limited - indicates full or limited membership for this
>>> - port. When omitted (or unrecognized) limited
>>> - membership is assumed.
>>> +<Partition Definition>:[<newline>]<Partition Properties>;
>>> +
>>> + Partition Definition:
>>> + [PartitionName][=PKey][,part_flag[=value]][,defmember=full|limited]
>>> +
>>> + PartitionName - string, will be used with logging. When omitted
>>> + empty string will be used.
>>> + PKey - P_Key value for this partition. Only low 15 bits will
>>> + be used. When omitted will be autogenerated.
>>> + part_flags - used to indicate/specify IPoIB capability of this partition.
>>> + defmember=full|limited - specifies default membership for port guid
>>> + list. Default is limited.
>>> +
>>> + part_flag:
>>> + ipoib - indicates that this partition may be used for IPoIB, as
>>> + result IPoIB capable MC group will be created.
>>> + rate=<val> - specifies rate for this IPoIB MC group
>>> + (default is 3 (10GBps))
>>> + mtu=<val> - specifies MTU for this IPoIB MC group
>>> + (default is 4 (2048))
>>> + sl=<val> - specifies SL for this IPoIB MC group
>>> + (default is 0)
>>> + scope=<val> - specifies scope for this IPoIB MC group
>>> + (default is 2 (link local)). Multiple scope settings
>>> + are permitted for a partition.
>>> +
>>> + Partition Properties:
>>> + [<Port list>|<MCast Group>]* | <Port list>
>>> +
>>> + Port list:
>>> + <Port Specifier>[,<Port Specifier>]
>>> +
>>> + Port Specifier:
>>> + <PortGUID>[=[full|limited]]
>>> +
>>> + PortGUID - GUID of partition member EndPort. Hexadecimal
>>> + numbers should start from 0x, decimal numbers
>>> + are accepted too.
>>> + full or limited - indicates full or limited membership for this
>>> + port. When omitted (or unrecognized) limited
>>> + membership is assumed.
>>> +
>>> + MCast Group:
>>> + mgid=gid[,mgroup_flag=val]*<newline>
>>> +
>>> + mgroup_flag:
>>> + rate=<val> - specifies rate for this MC group
>>> + (default is 3 (10GBps))
>>> + mtu=<val> - specifies MTU for this MC group
>>> + (default is 4 (2048))
>>> + sl=<val> - specifies SL for this MC group
>>> + (default is 0)
>>> + scope=<val> - specifies scope for this MC group
>>> + (default is 2 (link local)). Multiple scope settings
>>> + are permitted for a partition.
>>> + NOTE: This overwrites the scope nibble of the specified
>>> + mgid. Furthermore specifying multiple scope
>>> + settings will result in multiple MC groups
>>> + being created.
>>> + qkey=<val> - specifies the Q_Key for this MC group
>>> + (default: 0x0b1b for IP groups, 0 for other groups)
>>> + tclass=<val> - specifies tclass for this MC group
>>> + (default is 0)
>>> +
>>> + newline: '\n'
>>> +
>>> +
>>> +Note that values for rate, mtu, and scope, for both partitions and multicast
>>> +groups, should be specified as defined in the IBTA specification (for example,
>>> +mtu=4 for 2048).
>>>
>>> There are several useful keywords for PortGUID definition:
>>>
>>> @@ -577,9 +608,6 @@ Notes:
>>>
>>> White space is permitted between delimiters ('=', ',',':',';').
>>>
>>> -The line can be wrapped after ':' followed after Partition Definition and
>>> -between.
>>> -
>>> PartitionName does not need to be unique, PKey does need to be unique.
>>> If PKey is repeated then those partition configurations will be merged
>>> and first PartitionName will be used (see also next note).
>>> @@ -606,6 +634,15 @@ Examples:
>>> ShareIO = 0x80 , defmember=full : 0x123459, 0x12345a;
>>> ShareIO = 0x80 , defmember=full : 0x12345b, 0x12345c=limited, 0x12345d;
>>>
>>> + # multicast groups added to default
>>> + Default=0x7fff,ipoib:
>>> + mgid=ff12:401b::0707,sl=1 # random IPv4 group
>>> + mgid=ff12:601b::16 # MLDv2-capable routers
>>> + mgid=ff12:401b::16 # IGMP
>>> + mgid=ff12:601b::2 # All routers
>>> + mgid=ff12::1,sl=1,Q_Key=0xDEADBEEF,rate=3,mtu=2 # random group
>>> + ALL=full;
>>> +
>>>
>>> Note:
>>>
>>> diff --git a/opensm/osm_prtn.c b/opensm/osm_prtn.c
>>> index 3fd4fc0..2e2837a 100644
>>> --- a/opensm/osm_prtn.c
>>> +++ b/opensm/osm_prtn.c
>>> @@ -53,6 +53,8 @@
>>> #include <opensm/osm_node.h>
>>> #include <opensm/osm_sa.h>
>>> #include <opensm/osm_multicast.h>
>>> +#include <arpa/inet.h>
>>> +#include <errno.h>
>>>
>>> extern int osm_prtn_config_parse_file(osm_log_t * p_log, osm_subn_t * p_subn,
>>> const char *file_name);
>>> @@ -68,6 +70,8 @@ osm_prtn_t *osm_prtn_new(IN const char *name, IN uint16_t pkey)
>>> memset(p, 0, sizeof(*p));
>>> p->pkey = pkey;
>>> p->sl = OSM_DEFAULT_SL;
>>> + p->mgrps = NULL;
>>> + p->nmgrps = 0;
>>> cl_map_construct(&p->full_guid_tbl);
>>> cl_map_init(&p->full_guid_tbl, 32);
>>> cl_map_construct(&p->part_guid_tbl);
>>> @@ -81,14 +85,34 @@ osm_prtn_t *osm_prtn_new(IN const char *name, IN uint16_t pkey)
>>> return p;
>>> }
>>>
>>> -void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn)
>>> +void osm_prtn_delete(IN OUT osm_prtn_t ** pp_prtn, osm_subn_t * p_subn)
>>> {
>>> + char gid_str[INET6_ADDRSTRLEN];
>>> + int i = 0;
>>> osm_prtn_t *p = *pp_prtn;
>>>
>>> cl_map_remove_all(&p->full_guid_tbl);
>>> cl_map_destroy(&p->full_guid_tbl);
>>> cl_map_remove_all(&p->part_guid_tbl);
>>> cl_map_destroy(&p->part_guid_tbl);
>>> +
>>> + if (p->mgrps) {
>>> + /* Clean up mgrps */
>>> + for (i = 0; i < p->nmgrps; i++) {
>>> + /* osm_mgrp_cleanup will not delete
>>> + * "well_known" groups */
>>> + p->mgrps[i]->well_known = FALSE;
>>> + osm_mgrp_cleanup(p_subn, p->mgrps[i]);
>>> + OSM_LOG(&p_subn->p_osm->log, OSM_LOG_ERROR,
>>
>> Should this some other log level like VERBOSE or DEBUG ?
>
> Ooops, yep, DEBUG, that was left over from debugging. ;-)
>
> Also cleaned up indent of the parameters.
>
>>
>>> + "removing mgroup %s from partition (0x%x)\n",
>>> + inet_ntop(AF_INET6, p->mgrps[i]->mcmember_rec.mgid.raw,
>>> + gid_str, sizeof gid_str),
>>> + cl_hton16(p->pkey));
>>> + }
>>> +
>>> + free(p->mgrps);
>>> + }
>>> +
>>> free(p);
>>> *pp_prtn = NULL;
>>> }
>>> @@ -156,21 +180,47 @@ _err:
>>> return status;
>>> }
>>>
>>> -static const ib_gid_t osm_ipoib_mgid = {
>>> - {
>>> - 0xff, /* multicast field */
>>> - 0x12, /* non-permanent bit, link local scope */
>>> - 0x40, 0x1b, /* IPv4 signature */
>>> - 0xff, 0xff, /* 16 bits of P_Key (to be filled in) */
>>> - 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, /* 48 bits of zeros */
>>> - 0xff, 0xff, 0xff, 0xff, /* 32 bit IPv4 broadcast address */
>>> - },
>>> -};
>>> +static ib_api_status_t
>>> +track_mgrp_w_partition(osm_log_t *p_log, osm_prtn_t *p, osm_mgrp_t *mgrp,
>>> + osm_subn_t *p_subn, const ib_gid_t *mgid,
>>> + ib_net16_t pkey)
>>> +{
>>> + char gid_str[INET6_ADDRSTRLEN];
>>> + osm_mgrp_t **tmp;
>>> + int i = 0;
>>> +
>>> + /* check if we are already tracking this group */
>>> + for (i = 0; i< p->nmgrps; i++)
>>> + if (p->mgrps[i] == mgrp)
>>> + return (IB_SUCCESS);
>>> +
>>> + /* otherwise add it to our list */
>>> + tmp = realloc(p->mgrps, (p->nmgrps +1) * sizeof(*p->mgrps));
>>> + if (tmp) {
>>> + p->mgrps = tmp;
>>> + p->mgrps[p->nmgrps] = mgrp;
>>> + p->nmgrps++;
>>> + } else {
>>> + OSM_LOG(p_log, OSM_LOG_ERROR,
>>> + "ERR: realloc error to create MC group (%s) in "
>>
>> Other similar errors in this file don't have the "ERR: " part. There are
>> also other instances below but I snipped the rest.
>
> Do you feel "ERR:" should be removed? Or the other messages have "ERR:"
> added?
Removed.
> I just followed the convention with "ERR:" printed but did not have any
> numbers to follow...
I don't see "ERR: " in either osm_prtn.c or osm_prtn_config.c
-- Hal
> Alex, your call.
>
> Ira
>
>>
>> -- Hal
>>
>> <snip...>
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2011-11-01 13:46 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-10-07 0:13 [PATCH] opensm: Add the precreation of multicast groups Ira Weiny
[not found] ` <20111006171326.3b143b03.weiny2-i2BcT+NCU+M@public.gmane.org>
2011-10-27 19:41 ` Hal Rosenstock
[not found] ` <4EA9B401.5030101-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2011-10-31 20:12 ` Ira Weiny
[not found] ` <20111031131218.92b27d9f.weiny2-i2BcT+NCU+M@public.gmane.org>
2011-11-01 13:46 ` Hal Rosenstock
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox