adding path record wire format to libibverbs

adding path record wire format to libibverbs

[Sean Hefty]

Roland,

Would you accept a patch to libibverbs sa.h to add the wire format for a path
record, shown below?  I'm not ready to submit a patch yet, just need to find a
place for this.

- Sean


#define IB_PATH_RECORD_REVERSIBLE 0x80

struct ib_path_record
{
	uint64_t        service_id;
	union ibv_gid   dgid;
	union ibv_gid   sgid;
	uint16_t        dlid;
	uint16_t        slid;
	uint32_t        flowlabel_hoplimit; /* resv-31:28 flow label-27:8 hop
limit-7:0*/
	uint8_t         tclass;
	uint8_t         reversible_numpath; /* reversible-7:7 num path-6:0 */
	uint16_t        pkey;
	uint16_t        qosclass_sl;        /* qos class-15:4 sl-3:0 */
	uint8_t         mtu;                /* mtu selector-7:6 mtu-5:0 */
	uint8_t         rate;               /* rate selector-7:6 rate-5:0 */
	uint8_t         packetlifetime;     /* lifetime selector-7:6
lifetime-5:0 */
	uint8_t         preference;
	uint8_t         reserved[6];
};

#define IB_PATH_FLAGS_CM              (1<<0)
#define IB_PATH_FLAGS_PRIMARY         (1<<1)
#define IB_PATH_FLAGS_ALTERNATE       (1<<2)
#define IB_PATH_FLAGS_OUTBOUND        (1<<3)
#define IB_PATH_FLAGS_INBOUND         (1<<4)
#define IB_PATH_FLAGS_INBOUND_REVERSE (1<<5)
#define IB_PATH_FLAGS_BIDIRECTIONAL   (IB_PATH_FLAGS_OUTBOUND |     \
                                       IB_PATH_FLAGS_INBOUND_REVERSE)

struct ib_path_data
{
	uint32_t              flags;
	uint32_t              reserved;
	struct ib_path_record path;
};

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: adding path record wire format to libibverbs

[Jason Gunthorpe]

On Wed, Mar 17, 2010 at 10:48:30AM -0700, Sean Hefty wrote:

> Would you accept a patch to libibverbs sa.h to add the wire format for a path
> record, shown below?  I'm not ready to submit a patch yet, just need to find a
> place for this.

Could we use the bitfield versions of this I posted awhile back? :)

Sean made the kernel accept information in this format a few change
sets ago so it seems appropriate to me to be in libibverbs..

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: adding path record wire format to libibverbs

[Sean Hefty]

>Could we use the bitfield versions of this I posted awhile back? :)

Do you have this conveniently lying around?  If not, I'll search for it.

- Sean

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: adding path record wire format to libibverbs

[Roland Dreier]

Including this structure is no problem in principle.

 > Could we use the bitfield versions of this I posted awhile back? :)

I don't have that patch handy, but bitfields often end up ugly with
endian stuff, and also some compilers/architecture generate awful code
for it.  What's the advantage over Sean's patch?

 - R.
-- 
Roland Dreier  <rolandd-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org>
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: adding path record wire format to libibverbs

[Jason Gunthorpe]

On Thu, Mar 18, 2010 at 10:49:01AM -0700, Roland Dreier wrote:
> Including this structure is no problem in principle.
> 
>  > Could we use the bitfield versions of this I posted awhile back? :)
> 
> I don't have that patch handy, but bitfields often end up ugly with
> endian stuff, and also some compilers/architecture generate awful code
> for it.  What's the advantage over Sean's patch?

It is user space code, the interface should be trivially usable
without a huge hassle to the user. Bitfields can help provide
that. The LE/BE issue is easy to solve, but I admit the solution I
prefer (32 bit byte swap) might not fit very well with Sean's intended
use.

For instance if you want to override the hoplimit to 1:
 pr.flowlabel_hoplimit = htonl((ntohl(pr.flowlabel_hoplimit) & (~0xFF)) | 1);
vs
 swap32_all(&pr, sizeof(pr);
 pr.hopLimit = 1;
 swap32_all(&pr, sizeof(ptr));

For this application I don't think the codegen is a concern, and I've
seen that x86 and ppc generate acceptable code for this.. Certainly,
this approach is definately faster than the pack/unpack scheme I've
seen used in many place.

#ifdef BIG_ENDIAN
struct SAPathRecord {
    uint32_t rsv0;

    uint32_t rsv1;

    uint8_t DGID[16];

    uint8_t SGID[16];

    uint16_t DLID;
    uint16_t SLID;

    uint32_t rawTraffic:1;
    uint32_t rsv2:3;
    uint32_t flowLabel:20;
    uint32_t hopLimit:8;

    uint8_t TClass;
    uint8_t reversible:1;
    uint8_t numbPath:7;
    uint16_t PKey;

    uint16_t rsv3:12;
    uint16_t SL:4;
    uint8_t MTUSelector:2;
    uint8_t MTU:6;
    uint8_t rateSelector:2;
    uint8_t rate:6;

    uint8_t packetLifeTimeSelector:2;
    uint8_t packetLifeTime:6;
    uint8_t preference;
    uint16_t rsv4;

    uint32_t rsv5;
};
#else
// Must ntohl() every 32 bit word prior to appling this structure
struct SAPathRecord {
    uint32_t rsv0;

    uint32_t rsv1;

    uint8_t DGID[16];

    uint8_t SGID[16];

    uint16_t SLID;
    uint16_t DLID;

    uint32_t hopLimit:8;
    uint32_t flowLabel:20;
    uint32_t rsv2:3;
    uint32_t rawTraffic:1;

    uint16_t PKey;
    uint8_t numbPath:7;
    uint8_t reversible:1;
    uint8_t TClass;

    uint8_t rate:6;
    uint8_t rateSelector:2;
    uint8_t MTU:6;
    uint8_t MTUSelector:2;
    uint16_t SL:4;
    uint16_t rsv3:12;

    uint16_t rsv4;
    uint8_t preference;
    uint8_t packetLifeTime:6;
    uint8_t packetLifeTimeSelector:2;

    uint32_t rsv5;
};
#endif

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html