linux-rdma.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] opensm: event plig-in API fixed to compile with g++
@ 2010-06-24  8:10 Yevgeny Kliteynik
       [not found] ` <4C2312F0.8020308-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
  0 siblings, 1 reply; 21+ messages in thread
From: Yevgeny Kliteynik @ 2010-06-24  8:10 UTC (permalink / raw)
  To: Sasha Khapyorsky; +Cc: Linux RDMA, Yevgeny Kliteynik

Event API should have been able to be used by libraries
written both in C and C++. The problem is, one of the
fields in struct osm_event_plugin is called "delete".
Changing it to "destroy" and promoting the API version.

Signed-off-by: Yevgeny Kliteynik <kliteyn-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
---
 opensm/include/opensm/osm_event_plugin.h    |    2 +-
 opensm/opensm/osm_event_plugin.c            |    4 ++--
 opensm/osmeventplugin/libosmeventplugin.ver |    2 +-
 opensm/osmeventplugin/src/osmeventplugin.c  |    2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/opensm/include/opensm/osm_event_plugin.h b/opensm/include/opensm/osm_event_plugin.h
index 0b3464e..aa3fed4 100644
--- a/opensm/include/opensm/osm_event_plugin.h
+++ b/opensm/include/opensm/osm_event_plugin.h
@@ -145,7 +145,7 @@ typedef struct osm_api_ps_event {
 typedef struct osm_event_plugin {
 	const char *osm_version;
 	void *(*create) (struct osm_opensm *osm);
-	void (*delete) (void *plugin_data);
+	void (*destroy) (void *plugin_data);
 	void (*report) (void *plugin_data, osm_epi_event_id_t event_id,
 			void *event_data);
 } osm_event_plugin_t;
diff --git a/opensm/opensm/osm_event_plugin.c b/opensm/opensm/osm_event_plugin.c
index 2d67065..7f61960 100644
--- a/opensm/opensm/osm_event_plugin.c
+++ b/opensm/opensm/osm_event_plugin.c
@@ -141,8 +141,8 @@ DLOPENFAIL:
 void osm_epi_destroy(osm_epi_plugin_t * plugin)
 {
 	if (plugin) {
-		if (plugin->impl->delete)
-			plugin->impl->delete(plugin->plugin_data);
+		if (plugin->impl->destroy)
+			plugin->impl->destroy(plugin->plugin_data);
 		dlclose(plugin->handle);
 		free(plugin->plugin_name);
 		free(plugin);
diff --git a/opensm/osmeventplugin/libosmeventplugin.ver b/opensm/osmeventplugin/libosmeventplugin.ver
index f755ff6..0c3a85b 100644
--- a/opensm/osmeventplugin/libosmeventplugin.ver
+++ b/opensm/osmeventplugin/libosmeventplugin.ver
@@ -6,4 +6,4 @@
 # API_REV - advance on any added API
 # RUNNING_REV - advance any change to the vendor files
 # AGE - number of backward versions the API still supports
-LIBVERSION=1:0:0
+LIBVERSION=2:0:0
diff --git a/opensm/osmeventplugin/src/osmeventplugin.c b/opensm/osmeventplugin/src/osmeventplugin.c
index f40f7fe..a82be07 100644
--- a/opensm/osmeventplugin/src/osmeventplugin.c
+++ b/opensm/osmeventplugin/src/osmeventplugin.c
@@ -207,6 +207,6 @@ static void report(void *_log, osm_epi_event_id_t event_id, void *event_data)
 osm_event_plugin_t osm_event_plugin = {
       osm_version:OSM_VERSION,
       create:construct,
-      delete:destroy,
+      destroy:destroy,
       report:report
 };
-- 
1.5.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply related	[flat|nested] 21+ messages in thread
* RDMA synchronous operations
@ 2010-06-24 14:33 Andrea Gozzelino
  0 siblings, 0 replies; 21+ messages in thread
From: Andrea Gozzelino @ 2010-06-24 14:33 UTC (permalink / raw)
  To: linux-rdma-u79uwXL29TY76Z2rM5mHXA; +Cc: sean.hefty-ral2JQCrhuEAvxtiuMwx3w

Hi all,

I’m testing the latency time of an RDMA system (2 card and 1 switch).

The connection server/client is established and the code is just as
mirror on both sides.

The RDMA action scheme is below:
1-	client sends a buffer;
2-	server receives a buffer;
3-	server sends a buffer;
4-	client receives a buffer.

The scheme is in a for loop, so after step 4 client repeats step 1.
On client side I put a clock: t_start is before loop and t_end is after
loop.
I consider the round trip time RTT as (t_end-t_start)/NTX where NTX is
number of transfer (number of loop). The latency time is half RTT, if
transfer time is negligible, i.e. for small (8 Bytes – 4 Kbytes) buffer
size and high NTX value (> 10000) is true.

The RDMA communication stops casually with error status 5. It means that
the send or receive operation on one side (server or client) casually is
not in time. I summarize post_send and post_receive operation are
quasi-synchronous. I would like that RDMA operation are completely
synchronous. I use the completion queue and the acknowledgement event.

Do you see the same character in other RDMA use? How can I synchronize
all RDMA operation?
Could you please give me some suggestions?

Thank you very much,
Andrea

Example of structure code:
Ibv_recv_wr rrq(mr), *bad_rrq;
		assert(!conn->post_recv(&rrq, &bad_rrq, 0));
		assert(!conn->get_completion(&compl_queue));
		//printf("ok %p %s
", compl_queue, strerror(errno));
		assert(1 == compl_queue->poll(1, &wc));
		//printf("status %i
", (int)wc.status);
		if (wc.status !=0){
			printf("RDMA server closes. 
");
			break;
			}
-----------------
Ibv_send_wr srq((void *)rpd->address, rpd->rkey), *bad_srq;
  		srq.add_sge(mr);
  		assert(!conn.post_send(&srq, &bad_srq, 0));
  		assert(!conn.get_completion(&compl_queue));
  		assert(1 == compl_queue->poll(1, &wc));
  
  		//printf("ok %p %s status %i
", compl_queue, strerror(errno), wc.status);
  		//printf("loop %i
", i);
  
  		if (wc.status != 0){
  			//printf("KO %p %s status %i
", compl_queue, strerror(errno), wc.status);
  			printf("RDMA data sending problem in loop %i.
", i);
  			break;
  		}
-----------
Andrea Gozzelino

INFN - Laboratori Nazionali di Legnaro	(LNL)
Viale dell'Universita' 2 -I-35020 - Legnaro (PD)- ITALIA
Office: E-101
Tel: +39 049 8068346
Fax: +39 049 641925
Mail: andrea.gozzelino-PK20h7lG/Rc1GQ1Ptb7lUw@public.gmane.org
Cell: +39 3488245552				

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 21+ messages in thread
* RDMA synchronous operations
@ 2010-06-24 14:33 Andrea Gozzelino
  0 siblings, 0 replies; 21+ messages in thread
From: Andrea Gozzelino @ 2010-06-24 14:33 UTC (permalink / raw)
  To: linux-rdma-u79uwXL29TY76Z2rM5mHXA; +Cc: sean.hefty-ral2JQCrhuEAvxtiuMwx3w

Hi all,

I’m testing the latency time of an RDMA system (2 card and 1 switch).

The connection server/client is established and the code is just as
mirror on both sides.

The RDMA action scheme is below:
1-	client sends a buffer;
2-	server receives a buffer;
3-	server sends a buffer;
4-	client receives a buffer.

The scheme is in a for loop, so after step 4 client repeats step 1.
On client side I put a clock: t_start is before loop and t_end is after
loop.
I consider the round trip time RTT as (t_end-t_start)/NTX where NTX is
number of transfer (number of loop). The latency time is half RTT, if
transfer time is negligible, i.e. for small (8 Bytes – 4 Kbytes) buffer
size and high NTX value (> 10000) is true.

The RDMA communication stops casually with error status 5. It means that
the send or receive operation on one side (server or client) casually is
not in time. I summarize post_send and post_receive operation are
quasi-synchronous. I would like that RDMA operation are completely
synchronous. I use the completion queue and the acknowledgement event.

Do you see the same character in other RDMA use? How can I synchronize
all RDMA operation?
Could you please give me some suggestions?

Thank you very much,
Andrea

Example of structure code:
Ibv_recv_wr rrq(mr), *bad_rrq;
		assert(!conn->post_recv(&rrq, &bad_rrq, 0));
		assert(!conn->get_completion(&compl_queue));
		//printf("ok %p %s
", compl_queue, strerror(errno));
		assert(1 == compl_queue->poll(1, &wc));
		//printf("status %i
", (int)wc.status);
		if (wc.status !=0){
			printf("RDMA server closes. 
");
			break;
			}
-----------------
Ibv_send_wr srq((void *)rpd->address, rpd->rkey), *bad_srq;
  		srq.add_sge(mr);
  		assert(!conn.post_send(&srq, &bad_srq, 0));
  		assert(!conn.get_completion(&compl_queue));
  		assert(1 == compl_queue->poll(1, &wc));
  
  		//printf("ok %p %s status %i
", compl_queue, strerror(errno), wc.status);
  		//printf("loop %i
", i);
  
  		if (wc.status != 0){
  			//printf("KO %p %s status %i
", compl_queue, strerror(errno), wc.status);
  			printf("RDMA data sending problem in loop %i.
", i);
  			break;
  		}
-----------
Andrea Gozzelino

INFN - Laboratori Nazionali di Legnaro	(LNL)
Viale dell'Universita' 2 -I-35020 - Legnaro (PD)- ITALIA
Office: E-101
Tel: +39 049 8068346
Fax: +39 049 641925
Mail: andrea.gozzelino-PK20h7lG/Rc1GQ1Ptb7lUw@public.gmane.org
Cell: +39 3488245552				

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 21+ messages in thread
* RDMA synchronous operations
@ 2010-06-24 14:33 Andrea Gozzelino
  0 siblings, 0 replies; 21+ messages in thread
From: Andrea Gozzelino @ 2010-06-24 14:33 UTC (permalink / raw)
  To: linux-rdma-u79uwXL29TY76Z2rM5mHXA; +Cc: sean.hefty-ral2JQCrhuEAvxtiuMwx3w

Hi all,

I’m testing the latency time of an RDMA system (2 card and 1 switch).

The connection server/client is established and the code is just as
mirror on both sides.

The RDMA action scheme is below:
1-	client sends a buffer;
2-	server receives a buffer;
3-	server sends a buffer;
4-	client receives a buffer.

The scheme is in a for loop, so after step 4 client repeats step 1.
On client side I put a clock: t_start is before loop and t_end is after
loop.
I consider the round trip time RTT as (t_end-t_start)/NTX where NTX is
number of transfer (number of loop). The latency time is half RTT, if
transfer time is negligible, i.e. for small (8 Bytes – 4 Kbytes) buffer
size and high NTX value (> 10000) is true.

The RDMA communication stops casually with error status 5. It means that
the send or receive operation on one side (server or client) casually is
not in time. I summarize post_send and post_receive operation are
quasi-synchronous. I would like that RDMA operation are completely
synchronous. I use the completion queue and the acknowledgement event.

Do you see the same character in other RDMA use? How can I synchronize
all RDMA operation?
Could you please give me some suggestions?

Thank you very much,
Andrea

Example of structure code:
Ibv_recv_wr rrq(mr), *bad_rrq;
		assert(!conn->post_recv(&rrq, &bad_rrq, 0));
		assert(!conn->get_completion(&compl_queue));
		//printf("ok %p %s
", compl_queue, strerror(errno));
		assert(1 == compl_queue->poll(1, &wc));
		//printf("status %i
", (int)wc.status);
		if (wc.status !=0){
			printf("RDMA server closes. 
");
			break;
			}
-----------------
Ibv_send_wr srq((void *)rpd->address, rpd->rkey), *bad_srq;
  		srq.add_sge(mr);
  		assert(!conn.post_send(&srq, &bad_srq, 0));
  		assert(!conn.get_completion(&compl_queue));
  		assert(1 == compl_queue->poll(1, &wc));
  
  		//printf("ok %p %s status %i
", compl_queue, strerror(errno), wc.status);
  		//printf("loop %i
", i);
  
  		if (wc.status != 0){
  			//printf("KO %p %s status %i
", compl_queue, strerror(errno), wc.status);
  			printf("RDMA data sending problem in loop %i.
", i);
  			break;
  		}
-----------
Andrea Gozzelino

INFN - Laboratori Nazionali di Legnaro	(LNL)
Viale dell'Universita' 2 -I-35020 - Legnaro (PD)- ITALIA
Office: E-101
Tel: +39 049 8068346
Fax: +39 049 641925
Mail: andrea.gozzelino-PK20h7lG/Rc1GQ1Ptb7lUw@public.gmane.org
Cell: +39 3488245552				

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 21+ messages in thread
* RDMA synchronous operations
@ 2010-06-24 14:33 Andrea Gozzelino
  0 siblings, 0 replies; 21+ messages in thread
From: Andrea Gozzelino @ 2010-06-24 14:33 UTC (permalink / raw)
  To: linux-rdma-u79uwXL29TY76Z2rM5mHXA; +Cc: sean.hefty-ral2JQCrhuEAvxtiuMwx3w

Hi all,

I’m testing the latency time of an RDMA system (2 card and 1 switch).

The connection server/client is established and the code is just as
mirror on both sides.

The RDMA action scheme is below:
1-	client sends a buffer;
2-	server receives a buffer;
3-	server sends a buffer;
4-	client receives a buffer.

The scheme is in a for loop, so after step 4 client repeats step 1.
On client side I put a clock: t_start is before loop and t_end is after
loop.
I consider the round trip time RTT as (t_end-t_start)/NTX where NTX is
number of transfer (number of loop). The latency time is half RTT, if
transfer time is negligible, i.e. for small (8 Bytes – 4 Kbytes) buffer
size and high NTX value (> 10000) is true.

The RDMA communication stops casually with error status 5. It means that
the send or receive operation on one side (server or client) casually is
not in time. I summarize post_send and post_receive operation are
quasi-synchronous. I would like that RDMA operation are completely
synchronous. I use the completion queue and the acknowledgement event.

Do you see the same character in other RDMA use? How can I synchronize
all RDMA operation?
Could you please give me some suggestions?

Thank you very much,
Andrea

Example of structure code:
Ibv_recv_wr rrq(mr), *bad_rrq;
		assert(!conn->post_recv(&rrq, &bad_rrq, 0));
		assert(!conn->get_completion(&compl_queue));
		//printf("ok %p %s
", compl_queue, strerror(errno));
		assert(1 == compl_queue->poll(1, &wc));
		//printf("status %i
", (int)wc.status);
		if (wc.status !=0){
			printf("RDMA server closes. 
");
			break;
			}
-----------------
Ibv_send_wr srq((void *)rpd->address, rpd->rkey), *bad_srq;
  		srq.add_sge(mr);
  		assert(!conn.post_send(&srq, &bad_srq, 0));
  		assert(!conn.get_completion(&compl_queue));
  		assert(1 == compl_queue->poll(1, &wc));
  
  		//printf("ok %p %s status %i
", compl_queue, strerror(errno), wc.status);
  		//printf("loop %i
", i);
  
  		if (wc.status != 0){
  			//printf("KO %p %s status %i
", compl_queue, strerror(errno), wc.status);
  			printf("RDMA data sending problem in loop %i.
", i);
  			break;
  		}
-----------
Andrea Gozzelino

INFN - Laboratori Nazionali di Legnaro	(LNL)
Viale dell'Universita' 2 -I-35020 - Legnaro (PD)- ITALIA
Office: E-101
Tel: +39 049 8068346
Fax: +39 049 641925
Mail: andrea.gozzelino-PK20h7lG/Rc1GQ1Ptb7lUw@public.gmane.org
Cell: +39 3488245552				

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 21+ messages in thread
* RDMA synchronous operations
@ 2010-06-24 14:33 Andrea Gozzelino
  0 siblings, 0 replies; 21+ messages in thread
From: Andrea Gozzelino @ 2010-06-24 14:33 UTC (permalink / raw)
  To: linux-rdma-u79uwXL29TY76Z2rM5mHXA; +Cc: sean.hefty-ral2JQCrhuEAvxtiuMwx3w

Hi all,

I’m testing the latency time of an RDMA system (2 card and 1 switch).

The connection server/client is established and the code is just as
mirror on both sides.

The RDMA action scheme is below:
1-	client sends a buffer;
2-	server receives a buffer;
3-	server sends a buffer;
4-	client receives a buffer.

The scheme is in a for loop, so after step 4 client repeats step 1.
On client side I put a clock: t_start is before loop and t_end is after
loop.
I consider the round trip time RTT as (t_end-t_start)/NTX where NTX is
number of transfer (number of loop). The latency time is half RTT, if
transfer time is negligible, i.e. for small (8 Bytes – 4 Kbytes) buffer
size and high NTX value (> 10000) is true.

The RDMA communication stops casually with error status 5. It means that
the send or receive operation on one side (server or client) casually is
not in time. I summarize post_send and post_receive operation are
quasi-synchronous. I would like that RDMA operation are completely
synchronous. I use the completion queue and the acknowledgement event.

Do you see the same character in other RDMA use? How can I synchronize
all RDMA operation?
Could you please give me some suggestions?

Thank you very much,
Andrea

Example of structure code:
Ibv_recv_wr rrq(mr), *bad_rrq;
		assert(!conn->post_recv(&rrq, &bad_rrq, 0));
		assert(!conn->get_completion(&compl_queue));
		//printf("ok %p %s
", compl_queue, strerror(errno));
		assert(1 == compl_queue->poll(1, &wc));
		//printf("status %i
", (int)wc.status);
		if (wc.status !=0){
			printf("RDMA server closes. 
");
			break;
			}
-----------------
Ibv_send_wr srq((void *)rpd->address, rpd->rkey), *bad_srq;
  		srq.add_sge(mr);
  		assert(!conn.post_send(&srq, &bad_srq, 0));
  		assert(!conn.get_completion(&compl_queue));
  		assert(1 == compl_queue->poll(1, &wc));
  
  		//printf("ok %p %s status %i
", compl_queue, strerror(errno), wc.status);
  		//printf("loop %i
", i);
  
  		if (wc.status != 0){
  			//printf("KO %p %s status %i
", compl_queue, strerror(errno), wc.status);
  			printf("RDMA data sending problem in loop %i.
", i);
  			break;
  		}
-----------
Andrea Gozzelino

INFN - Laboratori Nazionali di Legnaro	(LNL)
Viale dell'Universita' 2 -I-35020 - Legnaro (PD)- ITALIA
Office: E-101
Tel: +39 049 8068346
Fax: +39 049 641925
Mail: andrea.gozzelino-PK20h7lG/Rc1GQ1Ptb7lUw@public.gmane.org
Cell: +39 3488245552				

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 21+ messages in thread
end of thread, other threads:[~2010-07-08 13:22 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-06-24  8:10 [PATCH] opensm: event plig-in API fixed to compile with g++ Yevgeny Kliteynik
     [not found] ` <4C2312F0.8020308-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2010-06-24 14:36   ` RDMA synchronous operations Andrea Gozzelino
     [not found]     ` <964270.1277390217198.SLOX.WebMail.wwwrun-XDIR3SKYeFbgKi2NxijLtw@public.gmane.org>
2010-06-24 19:21       ` Dotan Barak
2010-07-05 18:11   ` [PATCH] opensm: event plig-in API fixed to compile with g++ Sasha Khapyorsky
2010-07-05 18:30     ` Hal Rosenstock
     [not found]       ` <AANLkTilUABmkKA8W-vIVWyTuDR8Gte9jtxmDUL1SiAte-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2010-07-05 18:41         ` Sasha Khapyorsky
2010-07-05 18:49           ` Hal Rosenstock
     [not found]             ` <AANLkTina2lO8M65L465jXjO1ukaAudUO1ZNk8opfKtQc-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2010-07-05 22:52               ` Sasha Khapyorsky
2010-07-05 23:29                 ` Hal Rosenstock
     [not found]                   ` <AANLkTinWEuCtSSfVc1Ti0SjgmUl3vnEY-DDd1nkWtlne-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2010-07-06 16:21                     ` Hefty, Sean
     [not found]                       ` <CF9C39F99A89134C9CF9C4CCB68B8DDF25A116CA8F-osO9UTpF0USkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2010-07-06 17:00                         ` Roland Dreier
     [not found]                           ` <adamxu4mtov.fsf-BjVyx320WGW9gfZ95n9DRSW4+XlvGpQz@public.gmane.org>
2010-07-07 13:58                             ` Sasha Khapyorsky
2010-07-08 13:22                               ` Hal Rosenstock
2010-07-06 21:03           ` Ira Weiny
     [not found]             ` <20100706140335.81e25f4c.weiny2-i2BcT+NCU+M@public.gmane.org>
2010-07-06 22:19               ` Yevgeny Kliteynik
2010-07-07 11:22     ` Yevgeny Kliteynik
  -- strict thread matches above, loose matches on Subject: below --
2010-06-24 14:33 RDMA synchronous operations Andrea Gozzelino
2010-06-24 14:33 Andrea Gozzelino
2010-06-24 14:33 Andrea Gozzelino
2010-06-24 14:33 Andrea Gozzelino
2010-06-24 14:33 Andrea Gozzelino

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).