Re: OpenMPI over RoCEE

[Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>]

You know, I got it running by adding this:  --mca btl_openib_cpc_include 
rdmacm

Which basically sez use only the rdmacm to setup the connection.

Thanks,

Steve.

Jeff Squyres wrote:
> Does it work with Open MPI v1.4.2?
>
>
> On Jul 12, 2010, at 4:21 PM, Steve Wise wrote:
>
>   
>> I'm running OFED-1.5.1 with the RoCEE mlx4 drivers.  I can run low level
>> verbs programs ok, but when running open mpi, I'm getting this error. 
>> Anybody seen this?
>>
>> -----
>>
>> [ompi@escher ~]$ mpirun -np 2 -host 10.192.176.111,10.192.176.112 --mca
>> btl openib,sm,self /usr/mpi/gcc/openmpi-1.4.1/tests/IMB-3.2/IMB-MPI1
>> -msglen msglen.txt -iter 1000000 pingpong
>> [escher][[36356,1],1][connect/btl_openib_connect_oob.c:325:qp_connect_all]
>> error modifing QP to RTR errno says Invalid argument
>> [escher][[36356,1],1][connect/btl_openib_connect_oob.c:809:rml_recv_cb]
>> error in endpoint reply start connect
>> --------------------------------------------------------------------------
>> mpirun has exited due to process rank 1 with PID 4894 on
>> node escher exiting without calling "finalize". This may
>> have caused other processes in the application to be
>> terminated by signals sent by mpirun (as reported here).
>> --------------------------------------------------------------------------
>>
>>
>>     
>
>
>   

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html