Re: rsockets and fork

[Sridhar Samudrala <sri-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org>]

On 8/13/2012 4:39 PM, Hefty, Sean wrote:
>> I could not get fork enabled netperf to work with rsockets in the latest
>> librdmacm git repository.
>> After some debugging, i found that the child netserver process is blocked at
>> sem_wait() call in fork_passive().
>> It is not clear to me how this call is supposed to unblock as sem_post()
>> is done later in the same function.
> sem_open() should create the semaphore with an initial value of 1.  The sem_wait/sem_post calls serialize listening on the corresponding rsocket.  The name semaphore should remain until the system is rebooted.
Looks like a stale semaphore issue. I removed /dev/shm/sem.rsocket_fork 
and restarted netserver and it is working now.
>   
>> However, if i start another netperf instance while the other session is
>> still going on, it seems to hang and return with a very low throughput.
>> It looks as if the first session is starving all the other sessions.
>> The right behavior would be to have the available bw split across the
>> parallel instances.
> Can you verify that the second session is using rsockets and not falling back to a socket connection?
Yes. it is also using rsockets.
The second session always hangs after sending a fixed number of bytes 
(38469632).
rsend() blocks waiting for the CQ event.

Thanks
Sridhar
>
> You can try adjusting the polling time.  This can usually be done by writing a value (time in microseconds) to the following file:
>
> /etc/rdma/rsocket/polling_time
>
> By default, the value is 10.
>
> - Sean
>

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html