Re: [PATCH v2 3/4] rsockets: distribute completion queue vectors among multiple cores

[Sreedhar Kodali <srkodali-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>]

On 2014-09-09 17:58, Sreedhar Kodali wrote:
> On 2014-09-08 23:52, Bart Van Assche wrote:
>> On 09/08/14 16:27, Sreedhar Kodali wrote:
>>> Thanks for your insights.  Your suggestion of having a separate
>>> vector per process is really useful when multiple processes are
>>> simultaneously running in the same environment.  At the 
>>> implementation
>>> level this requires an extended configuration file format. Can't
>>> we possibly think of this as a next iteration?
>> 
>> This patch introduces a new configuration file
>> (/etc/rdma/rsocket/comp_vector). User expect that such configuration
>> files remain supported after a software upgrade. In other words,
>> introduction of a new configuration file deserves careful
>> consideration. Do we really need this new configuration file ? Why not
>> e.g. to change the default behavior of the rsockets library into
>> spreading the interrupt processing workload over all completion
>> vectors supported by the HCA ? Furthermore, the number of completion
>> vectors that is supported by an HCA not only depends on the HCA
>> firmware but also on the number of completion vectors a HCA could
>> allocate during driver initialization. The number of MSI-X vectors
>> available on an x86-64 system is limited (224). Dual-socket x86-64
>> motherboards with up to 11 PCIe slots are available commercially. If
>> all PCIe-slots are populated it's almost sure that one of the PCIe
>> cards won't be assigned as many MSI-X vectors as it could use.
>> 
>> Sorry but I'm not convinced that using the same completion vector mask
>> for all HCA's and all processes is a good idea. Or is there perhaps
>> something I have overlooked ?
>> 
>> Bart.
> 
> Hi Bart,
> 
> I agree with your suggestions - working on the revised patch 
> accordingly.
> 
> Thank You.
> 
> - Sreedhar
> 

Hi Bart,

I have sent the revised patch v4 that groups and assigns comp vectors 
per process as you suggested.  Please go through it.

Thank You.

- Sreedhar
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" 
> in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html