Re: [RFC 0/8] Reliably generate large request from SRP

David Dillow
2011-01-19T12:01:17Z

On Tue, 2011-01-18 at 21:31 -0800, Roland Dreier wrote:
> Hi Dave,
>
> > Now that at least one vendor is implementing full support for the SRP
> > indirect memory descriptor tables, we can safely expand the sg_tablesize,
> > and realize some performance gains, in many cases quite large. I don't
> > have vendor code that implements the full support needed for safety, but
> > the rareness of FMR mapping failures allows the mapping code to function,
> > at a risk, with existing targets.
>
> Have you considered using memory registration through a send queue (from
> the base memory management extensions)? mlx4 at least has support for
> this operation, which would let you pre-allocate everything and avoid
> the possibility of failure (I think).

I'd rather keep it to one code path if possible, rather than mix FMRs
and BMM. I also need this to work on mthca cards, though I haven't
checked if they support it.

Still, I'll check into it; avoiding the need for vendor support would be
nice.

> When do we get FMR mapping failures now?

I don't know that we ever do, but reading through the code in fmr_pool.c
and the HW implementations, there is no telling what goes on in ehca
since it is hidden in the hypervisor, or so it appeared to me last I
looked. There seemed to be error cases elsewhere, but they should also
be quite rare.
--
Dave Dillow
National Center for Computational Science
Oak Ridge National Laboratory
(865) 241-6602 office