Re: [ewg] nfsrdma fails to write big file,

[Tom Tucker <tom-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>]

Roland Dreier wrote:
>  > +               /* 
>  > +                * Add room for frmr register and invalidate WRs
>  > +                * Requests sometimes have two chunks, each chunk
>  > +                * requires to have different frmr. The safest
>  > +                * WRs required are max_send_wr * 6; however, we
>  > +                * get send completions and poll fast enough, it
>  > +                * is pretty safe to have max_send_wr * 4. 
>  > +                */
>  > +               ep->rep_attr.cap.max_send_wr *= 4;
>
> Seems like a bad design if there is a possibility of work queue
> overflow; if you're counting on events occurring in a particular order
> or completions being handled "fast enough", then your design is going to
> fail in some high load situations, which I don't think you want.
>
>   

Vu,

Would you please try the following:

- Set the multiplier to 5
- Set the number of buffer credits small as follows "echo 4 > 
/proc/sys/sunrpc/rdma_slot_table_entries"
- Rerun your test and see if you can reproduce the problem?

I did the above and was unable to reproduce, but I would like to see if 
you can to convince ourselves that 5 is the right number.

Thanks,
Tom

>  - R.
>   

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html