Re: [PATCH 0/3] block I/O when cluster is full

[Josh Durgin <josh.durgin@inktank.com>]

On 12/09/2013 04:19 PM, Gregory Farnum wrote:
> On Mon, Dec 9, 2013 at 4:11 PM, Josh Durgin <josh.durgin@inktank.com> wrote:
>> On 12/06/2013 06:24 PM, Gregory Farnum wrote:
>>>
>>> On Fri, Dec 6, 2013 at 6:16 PM, Josh Durgin <josh.durgin@inktank.com>
>>> wrote:
>>>> Don't bother trying to stop ENOSPC on the client side, since it'd need
>>>> some
>>>> restructuring in the kernel side and would be prone to screwing up
>>>> write ordering.
>>>>
>>>> Instead drop writes on the osd side when they have a map marked full,
>>>> and have clients resend all writes when a map goes transitions from
>>>> full -> nonfull. The userspace side is
>>>> https://github.com/ceph/ceph/pull/914
>>>
>>>
>>> Do previous client implementations already satisfy that requirement?
>>> We can't drop requests if older clients expect a response...
>>
>>
>> No, previous clients do not do this. For old rbd clients, this turns a
>> potential corruption into a hang, which is a good trade-off imo.
>>
>> For userspace clients, this only happens when the osd gets the FULL map
>> first, and rejects a write in flight before the client got a FULL map.
>>
>> The kernel client already rejects writes at the fs layer when the FULL
>> flag is set, so kcephfs will only be affected when it hits this race as
>> well.
>
> Hrm, do we have mechanisms in the kernel for re-sending ops that are
> waiting? Hanging instead of corrupting doesn't help us much if we have
> no way to get the proper state ondisk.

Yes, that's what 1/3 uses.