Re: [fuse-devel] 512 byte aligned write + O_DIRECT for xfstests

[Nikolaus Rath <Nikolaus@rath.org>]

On Jun 22 2020, Amir Goldstein <amir73il@gmail.com> wrote:
>> >> > https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_09_07
>> >>
>> >> Thanks for digging this up, I did not know about this.
>> >>
>> >> That leaves FUSE in a rather uncomfortable place though, doesn't it?
>> >> What does the kernel do when userspace issues a write request that's
>> >> bigger than FUSE userspace pipe? It sounds like either the request must
>> >> be splitted (so it becomes non-atomic), or you'd have to return a short
>> >> write (which IIRC is not supposed to happen for local filesystems).
>> >>
>> >
>> > What makes you say that short writes are not supposed to happen?
>>
>> I don't think it was an authoritative source, but I I've repeatedly read
>> that "you do not have to worry about short reads/writes when accessing
>> the local disk". I expect this to be a common expectation to be baked
>> into programs, no matter if valid or not.
>
> Even if that statement would have been considered true, since when can
> we speak of FUSE as a "local filesystem".
> IMO it follows all the characteristics of a "network filesystem".
>
>> > Seems like the options for FUSE are:
>> > - Take shared i_rwsem lock on read like XFS and regress performance of
>> >   mixed rw workload
>> > - Do the above only for non-direct and writeback_cache to minimize the
>> >   damage potential
>> > - Return short read/write for direct IO if request is bigger that FUSE
>> > buffer size
>> > - Add a FUSE mode that implements direct IO internally as something like
>> >   RWF_UNCACHED [2] - this is a relaxed version of "no caching" in client or
>> >   a stricter version of "cache write-through"  in the sense that
>> > during an ongoing
>> >   large write operation, read of those fresh written bytes only is served
>> >   from the client cache copy and not from the server.
>>
>> I didn't understand all of that, but it seems to me that there is a
>> fundamental problem with splitting up a single write into multiple FUSE
>> requests, because the second request may fail after the first one
>> succeeds.
>>
>
> I think you are confused by the use of the word "atomic" in the standard.
> It does not mean what the O_ATOMIC proposal means, that is - write everything
> or write nothing at all.
> It means if thread A successfully wrote data X over data Y, then thread B can
> either read X or Y, but not half X half Y.
> If A got an error on write, the content that B will read is probably undefined
> (excuse me for not reading what "the law" has to say about this).
> If A got a short (half) write, then surely B can read either half X or half Y
> from the first half range. Second half range I am not sure what to expect.
>
> So I do not see any fundamental problem with FUSE write requests.
> On the contrary - FUSE write requests are just like any network protocol write
> request or local disk IO request for that matter.
>
> Unless I am missing something...

Well, you're missing the point I was trying to make, which was that FUSE
is in an unfortunate spot if we want to avoid short writes *and* comply
with the standard. You are asserting that is perfectly fine for FUSE to
return short writes and I agree that in that case there is no problem
with making writes atomic.

I do not dispute that FUSE is within its right to return short
rights. What I am saying is that I'm sure that there are plenty of
userspace applications that don't expect short writes or reads when
reading *any* regular file, because people assume this is only a concern
for fds that represents sockets or pipes. Yes, this is wrong of
them. But it works almost all the time, so it would be unfortunate if it
suddenly stopped working for FUSE in the situations where it previously
worked.


Best,
-Nikolaus

-- 
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«