Re: [PATCH 0/10] fuse: An attempt to implement a write-back cache policy

[Pavel Emelyanov <xemul-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org>]

On 07/05/2012 11:31 PM, Anand Avati wrote:
> Doesn't such a change increase the "dirty debt" held in the kernel and 
> therefore increase the probability of resulting in a deadlock (when the 
> userspace filesystem's memory allocation request ends up waiting on a 
> dirty page writeout caused by this write-back feature)?

That's a very good question! I've tried to address this in the patch #10.
The balance_dirty_pages start throttling task which tries to put too many
dirty pages on a bdi. On FUSE this limit is set to a reasonably low value,
so that the dirty set of a FUSE mount won't grow too high.

> Why not  implement the write-back inside the userspace filesystem leaving the 
> kernel operate in write-through, which makes things overall less unsafe? 

Making yet another cache in the user space is extremely hard, since the
user space app has no idea at all when to start shrinking this cache. Neither
it knows which part of the cache (clean/dirty) is better candidate for that.
On the other hand, the in kernel cache gets shrunk when required and is balanced.

> Do you have performance numbers comparing write-back in kernel v/s 
> write-back in userspace?

I have only very preliminary measurements, I mostly played with the patches
stability. Pushing a big portion of data (4x times bigger than RAM) from one
VM to another via FUSE + networking backend with dd in 4k chunks increases
the reported by dd speed more than twice.

But, Kirill Korotaev (in Cc) was supposed to perform more accurate measurements,
hopefully he can provide the results.

> Avati

Thanks,
Pavel


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/