Re: [Lsf-pc] [LSF/MM ATTEND] Filesystems -- Btrfs, cgroups, Storage topics from Facebook

[Mel Gorman <mgorman@suse.de>]

On Tue, Dec 31, 2013 at 01:19:15PM +0000, Chris Mason wrote:
> On Tue, 2013-12-31 at 13:45 +0100, Jan Kara wrote:
> > On Tue 31-12-13 16:49:27, Zheng Liu wrote:
> > > Hi Chris,
> > > 
> > > On Mon, Dec 30, 2013 at 09:36:20PM +0000, Chris Mason wrote:
> > > > Hi everyone,
> > > > 
> > > > I'd like to attend the LSF/MM conference this year.  My current
> > > > discussion points include:
> > > > 
> > > > All things Btrfs!
> > > > 
> > > > Adding cgroups for more filesystem resources, especially to limit the
> > > > speed dirty pages are created.
> > > 
> > > Interesting.  If I remember correctly, IO-less dirty throttling has been
> > > applied into upstream kernel, which can limit the speed that dirty pages
> > > are created.  Does it has any defect?
> >   It works as it should. But as Jeff points out, the throttling isn't
> > cgroup aware. So it can happen that one memcg is full of dirty pages and
> > reclaim has problems with reclaiming pages for it. I guess what Chris asks
> > for is that we watch number of dirty pages in each memcg and throttle
> > processes creating dirty pages in memcg which is close to its limit on
> > dirty pages.
> 
> Right, the ioless dirty throttling is fantastic, but it's based on the
> BDI and you only get one of those per device.
> 

It's only partially related but we'll also need to keep in mind that
even with ioless dirty throttling that dirty_ratio and dirty_bytes have
been showing their age for a long time.  dirty_ratio was fine when 20%
of memory was still a few seconds of IO but it has not been the case in a
long time. dirty_bytes is also not a great interface because it ignores the
speed of the underlying device. While proposals to fix it have been raised
in the past, no one (including me) has put themselves on the firing line
to replace that interface with something like dirty_time -- do not dirty
more pages than it takes N seconds to writeback. When/if someone clears
their table sufficiently to tackle that problem they are likely to collide
with any IO controller work.

-- 
Mel Gorman
SUSE Labs