Re: [PATCH 2/2 v2] writeback: Add writeback stats for pages written

[Jan Kara <jack@suse.cz>]

On Mon 15-08-11 10:16:38, Curt Wohlgemuth wrote:
> On Mon, Aug 15, 2011 at 6:48 AM, Wu Fengguang <fengguang.wu@intel.com> wrote:
> > Curt,
> >
> > Some thoughts about the interface..before dipping into the code.
> >
> > On Sat, Aug 13, 2011 at 06:47:25AM +0800, Curt Wohlgemuth wrote:
> >> Add a new file, /proc/writeback/stats, which displays
> >
> > That's creating a new top directory in /proc. Do you have plans for
> > adding more files under it?
> 
> Good question.  We have several files under /proc/writeback in our
> kernels that we created at various times, some of which are probably
> no longer useful, but others seem to be.  For example:
>   - congestion: prints # of calls, # of jiffies slept in
> congestion_wait() / io_schedule_timeout() from various call points
>   - threshold_dirty : prints the current global FG threshold
>   - threshold_bg : prints the current global BG threshold
>   - pages_cleaned : prints the # pages sent to writeback -- same as
> 'nr_written' in /proc/vmstat (ours was earlier :-( )
>   - pages_dirtied (same as nr_dirtied in /proc/vmstat)
>   - prop_vm_XXX : print shift/events from vm_completions and vm_dirties
>
> I'm not sure right now if global FG/BG thresholds appear anywhere in a
> 3.1 kernel; if so, the two threshold files above are superfluous.  So
> are the pages_cleaned/dirtied.  The prop_vm files have not proven
> useful to me.  I think the congestion file has a lot of value,
> especially in an IO-less throttling world...
  /sys/kernel/debug/bdi/<dev>/stats has BdiDirtyThresh, DirtyThresh, and
BackgroundThresh. So we should already expose all you have in the threshold
files.

Regarding congestion_wait() statistics - do I get right that the numbers
gathered actually depend on the number of threads using the congested
device? They are something like
  \sum_{over threads} time_waited_for_bdi
How do you interpret the resulting numbers then?

								Honza

> >> machine global data for how many pages were cleaned for
> >> which reasons.  It also displays some additional counts for
> >> various writeback events.
> >>
> >> These data are also available for each BDI, in
> >> /sys/block/<device>/bdi/writeback_stats .
> >
> >> Sample output:
> >>
> >>    page: balance_dirty_pages           2561544
> >>    page: background_writeout              5153
> >>    page: try_to_free_pages                   0
> >>    page: sync                                0
> >>    page: kupdate                        102723
> >>    page: fdatawrite                    1228779
> >>    page: laptop_periodic                     0
> >>    page: free_more_memory                    0
> >>    page: fs_free_space                       0
> >>    periodic writeback                      377
> >>    single inode wait                         0
> >>    writeback_wb wait                         1
> >
> > That's already useful data, and could be further extended (in
> > future patches) to answer questions like "what's the writeback
> > efficiency in terms of effective chunk size?"
> >
> > So in future there could be lines like
> >
> >    pages: balance_dirty_pages           2561544
> >    chunks: balance_dirty_pages          XXXXXXX
> >    works: balance_dirty_pages           XXXXXXX
> >
> > or even derived lines like
> >
> >    pages_per_chunk: balance_dirty_pages         XXXXXXX
> >    pages_per_work: balance_dirty_pages          XXXXXXX
> >
> > Another question is, how can the display format be script friendly?
> > The current form looks not easily parse-able at least for "cut"..
> 
> I suppose you mean because of the variable number of tokens.  Yeah,
> this can be hard.  Of course, I always just use "awk '{print $NF}'"
> and it works for me :-) .  But I'd be happy to change these to use a
> consistent # of args.
> 
> Thanks,
> Curt
> 
> 
> > Thanks,
> > Fengguang
> >
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html