How to find out, what "pdflush" is working on

How to find out, what "pdflush" is working on

[Martin Knoblauch]

Hi,

 I find the following comment in mm/pdflush.c

/*
 * The pdflush threads are worker threads for writing back dirty data.
 * Ideally, we'd like one thread per active disk spindle.  But the disk
 * topology is very hard to divine at this level.   Instead, we take
 * care in various places to prevent more than one pdflush thread from
 * performing writeback against a single filesystem.  pdflush threads
 * have the PF_FLUSHER flag set in current->flags to aid in this.
 */

 Is there a way to find out what a certain instance of "pdflush" is working on? Like which block-device or which fliesystem it is writing to? I am still (2.6.27) trying to track down why writing a single file can make linux very sluggish and unresponsive. If that happens I usually see all possible 8 "pdflush" threads being in "D"-state. According to above comment only one of them should be really busy.

Cheers
Martin
------------------------------------------------------
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www:   http://www.knobisoft.de

Re: How to find out, what "pdflush" is working on

[Chris Snook]

Martin Knoblauch wrote:
> Hi,
> 
> I find the following comment in mm/pdflush.c
> 
> /* * The pdflush threads are worker threads for writing back dirty data. *
> Ideally, we'd like one thread per active disk spindle.  But the disk *
> topology is very hard to divine at this level.   Instead, we take * care in
> various places to prevent more than one pdflush thread from * performing
> writeback against a single filesystem.  pdflush threads * have the PF_FLUSHER
> flag set in current->flags to aid in this. */
> 
> Is there a way to find out what a certain instance of "pdflush" is working
> on? Like which block-device or which fliesystem it is writing to? I am still
> (2.6.27) trying to track down why writing a single file can make linux very
> sluggish and unresponsive. If that happens I usually see all possible 8
> "pdflush" threads being in "D"-state. According to above comment only one of
> them should be really busy.

The key word is "ideally".  We'd like it to work that way, but it doesn't. 
Patches to fix this are welcome.

-- Chris

Re: How to find out, what "pdflush" is working on

[Martin Knoblauch]

----- Original Message ----

> From: Chris Snook <csnook@redhat.com>
> To: Martin Knoblauch <knobi@knobisoft.de>
> Cc: linux-kernel@vger.kernel.org; Peter zijlstra <a.p.zijlstra@chello.nl>; Fengguang Wu <wfg@mail.ustc.edu.cn>
> Sent: Tuesday, September 16, 2008 9:07:29 PM
> Subject: Re: How to find out, what "pdflush" is working on
> 
> Martin Knoblauch wrote:
> > Hi,
> > 
> > I find the following comment in mm/pdflush.c
> > 
> > /* * The pdflush threads are worker threads for writing back dirty data. *
> > Ideally, we'd like one thread per active disk spindle.  But the disk *
> > topology is very hard to divine at this level.   Instead, we take * care in
> > various places to prevent more than one pdflush thread from * performing
> > writeback against a single filesystem.  pdflush threads * have the PF_FLUSHER
> > flag set in current->flags to aid in this. */
> > 
> > Is there a way to find out what a certain instance of "pdflush" is working
> > on? Like which block-device or which fliesystem it is writing to? I am still
> > (2.6.27) trying to track down why writing a single file can make linux very
> > sluggish and unresponsive. If that happens I usually see all possible 8
> > "pdflush" threads being in "D"-state. According to above comment only one of
> > them should be really busy.
> 
> The key word is "ideally".  We'd like it to work that way, but it doesn't. 
> Patches to fix this are welcome.
> 

 Now, I am pretty much aware that apparently things are not "ideal". Therefore my question. Because before doing patches, one has to analyze.

Cheers
Martin

Re: How to find out, what "pdflush" is working on

[Frank Ch. Eigler]

> [...]
>> Is there a way to find out what a certain instance of "pdflush" is
>> working on? Like which block-device or which fliesystem it is
>> writing to? [...]

Something based upon the following systemtap script may help:

probe kernel.function("__writeback_single_inode") {
  if (execname() == "pdflush")
     printf("pdflush %d writeback bdev %x inode %x\n", tid(), $inode->i_rdev, $inode->i_ino) 
}

Other fields are available as $inode->FOO.

- FChE