linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Sanjoy Mahajan <sanjoy@olin.edu>
To: Chris Mason <chris.mason@oracle.com>
Cc: Ingo Molnar <mingo@elte.hu>, Pekka Enberg <penberg@kernel.org>,
	Aidar Kultayev <the.aidar@gmail.com>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Jens Axboe <axboe@kernel.dk>,
	Peter.Zijl@MIT.EDU
Subject: Re: 2.6.36 io bring the system to its knees
Date: Tue, 2 Nov 2010 07:47:15 -0400	[thread overview]
Message-ID: <E1PDFKe-0005sq-2D@approx.mit.edu> (raw)
In-Reply-To: Your message of "Thu, 28 Oct 2010 13:01:32 EDT." <20101028170132.GY27796@think>

Chris Mason <chris.mason@oracle.com> wrote:

> > This has the appearance of some really bad IO or VM latency
> > problem. Unfixed and present in stable kernel versions going from
> > years ago all the way to v2.6.36.
> 
> Hmmm, the workload you're describing here has two special parts.
> First it dramatically overloads the disk, and then it has guis doing
> things waiting for the disk.

I think I see this same issue every few days when I back up my hard
drive to a USB hard drive using rsync.  While the backup is running, the
interactive response is bad.  A reproducible measurement of the badness
is starting an rxvt with F8 (bound to "rxvt &" in my .twmrc).  Often it
takes 8 seconds for the window to appear (as it just did about 2 minutes
ago)!  (Starting a subsequent rxvt is quick.)

The command for running the backup:

  rsync -av --delete /etc /home /media/usbdrive/bak > /tmp/homebackup.log

The hardware is a T60 w/ Intel graphics and wireless, 1.5GB RAM, 5400rpm
160GB harddrive w/ ext3 filesystems, and it's running vanilla 2.6.36.
There's not much memory pressure.  The swap is mostly empty, and there's
usually a Firefox eating 500MB of RAM.  Even Emacs at 50MB is in the
noise compared to the Firefox.

Here's the 'free' output:

             total       used       free     shared    buffers     cached
Mem:       1545292    1500288      45004          0      92848     713988
-/+ buffers/cache:     693452     851840
Swap:      2000088      22680    1977408

What tests or probes are worth running when the problem reappears in
order to find the root cause?

-Sanjoy

`Until lions have their historians, tales of the hunt shall always
 glorify the hunters.'  --African Proverb

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2010-11-02 11:47 UTC|newest]

Thread overview: 65+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <AANLkTimt7wzR9RwGWbvhiOmot_zzayfCfSh_-v6yvuAP@mail.gmail.com>
     [not found] ` <AANLkTikRKVBzO=ruy=JDmBF28NiUdJmAqb4-1VhK0QBX@mail.gmail.com>
     [not found]   ` <AANLkTinzJ9a+9w7G5X0uZpX2o-L8E6XW98VFKoF1R_-S@mail.gmail.com>
2010-10-28  6:09     ` 2.6.36 io bring the system to its knees Aidar Kultayev
2010-10-28  6:32       ` Pekka Enberg
2010-10-28  9:00         ` Ingo Molnar
2010-10-28  9:34           ` Pekka Enberg
2010-10-28 11:16           ` Pekka Enberg
2010-10-28 11:33             ` Aidar Kultayev
2010-10-28 11:48               ` Pekka Enberg
2010-10-28 12:18                 ` Aidar Kultayev
2010-10-28 13:46                 ` Christoph Hellwig
2010-10-28 13:54                   ` Ingo Molnar
2010-10-28 13:30             ` Ingo Molnar
2010-10-28 13:47               ` Christoph Hellwig
2010-10-28 13:50                 ` Ingo Molnar
2010-10-28 17:01               ` Chris Mason
2010-10-28 17:57                 ` Pekka Enberg
2010-10-29 14:52                   ` Ted Ts'o
2010-10-29 15:33                     ` Aidar Kultayev
2010-10-30  9:14                       ` Ingo Molnar
2010-10-30 13:02                         ` Aidar Kultayev
2010-10-30 19:06                           ` Chris Mason
2010-10-31  2:31                           ` Ted Ts'o
2010-10-31 17:49                             ` Corrado Zoccolo
2010-11-02  3:10                           ` Shaohua Li
2010-11-02 11:47                 ` Sanjoy Mahajan [this message]
2010-11-02 13:12                   ` Chris Mason
2010-11-04 16:05                     ` Sanjoy Mahajan
2010-11-04 23:35                       ` Steven Barrett
2010-11-04 23:44                 ` Jesper Juhl
2010-11-04 23:48                   ` Jesper Juhl
2010-11-05  1:43                     ` Dave Chinner
2010-11-05 12:48                       ` Sanjoy Mahajan
2010-11-06 14:10                         ` dave b
2010-11-06 15:12                           ` Dave Chinner
2010-11-07  6:06                             ` dave b
2010-11-07 12:08                           ` Jens Axboe
2010-11-07 15:50                             ` Linus Torvalds
2010-11-10  1:32                               ` Dave Chinner
2010-11-10  2:01                                 ` dave b
2010-11-10  8:08                                 ` Evgeniy Ivanov
2010-11-10  8:24                                   ` Dave Chinner
2010-11-10 14:22                                     ` Pavel Machek
2010-11-10 14:20                                 ` Pavel Machek
2010-11-10 14:27                                   ` Ingo Molnar
2010-11-10 14:55                                     ` Christoph Hellwig
2010-11-10 19:09                                       ` Pavel Machek
2010-11-10 14:33                                 ` Theodore Tso
2010-11-10 14:57                                   ` Christoph Hellwig
2010-11-10 15:00                                     ` Chris Mason
2010-11-10 23:36                                   ` Dave Chinner
2010-11-10 15:59                                 ` Linus Torvalds
2010-11-10 16:46                                   ` Alexey Dobriyan
2010-11-10 16:55                                     ` Linus Torvalds
2010-11-10 17:10                                       ` Alexey Dobriyan
2010-11-10 18:55                                         ` Mark Lord
2010-11-10 18:27                                     ` Mike Galbraith
2010-11-10 23:43                                   ` Dave Chinner
2010-11-06 19:10                         ` Arjan van de Ven
2010-11-07 17:16                       ` Jesper Juhl
2010-11-09 19:47                         ` Evgeniy Ivanov
2010-11-09 20:20                           ` Christoph Hellwig
2010-11-09 21:00                       ` Chris Mason
2010-10-31  1:22       ` Wu Fengguang
2010-10-31  1:51         ` Wu Fengguang
2010-11-01  1:09           ` Dimitrios Apostolou
2010-11-02  1:20             ` Wu Fengguang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=E1PDFKe-0005sq-2D@approx.mit.edu \
    --to=sanjoy@olin.edu \
    --cc=Peter.Zijl@MIT.EDU \
    --cc=akpm@linux-foundation.org \
    --cc=axboe@kernel.dk \
    --cc=chris.mason@oracle.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mingo@elte.hu \
    --cc=penberg@kernel.org \
    --cc=the.aidar@gmail.com \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).