All of lore.kernel.org
 help / color / mirror / Atom feed
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
To: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: kosaki.motohiro@jp.fujitsu.com,
	Ben Gamari <bgamari.foss@gmail.com>,
	linux-kernel@vger.kernel.org, rsync@lists.samba.org,
	linux-mm@kvack.org
Subject: Re: fadvise DONTNEED implementation (or lack thereof)
Date: Tue,  9 Nov 2010 17:03:09 +0900 (JST)	[thread overview]
Message-ID: <20101109170303.BC90.A69D9226@jp.fujitsu.com> (raw)
In-Reply-To: <20101109162525.BC87.A69D9226@jp.fujitsu.com>

> > I've recently been trying to track down the root cause of my server's
> > persistent issue of thrashing horribly after being left inactive. It
> > seems that the issue is likely my nightly backup schedule (using rsync)
> > which traverses my entire 50GB home directory. I was surprised to find
> > that rsync does not use fadvise to notify the kernel of its use-once
> > data usage pattern.
> > 
> > It looks like a patch[1] was written (although never merged, it seems)
> > incorporating fadvise support, but I found its implementation rather
> > odd, using mincore() and FADV_DONTNEED to kick out only regions brought
> > in by rsync. It seemed to me the simpler and more appropriate solution
> > would be to simply flag every touched file with FADV_NOREUSE and let the
> > kernel manage automatically expelling used pages.
> > 
> > After looking deeper into the kernel implementation[2] of fadvise() the
> > reason for using DONTNEED became more apparant. It seems that the kernel
> > implements NOREUSE as a noop. A little googling revealed[3] that I not
> > the first person to encounter this limitation. It looks like a few
> > folks[4] have discussed addressing the issue in the past, but nothing
> > has happened as of 2.6.36. Are there plans to implement this
> > functionality in the near future? It seems like the utility of fadvise
> > is severely limited by lacking support for NOREUSE.
> 
> btw, Other OSs seems to also don't implement it.
> example,

I've heared other OSs status of fadvise() from private mail.

NetBSD: no-op (as linux)
FreeBSD/DragonflyBSD/OpenBSD: don't exist posix_fadvise(2)




WARNING: multiple messages have this Message-ID (diff)
From: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
To: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Ben Gamari <bgamari.foss@gmail.com>,
	linux-kernel@vger.kernel.org, rsync@lists.samba.org,
	linux-mm@kvack.org
Subject: Re: fadvise DONTNEED implementation (or lack thereof)
Date: Tue,  9 Nov 2010 17:03:09 +0900 (JST)	[thread overview]
Message-ID: <20101109170303.BC90.A69D9226@jp.fujitsu.com> (raw)
In-Reply-To: <20101109162525.BC87.A69D9226@jp.fujitsu.com>

> > I've recently been trying to track down the root cause of my server's
> > persistent issue of thrashing horribly after being left inactive. It
> > seems that the issue is likely my nightly backup schedule (using rsync)
> > which traverses my entire 50GB home directory. I was surprised to find
> > that rsync does not use fadvise to notify the kernel of its use-once
> > data usage pattern.
> > 
> > It looks like a patch[1] was written (although never merged, it seems)
> > incorporating fadvise support, but I found its implementation rather
> > odd, using mincore() and FADV_DONTNEED to kick out only regions brought
> > in by rsync. It seemed to me the simpler and more appropriate solution
> > would be to simply flag every touched file with FADV_NOREUSE and let the
> > kernel manage automatically expelling used pages.
> > 
> > After looking deeper into the kernel implementation[2] of fadvise() the
> > reason for using DONTNEED became more apparant. It seems that the kernel
> > implements NOREUSE as a noop. A little googling revealed[3] that I not
> > the first person to encounter this limitation. It looks like a few
> > folks[4] have discussed addressing the issue in the past, but nothing
> > has happened as of 2.6.36. Are there plans to implement this
> > functionality in the near future? It seems like the utility of fadvise
> > is severely limited by lacking support for NOREUSE.
> 
> btw, Other OSs seems to also don't implement it.
> example,

I've heared other OSs status of fadvise() from private mail.

NetBSD: no-op (as linux)
FreeBSD/DragonflyBSD/OpenBSD: don't exist posix_fadvise(2)



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2010-11-09  8:03 UTC|newest]

Thread overview: 47+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-11-04  5:58 fadvise DONTNEED implementation (or lack thereof) Ben Gamari
2010-11-04  5:58 ` Ben Gamari
2010-11-06 16:23 ` Wayne Davison
2010-11-09  7:28 ` KOSAKI Motohiro
2010-11-09  7:28   ` KOSAKI Motohiro
2010-11-09  8:03   ` KOSAKI Motohiro [this message]
2010-11-09  8:03     ` KOSAKI Motohiro
2010-11-09 12:54   ` Ben Gamari
2010-11-09 12:54     ` Ben Gamari
2010-11-14  5:09     ` KOSAKI Motohiro
2010-11-14  5:09       ` KOSAKI Motohiro
2010-11-14  5:20       ` Ben Gamari
2010-11-14  5:20         ` Ben Gamari
2010-11-14 21:33         ` Brian K. White
2010-11-14 21:33           ` Brian K. White
2010-11-15  6:07       ` Minchan Kim
2010-11-15  6:07         ` Minchan Kim
2010-11-15  7:09         ` KOSAKI Motohiro
2010-11-15  7:09           ` KOSAKI Motohiro
2010-11-15  7:19           ` Minchan Kim
2010-11-15  7:19             ` Minchan Kim
2010-11-15  7:28             ` KOSAKI Motohiro
2010-11-15  7:28               ` KOSAKI Motohiro
2010-11-15  7:46               ` Minchan Kim
2010-11-15  7:46                 ` Minchan Kim
2010-11-15 12:46               ` Ben Gamari
2010-11-15 12:46                 ` Ben Gamari
2010-11-15  8:47         ` Peter Zijlstra
2010-11-15  8:47           ` Peter Zijlstra
2010-11-15  9:05           ` Minchan Kim
2010-11-15  9:05             ` Minchan Kim
2010-11-15 14:48             ` Rik van Riel
2010-11-15 14:48               ` Rik van Riel
2010-11-17 10:16               ` Minchan Kim
2010-11-17 10:16                 ` Minchan Kim
2010-11-17 11:15                 ` Minchan Kim
2010-11-17 11:15                   ` Minchan Kim
2010-11-17 16:22                 ` Rik van Riel
2010-11-17 16:22                   ` Rik van Riel
2010-11-18  2:47                   ` Minchan Kim
2010-11-18  2:47                     ` Minchan Kim
2010-11-18  3:24                     ` Rik van Riel
2010-11-18  3:24                       ` Rik van Riel
2010-11-18  3:46                       ` Minchan Kim
2010-11-18  3:46                         ` Minchan Kim
2010-11-15  9:10           ` KOSAKI Motohiro
2010-11-15  9:10             ` KOSAKI Motohiro

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20101109170303.BC90.A69D9226@jp.fujitsu.com \
    --to=kosaki.motohiro@jp.fujitsu.com \
    --cc=bgamari.foss@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=rsync@lists.samba.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.