public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* [PATCH 00/16] Adaptive read-ahead V7
@ 2005-11-09 13:49 Wu Fengguang
  2005-11-09 13:49 ` [PATCH 01/16] mm: delayed page activation Wu Fengguang
                   ` (16 more replies)
  0 siblings, 17 replies; 53+ messages in thread
From: Wu Fengguang @ 2005-11-09 13:49 UTC (permalink / raw)
  To: linux-kernel; +Cc: Andrew Morton

This is the 7th version of adaptive read-ahead patch.

There are various code cleanups and polish ups:
- new tunable parameters: readahead_hit_rate/readahead_live_chunk
- support sparse sequential accesses
- delay look-ahead in laptop mode
- disable look-ahead for loopback file
- make mandatory thrashing protection more simple and robust
- attempt to improve responsiveness on large I/O request size

Support for sparse reads is disabled by default. One must increase
/proc/sys/vm/readahead_hit_rate to explicitly enable it. Please
refer to Documentation/sysctl/vm.txt for details.

Currently the linux kernel does not support inter-file read-ahead.
Tero Grundstr?m takes an intresting approach that achieves it: pack
a dir of small files into a loopback file with reiserfs filesystem, and
turn on sparse read support. But be prepared to waste some memory by
this way :(

For crazy laptop users who prefer aggressive read-ahead, here is the way:

# echo 10000 > /proc/sys/vm/readahead_ratio
# blockdev --setra 524280 /dev/hda      # this is the max possible value

Notes:
- It is still an untested feature.
- It is safer to use blockdev+fadvise to increase ra-max for a single file,
  which needs patching your movie player.
- Be sure to restore them to sane values in normal operations!

Regards,
Wu

^ permalink raw reply	[flat|nested] 53+ messages in thread
* [PATCH 00/16] Adaptive read-ahead V9
@ 2005-12-03  7:14 Wu Fengguang
  2005-12-03  7:14 ` [PATCH 01/16] mm: delayed page activation Wu Fengguang
  0 siblings, 1 reply; 53+ messages in thread
From: Wu Fengguang @ 2005-12-03  7:14 UTC (permalink / raw)
  To: linux-kernel; +Cc: Andrew Morton

The current read-ahead logic uses an inflexible algorithm with 128KB
VM_MAX_READAHEAD. Less memory leads to thrashing, more memory helps no
throughput. The new logic is simply safer and faster. It makes sure
every single read-ahead request is safe for the current load. Memory
tight systems are expected to benefit a lot: no thrashing any more.
It can also help boost I/O throughput for large memory systems, for
VM_MAX_READAHEAD now defaults to 1MB. The value is no longer tightly
coupled with the thrashing problem, and therefore constrainted by it.

Changelog
=========

V9  2005-12-3

- standalone mmap read-around code, a little more smart and tunable
- make stateful method sensible of request size
- decouple readahead_ratio from live pages protection
- let readahead_ratio contribute to ra_size grow speed in stateful method
- account variance of ra_size

V8  2005-11-25

- balance zone aging only in page relaim paths and do it right
- do the aging of slabs in the same way as zones
- add debug code to dump the detailed page reclaim steps
- undo exposing of struct radix_tree_node and uninline related functions
- work better with nfsd
- generalize accelerated context based read-ahead
- account smooth read-ahead aging based on page referenced/activate bits
- avoid divide error in compute_thrashing_threshold()
- more low lantency efforts
- update some comments
- rebase debug actions on debugfs entries instead of magic readahead_ratio values

V7  2005-11-09

- new tunable parameters: readahead_hit_rate/readahead_live_chunk
- support sparse sequential accesses
- delay look-ahead if drive is spinned down in laptop mode
- disable look-ahead for loopback file
- make mandatory thrashing protection more simple and robust
- attempt to improve responsiveness on large read-ahead size

V6  2005-11-01

- cancel look-ahead in laptop mode
- increase read-ahead limit to 0xFFFF pages

V5  2005-10-28

- rewrite context based method to make it clean and robust
- improved accuracy of stateful thrashing threshold estimation
- make page aging equal to the number of code pages scanned
- sort out the thrashing protection logic
- enhanced debug/accounting facilities

V4  2005-10-15

- detect and save live chunks on page reclaim
- support database workload
- support reading backward
- radix tree lookup look-aside cache

V3  2005-10-06

- major code reorganization and documention
- stateful estimation of thrashing-threshold
- context method with accelerated grow up phase
- adaptive look-ahead
- early detection and rescue of pages in danger
- statitics data collection
- synchronized page aging between zones

V2  2005-09-15

- delayed page activation
- look-ahead: towards pipelined read-ahead

V1  2005-09-13

Initial release which features:
        o stateless (for now)
        o adapts to available memory / read speed
        o free of thrashing (in theory)

And handles:
        o large number of slow streams (FTP server)
	o open/read/close access patterns (NFS server)
        o multiple interleaved, sequential streams in one file
	  (multithread / multimedia / database)


Thanks,
Wu Fengguang
--
Dept. Automation                University of Science and Technology of China

^ permalink raw reply	[flat|nested] 53+ messages in thread

end of thread, other threads:[~2005-12-07 13:26 UTC | newest]

Thread overview: 53+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-11-09 13:49 [PATCH 00/16] Adaptive read-ahead V7 Wu Fengguang
2005-11-09 13:49 ` [PATCH 01/16] mm: delayed page activation Wu Fengguang
2005-11-10  0:21   ` Nick Piggin
2005-11-10  3:15     ` Wu Fengguang
2005-11-10  9:17   ` Peter Zijlstra
2005-11-10 10:30     ` Wu Fengguang
2005-11-09 13:49 ` [PATCH 02/16] mm: balance page aging between zones Wu Fengguang
2005-11-09 13:49 ` [PATCH 03/16] radixtree: sync with mainline Wu Fengguang
2005-11-09 13:49 ` [PATCH 04/16] radix-tree: look-aside cache Wu Fengguang
2005-11-09 23:31   ` Nick Piggin
2005-11-10  5:25     ` Wu Fengguang
2005-11-10  6:50       ` Nick Piggin
2005-11-10  8:30         ` Wu Fengguang
2005-11-18 11:25         ` Wu Fengguang
2005-11-18 12:12           ` Wu Fengguang
2005-11-09 13:49 ` [PATCH 05/16] readahead: some preparation Wu Fengguang
2005-11-18  7:46   ` 2.6.15-rc1-mm2 Andrew Morton
2005-11-18  8:56     ` 2.6.15-rc1-mm2 Benoit Boissinot
2005-11-18  9:04       ` 2.6.15-rc1-mm2 Andrew Morton
2005-11-18  9:13         ` 2.6.15-rc1-mm2 Benoit Boissinot
2005-11-18 13:43         ` 2.6.15-rc1-mm2 Rafael J. Wysocki
2005-11-18 10:10     ` 2.6.15-rc1-mm2 Mauro Carvalho Chehab
2005-11-18 10:55     ` 2.6.15-rc1-mm2 Wu Fengguang
2005-11-18 11:29     ` 2.6.15-rc1-mm2 Andy Whitcroft
2005-11-18 16:29     ` 2.6.15-rc1-mm2 Michael Krufky
2005-11-20  0:23     ` 2.6.15-rc1-mm2 Michal Piotrowski
2005-11-20  8:04       ` 2.6.15-rc1-mm2 Hugh Dickins
2005-11-20 12:53         ` 2.6.15-rc1-mm2 Michal Piotrowski
2005-11-09 13:49 ` [PATCH 06/16] readahead: call scheme Wu Fengguang
2005-11-09 13:49 ` [PATCH 07/16] readahead: tunable parameters Wu Fengguang
2005-11-09 13:49 ` [PATCH 08/16] readahead: state based method Wu Fengguang
2005-11-09 13:49 ` [PATCH 09/16] readahead: context " Wu Fengguang
2005-11-09 13:49 ` [PATCH 10/16] readahead: other methods Wu Fengguang
2005-11-09 13:49 ` [PATCH 11/16] readahead: mandatory thrashing protection Wu Fengguang
2005-11-09 13:49 ` [PATCH 12/16] readahead: events accounting Wu Fengguang
2005-11-09 13:49 ` [PATCH 13/16] readahead: page aging accounting Wu Fengguang
2005-11-09 13:49 ` [PATCH 14/16] readahead: laptop mode support Wu Fengguang
2005-11-09 13:49 ` [PATCH 15/16] readahead: disable look-ahead for loopback file Wu Fengguang
2005-11-09 13:49 ` [PATCH 16/16] io: reduce lantency Wu Fengguang
2005-11-09 20:39 ` [PATCH 00/16] Adaptive read-ahead V7 Christoph Lameter
2005-11-10 10:19   ` Wu Fengguang
  -- strict thread matches above, loose matches on Subject: below --
2005-12-03  7:14 [PATCH 00/16] Adaptive read-ahead V9 Wu Fengguang
2005-12-03  7:14 ` [PATCH 01/16] mm: delayed page activation Wu Fengguang
2005-12-04 12:11   ` Nikita Danilov
2005-12-04 13:48     ` Wu Fengguang
2005-12-04 15:03       ` Nikita Danilov
2005-12-04 19:10         ` Peter Zijlstra
2005-12-05  1:48         ` Wu Fengguang
2005-12-06 17:55           ` Nikita Danilov
2005-12-07  1:42             ` Wu Fengguang
2005-12-07  9:46               ` Andrew Morton
2005-12-07 10:36                 ` Wu Fengguang
2005-12-07 12:44               ` Nikita Danilov
2005-12-07 13:53                 ` Wu Fengguang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox