All of lore.kernel.org
 help / color / mirror / Atom feed
From: zwu.kernel@gmail.com
To: linux-fsdevel@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, linux-btrfs@vger.kernel.org,
	linux-ext4@vger.kernel.org, linuxram@linux.vnet.ibm.com,
	viro@zeniv.linux.org.uk, cmm@us.ibm.com, tytso@mit.edu,
	marco.stornelli@gmail.com, david@fromorbit.com,
	stroetmann@ontolinux.com, diegocg@gmail.com, chris@csamuel.org,
	Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
Subject: [RFC v2 10/10] vfs: add documentation
Date: Sun, 23 Sep 2012 20:56:35 +0800	[thread overview]
Message-ID: <1348404995-14372-11-git-send-email-zwu.kernel@gmail.com> (raw)
In-Reply-To: <1348404995-14372-1-git-send-email-zwu.kernel@gmail.com>

From: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>

Signed-off-by: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
---
 Documentation/filesystems/hot_tracking.txt |  106 ++++++++++++++++++++++++++++
 1 files changed, 106 insertions(+), 0 deletions(-)
 create mode 100644 Documentation/filesystems/hot_tracking.txt

diff --git a/Documentation/filesystems/hot_tracking.txt b/Documentation/filesystems/hot_tracking.txt
new file mode 100644
index 0000000..340df45
--- /dev/null
+++ b/Documentation/filesystems/hot_tracking.txt
@@ -0,0 +1,106 @@
+Hot Data Tracking
+
+Introduction
+-------------------------------------------------------------------------------
+
+  The feature adds experimental support for tracking data temperature
+information in VFS layer.  Essentially, this means maintaining some key
+stats(like number of reads/writes, last read/write time, frequency of
+reads/writes), then distilling those numbers down to a single
+"temperature" value that reflects what data is "hot," and using that
+temperature to move data to SSDs.
+
+  The long-term goal of the feature is to allow some FSs,
+e.g. Btrfs to intelligently utilize SSDs in a heterogenous volume.
+Incidentally, this project has been motivated by
+the Project Ideas page on the Btrfs wiki.
+
+  Of course, users are warned not to run this code outside of development
+environments. These patches are EXPERIMENTAL, and as such they might eat
+your data and/or memory. That said, the code should be relatively safe
+when the hottrack mount option are disabled.
+
+Motivation
+-------------------------------------------------------------------------------
+
+  The overall goal of enabling hot data relocation to SSD has been
+motivated by the Project Ideas page on the Btrfs wiki at
+<https://btrfs.wiki.kernel.org/index.php/Project_ideas>.
+It will divide into two steps. VFS provide hot data tracking function
+while specific FS will provide hot data relocation function.
+So as the first step of this goal, it is hoped that the patchset
+for hot data tracking will eventually mature into VFS.
+
+  This is essentially the traditional cache argument: SSD is fast and
+expensive; HDD is cheap but slow. ZFS, for example, can already take
+advantage of SSD caching. Btrfs should also be able to take advantage of
+hybrid storage without many broad, sweeping changes to existing code.
+
+Main Parts Description
+-------------------------------------------------------------------------------
+
+These include the following parts:
+    * Hooks in existing vfs functions to track data access frequency
+    * New rbtrees for tracking access frequency of inodes and sub-file
+ranges (hot_rb.c)
+    The relationship between super_block and rbtree is as below:
+super_block->s_hotinfo.hot_inode_tree
+    In include/linux/fs.h, one struct hot_info s_hotinfo is added to
+super_block struct. Each FS instance can find hot tracking info
+s_hotinfo via its super_block. In this hot_info, it store a lot of hot
+tracking info such as hot_inode_tree, inode and range hash list, etc.
+    * A hash list for indexing data by its temperature (hot_hash.c)
+    * A debugfs interface for dumping data from the rbtrees (hot_debugfs.c)
+    * A background kthread for updating inode heat info
+    * Mount options for enabling temperature tracking(-o hottrack,
+default mean disabled) (hot_track.c)
+    * An ioctl to retrieve the frequency information collected for a certain
+file
+    * Ioctls to enable/disable frequency tracking per inode.
+
+Git Development Tree
+-------------------------------------------------------------------------------
+
+  The feature is still on development and review, so if you're interested,
+you can pull from the git repository at the following location:
+  https://github.com/wuzhy/kernel.git hot_tracking
+  git://github.com/wuzhy/kernel.git hot_tracking
+
+Usage Example
+-------------------------------------------------------------------------------
+To use hot tracking, you should mount like this:
+
+$ mount -o hottrack /dev/sdb /mnt
+[ 1505.894078] device label test devid 1 transid 29 /dev/sdb
+[ 1505.952977] btrfs: disk space caching is enabled
+[ 1506.069678] vfs: turning on hot data tracking
+
+Mount debugfs at first:
+
+$ mount -t debugfs none /sys/kernel/debug
+$ ls -l /sys/kernel/debug/vfs_hotdata/
+total 0
+drwxr-xr-x 2 root root 0 Aug  8 04:40 sdb
+$ ls -l /sys/kernel/debug/vfs_hotdata/sdb
+total 0
+-rw-r--r-- 1 root root 0 Aug  8 04:40 inode_data
+-rw-r--r-- 1 root root 0 Aug  8 04:40 range_data
+
+View information about hot tracking from debugfs:
+
+$ echo "hot tracking test" > /mnt/file
+$ cat /sys/kernel/debug/hot_track/sdb/inode_data
+inode #279, reads 0, writes 1, avg read time 18446744073709551615,
+avg write time 5251566408153596, temp 109
+$ cat /sys/kernel/debug/hot_track/sdb/range_data
+inode #279, range start 0 (range len 1048576) reads 0, writes 1,
+avg read time 18446744073709551615, avg write time 1128690176623144209, temp 64
+
+$ echo "hot data tracking test" >> /mnt/file
+$ cat /sys/kernel/debug/hot_track/sdb/inode_data
+inode #279, reads 0, writes 2, avg read time 18446744073709551615,
+avg write time 4923343766042451, temp 109
+$ cat /sys/kernel/debug/hot_track/sdb/range_data
+inode #279, range start 0 (range len 1048576) reads 0, writes 2,
+avg read time 18446744073709551615, avg write time 1058147040842596150, temp 64
+
-- 
1.7.6.5


      parent reply	other threads:[~2012-09-23 12:58 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-09-23 12:56 [RFC v2 00/10] vfs: hot data tracking zwu.kernel
2012-09-23 12:56 ` [RFC v2 01/10] vfs: introduce private rb structures zwu.kernel
2012-09-25  7:37   ` Dave Chinner
2012-09-25  7:57     ` Zhi Yong Wu
2012-09-25  8:00     ` Zhi Yong Wu
2012-09-25 10:20   ` Ram Pai
2012-09-26  3:20     ` Zhi Yong Wu
2012-09-23 12:56 ` [RFC v2 02/10] vfs: add support for updating access frequency zwu.kernel
2012-09-25  9:17   ` Dave Chinner
2012-09-26  2:53     ` Zhi Yong Wu
2012-09-27  2:19       ` Dave Chinner
2012-09-27  2:30         ` Zhi Yong Wu
2012-09-23 12:56 ` [RFC v2 03/10] vfs: add one new mount option '-o hottrack' zwu.kernel
2012-09-25  9:28   ` Dave Chinner
2012-09-26  2:56     ` Zhi Yong Wu
2012-09-27  2:20       ` Dave Chinner
2012-09-27  2:30         ` Zhi Yong Wu
2012-09-27  5:25     ` Zhi Yong Wu
2012-09-27  7:05       ` Dave Chinner
2012-09-27  7:21         ` Zhi Yong Wu
2012-09-23 12:56 ` [RFC v2 04/10] vfs: add init and exit support zwu.kernel
2012-09-27  2:27   ` Dave Chinner
2012-09-23 12:56 ` [RFC v2 05/10] vfs: introduce one hash table zwu.kernel
2012-09-25  9:54   ` Ram Pai
2012-09-26  4:08     ` Zhi Yong Wu
2012-09-27  3:43   ` Dave Chinner
2012-09-27  6:23     ` Zhi Yong Wu
2012-09-27  6:57       ` Dave Chinner
2012-09-27  7:10         ` Zhi Yong Wu
2012-09-23 12:56 ` [RFC v2 06/10] vfs: enable hot data tracking zwu.kernel
2012-09-27  3:54   ` Dave Chinner
2012-09-27  6:28     ` Zhi Yong Wu
2012-09-27  6:59       ` Dave Chinner
2012-09-27  7:12         ` Zhi Yong Wu
2012-09-23 12:56 ` [RFC v2 07/10] vfs: fork one kthread to update data temperature zwu.kernel
2012-09-27  4:03   ` Dave Chinner
2012-09-27  6:54     ` Zhi Yong Wu
2012-09-27  7:01       ` Dave Chinner
2012-09-27  7:19         ` Zhi Yong Wu
2012-09-23 12:56 ` [RFC v2 08/10] vfs: add 3 new ioctl interfaces zwu.kernel
2012-09-23 12:56 ` [RFC v2 09/10] vfs: add debugfs support zwu.kernel
2012-09-23 12:56 ` zwu.kernel [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1348404995-14372-11-git-send-email-zwu.kernel@gmail.com \
    --to=zwu.kernel@gmail.com \
    --cc=chris@csamuel.org \
    --cc=cmm@us.ibm.com \
    --cc=david@fromorbit.com \
    --cc=diegocg@gmail.com \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linuxram@linux.vnet.ibm.com \
    --cc=marco.stornelli@gmail.com \
    --cc=stroetmann@ontolinux.com \
    --cc=tytso@mit.edu \
    --cc=viro@zeniv.linux.org.uk \
    --cc=wuzhy@linux.vnet.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.