linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Mark Fasheh <mfasheh@suse.de>
To: linux-btrfs@vger.kernel.org
Cc: David Sterba <dsterba@suse.com>, Chris Mason <clm@fb.com>
Subject: [PATCH] btrfs-progs: add 'du' command
Date: Wed, 20 Jan 2016 13:49:24 -0800	[thread overview]
Message-ID: <1453326567-20454-1-git-send-email-mfasheh@suse.de> (raw)

Hi,

This patch adds a 'du' subcommand to btrfs. 'btrfs fi du' will
calculate disk usage of the target files using fiemap. For individual
files, it will report a count of total bytes, and exclusive (not
shared) bytes. We also calculate a 'set shared' value which is
described below.

Each argument to 'btrfs fi du' will have a 'set shared' value
calculated for it. We define each 'set' as those files found by a
recursive search of an argument to btrfs fi du. The 'set shared' value
then is a sum of all shared space referenced by the set.

'set shared' takes into account overlapping shared extents, hence it
isn't as simple as adding up shared extents. To efficiently find
overlapping regions, we store them in an interval tree. When the scan
of a file set is complete, we can walk the tree and calculate our
actual shared bytes while also taking into account any duplicate or
overlapping extents.

The interval tree implementation is taken from Linux v4.0. I went
ahead and made some small comment updates to rbtree.h and
rbtree_augmented.h while I was importing this code as both are used by
the interval tree and I needed to check for any code changes in those
headers.

Following this paragraph is a very simple example. I started with a
clean btrfs fs in which i copied vmlinuz from /boot. I then made a
snapshot of the fs root in 'snap1'. After the snapshot, I made a 2nd
copy of vmlinuz into the main fs to give us some not-shared data. The
output below shows a sum of all the space, and a 'set shared' with len
exactly equal to that of the single shared file.

# btrfs fi du .
total	exclusive	set shared	filename
76386304	0			./vmlinuz
76386304	0			./snap1/vmlinuz
76386304	0			./snap1
0	0			./vmlinuz.copy
152772608	0	76386304	.


A git tree of the patches can be found here:

https://github.com/markfasheh/btrfs-progs-patches/tree/du

or if you prefer to pull:

git pull https://github.com/markfasheh/btrfs-progs-patches du

Comments/feedback appreciated.
    --Mark

             reply	other threads:[~2016-01-20 21:49 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-01-20 21:49 Mark Fasheh [this message]
2016-01-20 21:49 ` [PATCH 1/3] btrfs-progs: Import interval tree implemenation from Linux v4.0-rc7 Mark Fasheh
2016-01-20 21:49 ` [PATCH 2/3] btrfs-progs: add 'du' command Mark Fasheh
2016-02-25  8:57   ` Jérôme Poulin
2016-02-25  9:25     ` David Sterba
2016-01-20 21:49 ` [PATCH 3/3] btrfs-progs du: Calculate space shared by each directory arguments file set Mark Fasheh
2016-02-01 23:43 ` [PATCH] btrfs-progs: add 'du' command David Sterba
2016-02-02 20:09   ` Mark Fasheh
2016-02-24 15:28     ` David Sterba

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1453326567-20454-1-git-send-email-mfasheh@suse.de \
    --to=mfasheh@suse.de \
    --cc=clm@fb.com \
    --cc=dsterba@suse.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).