linux-ext4.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Shapor Naghibzadeh <shapor@shapor.com>
To: linux-ext4@vger.kernel.org
Subject: poor performance of mount due to libblkid
Date: Wed, 9 May 2007 17:06:46 -0500	[thread overview]
Message-ID: <20070509170646.C12805@cbr.shaptech.com> (raw)


There is a serious performance degradation with the mount command after
mounting many unique devices when compiled with libblkid support.  A simple
"mount" command to display the list of mounted filesystem can take minutes to
run.  This is due to a call to libblkid's blkid_get_cache and a
relatively large /etc/blkid.tab (tens of thousands of lines, a few megs in
size).

The file is able to grow to a large size because it does not know that device
mapper devices have been removed and will never be created again.  Neither the
libblkid api nor blkid command line appear to even provide a facility for
removing entries if you wanted to do so manually on device removal.

Combined with no (reasonable) bound on the size of the blkid.tab file, this
causes the mount command to get slower over time.  To make matters worse, the
cost of reading the file in to memory is n-squared (which happens every time
the mount command is run, even with "-h" for help!).  I have provided a simple
shell script which reproduces the problem and generates a graph of mount's
running time using gnuplot: http://www.shapor.com/libblkid/

I intended to run 100,000 iterations but even 16,000 took almost a day to run
on a fast computer (2.8GHz P4).  Simply removing the blkid.tab immediately
restores the original performance at iteration zero.

Conclusions:
1) mount has libblkid support hacked in sloppily.  It shouldn't attempt to
read the blkid.tab unless it is trying to guess the filesystem type.  Even if
it is, what is the point?  Is reading blkid.tab and parsing xml really an
optimization over reading the superblock (which we are about to do when we
mount the filesystem) and determining the fs type?  It doesn't even seem to
help the normal case, and really hurts the worst case badly.  If mount is to
use the file, it should scan through it only in the case it is actually
trying to detect the filesystem type, and stop when it finds the entry.

2) The in-memory data structure of the blkid cache was not designed with scale
in mind.  It should not have to scan the entire list locate a device,
which happens on every insert when reading it.

3) The use of XML in /etc is not very unixy.  It is difficult for both
computers and humans to parse.

This list appeared to be the best place to post my findings with mount and the
libblkid component of e2fsprogs.  If there is a better place, please let me
know.

Regards,
Shapor

             reply	other threads:[~2007-05-09 22:04 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-05-09 22:06 Shapor Naghibzadeh [this message]
2007-05-10  0:30 ` poor performance of mount due to libblkid Andreas Dilger
2007-05-10  4:45   ` Shapor Naghibzadeh
2007-05-10  6:44     ` Theodore Tso
2007-05-14 21:40       ` Shapor Naghibzadeh
2007-05-18  3:20         ` Theodore Tso

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20070509170646.C12805@cbr.shaptech.com \
    --to=shapor@shapor.com \
    --cc=linux-ext4@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).