linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Phil Davis <pmdhazy@gmail.com>
To: linux-btrfs@vger.kernel.org
Subject: btrfs send 'leaks' open files
Date: Sun, 20 Oct 2013 11:33:56 +0100	[thread overview]
Message-ID: <CAERquQ78pOig3oQoom2e0Jcsv1i3-_8ET0ewAWUUo+nt2vk4Mg@mail.gmail.com> (raw)

Setup:
Fedora 19 with kernel 3.11.4-201.fc19.x86_64
btrfs-progs-0.20.rc1.20130917git194aa4a-1.fc19.x86_64

I am trying to do an initial btrfs send of a large (206GB, 1392221
paths) snapshot to a single file on a
different filesystem (a slow external USB drive with btrfs):

   btrfs send /tank/backups/snapshots/test1 > /extpool/filetest/test5.btr

What happens is eventually get following error in in /var/log/messages:

  VFS: file-max limit 202149 reached

I worked around this by:

echo 900000 > /proc/sys/fs/file-max

but then it hit that so tried 1800000 and left it running. This
produced a 377GB test5.btr file
before system ran out of memory and crashed (oom killer etc.)

This is reproducible but the btrfs send takes about 6+ hours (USB 3.0
card on order...)

The reason I think btrfs send is leaking open files is if you watch
/proc/sys/fs/file-nr you see the
number of open files increasing  but if you kill the btrfs send
process then the open
files count reduces back down.  In fact suspending the process also
reduces the open file count but
resuming it then makes the count start increasing again.

I also found Robert Buhren reporting very similar issue back in April 2013:
http://comments.gmane.org/gmane.comp.file-systems.btrfs/24795

If further information is needed, i'd be happy to help.

-- 
Phil Davis

             reply	other threads:[~2013-10-20 10:34 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-10-20 10:33 Phil Davis [this message]
2013-10-21 13:02 ` btrfs send 'leaks' open files David Sterba
2013-10-22 17:22 ` Al Viro
2013-10-22 17:39   ` Al Viro
2013-10-22 20:41   ` Al Viro
2013-10-22 21:07     ` Josef Bacik
2013-10-22 21:22   ` Zach Brown

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAERquQ78pOig3oQoom2e0Jcsv1i3-_8ET0ewAWUUo+nt2vk4Mg@mail.gmail.com \
    --to=pmdhazy@gmail.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).