All of lore.kernel.org
 help / color / mirror / Atom feed
From: Krister Johansen <kjlx-6woCzk5+qv5TrMCiz+cRkdBPR1lH4CV8@public.gmane.org>
To: "Eric W. Biederman" <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
Cc: linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org,
	Al Viro <viro-3bDd1+5oDREiFSDQTTA3OLVCufUGDwFn@public.gmane.org>
Subject: Re: Possible bug: detached mounts difficult to cleanup
Date: Tue, 10 Jan 2017 19:07:53 -0800	[thread overview]
Message-ID: <20170111030753.GC2497@templeofstupid.com> (raw)
In-Reply-To: <87r34a5p3t.fsf-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>

On Wed, Jan 11, 2017 at 03:04:22PM +1300, Eric W. Biederman wrote:
> Any chance you have a trivial reproducer script?
> 
> From you description I don't quite see the problem.  I know where to
> look but if could give a script that reproduces the conditions you
> see that would make it easier for me to dig into, and would certainly
> would remove ambiguity.   Ideally such a script would be runnable
> under unshare -Urm for easy repeated testing.

My apologies.  I don't have something that fits into a shell script, but
I can walk you through the simplest test case that I used when I was
debugging this.

Create net a ns:

    $ sudo unshare -n bash
    # echo $$
    2771

In another terminal bind mount that ns onto a file:

    # mkdir /run/testns
    # touch /run/testns/ns1
    # mount --bind /proc/2771/ns/net /run/testns/ns1

Back in first terminal, create a new ns, pivot root, and umount detach:

    # exit
    $ unshare -U -m -n --propagation slave --map-root-user bash
    # mkdir binddir
    # mount --bind binddir binddir
    # cp busybox binddir
    # mkdir binddir/old_root
    # cd binddir
    # pivot_root . old_root
    # ./busybox umount -l old_root

Back in second terminal:

    # umount /run/testns/ns1
[ watch for ns cleanup -- not seen if mnt is locked ]
    # rm /run/testns/ns1
[ now we see it ]


For the observability stuff, I went back and forth between using 'perf
probe' to place a kprobe on nsfs_evict, and using a bcc script to
watch events on the same kprobe.  I can send along the script, if you're
a bcc user.

At least when I debugged this, I found that when the mount was
MNT_LOCKED, disconnect_mount() returned false so the actual unmount
didn't happen until the mountpoint was rm'd in the host container.

I'm not sure if this is actually a bug, or a case where the cleanup is
just conservative.  However, it looked like in the case where we call
pivot_root, the detached mounts get marked private but otherwise aren't
in use in the container's namespace any longer.

-K

WARNING: multiple messages have this Message-ID (diff)
From: Krister Johansen <kjlx@templeofstupid.com>
To: "Eric W. Biederman" <ebiederm@xmission.com>
Cc: Krister Johansen <kjlx@templeofstupid.com>,
	Al Viro <viro@ZenIV.linux.org.uk>,
	linux-fsdevel@vger.kernel.org,
	containers@lists.linux-foundation.org
Subject: Re: Possible bug: detached mounts difficult to cleanup
Date: Tue, 10 Jan 2017 19:07:53 -0800	[thread overview]
Message-ID: <20170111030753.GC2497@templeofstupid.com> (raw)
In-Reply-To: <87r34a5p3t.fsf@xmission.com>

On Wed, Jan 11, 2017 at 03:04:22PM +1300, Eric W. Biederman wrote:
> Any chance you have a trivial reproducer script?
> 
> From you description I don't quite see the problem.  I know where to
> look but if could give a script that reproduces the conditions you
> see that would make it easier for me to dig into, and would certainly
> would remove ambiguity.   Ideally such a script would be runnable
> under unshare -Urm for easy repeated testing.

My apologies.  I don't have something that fits into a shell script, but
I can walk you through the simplest test case that I used when I was
debugging this.

Create net a ns:

    $ sudo unshare -n bash
    # echo $$
    2771

In another terminal bind mount that ns onto a file:

    # mkdir /run/testns
    # touch /run/testns/ns1
    # mount --bind /proc/2771/ns/net /run/testns/ns1

Back in first terminal, create a new ns, pivot root, and umount detach:

    # exit
    $ unshare -U -m -n --propagation slave --map-root-user bash
    # mkdir binddir
    # mount --bind binddir binddir
    # cp busybox binddir
    # mkdir binddir/old_root
    # cd binddir
    # pivot_root . old_root
    # ./busybox umount -l old_root

Back in second terminal:

    # umount /run/testns/ns1
[ watch for ns cleanup -- not seen if mnt is locked ]
    # rm /run/testns/ns1
[ now we see it ]


For the observability stuff, I went back and forth between using 'perf
probe' to place a kprobe on nsfs_evict, and using a bcc script to
watch events on the same kprobe.  I can send along the script, if you're
a bcc user.

At least when I debugged this, I found that when the mount was
MNT_LOCKED, disconnect_mount() returned false so the actual unmount
didn't happen until the mountpoint was rm'd in the host container.

I'm not sure if this is actually a bug, or a case where the cleanup is
just conservative.  However, it looked like in the case where we call
pivot_root, the detached mounts get marked private but otherwise aren't
in use in the container's namespace any longer.

-K

  parent reply	other threads:[~2017-01-11  3:07 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-01-11  1:24 Possible bug: detached mounts difficult to cleanup Krister Johansen
     [not found] ` <20170111012454.GB2497-6woCzk5+qv5TrMCiz+cRkdBPR1lH4CV8@public.gmane.org>
2017-01-11  2:04   ` Eric W. Biederman
2017-01-11  2:04     ` Eric W. Biederman
     [not found]     ` <87r34a5p3t.fsf-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
2017-01-11  3:07       ` Krister Johansen [this message]
2017-01-11  3:07         ` Krister Johansen
     [not found]         ` <20170111030753.GC2497-6woCzk5+qv5TrMCiz+cRkdBPR1lH4CV8@public.gmane.org>
2017-01-13  0:37           ` Andrei Vagin
2017-01-13  0:37             ` Andrei Vagin
2017-01-13 23:28             ` Krister Johansen
     [not found]             ` <CANaxB-zMzS-euqR1_LvZSoEsO-Y6q=_qGNTJZCKZTL5WfFF16g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2017-01-13 23:28               ` Krister Johansen
2017-01-11  2:27   ` Eric W. Biederman
2017-01-11  2:27 ` Eric W. Biederman
     [not found]   ` <87fukqwcue.fsf-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
2017-01-11  2:37     ` Eric W. Biederman
2017-01-11  2:37       ` Eric W. Biederman
     [not found]       ` <87shoqtj7z.fsf-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
2017-01-12  6:15         ` Krister Johansen
2017-01-12  6:15           ` Krister Johansen
     [not found]           ` <20170112061539.GA2345-6woCzk5+qv5TrMCiz+cRkdBPR1lH4CV8@public.gmane.org>
2017-01-12  8:26             ` Eric W. Biederman
2017-01-12  8:26               ` Eric W. Biederman
     [not found]               ` <87r348y98z.fsf-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
2017-01-13 23:28                 ` Krister Johansen
2017-01-13 23:28                   ` Krister Johansen
2017-01-11  2:51     ` Al Viro
2017-01-11  2:51   ` Al Viro
  -- strict thread matches above, loose matches on Subject: below --
2017-01-11  1:24 Krister Johansen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170111030753.GC2497@templeofstupid.com \
    --to=kjlx-6woczk5+qv5trmciz+crkdbpr1lh4cv8@public.gmane.org \
    --cc=containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \
    --cc=ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org \
    --cc=linux-fsdevel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=viro-3bDd1+5oDREiFSDQTTA3OLVCufUGDwFn@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.