* umount doesn't seem to really unmount
@ 2012-08-26 9:32 Stefan Ring
2012-08-26 20:11 ` Eric Sandeen
0 siblings, 1 reply; 9+ messages in thread
From: Stefan Ring @ 2012-08-26 9:32 UTC (permalink / raw)
To: Linux fs XFS
Today I had some data loss with an XFS filesystem, apparently because
it does not write everything to disk during unmount. Yesterday, I shut
down my machine for the night. My XFS lives on an iSCSI volume which I
shut down shortly, but not instantly, after unmounting the XFS
filesystem. There has been no noticable disk activity after the
umount. Today, some recently written files all had size zero.
Additional indicators that something is amiss is, that during startup,
when mounting the XFS filesystem, it would often claim that it has
done log recovery, which should not be the case if it had been
unmounted correctly. Also, when I just mount and unmount the
filesystem repeatedly, I would at least expect "ending clean mount"
messages in dmesg, like I get on CentOS 6. Not so on my system.
This is with kernel 3.5.2-3.fc17 from Fedora. Is there a known problem?
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: umount doesn't seem to really unmount
2012-08-26 9:32 umount doesn't seem to really unmount Stefan Ring
@ 2012-08-26 20:11 ` Eric Sandeen
2012-08-27 6:47 ` Stefan Ring
0 siblings, 1 reply; 9+ messages in thread
From: Eric Sandeen @ 2012-08-26 20:11 UTC (permalink / raw)
To: Stefan Ring; +Cc: Linux fs XFS
On Aug 26, 2012, at 2:32 AM, Stefan Ring <stefanrin@gmail.com> wrote:
> Today I had some data loss with an XFS filesystem, apparently because
> it does not write everything to disk during unmount. Yesterday, I shut
> down my machine for the night. My XFS lives on an iSCSI volume which I
> shut down shortly, but not instantly, after unmounting the XFS
> filesystem. There has been no noticable disk activity after the
> umount. Today, some recently written files all had size zero.
> Additional indicators that something is amiss is, that during startup,
> when mounting the XFS filesystem, it would often claim that it has
> done log recovery, which should not be the case if it had been
> unmounted correctly. Also, when I just mount and unmount the
> filesystem repeatedly, I would at least expect "ending clean mount"
> messages in dmesg, like I get on CentOS 6. Not so on my system.
>
> This is with kernel 3.5.2-3.fc17 from Fedora. Is there a known problem?
>
Do you have barriers on and does iscsi pass them through to the target, and does the target propagate them to the disk itself?
Eric
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: umount doesn't seem to really unmount
2012-08-26 20:11 ` Eric Sandeen
@ 2012-08-27 6:47 ` Stefan Ring
2012-08-27 6:56 ` Gabriel VLASIU
0 siblings, 1 reply; 9+ messages in thread
From: Stefan Ring @ 2012-08-27 6:47 UTC (permalink / raw)
To: Linux fs XFS
> Do you have barriers on and does iscsi pass them through to the target, and does the target propagate them to the disk itself?
Regardless of the barriers, it doesn't explain this:
$ sudo mount -o barrier=1,inode64 /dev/mapper/vg_xfs2-lvxfs1 /xfs1
$ sudo umount /xfs1
$ sudo xfs_logprint /dev/mapper/vg_xfs2-lvxfs1
xfs_logprint:
xfs_logprint: /dev/mapper/vg_xfs2-lvxfs1 contains a mounted and
writable filesystem
data device: 0xfd01
log device: 0xfd01 daddr: 83882016 length: 81912
It also happens in kernel 3.4.3-1.fc17. There are no other machines
which access the iSCSI target.
Right after boot, I can mount and umount it all that I want, and it
will work normally, i.e., say "Mounting Filesystem / Ending clean
mount" in dmesg, xfs_logprint NOT saying "... contains a mounted and
writable filesystem".
As crazy as that sounds, I've just found out what triggers the
situation. The weird behavior only manifests itself after this:
$ sudo service mysqld start
>From then on, it behaves as described earlier. Not that mysqld has
anything to do with XFS. It's just the plain package from Fedora, with
everything in the default place on the root filesystem, which is ext4.
As I've just found out, the behavior even returns to normal when I
shut down mysqld. So for now, my workaround is shutting down mysqld
myself before rebooting.
I'll try and see if this is easily reproducible inside a clean F17
install inside a KVM guest.
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: umount doesn't seem to really unmount
2012-08-27 6:47 ` Stefan Ring
@ 2012-08-27 6:56 ` Gabriel VLASIU
2012-08-27 7:13 ` Stefan Ring
0 siblings, 1 reply; 9+ messages in thread
From: Gabriel VLASIU @ 2012-08-27 6:56 UTC (permalink / raw)
To: xfs
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Mon, 27 Aug 2012, Stefan Ring wrote:
> Regardless of the barriers, it doesn't explain this:
>
> $ sudo mount -o barrier=1,inode64 /dev/mapper/vg_xfs2-lvxfs1 /xfs1
> $ sudo umount /xfs1
> $ sudo xfs_logprint /dev/mapper/vg_xfs2-lvxfs1
> xfs_logprint:
> xfs_logprint: /dev/mapper/vg_xfs2-lvxfs1 contains a mounted and
> writable filesystem
> data device: 0xfd01
> log device: 0xfd01 daddr: 83882016 length: 81912
Same here. Plain sata hdd. 3.5.2 (Fedora 17).
# mount | grep /saves
/dev/sdb1 on /saves type xfs (rw,seclabel,nosuid,nodev,noexec,relatime,attr2,noquota)
# umount /saves
# xfs_repair /dev/sdb1
xfs_repair: /dev/sdb1 contains a mounted filesystem
fatal error -- couldn't initialize XFS library
> As crazy as that sounds, I've just found out what triggers the
> situation. The weird behavior only manifests itself after this:
>
> $ sudo service mysqld start
Does not work for me. It must be something else.
Sincerely,
Gabriel
- --
// Gabriel VLASIU
//
// OpenGPG-KeyID : 44952F15
// OpenGPG-Fingerprint: 4AC5 7C26 2FE9 02DA 4906 24B2 D32B 7ED7 4495 2F15
// OpenGPG-URL : http://www.vlasiu.net/public.key
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQIcBAEBAgAGBQJQOxoRAAoJENMrftdElS8VlCYQAKiLaCp0KDsnkzr0nHGIBrbU
Xza/Ph2yIPw3BAZtXCPYGxH5gn7vemCnwZUhz+Oia9V9XFnlEjALvzdA1zhhdiYO
SRQi9n0qcuVLwuF/Mtp8w6B2K8S7DATeDmVBR3HGKb2SgvnOeJD3FaJYIPp98FKF
Kg9KhEmvOVyyXG7btCwqj0w7Uk5TxlPbAq+RVDjSeQLyo879fyQbdEmT6+jFk73Z
4afK0LhiOlzMGUGoU1Wr/hY16JIIeY2RIyoW6aS4QJQJMdLq8oiH3wgi+v+4lpJB
d1EOfKsqibK8+iqMiOkUP18VhK6xTP9nGthIB3wtmNyF/8StOc071GluJHOE2I7y
klxO807oj6pnKcYZrH/xxFEwf2iABJd6faFuu1pUtBr+ExlEQiJ2B5fGxWPFIysv
17/NtM/jNyNzhHQjImeYJvM0f2C0kXqhHsZ6U33Qk58OIjV8AEXVKHAPYMy5CXn2
GKD4catqrTsklzNW3Mi34cSXeMPFd7KqsuXQ3u6tekmal3TK01TMZJfdMlZ6IJqA
i+fUGk7iBfgamcwbn1wdy+odz5ZukvzkcEGlQZ9BGdSIcNz0p9/Tbk9hc5jCcfPw
gBwWSzTUFMdvPO9ccVE9hVs2285sL+p81ojk1wos71SPK3cfJmAQz715808zRlt1
VgQV6MwXCmZQlF/9Y2C0
=oamX
-----END PGP SIGNATURE-----
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: umount doesn't seem to really unmount
2012-08-27 6:56 ` Gabriel VLASIU
@ 2012-08-27 7:13 ` Stefan Ring
2012-08-27 7:23 ` Gabriel VLASIU
0 siblings, 1 reply; 9+ messages in thread
From: Stefan Ring @ 2012-08-27 7:13 UTC (permalink / raw)
To: Gabriel VLASIU; +Cc: xfs
>> $ sudo service mysqld start
> Does not work for me. It must be something else.
It seems to happen with any service that has PrivateTmp set:
$ grep PrivateTmp /usr/lib/systemd/system/*.service
/usr/lib/systemd/system/colord-sane.service:PrivateTmp=yes
/usr/lib/systemd/system/colord.service:PrivateTmp=yes
/usr/lib/systemd/system/cups.service:PrivateTmp=true
/usr/lib/systemd/system/httpd-event.service:PrivateTmp=true
/usr/lib/systemd/system/httpd.service:PrivateTmp=true
/usr/lib/systemd/system/httpd-worker.service:PrivateTmp=true
/usr/lib/systemd/system/mysqld.service:PrivateTmp=true
/usr/lib/systemd/system/openvpn@.service:PrivateTmp=true
I guess that the XFS filesystem must be mounted at the time the
service gets started for this to happen. In that case, the service
seems to hold a hidden reference to XFS which get released when the
service is stopped.
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: umount doesn't seem to really unmount
2012-08-27 7:13 ` Stefan Ring
@ 2012-08-27 7:23 ` Gabriel VLASIU
2012-08-27 23:40 ` Eric Sandeen
0 siblings, 1 reply; 9+ messages in thread
From: Gabriel VLASIU @ 2012-08-27 7:23 UTC (permalink / raw)
To: xfs
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Mon, 27 Aug 2012, Stefan Ring wrote:
> It seems to happen with any service that has PrivateTmp set:
Thanks. Stopping all services using PrivateTmp seems to fix the problem.
systemd again... :-(
Sincerely,
Gabriel
- --
// Gabriel VLASIU
//
// OpenGPG-KeyID : 44952F15
// OpenGPG-Fingerprint: 4AC5 7C26 2FE9 02DA 4906 24B2 D32B 7ED7 4495 2F15
// OpenGPG-URL : http://www.vlasiu.net/public.key
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQIcBAEBAgAGBQJQOyBWAAoJENMrftdElS8VgAoP/1q9BlGcs/th/+Ex0lG6HWmj
ApBjpnS1JCzKDCrtZjtTpzWOF0FZe84yHJ/l8/T0Fze37QWJ7k8KRizMtn0gcAXQ
HeqBBxdbiNWeaqyE89n9sBX30yTvFRNGqHzXseIEHo1VfAwven2zrhPpMzFdz++D
qPxl/7HqnsYSb6oGX03m9nDoJhlKvE1YLNHaqlJXQI8UjgJIMu9aQvgTJKIDF6i1
q2oTVOpOvXjbt4/A1n3aO0qMlS7h7sztokkGstalscHYVVqgMlNjI0r+bgy5CwSh
gkztUg+l1/ga5HcGurkqJW2TWz2CxYOE40JfGzjqx02dEnDMW1hMObmlkSPkWu/m
sLYXsUvvlT58b/uSTjTcRDHkt1SQEERNp85JlyEBTIbVwN4HQnpZnXNGzKjvllw+
L47oQwk59bBb1/LSuFyJtT8nXn6ODpKglsMK4pc82SO9Zney1gpCCGcGUQamp6/w
koRqcSVADJy4YFZ61TQVMxxafPPAFrmvCDX34QT/I96iGMLwheg5Na8LGKgWkczl
YRg9abG8jvUuO4B34mfLnUjm9p1nepEmn8xYnzE6batLg00gkxiB2T2ThyAOW9co
T+utXpBJM+5meCzsR0w+X0bTFnqganbZi95Marm91BzehaG1L3pXbNkVk52V0yTs
H8IwCc+WaFo+xEFxwR6i
=TiH/
-----END PGP SIGNATURE-----
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: umount doesn't seem to really unmount
2012-08-27 7:23 ` Gabriel VLASIU
@ 2012-08-27 23:40 ` Eric Sandeen
2012-08-28 5:45 ` Stefan Ring
2012-08-28 7:18 ` Gabriel VLASIU
0 siblings, 2 replies; 9+ messages in thread
From: Eric Sandeen @ 2012-08-27 23:40 UTC (permalink / raw)
To: Gabriel VLASIU; +Cc: xfs
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 8/27/12 2:23 AM, Gabriel VLASIU wrote:
> On Mon, 27 Aug 2012, Stefan Ring wrote:
>
>> It seems to happen with any service that has PrivateTmp set:
> Thanks. Stopping all services using PrivateTmp seems to fix the problem.
> systemd again... :-(
>
Gah, is umount replaced by something via systemd? I mean.... umount is
still umount, no?
IOW is this just doing a lazy unmount behind your back?
- -Eric
> Sincerely,
> Gabriel
>
>
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQIcBAEBAgAGBQJQPAWCAAoJECCuFpLhPd7gUgsQALWEEVT2MBfD/EAGkZn1OayZ
NXpJZeedYYA2mhoBxx7CD0QfJgzhmeZxMq2q/mmUIb7XWibnlOPeGJ9ieq6IJ7u3
0Izxs2VOKpeE93j5AOMoaEdoY7ncTPfiIq6pNqNMTK/LbW8iN2ej5vW7mSkErcTs
KmiyOubCoj27a8JXVyc1ZY/oFp82HmCsx0WLe5nxhBYSto1ochUZY2vXZy0sdem0
VGtrCa8uLBh4YDzq9s3aDEukYdqsSFGjCYvdVOGqQfLaDj78JVn8jqLVplcGmnrF
Knqv+suxuVwt/WVnp4vzK7707YxV2g/ys9RBit3CFjrj3AWnyPMdwzL6i+kKkDPI
h7xYpeUNSDfwdNrxdjmPOD7pegUxwNKc5Id0gIei+C8vbUgtaaMIkgRbQnlSa7e8
4T8iF+LS4VAi480svOEUVD0Yh4jznCqBmjh3jjO8wrgP4Md12rdV8HKCFYtcq3Uy
EWhWy4dlG9H0w1nRG7jqrcj2hjWVhYRnY3DkAg+E9adfr6LFoEejEFjSH0HTjs3j
h8niXZt19EFUNijbHkNbmpIyAuloEyK4qwJ0E1myE2llh2xax7licRcXs9rirJvU
biD1kW0x31Dhb1QuI3yRrD35peczWtdr6xYhn6+iWcJdq8xkS8hxooObzYjc5PaO
6VHp3AtZEHXekvJFvUPq
=YlDt
-----END PGP SIGNATURE-----
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: umount doesn't seem to really unmount
2012-08-27 23:40 ` Eric Sandeen
@ 2012-08-28 5:45 ` Stefan Ring
2012-08-28 7:18 ` Gabriel VLASIU
1 sibling, 0 replies; 9+ messages in thread
From: Stefan Ring @ 2012-08-28 5:45 UTC (permalink / raw)
To: Eric Sandeen; +Cc: Linux fs XFS
> Gah, is umount replaced by something via systemd? I mean.... umount is
> still umount, no?
>
> IOW is this just doing a lazy unmount behind your back?
My interpretation is that at the time a service is started, the entire
filesystem "namespace" is copied to a secret place, including the
mounts. Then, when I issue umount, it is actually unmounted from the
main namespace, but from the viewpoint of the filesystem, it's still
mounted, precisely in the hidden namespace that got created for the
running service. Only when this service quits, its namespace will be
destroyed, and with it the filesystem's refcount will drop to 0. At
this point, the real "unmount", or rather "close" will happen.
Apparently, this is not usually a problem, but for me it is, because
at that time, the iSCSI storage device is gone. Gabriel has filed a
bug here: [1]. Unfortunately, I have no hope that this will ever be
"fixed", because it behaves just the way it was designed :(.
At least it has become clear that this has nothing to do with XFS,
although subjectively my feeling is that the effects (i.e., data loss)
are felt more with XFS than with ext4.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=851970
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: umount doesn't seem to really unmount
2012-08-27 23:40 ` Eric Sandeen
2012-08-28 5:45 ` Stefan Ring
@ 2012-08-28 7:18 ` Gabriel VLASIU
1 sibling, 0 replies; 9+ messages in thread
From: Gabriel VLASIU @ 2012-08-28 7:18 UTC (permalink / raw)
To: Eric Sandeen; +Cc: xfs
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Mon, 27 Aug 2012, Eric Sandeen wrote:
> Gah, is umount replaced by something via systemd? I mean.... umount is
> still umount, no?
It's still umount. For how long... I don't know.
Seeing the latest and "greatest" achievements of this Poettering guy I
just hope that Fedora 28 will not have just only one rpm package
systemd-everything.
> IOW is this just doing a lazy unmount behind your back?
Yes. And it's driving me crazy. I mount a lot of file-systems via loop.
And after a while, most of the time, I can umount the file-system but I
cannot detach the loop device anymore. So a reboot is required. I did not
have time to check if stopping all services with PrivateTmp fix this but
I'm pretty sure it will.
Sincerely,
Gabriel
- --
// Gabriel VLASIU
//
// OpenGPG-KeyID : 44952F15
// OpenGPG-Fingerprint: 4AC5 7C26 2FE9 02DA 4906 24B2 D32B 7ED7 4495 2F15
// OpenGPG-URL : http://www.vlasiu.net/public.key
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQIcBAEBAgAGBQJQPHDHAAoJENMrftdElS8VUugP/1VRGg9L77VxYrQeG+o35x96
zMh5LxFqIL7h8gloNHI2xkHb8LlQbHqyOJn7jrpRoSDnxymDVefuCzE2dZDP0mx4
kEVyIYn12ElMqMmzdXMBJJI/4av17DTgohbYYHXM3FGyN0uow+6GihV2PcIOXsP6
dkFZ48O55Lc4hhJ4vNyrNl9K+MHDfcxPtQTx/aZ5JuNeMLK4ywROcRycsML6LPl4
5gfHzlg3dMWSKg9gCWZCETC82u4MCvaQ7L8mhTwsDCM/Kn4mFiEaYUpCDL/poDqI
OhUMpG+zt0qgoJ7rUjjQFHRO0ianiubAeV7ajnw4n4Nd+6+r9iohR65lAEclfgID
wskVEI3sAD3ZOdK2hL0SUaR4ZHQfx8ku/4BdmTG0PHA6+eO0Q+THXzWgP4dE5eQX
qK+ghTHFyiDd29b2fbKS5qLmz84hQQAHT4I/IszSJ85v2siuCxqhJ/xOfVRsvu0c
dNQc9h8KYTORvc1QVLs4l1cAIUuHqnk4LumLjvEFIly7+aXHg2rRZreqNkdRlkRf
MnXvlfKwxI6oSYqp09bDPy/nn8fnYcechALQeDPglSpADHLjGrNNjLchoA2y8Thi
hV6ytXpsEMp1vrcoKSLmVbE6bYThGNjKykMwKAXw3OnWWs3hpMCZYivgHi+G2TGa
A0FK89doR0ysDJtGxIdZ
=No5Y
-----END PGP SIGNATURE-----
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2012-08-28 7:17 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-08-26 9:32 umount doesn't seem to really unmount Stefan Ring
2012-08-26 20:11 ` Eric Sandeen
2012-08-27 6:47 ` Stefan Ring
2012-08-27 6:56 ` Gabriel VLASIU
2012-08-27 7:13 ` Stefan Ring
2012-08-27 7:23 ` Gabriel VLASIU
2012-08-27 23:40 ` Eric Sandeen
2012-08-28 5:45 ` Stefan Ring
2012-08-28 7:18 ` Gabriel VLASIU
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox