From: "Arkadiusz Miśkiewicz" <arekm@maven.pl>
To: "Fran Tsao Santín" <tsao@members.fsf.org>
Cc: xfs@oss.sgi.com
Subject: Re: "No space left on device" from hell
Date: Wed, 24 Sep 2014 10:54:05 +0200 [thread overview]
Message-ID: <201409241054.06983.arekm@maven.pl> (raw)
In-Reply-To: <CAM5tkX-mOO6EtJczQZC-23-vk7LwSczTL8CB6pgLTSLH0q94fg@mail.gmail.com>
On Wednesday 24 of September 2014, Fran Tsao Santín wrote:
> 2014-09-24 9:03 GMT+02:00 Arkadiusz Miśkiewicz <arekm@maven.pl>:
> > Hello.
> >
> > 3.10.40 kernel and creating files causes "No space left on device". After
> > deleting some files I can create new files again until the problem
> > happens again.
>
> Are you sure the problem is the filesystem device? Maybe you are running a
> process that fills up another device i.e. shm.
Dave Chinner figured out the problem already (on irc).
Basically this fs was grown from 300GB to 600GB but due to bug (fixed by
commit 9de67c3 ("xfs: allow inode allocations in post-growfs disk space"))
that new space was never used for new inodes.
So ended up with full AGs 0-3 (xfs_db -r -c "freesp -s -a XX" /dev/xyz shows
that for each AG XX).
Fs was never unmounted after grows but fortunately mount -o remount,inode32
... followed by mount -o remount,inode64 cured it without a need to reboot, so
now new inodes are distributed to all AGs.
I still need to free some space in AGs 0-3 though (by moving data to new dirs
allocated in new AGs; xfs_db -r -c "convert inode DIR_INODE_NR agno" /dev/xyz
shows in which AG new dir got allocated).
There are more bugs hiding (like if ENOSPC on some AG happens it should try
next AG instead of failing). Dave knows all that magic.
Big thanks to Dave!
--
Arkadiusz Miśkiewicz, arekm / maven.pl
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
prev parent reply other threads:[~2014-09-24 8:54 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-24 7:03 "No space left on device" from hell Arkadiusz Miśkiewicz
2014-09-24 8:42 ` Fran Tsao Santín
2014-09-24 8:54 ` Arkadiusz Miśkiewicz [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=201409241054.06983.arekm@maven.pl \
--to=arekm@maven.pl \
--cc=tsao@members.fsf.org \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox