* ENOSPC on file deletion with 3.1.6
@ 2012-01-03 13:49 Arie Peterson
2012-01-03 14:06 ` Sander
0 siblings, 1 reply; 6+ messages in thread
From: Arie Peterson @ 2012-01-03 13:49 UTC (permalink / raw)
To: linux-btrfs
Hi,
After upgrading my kernel from 2.6.38 (which has worked fine for months) to
3.1.6, I got ENOSPC on recompiling gcc (even though df says there is 16G free
of 50G; this is a raid1 setup, so in fact it's 8 of 25).
After this error, I tried to remove the compilation directory (with "rm -r"):
this also gives ENOSPC. I am trying to work around this by first truncating
files using "echo > $file", but this fails for some files, again with ENOSPC.
Also, removal of files is very slow even if it succeeds.
Moreover, any write operation on the file system now fails with ENOSPC.
Reverting to my old kernel does not help: it now shows the same problem.
Is this a known issue? Is there a way to make this file system unstuck? (I have
backups, but I'd like to preserve snapshot information if possible.) Should I
try upgrading to an even newer kernel?
Kind regards,
Arie
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ENOSPC on file deletion with 3.1.6
2012-01-03 13:49 ENOSPC on file deletion with 3.1.6 Arie Peterson
@ 2012-01-03 14:06 ` Sander
2012-01-03 14:12 ` Arie Peterson
0 siblings, 1 reply; 6+ messages in thread
From: Sander @ 2012-01-03 14:06 UTC (permalink / raw)
To: Arie Peterson; +Cc: linux-btrfs
Arie Peterson wrote (ao):
> After upgrading my kernel from 2.6.38 (which has worked fine for months) to
> 3.1.6, I got ENOSPC on recompiling gcc (even though df says there is 16G free
> of 50G; this is a raid1 setup, so in fact it's 8 of 25).
>
> After this error, I tried to remove the compilation directory (with "rm -r"):
> this also gives ENOSPC. I am trying to work around this by first truncating
> files using "echo > $file", but this fails for some files, again with ENOSPC.
> Also, removal of files is very slow even if it succeeds.
>
> Moreover, any write operation on the file system now fails with ENOSPC.
>
> Reverting to my old kernel does not help: it now shows the same problem.
>
> Is this a known issue? Is there a way to make this file system unstuck? (I have
> backups, but I'd like to preserve snapshot information if possible.) Should I
> try upgrading to an even newer kernel?
Maybe your snapshots take up space. Can you show 'btrfs filesystem df /' ?
FWIW, I also had a disk full just a few days ago. Removed all snapshots
and some big files, but to no avail. Likely the background cleanup took
too much time. A reboot fixed this.
Sander
--
Humilis IT Services and Solutions
http://www.humilis.net
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ENOSPC on file deletion with 3.1.6
2012-01-03 14:06 ` Sander
@ 2012-01-03 14:12 ` Arie Peterson
2012-01-03 14:22 ` Sander
2012-01-03 14:44 ` cwillu
0 siblings, 2 replies; 6+ messages in thread
From: Arie Peterson @ 2012-01-03 14:12 UTC (permalink / raw)
To: linux-btrfs
On Tuesday 03 January 2012 15:06:43 Sander wrote:
> Maybe your snapshots take up space. Can you show 'btrfs filesystem df /' ?
Data, RAID1: total=22.72GB, used=14.73GB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=12.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=2.25GB, used=1.88GB
Metadata: total=8.00MB, used=0.00
> FWIW, I also had a disk full just a few days ago. Removed all snapshots
> and some big files, but to no avail. Likely the background cleanup took
> too much time. A reboot fixed this.
OK, I'll keep this in mind. I'm a bit anxious to reboot, because I'm afraid
booting will fail if the root file system cannot be written to.
> Sander
Thanks,
Arie
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ENOSPC on file deletion with 3.1.6
2012-01-03 14:12 ` Arie Peterson
@ 2012-01-03 14:22 ` Sander
2012-01-03 14:44 ` Arie Peterson
2012-01-03 14:44 ` cwillu
1 sibling, 1 reply; 6+ messages in thread
From: Sander @ 2012-01-03 14:22 UTC (permalink / raw)
To: Arie Peterson; +Cc: linux-btrfs
Arie Peterson wrote (ao):
> On Tuesday 03 January 2012 15:06:43 Sander wrote:
> > Maybe your snapshots take up space. Can you show 'btrfs filesystem df /' ?
>
> Data, RAID1: total=22.72GB, used=14.73GB
> Data: total=8.00MB, used=0.00
> System, RAID1: total=8.00MB, used=12.00KB
> System: total=4.00MB, used=0.00
> Metadata, RAID1: total=2.25GB, used=1.88GB
> Metadata: total=8.00MB, used=0.00
Hm, not full.
> > FWIW, I also had a disk full just a few days ago. Removed all snapshots
> > and some big files, but to no avail. Likely the background cleanup took
> > too much time. A reboot fixed this.
>
> OK, I'll keep this in mind. I'm a bit anxious to reboot, because I'm afraid
> booting will fail if the root file system cannot be written to.
But you did already reboot as you said the old kernel exposed the same
behavior?
Sander
--
Humilis IT Services and Solutions
http://www.humilis.net
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ENOSPC on file deletion with 3.1.6
2012-01-03 14:12 ` Arie Peterson
2012-01-03 14:22 ` Sander
@ 2012-01-03 14:44 ` cwillu
1 sibling, 0 replies; 6+ messages in thread
From: cwillu @ 2012-01-03 14:44 UTC (permalink / raw)
To: Arie Peterson; +Cc: linux-btrfs
On Tue, Jan 3, 2012 at 8:12 AM, Arie Peterson <ariep@xs4all.nl> wrote:
> On Tuesday 03 January 2012 15:06:43 Sander wrote:
>
>> Maybe your snapshots take up space. Can you show 'btrfs filesystem df /' ?
>
> Data, RAID1: total=22.72GB, used=14.73GB
> Data: total=8.00MB, used=0.00
> System, RAID1: total=8.00MB, used=12.00KB
> System: total=4.00MB, used=0.00
> Metadata, RAID1: total=2.25GB, used=1.88GB
> Metadata: total=8.00MB, used=0.00
>
>> FWIW, I also had a disk full just a few days ago. Removed all snapshots
>> and some big files, but to no avail. Likely the background cleanup took
>> too much time. A reboot fixed this.
>
> OK, I'll keep this in mind. I'm a bit anxious to reboot, because I'm afraid
> booting will fail if the root file system cannot be written to.
I'd probably run a "btrfs fi balance /", it should be able to recover
the space. I'd typically be a little anxious if it was a large
filesystem as it's not interruptable except via power-button (in
principle it shouldn't matter, but...), but given that your filesystem
is quite small, it shouldn't take more than an hour or so.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: ENOSPC on file deletion with 3.1.6
2012-01-03 14:22 ` Sander
@ 2012-01-03 14:44 ` Arie Peterson
0 siblings, 0 replies; 6+ messages in thread
From: Arie Peterson @ 2012-01-03 14:44 UTC (permalink / raw)
To: linux-btrfs
On Tuesday 03 January 2012 15:22:58 Sander wrote:
> Hm, not full.
>
> > OK, I'll keep this in mind. I'm a bit anxious to reboot, because I'm
> > afraid booting will fail if the root file system cannot be written to.
>
> But you did already reboot as you said the old kernel exposed the same
> behavior?
You are right; the full history of events was:
(- compile new kernel (3.1.6))
- boot new kernel;
- recompile gcc: problem occurs;
- solve problem by removing compilation directory;
- boot old kernel;
- recompile gcc: problem occurs for this kernel as well.
After trying to remove the problematic files for some time, I took the chance
and rebooted. After the reboot, the file system still gave ENOSPC on any write
operation. However, it was able to boot anyway, and now the removal of the
problematic files went much faster and without new ENOSPC. The compilation
directory was completely removed, and immediately afterwards, the file system
became writeable again.
Sander, thanks for your help.
I am still curious if this a known problem, and if upgrading to a newer kernel
might prevent it from reoccurring. (I was planning to wait for 3.2 to be
released and included in Gentoo's repository; maybe I shouldn't wait for
this...)
Kind regards,
Arie
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-01-03 14:44 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-01-03 13:49 ENOSPC on file deletion with 3.1.6 Arie Peterson
2012-01-03 14:06 ` Sander
2012-01-03 14:12 ` Arie Peterson
2012-01-03 14:22 ` Sander
2012-01-03 14:44 ` Arie Peterson
2012-01-03 14:44 ` cwillu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).