Can't remove empty directory after kernel panic, no errors in dmesg

Can't remove empty directory after kernel panic, no errors in dmesg

[Niklas Schnelle]

Hi List,

so first the basics. I'm running Arch Linux with 3.13-rc2, btrfs-progs
0.20rc1.3-2 from the repo and I'm using a SSD.
So I was having kernel panics with my USB 3.0 Gigabit card and was
trying to get a panic output. These panics are intermittent and most
often happen while using Chromium. Anyway so my system paniced while I
was in Chromium.
After the reboot Chromium reported that its preferences are corrupted,
thankfully I've both backups and an older snapshot. So I wanted to
copy over my ~/.config/chromium from the snapshot.
However I couldn't delete that directory, rm -rf reported it to not be
empty. Renaming worked via
"mv chromium bad" but now I can't delete the bad directory, this is the output:
http://pastebin.com/FWTPGGH1

any idea how to get that directory deleted or how to obtain more information?

Greetings Niklas

Re: Can't remove empty directory after kernel panic, no errors in dmesg

[Duncan]

Niklas Schnelle posted on Sat, 07 Dec 2013 11:36:45 +0100 as excerpted:

> Hi List,
> 
> so first the basics. I'm running Arch Linux with 3.13-rc2, btrfs-progs
> 0.20rc1.3-2 from the repo and I'm using a SSD.
> So I was having kernel panics with my USB 3.0 Gigabit card and was
> trying to get a panic output. These panics are intermittent and most
> often happen while using Chromium. Anyway so my system paniced while I
> was in Chromium.
> After the reboot Chromium reported that its preferences are corrupted,
> thankfully I've both backups and an older snapshot. So I wanted to copy
> over my ~/.config/chromium from the snapshot.
> However I couldn't delete that directory, rm -rf reported it to not be
> empty. Renaming worked via "mv chromium bad" but now I can't delete the
> bad directory, this is the output:
> http://pastebin.com/FWTPGGH1
> 
> any idea how to get that directory deleted or how to obtain more
> information?

That sort of behavior is a(n almost[1]) sure sign of filesystem 
corruption.  On a normal filesystem, you'd fsck it and hope that fixed 
the errors, and you can try btrfsck too, first without the --repair 
option, just to see what it gives you, then if you want to risk it 
(btrfsck still not being fully tested yet, see the manpage), with the 
option.

But before you try that repair option, you can try a few other things 
first.  Here's a link to a post with a list of things to try, in the 
order of least to greatest risk.  (It that case IIRC the filesystem 
wouldn't mount at all, so the problem was worse.  But the point is, 
there's other things you can try first -- btrfsck --repair isn't always 
the first recommended option.)

http://permalink.gmane.org/gmane.comp.file-systems.btrfs/27999

Meanwhile, FWIW I have my btrfs filesystems (also on ssd, actually dual 
SSD in btrfs raid1 mode) split up into independent filesystems on 
separate partitions, so all my data eggs aren't in the same basket, and 
recovery from one going bad isn't so difficult.  As a result, since most 
of it's still readable, I'd probably first do a scrub (raid1 mode both 
data and metadata so hopefully one copy is good), then if that didn't 
work I'd ensure my backups were current, then do a balance and/or btrfsck 
--repair, hoping that would fix it.  If that didn't fix it, I'd probably 
simply blow it away and restore from backup.  Since I have things splitup 
into multiple independent filesystems, the biggest is only double-digit 
gigs, and being on SSD, doing a mkfs.btrfs on the partition automatically 
does a trim/discard on the entire partition, zeroing it out, and copying 
over the tens of gigs from the backup will only take a few minutes.  It's 
not like the multiple TB btrfs filesystems on spinning rust I see people 
reporting as taking a good fraction of a day or longer.

---
[1] Almost: Barring something like selinux or the like, where root is 
/not/ necessarily all powerful!  I also once had problems getting 
something to execute, even tho execute permissions were set... until I 
remembered that partition was mounted noexec!  Of course the equivalent 
here would be a read-only mount, but that can't be it or you'd not have 
been able to rename/move the directory, either.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Can't remove empty directory after kernel panic, no errors in dmesg

[Niklas Schnelle]

Ok, so I tried btrfsck from a live stick and I got the following:
After btrfsck --repair the errors are displayed again though.

 btrfsck ./btrfsroot
Checking filesystem on ./btrfsroot
UUID: 2966e7fc-22dd-414a-9197-ec512d94a622
checking extents
checking free space cache
cache and super generation don't match, space cache will be invalidated
checking fs roots
root 470 inode 60984 errors 200
root 470 inode 62463 errors 200
root 470 inode 62522 errors 200
root 470 inode 62536 errors 200
root 470 inode 62561 errors 200
root 470 inode 62594 errors 200
root 470 inode 68067 errors 400
root 470 inode 318806 errors 400
root 470 inode 318814 errors 400
root 470 inode 653218 errors 400
root 474 inode 60984 errors 200
root 474 inode 62463 errors 200
root 474 inode 62522 errors 200
root 474 inode 62536 errors 200
root 474 inode 62561 errors 200
root 474 inode 62594 errors 200
found 38006717310 bytes used err is 1
total csum bytes: 114009592
total tree bytes: 2296799232
total fs tree bytes: 2007769088
total extent tree bytes: 148426752
btree space waste bytes: 525760756
file data blocks allocated: 221531045888
 referenced 176664707072
Btrfs v0.20-rc1



On Sat, Dec 7, 2013 at 11:23 AM, Duncan <1i5t5.duncan@cox.net> wrote:
> Niklas Schnelle posted on Sat, 07 Dec 2013 11:36:45 +0100 as excerpted:
>
>> Hi List,
>>
>> so first the basics. I'm running Arch Linux with 3.13-rc2, btrfs-progs
>> 0.20rc1.3-2 from the repo and I'm using a SSD.
>> So I was having kernel panics with my USB 3.0 Gigabit card and was
>> trying to get a panic output. These panics are intermittent and most
>> often happen while using Chromium. Anyway so my system paniced while I
>> was in Chromium.
>> After the reboot Chromium reported that its preferences are corrupted,
>> thankfully I've both backups and an older snapshot. So I wanted to copy
>> over my ~/.config/chromium from the snapshot.
>> However I couldn't delete that directory, rm -rf reported it to not be
>> empty. Renaming worked via "mv chromium bad" but now I can't delete the
>> bad directory, this is the output:
>> http://pastebin.com/FWTPGGH1
>>
>> any idea how to get that directory deleted or how to obtain more
>> information?
>
> That sort of behavior is a(n almost[1]) sure sign of filesystem
> corruption.  On a normal filesystem, you'd fsck it and hope that fixed
> the errors, and you can try btrfsck too, first without the --repair
> option, just to see what it gives you, then if you want to risk it
> (btrfsck still not being fully tested yet, see the manpage), with the
> option.
>
> But before you try that repair option, you can try a few other things
> first.  Here's a link to a post with a list of things to try, in the
> order of least to greatest risk.  (It that case IIRC the filesystem
> wouldn't mount at all, so the problem was worse.  But the point is,
> there's other things you can try first -- btrfsck --repair isn't always
> the first recommended option.)
>
> http://permalink.gmane.org/gmane.comp.file-systems.btrfs/27999
>
> Meanwhile, FWIW I have my btrfs filesystems (also on ssd, actually dual
> SSD in btrfs raid1 mode) split up into independent filesystems on
> separate partitions, so all my data eggs aren't in the same basket, and
> recovery from one going bad isn't so difficult.  As a result, since most
> of it's still readable, I'd probably first do a scrub (raid1 mode both
> data and metadata so hopefully one copy is good), then if that didn't
> work I'd ensure my backups were current, then do a balance and/or btrfsck
> --repair, hoping that would fix it.  If that didn't fix it, I'd probably
> simply blow it away and restore from backup.  Since I have things splitup
> into multiple independent filesystems, the biggest is only double-digit
> gigs, and being on SSD, doing a mkfs.btrfs on the partition automatically
> does a trim/discard on the entire partition, zeroing it out, and copying
> over the tens of gigs from the backup will only take a few minutes.  It's
> not like the multiple TB btrfs filesystems on spinning rust I see people
> reporting as taking a good fraction of a day or longer.
>
> ---
> [1] Almost: Barring something like selinux or the like, where root is
> /not/ necessarily all powerful!  I also once had problems getting
> something to execute, even tho execute permissions were set... until I
> remembered that partition was mounted noexec!  Of course the equivalent
> here would be a read-only mount, but that can't be it or you'd not have
> been able to rename/move the directory, either.
>
> --
> Duncan - List replies preferred.   No HTML msgs.
> "Every nonfree program has a lord, a master --
> and if you use the program, he is your master."  Richard Stallman
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Can't remove empty directory after kernel panic, no errors in dmesg

[Duncan]

Niklas Schnelle posted on Sat, 07 Dec 2013 12:37:05 +0000 as excerpted:

> root 470 inode 60984 errors 200
> root 470 inode 62463 errors 200

Three quick things to note:

1) As another thread pointed out recently, that's not 200 errors, but an 
error type bitmask, with the 0x200 bit set.  Based on that other thread 
(I'm not a dev and haven't actually looked myself), there's comments/
varnames in the code (unfortunately only, no non-dev admin-level 
documentation anywhere that I know of) saying what each bitflag 
represents.

2) I had meant to mention earlier but forgot:  As announced on this list 
IIRC shortly after the kernel 3.12 release, btrfs-progs is now versioned 
similar to the kernel and will follow a similar release schedule as long 
as the level of new code warrants (possibly skipping kernel versions here 
and there as the code ultimately stabilizes and churn slows down), with 
the first release following that version scheme being 3.12.

Your reported version number is v0.20-rc1, no git commit snapshot 
indication, and IIRC that was released late last year, so it's about a 
year old now.  You may wish to try something a bit newer, to match your 
3.13-release-candidate kernel version.  There have been a number of fixes 
since v0.20-rc1 (including btrfsck being made part of the main btrfs 
command now, as btrfs check) and it's just possible a current version may 
fix your issues.

3) You asked in what might have been a private mail as I didn't see it 
hit the list, what liveCD, etc, to use, since it's a rootfs btrfs that 
you're working with.  The list mail I'm replying to says you tried a live 
stick (doesn't say the version), so you've worked around that to some 
extent, but as a more general followup based on the multiple independent 
btrfs partitions scheme I use that I mentioned earlier in the thread...

One of the best setups I've come up with over a decade's worth of 
experience here is, as I said, multiple independent partitions, with the 
first level backup actually on an identically sized partition on the same 
physical device.

So I have a working rootfs and a rootbak, identically sized independent 
partitions, with snapshot copy of the working root taken at a point I'm 
comfortable that it's stable.  There's further copies on other (also 
bootable) media, on reiserfs in case the after all still under heavy 
development btrfs blows up both my working root and primary backup.

That very nicely solves the whole rescue disk issue, since I effectively 
have my entire installed and configured system, *NOT* just a few rescue 
tools, as a snapshot taken when I did the backup, as ready to go now as 
it was when I did the backup, even if it's slightly out of date now and 
would need an update to bring it current.  That means all documentation, 
a fully configured X and kde install, media players so I can listen to 
music while I'm fixing the system, it's all there and ready to go, just 
as it was at the time I did the backup.

(I actually keep multiple fstab layouts around too, maintained with a 
script so I can update one and run the script to update the others, with 
fstab itself actually being a symlink to the active one, making selecting 
an fstab layout as easy as updating a symlink from single-user-mode.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Can't remove empty directory after kernel panic, no errors in dmesg

[Niklas Schnelle]

Well this system is a laptop with an mSATA disk I don't want to waste too
much space on. However I do use btrfs send, to the laptops hard drive as well
as to another system, as well as normal backups for everything that is not
in git anyway.
I'm currently working on a university course paper, so I had one of those
"Thank the great Turing that I am doing that with LaTeX and have everything
is in git on another system" moments. At least that also makes sure
that those files are not corrupted.

Anyway yes I used an Ubuntu 13.10 Live system, I will have to see whether
it actually got "btrfs check".

On Sat, Dec 7, 2013 at 2:59 PM, Duncan <1i5t5.duncan@cox.net> wrote:
> Niklas Schnelle posted on Sat, 07 Dec 2013 12:37:05 +0000 as excerpted:
>
>> root 470 inode 60984 errors 200
>> root 470 inode 62463 errors 200
>
> Three quick things to note:
>
> 1) As another thread pointed out recently, that's not 200 errors, but an
> error type bitmask, with the 0x200 bit set.  Based on that other thread
> (I'm not a dev and haven't actually looked myself), there's comments/
> varnames in the code (unfortunately only, no non-dev admin-level
> documentation anywhere that I know of) saying what each bitflag
> represents.
>
> 2) I had meant to mention earlier but forgot:  As announced on this list
> IIRC shortly after the kernel 3.12 release, btrfs-progs is now versioned
> similar to the kernel and will follow a similar release schedule as long
> as the level of new code warrants (possibly skipping kernel versions here
> and there as the code ultimately stabilizes and churn slows down), with
> the first release following that version scheme being 3.12.
>
> Your reported version number is v0.20-rc1, no git commit snapshot
> indication, and IIRC that was released late last year, so it's about a
> year old now.  You may wish to try something a bit newer, to match your
> 3.13-release-candidate kernel version.  There have been a number of fixes
> since v0.20-rc1 (including btrfsck being made part of the main btrfs
> command now, as btrfs check) and it's just possible a current version may
> fix your issues.
>
> 3) You asked in what might have been a private mail as I didn't see it
> hit the list, what liveCD, etc, to use, since it's a rootfs btrfs that
> you're working with.  The list mail I'm replying to says you tried a live
> stick (doesn't say the version), so you've worked around that to some
> extent, but as a more general followup based on the multiple independent
> btrfs partitions scheme I use that I mentioned earlier in the thread...
>
> One of the best setups I've come up with over a decade's worth of
> experience here is, as I said, multiple independent partitions, with the
> first level backup actually on an identically sized partition on the same
> physical device.
>
> So I have a working rootfs and a rootbak, identically sized independent
> partitions, with snapshot copy of the working root taken at a point I'm
> comfortable that it's stable.  There's further copies on other (also
> bootable) media, on reiserfs in case the after all still under heavy
> development btrfs blows up both my working root and primary backup.
>
> That very nicely solves the whole rescue disk issue, since I effectively
> have my entire installed and configured system, *NOT* just a few rescue
> tools, as a snapshot taken when I did the backup, as ready to go now as
> it was when I did the backup, even if it's slightly out of date now and
> would need an update to bring it current.  That means all documentation,
> a fully configured X and kde install, media players so I can listen to
> music while I'm fixing the system, it's all there and ready to go, just
> as it was at the time I did the backup.
>
> (I actually keep multiple fstab layouts around too, maintained with a
> script so I can update one and run the script to update the others, with
> fstab itself actually being a symlink to the active one, making selecting
> an fstab layout as easy as updating a symlink from single-user-mode.)
>
> --
> Duncan - List replies preferred.   No HTML msgs.
> "Every nonfree program has a lord, a master --
> and if you use the program, he is your master."  Richard Stallman
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Can't remove empty directory after kernel panic, no errors in dmesg

[Niklas Schnelle]

Ok here is yet another update, I compiled the current git version
of btrfs progs on the Ubuntu Live System and ran that on my filesystem
first as btrfs check then with repair.
Here is the output after repair:
http://niklas.sceneproject.org/btrfs_out.txt

Somehow I still don't see btrfs errors when actually running with that
fillesystem,
even though that looks like a lot of errors to me. I hope
this information is useful. I guess I'll just try my luck until
my course paper is due and then it might be best to just get my
btrfs sent backups even though they are a few weeks old,
thankfully I'm pretty sure everything changed since then is
either updates, in git or in Dropbox.

Thanks for the information and help anyway.

On Sat, Dec 7, 2013 at 2:59 PM, Duncan <1i5t5.duncan@cox.net> wrote:
> Niklas Schnelle posted on Sat, 07 Dec 2013 12:37:05 +0000 as excerpted:
>
>> root 470 inode 60984 errors 200
>> root 470 inode 62463 errors 200
>
> Three quick things to note:
>
> 1) As another thread pointed out recently, that's not 200 errors, but an
> error type bitmask, with the 0x200 bit set.  Based on that other thread
> (I'm not a dev and haven't actually looked myself), there's comments/
> varnames in the code (unfortunately only, no non-dev admin-level
> documentation anywhere that I know of) saying what each bitflag
> represents.
>
> 2) I had meant to mention earlier but forgot:  As announced on this list
> IIRC shortly after the kernel 3.12 release, btrfs-progs is now versioned
> similar to the kernel and will follow a similar release schedule as long
> as the level of new code warrants (possibly skipping kernel versions here
> and there as the code ultimately stabilizes and churn slows down), with
> the first release following that version scheme being 3.12.
>
> Your reported version number is v0.20-rc1, no git commit snapshot
> indication, and IIRC that was released late last year, so it's about a
> year old now.  You may wish to try something a bit newer, to match your
> 3.13-release-candidate kernel version.  There have been a number of fixes
> since v0.20-rc1 (including btrfsck being made part of the main btrfs
> command now, as btrfs check) and it's just possible a current version may
> fix your issues.
>
> 3) You asked in what might have been a private mail as I didn't see it
> hit the list, what liveCD, etc, to use, since it's a rootfs btrfs that
> you're working with.  The list mail I'm replying to says you tried a live
> stick (doesn't say the version), so you've worked around that to some
> extent, but as a more general followup based on the multiple independent
> btrfs partitions scheme I use that I mentioned earlier in the thread...
>
> One of the best setups I've come up with over a decade's worth of
> experience here is, as I said, multiple independent partitions, with the
> first level backup actually on an identically sized partition on the same
> physical device.
>
> So I have a working rootfs and a rootbak, identically sized independent
> partitions, with snapshot copy of the working root taken at a point I'm
> comfortable that it's stable.  There's further copies on other (also
> bootable) media, on reiserfs in case the after all still under heavy
> development btrfs blows up both my working root and primary backup.
>
> That very nicely solves the whole rescue disk issue, since I effectively
> have my entire installed and configured system, *NOT* just a few rescue
> tools, as a snapshot taken when I did the backup, as ready to go now as
> it was when I did the backup, even if it's slightly out of date now and
> would need an update to bring it current.  That means all documentation,
> a fully configured X and kde install, media players so I can listen to
> music while I'm fixing the system, it's all there and ready to go, just
> as it was at the time I did the backup.
>
> (I actually keep multiple fstab layouts around too, maintained with a
> script so I can update one and run the script to update the others, with
> fstab itself actually being a symlink to the active one, making selecting
> an fstab layout as easy as updating a symlink from single-user-mode.)
>
> --
> Duncan - List replies preferred.   No HTML msgs.
> "Every nonfree program has a lord, a master --
> and if you use the program, he is your master."  Richard Stallman
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Can't remove empty directory after kernel panic, no errors in dmesg

[Niklas Schnelle]

Sorry for the high frequency but I just wanted to add that I was able to delete
the bad directory now.

On Sat, Dec 7, 2013 at 4:35 PM, Niklas Schnelle
<niklas.schnelle@gmail.com> wrote:
> Ok here is yet another update, I compiled the current git version
> of btrfs progs on the Ubuntu Live System and ran that on my filesystem
> first as btrfs check then with repair.
> Here is the output after repair:
> http://niklas.sceneproject.org/btrfs_out.txt
>
> Somehow I still don't see btrfs errors when actually running with that
> fillesystem,
> even though that looks like a lot of errors to me. I hope
> this information is useful. I guess I'll just try my luck until
> my course paper is due and then it might be best to just get my
> btrfs sent backups even though they are a few weeks old,
> thankfully I'm pretty sure everything changed since then is
> either updates, in git or in Dropbox.
>
> Thanks for the information and help anyway.
>
> On Sat, Dec 7, 2013 at 2:59 PM, Duncan <1i5t5.duncan@cox.net> wrote:
>> Niklas Schnelle posted on Sat, 07 Dec 2013 12:37:05 +0000 as excerpted:
>>
>>> root 470 inode 60984 errors 200
>>> root 470 inode 62463 errors 200
>>
>> Three quick things to note:
>>
>> 1) As another thread pointed out recently, that's not 200 errors, but an
>> error type bitmask, with the 0x200 bit set.  Based on that other thread
>> (I'm not a dev and haven't actually looked myself), there's comments/
>> varnames in the code (unfortunately only, no non-dev admin-level
>> documentation anywhere that I know of) saying what each bitflag
>> represents.
>>
>> 2) I had meant to mention earlier but forgot:  As announced on this list
>> IIRC shortly after the kernel 3.12 release, btrfs-progs is now versioned
>> similar to the kernel and will follow a similar release schedule as long
>> as the level of new code warrants (possibly skipping kernel versions here
>> and there as the code ultimately stabilizes and churn slows down), with
>> the first release following that version scheme being 3.12.
>>
>> Your reported version number is v0.20-rc1, no git commit snapshot
>> indication, and IIRC that was released late last year, so it's about a
>> year old now.  You may wish to try something a bit newer, to match your
>> 3.13-release-candidate kernel version.  There have been a number of fixes
>> since v0.20-rc1 (including btrfsck being made part of the main btrfs
>> command now, as btrfs check) and it's just possible a current version may
>> fix your issues.
>>
>> 3) You asked in what might have been a private mail as I didn't see it
>> hit the list, what liveCD, etc, to use, since it's a rootfs btrfs that
>> you're working with.  The list mail I'm replying to says you tried a live
>> stick (doesn't say the version), so you've worked around that to some
>> extent, but as a more general followup based on the multiple independent
>> btrfs partitions scheme I use that I mentioned earlier in the thread...
>>
>> One of the best setups I've come up with over a decade's worth of
>> experience here is, as I said, multiple independent partitions, with the
>> first level backup actually on an identically sized partition on the same
>> physical device.
>>
>> So I have a working rootfs and a rootbak, identically sized independent
>> partitions, with snapshot copy of the working root taken at a point I'm
>> comfortable that it's stable.  There's further copies on other (also
>> bootable) media, on reiserfs in case the after all still under heavy
>> development btrfs blows up both my working root and primary backup.
>>
>> That very nicely solves the whole rescue disk issue, since I effectively
>> have my entire installed and configured system, *NOT* just a few rescue
>> tools, as a snapshot taken when I did the backup, as ready to go now as
>> it was when I did the backup, even if it's slightly out of date now and
>> would need an update to bring it current.  That means all documentation,
>> a fully configured X and kde install, media players so I can listen to
>> music while I'm fixing the system, it's all there and ready to go, just
>> as it was at the time I did the backup.
>>
>> (I actually keep multiple fstab layouts around too, maintained with a
>> script so I can update one and run the script to update the others, with
>> fstab itself actually being a symlink to the active one, making selecting
>> an fstab layout as easy as updating a symlink from single-user-mode.)
>>
>> --
>> Duncan - List replies preferred.   No HTML msgs.
>> "Every nonfree program has a lord, a master --
>> and if you use the program, he is your master."  Richard Stallman
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Can't remove empty directory after kernel panic, no errors in dmesg

[Chris Murphy]

On Dec 7, 2013, at 3:36 AM, Niklas Schnelle <niklas.schnelle@gmail.com> wrote:

> Hi List,
> 
> so first the basics. I'm running Arch Linux with 3.13-rc2, btrfs-progs
> 0.20rc1.3-2 from the repo and I'm using a SSD.

I don't see a dmesg, which should contain some information if there are correct or uncorrected problems at mount time.

The first thing to try is the mount time recovery option. It's interesting considering how to do this for rootfs on btrfs, but I think simplest is to just add it to fstab and reboot. Then remove it. Again, check dmesg to see how the recovery went, and then check to see if you have the same problems.

Another way to do it is to change the kernel parameters editing the e.g. GRUB menu entry at boot time. Change the ro to rw, and add recovery to rootflags=. So if the current one is rootflags=subvol=root, then change to rootflags=recovery,subvol=root. This acts as a one time thing.

Ideally I'd like to see the kernel spit out something systemd could use that would cause it to generate a service to remount using recovery (and if multiple devices, possibly degraded, and if either of those fail, to then generate a remount service for ro,recovery and fallback to emergency.target). That way more of this can be done automagically.

So try the recovery mount option first. Before you try btrfs check (or btrfsck), you should upgrade your btrfs-progs to something more recent. v3.12 is current and is what distros should build as of kernel 3.12.


Chris Murphy

Re: Can't remove empty directory after kernel panic, no errors in dmesg

[Chris Murphy]

On Dec 9, 2013, at 11:54 AM, Niklas Schnelle <niklas.schnelle@gmail.com> wrote:

> On Sat, Dec 7, 2013 at 11:30 PM, Chris Murphy <lists@colorremedies.com> wrote:
>> 
>> On Dec 7, 2013, at 3:36 AM, Niklas Schnelle <niklas.schnelle@gmail.com> wrote:
>> 
>>> Hi List,
>>> 
>>> so first the basics. I'm running Arch Linux with 3.13-rc2, btrfs-progs
>>> 0.20rc1.3-2 from the repo and I'm using a SSD.
>> 
>> I don't see a dmesg, which should contain some information if there are correct or uncorrected problems at mount time.
>> 
>> The first thing to try is the mount time recovery option. It's interesting considering how to do this for rootfs on btrfs, but I think simplest is to just add it to fstab and reboot. Then remove it. Again, check dmesg to see how the recovery went, and then check to see if you have the same problems.
>> 
>> Another way to do it is to change the kernel parameters editing the e.g. GRUB menu entry at boot time. Change the ro to rw, and add recovery to rootflags=. So if the current one is rootflags=subvol=root, then change to rootflags=recovery,subvol=root. This acts as a one time thing.
>> 
>> Ideally I'd like to see the kernel spit out something systemd could use that would cause it to generate a service to remount using recovery (and if multiple devices, possibly degraded, and if either of those fail, to then generate a remount service for ro,recovery and fallback to emergency.target). That way more of this can be done automagically.
>> 
>> So try the recovery mount option first. Before you try btrfs check (or btrfsck), you should upgrade your btrfs-progs to something more recent. v3.12 is current and is what distros should build as of kernel 3.12.

> As it says in the threads subject there were no errors in dmesg. I
> tried the recovery mount option now but also
> got no notable output in dmesg. Any idea how to procede? By the since
> I was able to delete the undeleteable directory
> after running btrfs check --repair (freshly compiled git version) my
> filesystem has shown no odd behavior, though
> obviously I'm not really trusting it as there were errors in btrfs
> check when running it again after the repair.

Since you've made some changes, I think we need an update on what errors you're still receiving. I'm a bit baffled that mounting rw,recovery shows nothing in dmesg, yet you get errors with btrfs check. Maybe someone else has some ideas how to get more info. Maybe add kernel parameters 'debug ignore_loglevel' ?


Chris Murphy