mount problem

mount problem

[Simone Ferretti]

Hi all,

we're testing BTRFS on our Debian server.  After a lot of operations
simulating a RAID1 failure, every time I mount my BTRFS RAID1 volume
the kernel logs these messages:

[73894.436173] BTRFS: bdev /dev/etherd/e30.20 errs: wr 33036, rd 0, flush 0, corrupt 2806, gen 0
[73894.436181] BTRFS: bdev /dev/etherd/e60.28 errs: wr 244165, rd 0, flush 0, corrupt 1, gen 4

Everything seems to work nice but I'm courious to know what these
messages mean (in particular what do "gen" and "corrupt" mean?).

# uname -a
Linux dub 3.16-2-amd64 #1 SMP Debian 3.16.3-2 (2014-09-20) x86_64 GNU/Linux

# btrfs --version
Btrfs v3.16

# btrfs fi show
Label: 'btrfs_multiappliance'  uuid: 3452ffdd-c09b-43dd-9adb-cffde8518a72
        Total devices 2 FS bytes used 20.03GiB
        devid    1 size 1.82TiB used 24.03GiB path /dev/etherd/e30.20
        devid    2 size 3.64TiB used 24.03GiB path /dev/etherd/e60.28

# btrfs fi df /media/multiapp
Data, RAID1: total=22.00GiB, used=20.01GiB
System, RAID1: total=32.00MiB, used=16.00KiB
Metadata, RAID1: total=2.00GiB, used=21.23MiB
unknown, single: total=16.00MiB, used=0.00

# dmesg
[82932.655078] BTRFS info (device etherd/e30.20): disk space caching is enabled
[82932.678380] BTRFS: bdev /dev/etherd/e30.20 errs: wr 33036, rd 0, flush 0, corrupt 2806, gen 0
[82932.678388] BTRFS: bdev /dev/etherd/e60.28 errs: wr 244165, rd 0, flush 0, corrupt 1, gen 4


-- 
Thanks in advance,
Simone Ferretti

Re: mount problem

[Duncan]

Simone Ferretti posted on Tue, 23 Sep 2014 14:06:41 +0200 as excerpted:

> we're testing BTRFS on our Debian server.  After a lot of operations
> simulating a RAID1 failure, every time I mount my BTRFS RAID1 volume the
> kernel logs these messages:
> 
> [73894.436173] BTRFS: bdev /dev/etherd/e30.20 errs:
> wr 33036, rd 0, flush 0, corrupt 2806, gen 0
> [73894.436181] BTRFS: bdev /dev/etherd/e60.28 errs:
> wr 244165, rd 0, flush 0, corrupt 1, gen 4
> 
> Everything seems to work nice but I'm courious to know what these
> messages mean (in particular what do "gen" and "corrupt" mean?).

Gen=generation.  The generation or transaction-ID (different names for 
the exact same thing) is a monotonically increasing integer that gets 
updated every time a tree update reaches all the way to the superblock.  
In the error context, it means the superblock had one generation number 
but N other blocks had a different (presumably older) generation number.

Corrupt is simply the number of blocks where the calculated checksum 
didn't match the recorded checksum, thus indicating an error.

Of course rd=read, wr=write...

In raid1 mode scrub can typically find and fix many of these errors.  My 
brtfs are mostly raid1 mode, and when I crash and reboot, scrub nearly 
always finds and fixes errors on the two btrfs (independent btrfs full-
filesystems, not subvolumes, /var/log and /home) I normally have mounted 
rw.

But do note that this is the HISTORIC count, counting all errors since 
the counts were reset.  Thus, they'll still be reported after scrub or 
whatever has fixed them.  As long as the numbers don't increase, you're 
good.  Any increase indicates additional problems.

See btrfs device stats -z to reset the numbers to zero (after printing 
them one last time).

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: mount problem

[Simone Ferretti]

Wed, Sep 24, 2014 at 01:23:32PM +0000, Duncan wrote:
> Simone Ferretti posted on Tue, 23 Sep 2014 14:06:41 +0200 as excerpted:
> 
> > we're testing BTRFS on our Debian server.  After a lot of operations
> > simulating a RAID1 failure, every time I mount my BTRFS RAID1 volume the
> > kernel logs these messages:
> > 
> > [73894.436173] BTRFS: bdev /dev/etherd/e30.20 errs:
> > wr 33036, rd 0, flush 0, corrupt 2806, gen 0
> > [73894.436181] BTRFS: bdev /dev/etherd/e60.28 errs:
> > wr 244165, rd 0, flush 0, corrupt 1, gen 4
> > 
> > Everything seems to work nice but I'm courious to know what these
> > messages mean (in particular what do "gen" and "corrupt" mean?).
> 
> Gen=generation.  The generation or transaction-ID (different names for 
> the exact same thing) is a monotonically increasing integer that gets 
> updated every time a tree update reaches all the way to the superblock.  
> In the error context, it means the superblock had one generation number 
> but N other blocks had a different (presumably older) generation number.
> 
> Corrupt is simply the number of blocks where the calculated checksum 
> didn't match the recorded checksum, thus indicating an error.
>
> See btrfs device stats -z to reset the numbers to zero (after printing 
> them one last time).


Thank you much for your quick and illuminating answer.

I'm wondering if you (or anyone else of course) know if there is btrfs
documentation/papers/anything (besides wiki I did not find anything),
in which it's possible to learn this kind of informations?

-- 
Bye,
Simone

Re: mount problem

[Duncan]

Simone Ferretti posted on Wed, 24 Sep 2014 16:28:35 +0200 as excerpted:

> Wed, Sep 24, 2014 at 01:23:32PM +0000, Duncan wrote:
>> Simone Ferretti posted on Tue, 23 Sep 2014 14:06:41 +0200 as excerpted:
>> 
>>> we're testing BTRFS on our Debian server.  After a lot of operations
>>> simulating a RAID1 failure, every time I mount my BTRFS RAID1 volume
>>> the kernel logs these messages:
>>> 
>>> [73894.436173] BTRFS: bdev /dev/etherd/e30.20 errs:
>>> wr 33036, rd 0, flush 0, corrupt 2806, gen 0
>>> [73894.436181] BTRFS: bdev /dev/etherd/e60.28 errs:
>>> wr 244165, rd 0, flush 0, corrupt 1, gen 4
>>> 
>>> Everything seems to work nice but I'm courious to know what these
>>> messages mean (in particular what do "gen" and "corrupt" mean?).
>> 
>> Gen=generation.  The generation or transaction-ID (different names for
>> the exact same thing) is a monotonically increasing integer that gets
>> updated every time a tree update reaches all the way to the superblock.
>> In the error context, it means the superblock had one generation number
>> but N other blocks had a different (presumably older) generation
>> number.
>> 
>> Corrupt is simply the number of blocks where the calculated checksum
>> didn't match the recorded checksum, thus indicating an error.
>>
>> See btrfs device stats -z to reset the numbers to zero (after printing
>> them one last time).
> 
> 
> Thank you much for your quick and illuminating answer.
> 
> I'm wondering if you (or anyone else of course) know if there is btrfs
> documentation/papers/anything (besides wiki I did not find anything), in
> which it's possible to learn this kind of informations?

I've learned it from the list and wiki, and from general background 
experience and by reading between the lines at times.

For the monotonically increasing counts and a zero-out option case, the 
manpage and help information for btrfs device stats -z, that indicates -z 
resets counts to zero, implies that they continue to count up otherwise.  
At one point I think a dev did confirm that on-list, but it's easy enough 
to read the implication without such confirmation, particularly when it 
matches observed behavior, as it does.


The gen/trans-id thing is in fact covered in the wiki, but at least on 
the user-wiki side, I believe only in passing as it is mentioned on the 
btrfs restore page, here:

https://btrfs.wiki.kernel.org/index.php/Restore

(That is in turn linked from the problem-faq, filesystem won't mount and 
none of the above helped, is there any hope, entry, as well as from the 
built-in-tools section of the main page.)

Of course people only searching for specific things instead of doing 
general research before diving head-first into a new filesystem, thus 
reading most of at least the user section of the wiki, as I did, might 
miss it.

But while it's there, it took an actual problem and trying to actually 
use restore on my own system before the equivalence of trans-id and 
generation actually sunk in.

The corrupt thing probably came from my previous experience, working with 
mdraid and its scrub, and with ECC RAM and the related BIOS scrub 
features.  In general, any admin who has worked with (and understood) any 
sort of checksumming and error detection and correction should have a 
general idea what's going on there, at least after reading the
btrfs-scrub manpage and running it to correct errors a few times, thus 
seeing how its output matches that of the corresponding stats.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

mount problem

[Leonidas Spyropoulos]

Hey all,

I have a weird error with my RAID 0 btrfs partition.
Information for the partitions follow:

# btrfs filesystem show
failed to read /dev/sr0
Label: none  uuid: 1882b025-58e4-4287-98a3-9b772af0ad76
	Total devices 2 FS bytes used 108.16GB
	devid    2 size 74.53GB used 55.26GB path /dev/sdd2
	devid    3 size 74.53GB used 55.26GB path /dev/sde2

Btrfs v0.19-35-g1b444cd

# btrfs device scan
Scanning for Btrfs filesystems
failed to read /dev/sr0

# cat /etc/fstab | grep btrfs
/dev/disk/by-uuid/1882b025-58e4-4287-98a3-9b772af0ad76 	/media/data
	btrfs 	rw,user 0 0

# blkid
/dev/sdd2: UUID="1882b025-58e4-4287-98a3-9b772af0ad76"
UUID_SUB="468b49fa-a0b6-4e11-a312-ef0cafd2890a" TYPE="btrfs"
/dev/sde2: UUID="1882b025-58e4-4287-98a3-9b772af0ad76"
UUID_SUB="cf534558-a317-4259-808b-d950a155fb5d" TYPE="btrfs"

Although I have the config within fstab it doesn't mount on bootup with error:
mount: wrong fs type, bad option, bad superblock on /dev/sde2,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

Kernel messages:
device fsid 8742e45825b08218-76adf02a779ba398 devid 3 transid 4778 /dev/sde2
btrfs: failed to read the system array on sde2

No matter how many times I try with sudo mount -a or sudo mount
/media/data I got the same error.

Now the weird stuff:
If I do a blkid and fdisk -l I can then mount normally the partition.
Any ideas?

Kernel info: Linux woofy 2.6.36-ARCH #1 SMP PREEMPT Sat Jan 8 14:15:27
CET 2011 x86_64 Dual Core AMD Opteron(tm) Processor 165 AuthenticAMD
GNU/Linux
btrfs-progs: Latest from git
I don't know which revision/patch Arch stock kernel has of btrfs.


Thanks,
Leonidas



-- 
Caution: breathing may be hazardous to your health.

Re: mount problem

[Tsutomu Itoh]

(2011/01/12 9:25), Leonidas Spyropoulos wrote:
> Hey all,
> 
> I have a weird error with my RAID 0 btrfs partition.
> Information for the partitions follow:
> 
> # btrfs filesystem show
> failed to read /dev/sr0
> Label: none  uuid: 1882b025-58e4-4287-98a3-9b772af0ad76
> 	Total devices 2 FS bytes used 108.16GB
> 	devid    2 size 74.53GB used 55.26GB path /dev/sdd2
> 	devid    3 size 74.53GB used 55.26GB path /dev/sde2
> 
> Btrfs v0.19-35-g1b444cd
> 
> # btrfs device scan
> Scanning for Btrfs filesystems
> failed to read /dev/sr0
> 
> # cat /etc/fstab | grep btrfs
> /dev/disk/by-uuid/1882b025-58e4-4287-98a3-9b772af0ad76 	/media/data
> 	btrfs 	rw,user 0 0
> 
> # blkid
> /dev/sdd2: UUID="1882b025-58e4-4287-98a3-9b772af0ad76"
> UUID_SUB="468b49fa-a0b6-4e11-a312-ef0cafd2890a" TYPE="btrfs"
> /dev/sde2: UUID="1882b025-58e4-4287-98a3-9b772af0ad76"
> UUID_SUB="cf534558-a317-4259-808b-d950a155fb5d" TYPE="btrfs"
> 
> Although I have the config within fstab it doesn't mount on bootup with error:
> mount: wrong fs type, bad option, bad superblock on /dev/sde2,
>        missing codepage or helper program, or other error
>        In some cases useful info is found in syslog - try
>        dmesg | tail  or so
> 
> Kernel messages:
> device fsid 8742e45825b08218-76adf02a779ba398 devid 3 transid 4778 /dev/sde2
> btrfs: failed to read the system array on sde2

Please see Problem_FAQ in btrfs wiki.
(https://btrfs.wiki.kernel.org/index.php/Problem_FAQ)

Thanks,
Itoh

> 
> No matter how many times I try with sudo mount -a or sudo mount
> /media/data I got the same error.
> 
> Now the weird stuff:
> If I do a blkid and fdisk -l I can then mount normally the partition.
> Any ideas?
> 
> Kernel info: Linux woofy 2.6.36-ARCH #1 SMP PREEMPT Sat Jan 8 14:15:27
> CET 2011 x86_64 Dual Core AMD Opteron(tm) Processor 165 AuthenticAMD
> GNU/Linux
> btrfs-progs: Latest from git
> I don't know which revision/patch Arch stock kernel has of btrfs.
> 
> 
> Thanks,
> Leonidas
> 

Re: mount problem

[Leonidas Spyropoulos]

2011/1/12 Tsutomu Itoh <t-itoh@jp.fujitsu.com>:
> (2011/01/12 9:25), Leonidas Spyropoulos wrote:
>> Hey all,
>>
>> I have a weird error with my RAID 0 btrfs partition.
>> Information for the partitions follow:
>>
>> # btrfs filesystem show
>> failed to read /dev/sr0
>> Label: none =A0uuid: 1882b025-58e4-4287-98a3-9b772af0ad76
>> =A0 =A0 =A0 Total devices 2 FS bytes used 108.16GB
>> =A0 =A0 =A0 devid =A0 =A02 size 74.53GB used 55.26GB path /dev/sdd2
>> =A0 =A0 =A0 devid =A0 =A03 size 74.53GB used 55.26GB path /dev/sde2
>>
>> Btrfs v0.19-35-g1b444cd
>>
>> # btrfs device scan
>> Scanning for Btrfs filesystems
>> failed to read /dev/sr0
>>
>> # cat /etc/fstab | grep btrfs
>> /dev/disk/by-uuid/1882b025-58e4-4287-98a3-9b772af0ad76 =A0 =A0 =A0 =A0=
/media/data
>> =A0 =A0 =A0 btrfs =A0 rw,user 0 0
>>
>> # blkid
>> /dev/sdd2: UUID=3D"1882b025-58e4-4287-98a3-9b772af0ad76"
>> UUID_SUB=3D"468b49fa-a0b6-4e11-a312-ef0cafd2890a" TYPE=3D"btrfs"
>> /dev/sde2: UUID=3D"1882b025-58e4-4287-98a3-9b772af0ad76"
>> UUID_SUB=3D"cf534558-a317-4259-808b-d950a155fb5d" TYPE=3D"btrfs"
>>
>> Although I have the config within fstab it doesn't mount on bootup w=
ith error:
>> mount: wrong fs type, bad option, bad superblock on /dev/sde2,
>> =A0 =A0 =A0 =A0missing codepage or helper program, or other error
>> =A0 =A0 =A0 =A0In some cases useful info is found in syslog - try
>> =A0 =A0 =A0 =A0dmesg | tail =A0or so
>>
>> Kernel messages:
>> device fsid 8742e45825b08218-76adf02a779ba398 devid 3 transid 4778 /=
dev/sde2
>> btrfs: failed to read the system array on sde2
>
> Please see Problem_FAQ in btrfs wiki.
> (https://btrfs.wiki.kernel.org/index.php/Problem_FAQ)
>
> Thanks,
> Itoh
>
>>
>> No matter how many times I try with sudo mount -a or sudo mount
>> /media/data I got the same error.
>>
>> Now the weird stuff:
>> If I do a blkid and fdisk -l I can then mount normally the partition=
=2E
>> Any ideas?
>>
>> Kernel info: Linux woofy 2.6.36-ARCH #1 SMP PREEMPT Sat Jan 8 14:15:=
27
>> CET 2011 x86_64 Dual Core AMD Opteron(tm) Processor 165 AuthenticAMD
>> GNU/Linux
>> btrfs-progs: Latest from git
>> I don't know which revision/patch Arch stock kernel has of btrfs.
>>
>>
>> Thanks,
>> Leonidas
>>
>
>

Hey cwillu and Itoh,

Thanks both for the answers, so as I can see I have 2 options:
Either find the startup scripts of Arch Linux and run the scan command
before parsing fstab (this should be in /etc/rc.conf ?)
OR
edit fstab with parameters the correct devices like:
 /dev/disk/by-uuid/1882b025-58e4-4287-98a3-9b772af0ad76 =A0/media/data
btrfs   device=3D/dev/sde2,device=3D/dev/sdd2,rw,user 0 0

I think though that the devices names are randomly chosen in startup
from udev, right?
(this is the reason I use uuids)
so effectivly i have just an option.

Thanks,
Leonidas

--=20
Caution: breathing may be hazardous to your health.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: mount problem

[Leonidas Spyropoulos]

On 12 January 2011 00:58, Leonidas Spyropoulos <artafinde@gmail.com> wr=
ote:
> 2011/1/12 Tsutomu Itoh <t-itoh@jp.fujitsu.com>:
>> (2011/01/12 9:25), Leonidas Spyropoulos wrote:
>>> Hey all,
>>>
>>> I have a weird error with my RAID 0 btrfs partition.
>>> Information for the partitions follow:
>>>
>>> # btrfs filesystem show
>>> failed to read /dev/sr0
>>> Label: none =A0uuid: 1882b025-58e4-4287-98a3-9b772af0ad76
>>> =A0 =A0 =A0 Total devices 2 FS bytes used 108.16GB
>>> =A0 =A0 =A0 devid =A0 =A02 size 74.53GB used 55.26GB path /dev/sdd2
>>> =A0 =A0 =A0 devid =A0 =A03 size 74.53GB used 55.26GB path /dev/sde2
>>>
>>> Btrfs v0.19-35-g1b444cd
>>>
>>> # btrfs device scan
>>> Scanning for Btrfs filesystems
>>> failed to read /dev/sr0
>>>
>>> # cat /etc/fstab | grep btrfs
>>> /dev/disk/by-uuid/1882b025-58e4-4287-98a3-9b772af0ad76 =A0 =A0 =A0 =
=A0/media/data
>>> =A0 =A0 =A0 btrfs =A0 rw,user 0 0
>>>
>>> # blkid
>>> /dev/sdd2: UUID=3D"1882b025-58e4-4287-98a3-9b772af0ad76"
>>> UUID_SUB=3D"468b49fa-a0b6-4e11-a312-ef0cafd2890a" TYPE=3D"btrfs"
>>> /dev/sde2: UUID=3D"1882b025-58e4-4287-98a3-9b772af0ad76"
>>> UUID_SUB=3D"cf534558-a317-4259-808b-d950a155fb5d" TYPE=3D"btrfs"
>>>
>>> Although I have the config within fstab it doesn't mount on bootup =
with error:
>>> mount: wrong fs type, bad option, bad superblock on /dev/sde2,
>>> =A0 =A0 =A0 =A0missing codepage or helper program, or other error
>>> =A0 =A0 =A0 =A0In some cases useful info is found in syslog - try
>>> =A0 =A0 =A0 =A0dmesg | tail =A0or so
>>>
>>> Kernel messages:
>>> device fsid 8742e45825b08218-76adf02a779ba398 devid 3 transid 4778 =
/dev/sde2
>>> btrfs: failed to read the system array on sde2
>>
>> Please see Problem_FAQ in btrfs wiki.
>> (https://btrfs.wiki.kernel.org/index.php/Problem_FAQ)
>>
>> Thanks,
>> Itoh
>>
>>>
>>> No matter how many times I try with sudo mount -a or sudo mount
>>> /media/data I got the same error.
>>>
>>> Now the weird stuff:
>>> If I do a blkid and fdisk -l I can then mount normally the partitio=
n.
>>> Any ideas?
>>>
>>> Kernel info: Linux woofy 2.6.36-ARCH #1 SMP PREEMPT Sat Jan 8 14:15=
:27
>>> CET 2011 x86_64 Dual Core AMD Opteron(tm) Processor 165 AuthenticAM=
D
>>> GNU/Linux
>>> btrfs-progs: Latest from git
>>> I don't know which revision/patch Arch stock kernel has of btrfs.
>>>
>>>
>>> Thanks,
>>> Leonidas
>>>
>>
>>
>
> Hey cwillu and Itoh,
>
> Thanks both for the answers, so as I can see I have 2 options:
> Either find the startup scripts of Arch Linux and run the scan comman=
d
> before parsing fstab (this should be in /etc/rc.conf ?)
> OR
> edit fstab with parameters the correct devices like:
> =A0/dev/disk/by-uuid/1882b025-58e4-4287-98a3-9b772af0ad76 =A0/media/d=
ata
> btrfs =A0 device=3D/dev/sde2,device=3D/dev/sdd2,rw,user 0 0
>
> I think though that the devices names are randomly chosen in startup
> from udev, right?
> (this is the reason I use uuids)
> so effectivly i have just an option.
>
> Thanks,
> Leonidas
>
> --
> Caution: breathing may be hazardous to your health.
>

I searched around and after consulting with #archlinux irc channel the
best way to do that is to add a hook in mkinitcpio.

This tool already has a btrfs hook current version so simply add it
before filesystem hook and recreate the initramfs

=46or my stock kernel: mkinitcpio -p kernel26

Worked fine.

Thanks
Leonidas
--=20
Caution: breathing may be hazardous to your health.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html