possible bug in balance

possible bug in balance

[luvar]

Hi,
I have today added one device and I have converted metadata to raid1. Than I wanted to convert to raid1 also some data (with balance filter) and try if there is some speedup when reading files (starting programs)... I have issued this command:

luvar@blackdawn:~$ sudo time btrfs balance start -dconvert=raid1 -dusage=20 /home/luvar/programs/
[sudo] password for luvar: 
ERROR: error during balancing '/home/luvar/programs/' - Input/output error
There may be more info in syslog - try dmesg | tail
Command exited with non-zero status 19
0.00user 0.08system 0:08.29elapsed 1%CPU (0avgtext+0avgdata 768maxresident)k
14696inputs+6584outputs (2major+253minor)pagefaults 0swaps


Part of df command:
luvar@blackdawn:~$ df -h
Filesystem                   Size  Used Avail Use% Mounted on
/dev/sdb2                    458G  177G   59G  75% /
/dev/sdb2                    458G  177G   59G  75% /home
/dev/sdb1                    226M   96M  114M  46% /boot
/dev/sdb2                    458G  177G   59G  75% /home/luvar/eclipseWorkspaceAndroid
/dev/sdb2                    458G  177G   59G  75% /home/luvar/eclipseWorkspaceErlang
/dev/sdb2                    458G  177G   59G  75% /home/luvar/programs

root@blackdawn:/home/luvar# dmesg|tail -n 50
[ 8107.693414] attempt to access beyond end of device
[ 8107.693425] sdb2: rw=32, want=480102272, limit=473956352
[ 8107.711854] attempt to access beyond end of device
[ 8107.711863] sdb2: rw=1041, want=480102272, limit=473956352
[ 8107.771103] attempt to access beyond end of device
[ 8107.771114] sdb2: rw=32, want=482410504, limit=473956352
[ 8107.784037] attempt to access beyond end of device
[ 8107.784045] sdb2: rw=1041, want=482410504, limit=473956352
[ 8107.804923] attempt to access beyond end of device
[ 8107.804933] sdb2: rw=32, want=478657496, limit=473956352
[ 8107.817134] attempt to access beyond end of device
[ 8107.817142] sdb2: rw=1041, want=478657496, limit=473956352
[ 8107.835377] attempt to access beyond end of device
[ 8107.835384] sdb2: rw=32, want=480795752, limit=473956352
[ 8107.842977] attempt to access beyond end of device
[ 8107.842985] sdb2: rw=1041, want=480795752, limit=473956352
[ 8107.887768] attempt to access beyond end of device
[ 8107.887778] sdb2: rw=32, want=478931480, limit=473956352
[ 8107.898939] attempt to access beyond end of device
[ 8107.898946] sdb2: rw=1041, want=478931480, limit=473956352
[ 8107.958691] attempt to access beyond end of device
[ 8107.958699] sdb2: rw=32, want=479426840, limit=473956352
[ 8107.966368] attempt to access beyond end of device
[ 8107.966375] sdb2: rw=1041, want=479426840, limit=473956352
[ 8116.097908] attempt to access beyond end of device
[ 8116.097919] sdb2: rw=32, want=478334096, limit=473956352
[ 8116.097923] btrfs_dev_stat_print_on_error: 12 callbacks suppressed
[ 8116.097926] btrfs: bdev /dev/sdb2 errs: wr 638625, rd 65863, flush 0, corrupt 0, gen 0
[ 8116.133108] attempt to access beyond end of device
[ 8116.133118] sdb2: rw=1041, want=478334096, limit=473956352
[ 8116.133124] btrfs: bdev /dev/sdb2 errs: wr 638626, rd 65863, flush 0, corrupt 0, gen 0
[ 8125.065061] attempt to access beyond end of device
[ 8125.065073] sdb2: rw=32, want=481418928, limit=473956352
[ 8125.065077] btrfs: bdev /dev/sdb2 errs: wr 638626, rd 65864, flush 0, corrupt 0, gen 0
[ 8125.084522] attempt to access beyond end of device
[ 8125.084533] sdb2: rw=1041, want=481418928, limit=473956352
[ 8125.084539] btrfs: bdev /dev/sdb2 errs: wr 638627, rd 65864, flush 0, corrupt 0, gen 0
[ 8131.848768] btrfs: relocating block group 472710643712 flags 1
[ 8133.866427] attempt to access beyond end of device
[ 8133.866436] sdb2: rw=0, want=476739152, limit=473956352
[ 8133.866441] btrfs: bdev /dev/sdb2 errs: wr 638627, rd 65865, flush 0, corrupt 0, gen 0
[ 8133.866516] attempt to access beyond end of device
[ 8133.866520] sdb2: rw=0, want=476739152, limit=473956352
[ 8133.866523] btrfs: bdev /dev/sdb2 errs: wr 638627, rd 65866, flush 0, corrupt 0, gen 0
[ 8159.272179] attempt to access beyond end of device
[ 8159.272191] sdb2: rw=32, want=480110048, limit=473956352
[ 8159.272196] btrfs: bdev /dev/sdb2 errs: wr 638627, rd 65867, flush 0, corrupt 0, gen 0
[ 8159.300427] attempt to access beyond end of device
[ 8159.300434] sdb2: rw=1041, want=480110048, limit=473956352
[ 8159.300440] btrfs: bdev /dev/sdb2 errs: wr 638628, rd 65867, flush 0, corrupt 0, gen 0

root@blackdawn:/home/luvar# uname -a
Linux blackdawn 3.13.0-30-generic #55-Ubuntu SMP Fri Jul 4 21:40:53 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

root@blackdawn:/home/luvar# btrfs v
Btrfs v0.20-rc1-189-g704a08c


Am I doing something forbidden (I have not see any structure where raid type is stored per file/subvolume item), or I just hit some problem? What should I try?

PS: After all I will convert all data to raid1, but I want to play first :-)

Re: possible bug in balance

[Duncan]

luvar posted on Fri, 02 Jan 2015 15:42:29 +0100 as excerpted:

> root@blackdawn:/home/luvar# uname -a
> [...] 3.13.0-30-generic [...]
> 
> root@blackdawn:/home/luvar# btrfs v
> Btrfs v0.20-rc1-189-g704a08c
> 
> 
> Am I doing something forbidden [...]

Those versions are your problem.  Do you know how fast btrfs is 
improving, how old those are (hint, there are about five kernel release 
series a year and we're now on the 3.19 development series with 3.18 
stable, so 3.13 is... about five releases back and thus about a year old!
), and how many bugs have been fixed since then?

Try something newer than the paleolithic age, say a current 3.18.x stable 
series kernel or a 3.19-rc, and the recent btrfs-progs 3.18.0, and you 
should see rather better results.  There are valid reasons to stick to 
old and stable, but they simply don't mix well with a new and still not 
completely stable filesystem that still sees regular bug fixes, and not 
just for code introduced in the last kernel, either!  Therefore, if you 
want old and stable, choose a different filesystem; while if you want to 
run btrfs, staying current is strongly recommended.

Meanwhile, if you haven't yet, you may want to spend some time on the 
wiki reading up.  Take a look at the stuff under section 5.1, Guides and 
usage information, especially.  It'll answer a lot of questions and could 
save your data too. =:^)

https://btrfs.wiki.kernel.org

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: possible bug in balance

[Erkki Seppala]

luvar@plaintext.sk writes:

Hello,

> luvar@blackdawn:~$ sudo time btrfs balance start -dconvert=raid1 -dusage=20 /home/luvar/programs/

> Am I doing something forbidden (I have not see any structure where
> raid type is stored per file/subvolume item), or I just hit some
> problem? What should I try?

btrfs doesn't yet support per-subvolume RAID1 levels. I'm not sure how
it should behave with your command line. It probably tries to
rebalance the whole filesystem.

> Than I wanted to convert to raid1 also some data (with balance
> filter) and try if there is some speedup when reading files
> (starting programs)...

Though I can already tell that no, there won't be a speedup, as btrfs
scheduler chooses the device to access by using the process id as a
seed. Therefore a single thread is never able to use 100% RAID1 input
capability. Perhaps in future there will be more sophisticated
schedulers. You may try to use MD raid1 for extra speed, but you would
lose the automatic error recovery of btrfs (but you would still notice
if data gets corrupted).

> [ 8159.300427] attempt to access beyond end of device
> [ 8159.300434] sdb2: rw=1041, want=480110048, limit=473956352
> [ 8159.300440] btrfs: bdev /dev/sdb2 errs: wr 638628, rd 65867, flush 0, corrupt 0, gen 0

I have noticed that 'attempt to access beyond end of device' typically
indicates (with other file systems, I haven't seen that with btrfs)
that the partition table and the filesystem size don't
match. Typically such a situation could occur when one modifies
partition table after creating the file system, though I'm sure there
are other ways to get into such a situation. You may find the
filesystem size with btrfs filesystem show and partition sizes with
cat /proc/partitions (multiply by block size = 1024 bytes).

Should the partition sizes and filesystem sizes match, I would be
quite certain this would indeed be a btrfs bug. But,

> root@blackdawn:/home/luvar# uname -a
> Linux blackdawn 3.13.0-30-generic #55-Ubuntu SMP Fri Jul 4 21:40:53 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
> root@blackdawn:/home/luvar# btrfs v
> Btrfs v0.20-rc1-189-g704a08c

should this turn out to be a bug, I'm certain trying a more recent
kernel version is a terrific idea :). 3.18.x or 3.17.y where y>2 (I
think those were the two versions that were bad in 3.17
series..). They won't have support for raid1'n a subvolume either,
though, as far as I know.

Remember backups :).

-- 
  _____________________________________________________________________
     / __// /__ ____  __               http://www.modeemi.fi/~flux/\   \
    / /_ / // // /\ \/ /                                            \  /
   /_/  /_/ \___/ /_/\_\@modeemi.fi                                  \/