kernel BUG at volumes.c:1705

kernel BUG at volumes.c:1705

[Thomas Lindroth]

Hello

I'm using a recently pulled stable checkout on Gentoo i386 with kernel
2.6.24 and I get this message:

16:30:23 multivac [warning] btrfs csum failed ino 267 off 495616 csum
1766296807 private 3283918985
16:30:23 multivac [warning] btrfs csum failed ino 267 off 491520 csum
1623523565 private 3537565334
16:30:23 multivac [warning] btrfs csum failed ino 267 off 487424 csum
3058677931 private 870723426
16:30:23 multivac [warning] btrfs csum failed ino 267 off 487424 csum
3058677931 private 870723426
16:30:23 multivac [warning] btrfs csum failed ino 267 off 491520 csum
1623523565 private 3537565334
16:30:23 multivac [warning] btrfs csum failed ino 267 off 495616 csum
1766296807 private 3283918985
16:30:23 multivac [warning] btrfs csum failed ino 267 off 487424 csum
3058677931 private 870723426
16:30:23 multivac [warning] btrfs csum failed ino 267 off 487424 csum
3058677931 private 870723426
16:30:23 multivac [warning] btrfs csum failed ino 267 off 487424 csum
3058677931 private 870723426
16:30:23 multivac [warning] btrfs csum failed ino 267 off 487424 csum
3058677931 private 870723426
16:30:23 multivac [warning] btrfs csum failed ino 267 off 487424 csum
3058677931 private 870723426
16:30:23 multivac [warning] btrfs csum failed ino 267 off 487424 csum
3058677931 private 870723426
16:30:23 multivac [warning] btrfs csum failed ino 267 off 487424 csum
3058677931 private 870723426
16:30:23 multivac [warning] btrfs csum failed ino 267 off 487424 csum
3058677931 private 870723426
16:31:27 multivac [emerg] ------------[ cut here ]------------
16:31:27 multivac [crit] kernel BUG at
/home/cocobo/btrfs_dev/kernel/volumes.c:1705!
16:31:27 multivac [emerg] invalid opcode: 0000 [#1] PREEMPT SMP
16:31:27 multivac [warning] Modules linked in: btrfs crc32c libcrc32c
aes_i586 aes_generic nvidia(P) fan k8temp gspca snd_intel8x0 button
16:31:27 multivac [warning]
16:31:27 multivac [warning] Pid: 19052, comm: ctorrent Tainted: P
  (2.6.24-dirty #9)
16:31:27 multivac [warning] EIP: 0060:[<f966bfc9>] EFLAGS: 00010246 CPU: 0
16:31:27 multivac [warning] EIP is at btrfs_num_copies+0xc9/0xd0 [btrfs]
16:31:27 multivac [warning] EAX: 00000000 EBX: f522c140 ECX: 00000000
EDX: 00000001
16:31:27 multivac [warning] ESI: f522c148 EDI: ac5e4881 EBP: 3dba8102
ESP: f20a5abc
16:31:27 multivac [warning] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
16:31:27 multivac [emerg] Process ctorrent (pid: 19052, ti=f20a4000
task=f2bb4000 task.ti=f20a4000)
16:31:27 multivac [emerg] Stack: 00001000 00000000 00000000 f1ac4bd0
00000000 fffffffb c625202c f964c325
16:31:27 multivac [emerg] 00001000 00000000 00000001 f964da70 00000000
f50b9600 f1ac4bd0 f50b9600
16:31:27 multivac [emerg] f50b9600 f1ac48c0 f964c999 00000000 00000000
29301005 1032a0a7 00001000
16:31:27 multivac [emerg] Call Trace:
16:31:27 multivac [emerg] [<f964c325>]
btree_read_extent_buffer_pages+0x95/0xb0 [btrfs]
16:31:27 multivac [emerg] [<f964da70>] btree_get_extent+0x0/0x250 [btrfs]
16:31:27 multivac [emerg] [<f964c999>] read_tree_block+0x49/0x60 [btrfs]
16:31:27 multivac [emerg] [<f963bf8f>] read_node_slot+0xdf/0xf0 [btrfs]
16:31:27 multivac [emerg] [<f9640c2f>] btrfs_search_slot+0x13f/0xec0 [btrfs]
16:31:27 multivac [emerg] [<f966ab50>] alloc_extent_buffer+0x210/0x2e0 [btrfs]
16:31:27 multivac [emerg] [<f9657945>] btrfs_get_extent+0x485/0xf30 [btrfs]
16:31:27 multivac [emerg] [<f9666521>] insert_state+0x101/0x190 [btrfs]
16:31:27 multivac [emerg] [<f964ad8e>] btrfs_lookup_csum+0x5e/0x180 [btrfs]
16:31:27 multivac [emerg] [<f9655ca2>] btrfs_readpage_io_hook+0xa2/0x150 [btrfs]
16:31:27 multivac [emerg] [<f9655c00>] btrfs_readpage_io_hook+0x0/0x150 [btrfs]
16:31:27 multivac [emerg] [<f9667829>]
__extent_read_full_page+0x439/0x7e0 [btrfs]
16:31:27 multivac [emerg] [<c012ee85>] __do_softirq+0x75/0xf0
16:31:27 multivac [emerg] [<f96574c0>] btrfs_get_extent+0x0/0xf30 [btrfs]
16:31:27 multivac [emerg] [<c04b81ca>] _write_unlock_irq+0xa/0x20
16:31:27 multivac [emerg] [<f966a8cf>] extent_readpages+0xcf/0x140 [btrfs]
16:31:27 multivac [emerg] [<c015ffa3>] __alloc_pages+0x63/0x370
16:31:27 multivac [emerg] [<f965493d>] btrfs_readpages+0x1d/0x30 [btrfs]
16:31:27 multivac [emerg] [<f96574c0>] btrfs_get_extent+0x0/0xf30 [btrfs]
16:31:27 multivac [emerg] [<f9654920>] btrfs_readpages+0x0/0x30 [btrfs]
16:31:27 multivac [emerg] [<c016237c>] __do_page_cache_readahead+0x18c/0x250
16:31:27 multivac [emerg] [<c01627bb>] page_cache_sync_readahead+0x2b/0x30
16:31:27 multivac [emerg] [<c015b0d3>] do_generic_mapping_read+0x2b3/0x460
16:31:27 multivac [emerg] [<c015a540>] file_read_actor+0x0/0x100
16:31:27 multivac [emerg] [<c015cbbe>] generic_file_aio_read+0xce/0x1c0
16:31:27 multivac [emerg] [<c015a540>] file_read_actor+0x0/0x100
16:31:27 multivac [emerg] [<c017d6e5>] do_sync_read+0xd5/0x120
16:31:27 multivac [emerg] [<c013de50>] autoremove_wake_function+0x0/0x50
16:31:27 multivac [emerg] [<c017dfe5>] vfs_read+0xb5/0x160
16:31:27 multivac [emerg] [<c017d1ec>] vfs_llseek+0x3c/0x50
16:31:27 multivac [emerg] [<c017d610>] do_sync_read+0x0/0x120
16:31:27 multivac [emerg] [<c017e521>] sys_read+0x41/0x70
16:31:27 multivac [emerg] [<c01042ee>] sysenter_past_esp+0x5f/0x85
16:31:27 multivac [emerg] =======================
16:31:27 multivac [emerg] Code: 08 e8 ac fd fe ff 89 d8 83 c4 0c 5b 5e
5f 5d c3 31 c9 eb c9 89 c8 31 d2 bb 01 00 00 00 83 e0 40 89 d1 09 c1
74 d7 8b 5e 1c eb d2 <0f> 0b eb fe 8d 76 00 55 89 c5 57 56 53 83 ec 0c
89 54 24 04 89
16:31:27 multivac [emerg] EIP: [<f966bfc9>] btrfs_num_copies+0xc9/0xd0
[btrfs] SS:ESP 0068:f20a5abc
16:31:27 multivac [warning] ---[ end trace d88035f76ac38fa5 ]---


Steps I took leading up to this message was:
Create a new FS on an ordinary block device: mkfs.btrfs /dev/sdb5
Mount it without any options: mount /dev/sdb5 /mnt/temp-test
Place a .torrent file on the FS and start download using the ctorrent
bittorrent client.
The message occurred soon after and there was plenty of space left on the FS.

The FS now appears to be in an inconsistent state.
# uname -a
Linux multivac 2.6.24-dirty #9 SMP PREEMPT Tue Jun 10 12:12:45 CEST
2008 i686 AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ AuthenticAMD
GNU/Linux
# btrfs-show
failed to read /dev/sr1
failed to read /dev/sr0
Label: none  uuid: e9558f43-b22a-4318-90d2-ed3d8454378e
        Total devices 1 FS bytes used 3.02GB
        devid    1 size 27.94GB used 8.04GB path /dev/sdb5

# btrfsck /dev/sdb5
Segmentation fault

# debug-tree /dev/sdb5
<lots of output>
leaf 60243968 ptrs 2 free space 2425 generation 217 owner 5
fs uuid e9558f43-b22a-4318-90d2-ed3d8454378e
chunk uuid 4ab6cb38-0c05-418a-83a2-65368c0e1428
        item 0 key (262 13 4034560) itemoff 3847 itemsize 148
                csum item
        item 1 key (262 13 8376320) itemoff 2475 itemsize 1372
                csum item
Segmentation fault


I got a similar message yesterday using plain 0.15 without any
hotfixes. The message occurred when I was unrar:ing files on the FS.
Both messages show crc errors but I don't think there is anything
wrong with the disk.

22:04:08 multivac [warning] btrfs csum failed ino 126244 off 14999552
csum 372251014 private 2566472073
22:04:08 multivac [warning] btrfs csum failed ino 126244 off 14999552
csum 372251014 private 2566472073
22:04:58 multivac [crit] kernel BUG at
/var/tmp/portage/sys-fs/btrfs-0.15-r1/work/btrfs-0.15/volumes.c:1705!
22:04:58 multivac [emerg] invalid opcode: 0000 [#1] PREEMPT SMP
22:04:58 multivac [warning] Modules linked in: btrfs libcrc32c
aes_i586 aes_generic nvidia(P) k8temp fan gspca button snd_intel8x0
22:04:58 multivac [warning]
22:04:58 multivac [warning] Pid: 9026, comm: unrar Tainted: P
(2.6.24-dirty #9)
22:04:58 multivac [warning] EIP: 0060:[<f962cfa9>] EFLAGS: 00210246 CPU: 1
22:04:58 multivac [warning] EIP is at btrfs_num_copies+0xc9/0x140 [btrfs]
22:04:58 multivac [warning] EAX: 00000000 EBX: ee530140 ECX: 00000000
EDX: 00000001
22:04:58 multivac [warning] ESI: ee530148 EDI: 0f1e3080 EBP: 26e012ff
ESP: f71cbad0
22:04:58 multivac [warning] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
22:04:58 multivac [emerg] Process unrar (pid: 9026, ti=f71ca000
task=c5485a90 task.ti=f71ca000)
22:04:58 multivac [emerg] Stack: 00001000 00000000 00000000 f43235b0
00000000 fffffffb f4cae02c f960d325
22:04:58 multivac [emerg] 00001000 00000000 00000001 f960ea70 00000000
f7005e00 f43235b0 f7005e00
22:04:58 multivac [emerg] f7005e00 f4323380 f960d999 00000000 00000000
6620050e 4f9f4410 00001000
22:04:58 multivac [emerg] Call Trace:
22:04:58 multivac [emerg] [<f960d325>] btrfs_csum_final+0x245/0x260 [btrfs]
22:04:58 multivac [emerg] [<f960ea70>] btree_get_extent+0x0/0x270 [btrfs]
22:04:58 multivac [emerg] [<f960d999>] read_tree_block+0x49/0x130 [btrfs]
22:04:58 multivac [emerg] [<f95fcf8f>] btrfs_release_path+0x3ef/0x400 [btrfs]
22:04:58 multivac [emerg] [<f9601c2f>] btrfs_search_slot+0x13f/0xec0 [btrfs]
22:04:58 multivac [emerg] [<c0462009>] ip_route_input+0x39/0xda0
22:04:58 multivac [emerg] [<c047e9fb>] tcp_v4_do_rcv+0xab/0x360
22:04:58 multivac [emerg] [<f9618925>] btrfs_get_extent+0x485/0x1590 [btrfs]
22:04:58 multivac [emerg] [<f9627501>]
wait_on_extent_buffer_writeback+0x2a1/0x330 [btrfs]
22:04:58 multivac [emerg] [<c0464c84>] ip_local_deliver_finish+0xc4/0x170
22:04:58 multivac [emerg] [<f960bd8e>] btrfs_lookup_csum+0x5e/0x180 [btrfs]
22:04:58 multivac [emerg] [<f9616c82>] btrfs_readpage_io_hook+0xa2/0xb90 [btrfs]
22:04:58 multivac [emerg] [<f9616be0>] btrfs_readpage_io_hook+0x0/0xb90 [btrfs]
22:04:58 multivac [emerg] [<f9628809>] unlock_extent+0x489/0x830 [btrfs]
22:04:58 multivac [emerg] [<f96184a0>] btrfs_get_extent+0x0/0x1590 [btrfs]
22:04:58 multivac [emerg] [<f962b8af>] extent_readpages+0xcf/0x140 [btrfs]
22:04:58 multivac [emerg] [<c015ffa3>] __alloc_pages+0x63/0x370
22:04:58 multivac [emerg] [<c0479edb>] tcp_send_ack+0x2b/0x100
22:04:58 multivac [emerg] [<c0426948>] __kfree_skb+0x8/0x80
22:04:58 multivac [emerg] [<f961591d>]
btrfs_readpage_end_io_hook+0x77d/0x810 [btrfs]
22:04:58 multivac [emerg] [<f96184a0>] btrfs_get_extent+0x0/0x1590 [btrfs]
22:04:58 multivac [emerg] [<f9615900>]
btrfs_readpage_end_io_hook+0x760/0x810 [btrfs]
22:04:58 multivac [emerg] [<c016237c>] __do_page_cache_readahead+0x18c/0x250
22:04:58 multivac [emerg] [<c015af86>] do_generic_mapping_read+0x166/0x460
22:04:58 multivac [emerg] [<c015a540>] file_read_actor+0x0/0x100
22:04:58 multivac [emerg] [<c015cbbe>] generic_file_aio_read+0xce/0x1c0
22:04:58 multivac [emerg] [<c015a540>] file_read_actor+0x0/0x100
22:04:58 multivac [emerg] [<c017d6e5>] do_sync_read+0xd5/0x120
22:04:58 multivac [emerg] [<c013de50>] autoremove_wake_function+0x0/0x50
22:04:58 multivac [emerg] [<c042e089>] net_tx_action+0x49/0xf0
22:04:58 multivac [emerg] [<f9a11111>] nv_kern_isr+0x51/0xb0 [nvidia]
22:04:58 multivac [emerg] [<c017dfe5>] vfs_read+0xb5/0x160
22:04:58 multivac [emerg] [<c017d610>] do_sync_read+0x0/0x120
22:04:58 multivac [emerg] [<c017e521>] sys_read+0x41/0x70
22:04:58 multivac [emerg] [<c01042ee>] sysenter_past_esp+0x5f/0x85
22:04:58 multivac [emerg] =======================
22:04:58 multivac [emerg] Code: 08 e8 ac fd fe ff 89 d8 83 c4 0c 5b 5e
5f 5d c3 31 c9 eb c9 89 c8 31 d2 bb 01 00 00 00 83 e0 40 89 d1 09 c1
74 d7 8b 5e 1c eb d2 <0f> 0b eb fe 8d 76 00 55 89 c5 57 56 53 83 ec 0c
89 54 24 04 89
22:04:58 multivac [emerg] EIP: [<f962cfa9>]
btrfs_num_copies+0xc9/0x140 [btrfs] SS:ESP 0068:f71cbad0
22:04:58 multivac [warning] ---[ end trace 8259e45f08f9ee51 ]---

Re: kernel BUG at volumes.c:1705

[Chris Mason]

On Thu, 2008-06-12 at 18:07 +0200, Thomas Lindroth wrote:
> Hello
> 
> I'm using a recently pulled stable checkout on Gentoo i386 with kernel
> 2.6.24 and I get this message:

> 
> I got a similar message yesterday using plain 0.15 without any
> hotfixes. The message occurred when I was unrar:ing files on the FS.
> Both messages show crc errors but I don't think there is anything
> wrong with the disk.
> 

The hotfixes do fix a memory corruption that was present in 0.14.  Could
you please try the hotfixes?

-chris

Re: kernel BUG at volumes.c:1705

[Thomas Lindroth]

On 6/13/08, Chris Mason <chris.mason@oracle.com> wrote:
> On Thu, 2008-06-12 at 18:07 +0200, Thomas Lindroth wrote:
>  > Hello
>  >
>  > I'm using a recently pulled stable checkout on Gentoo i386 with kernel
>  > 2.6.24 and I get this message:
>
>
>
> > I got a similar message yesterday using plain 0.15 without any
>  > hotfixes. The message occurred when I was unrar:ing files on the FS.
>  > Both messages show crc errors but I don't think there is anything
>  > wrong with the disk.
>  >
>
>
> The hotfixes do fix a memory corruption that was present in 0.14.  Could
>  you please try the hotfixes?

The first time I got the message I was using 0.15 without hotfixes but
after that I started using the stable repository from
http://www.kernel.org/hg/btrfs/kernel and still got the same or a
similar message. It looks like the stable repository contain the
hotfixes. I included both messages for reference in my previous post.
If a message contains the line
kernel BUG at /home/cocobo/btrfs_dev/kernel/volumes.c:1705!
then that message is from when I was using the stable repository version.

Re: kernel BUG at volumes.c:1705

[Thomas Lindroth]

Chris Mason wrote:
> So, is the bittorrent program run every time you see the memory
> corruption?
> 
> Could you please send along a few of the most common errors you see?


It's possible to recreate it with mv, cp and other tools without having
the torrent program running.

I've found a way to trigger the problem every time. If I copy a folder
with music videos to a freshly created FS it will always fail on the
same file when about 5.6G of data has been copied. The FS is 30G and
there is nothing special with that file. If I only copy that file to a
fresh FS everything works. I've also tried to run
dd if=/dev/zero of=test to a fresh FS and I was able to write 15G, but I
was not able to remove that file.


cocobo@multivac /mnt/temp-test $ dd if=/dev/zero of=test bs=16384
930632+0 records in
930632+0 records out
15247474688 bytes (15 GB) copied, 374.052 s, 40.8 MB/s

cocobo@multivac /mnt/temp-test $ echo sdlfkj > sdlfkj
cocobo@multivac /mnt/temp-test $ echo sdlfkj > sdlfkjdf
cocobo@multivac /mnt/temp-test $ echo sdlfkj > sdlfkjdfer
cocobo@multivac /mnt/temp-test $ d
sdlfkj  sdlfkjdf  sdlfkjdfer  test
cocobo@multivac /mnt/temp-test $ rm *
Segmentation fault

Resulting backtrace:

[crit] kernel BUG at /home/cocobo/btrfs_dev/kernel/inode.c:962!
[emerg] invalid opcode: 0000 [#1] PREEMPT SMP
[warning] Modules linked in: btrfs crc32c libcrc32c aes_i586 aes_generic nvidia(P) fan k8temp button snd_intel8x0
[warning]
[warning] Pid: 7187, comm: rm Tainted: P        (2.6.24-dirty #9)
[warning] EIP: 0060:[<f95b2b45>] EFLAGS: 00210246 CPU: 1
[warning] EIP is at btrfs_truncate_in_trans+0x725/0x7d0 [btrfs]
[warning] EAX: 00000000 EBX: 00000001 ECX: e18acd80 EDX: e18acd80
[warning] ESI: 00000001 EDI: 00000001 EBP: 00000002 ESP: f5b2be58
[warning] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
[emerg] Process rm (pid: 7187, ti=f5b2a000 task=f5939540 task.ti=f5b2a000)
[emerg] Stack: e18acd80 ffffffff 00000001 f5b2becc c015cd0a 0000000e e18cd36c 00000000
[emerg] f5b2bec4 00000000 e18cd35c c0163102 f5b2becc 3be54000 00000002 e18cd2ac
[emerg] f58cd200 e1a02c80 e18acd80 00000013 f5c4bb60 00000000 00000000 00000000
[emerg] Call Trace:
[emerg] [<c015cd0a>] find_get_pages+0x5a/0x70
[emerg] [<c0163102>] pagevec_lookup+0x22/0x30
[emerg] [<f95b634f>] btrfs_delete_inode+0x7f/0x120 [btrfs]
[emerg] [<f95b62d0>] btrfs_delete_inode+0x0/0x120 [btrfs]
[emerg] [<c0191265>] generic_delete_inode+0x65/0xd0
[emerg] [<c0190a3c>] iput+0x5c/0x70
[emerg] [<c0187b4f>] do_unlinkat+0xef/0x150
[emerg] [<c01042ee>] sysenter_past_esp+0x5f/0x85
[emerg] =======================
[emerg] Code: 74 29 8b 4c 24 48 48 89 41 20 e9 16 fa ff ff 8b 7c 24 78 85 ff 0f 85 3c fb ff ff 8b 44 24 40 89 f2 e8 60 a0 fe ff e9 be f9 ff ff <0f> 0b eb fe 0f 0b eb fe 29 f8 8b 4c 24 34 8b 5c 24 38 89 84 24
[emerg] EIP: [<f95b2b45>] btrfs_truncate_in_trans+0x725/0x7d0 [btrfs] SS:ESP 0068:f5b2be58
[warning] ---[ end trace e9838122963995d9 ]---


When I copy music videos I get a backtrace like this:

[warning] bad mapping eb start 29790208 len 4096, wanted 3655755309 4
[warning] WARNING: at /home/cocobo/btrfs_dev/kernel/extent_io.c:3225 map_private_extent_buffer()
[warning] Pid: 8230, comm: cp Tainted: P        2.6.24-dirty #9
[warning] [<f95c1b16>] map_private_extent_buffer+0x126/0x170 [btrfs]
[warning] [<f95ba10d>] btrfs_item_offset+0x5d/0xe0 [btrfs]
[warning] [<f9599436>] leaf_space_used+0x96/0xe0 [btrfs]
[warning] [<f95994c2>] btrfs_leaf_free_space+0x42/0xa0 [btrfs]
[warning] [<f959ac6a>] push_leaf_right+0x8a/0x650 [btrfs]
[warning] [<c013dea0>] wake_bit_function+0x0/0x60
[warning] [<f95c28a2>] add_lru+0x82/0x90 [btrfs]
[warning] [<f95c1a7e>] map_private_extent_buffer+0x8e/0x170 [btrfs]
[warning] [<f95a915d>] verify_parent_transid+0x6d/0x1a0 [btrfs]
[warning] [<f95c7b50>] alloc_extent_buffer+0x210/0x2e0 [btrfs]
[warning] [<f959b694>] split_leaf+0x84/0x940 [btrfs]
[warning] [<f9599436>] leaf_space_used+0x96/0xe0 [btrfs]
[warning] [<f95994c2>] btrfs_leaf_free_space+0x42/0xa0 [btrfs]
[warning] [<f959e5f4>] btrfs_search_slot+0xb04/0xec0 [btrfs]
[warning] [<f95c1a7e>] map_private_extent_buffer+0x8e/0x170 [btrfs]
[warning] [<f95c3320>] tree_insert+0x60/0x90 [btrfs]
[warning] [<f9599436>] leaf_space_used+0x96/0xe0 [btrfs]
[warning] [<f95c2a19>] merge_state+0xb9/0x110 [btrfs]
[warning] [<f95c3a24>] set_extent_bit+0x3f4/0x460 [btrfs]
[warning] [<f959ea2d>] btrfs_insert_empty_items+0x7d/0x420 [btrfs]
[warning] [<f95c1a7e>] map_private_extent_buffer+0x8e/0x170 [btrfs]
[warning] [<f95c3dbf>] set_extent_dirty+0x3f/0x50 [btrfs]
[warning] [<f95a73d3>] insert_with_overflow+0x33/0x130 [btrfs]
[warning] [<f95a7982>] btrfs_insert_dir_item+0x172/0x240 [btrfs]
[warning] [<f95afb77>] btrfs_add_link+0x87/0x120 [btrfs]
[warning] [<f95afc31>] btrfs_add_nondir+0x21/0x50 [btrfs]
[warning] [<f95b57e8>] btrfs_create+0x1c8/0x2b0 [btrfs]
[warning] [<c018626a>] vfs_create+0xaa/0xf0
[warning] [<c01888f9>] open_namei+0x559/0x5d0
[warning] [<c017bdfe>] do_filp_open+0x2e/0x60
[warning] [<c017bae0>] get_unused_fd_flags+0xb0/0xd0
[warning] [<c017be7c>] do_sys_open+0x4c/0xf0
[warning] [<c017bf5c>] sys_open+0x1c/0x20
[warning] [<c01042ee>] sysenter_past_esp+0x5f/0x85
[warning] =======================
[alert] BUG: unable to handle kernel NULL pointer dereference at virtual address 00000000
[alert] printing eip: c0120e51 *pde = 00000000
[emerg] Oops: 0000 [#1] PREEMPT SMP
[warning] Modules linked in: btrfs crc32c libcrc32c aes_i586 aes_generic nvidia(P) k8temp fan button snd_intel8x0
[warning]
[warning] Pid: 8230, comm: cp Tainted: P        (2.6.24-dirty #9)
[warning] EIP: 0060:[<c0120e51>] EFLAGS: 00210202 CPU: 1
[warning] EIP is at kmap_atomic_prot+0x11/0xa0
[warning] EAX: f340c000 EBX: 00000000 ECX: 00000163 EDX: 00000004
[warning] ESI: 00000000 EDI: 00000163 EBP: f7822c78 ESP: f340da84
[warning] DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
[emerg] Process cp (pid: 8230, ti=f340c000 task=f75cc550 task.ti=f340c000)
[emerg] Stack: f3302154 00000000 f3302144 f95c1a7e f95cf135 f95ce938 00000c99 f95cdab9
[emerg] d9e6662d 00000004 00000000 00001000 d9e6661c f7822c78 d9e6662d 00000000
[emerg] f95ba10d f340daec f340dae8 f340dae4 f340dae0 00000004 92109210 00001000
[emerg] Call Trace:
[emerg] [<f95c1a7e>] map_private_extent_buffer+0x8e/0x170 [btrfs]
[emerg] [<f95ba10d>] btrfs_item_offset+0x5d/0xe0 [btrfs]
[emerg] [<f9599436>] leaf_space_used+0x96/0xe0 [btrfs]
[emerg] [<f95994c2>] btrfs_leaf_free_space+0x42/0xa0 [btrfs]
[emerg] [<f959ac6a>] push_leaf_right+0x8a/0x650 [btrfs]
[emerg] [<c013dea0>] wake_bit_function+0x0/0x60
[emerg] [<f95c28a2>] add_lru+0x82/0x90 [btrfs]
[emerg] [<f95c1a7e>] map_private_extent_buffer+0x8e/0x170 [btrfs]
[emerg] [<f95a915d>] verify_parent_transid+0x6d/0x1a0 [btrfs]
[emerg] [<f95c7b50>] alloc_extent_buffer+0x210/0x2e0 [btrfs]
[emerg] [<f959b694>] split_leaf+0x84/0x940 [btrfs]
[emerg] [<f9599436>] leaf_space_used+0x96/0xe0 [btrfs]
[emerg] [<f95994c2>] btrfs_leaf_free_space+0x42/0xa0 [btrfs]
[emerg] [<f959e5f4>] btrfs_search_slot+0xb04/0xec0 [btrfs]
[emerg] [<f95c1a7e>] map_private_extent_buffer+0x8e/0x170 [btrfs]
[emerg] [<f95c3320>] tree_insert+0x60/0x90 [btrfs]
[emerg] [<f9599436>] leaf_space_used+0x96/0xe0 [btrfs]
[emerg] [<f95c2a19>] merge_state+0xb9/0x110 [btrfs]
[emerg] [<f95c3a24>] set_extent_bit+0x3f4/0x460 [btrfs]
[emerg] [<f959ea2d>] btrfs_insert_empty_items+0x7d/0x420 [btrfs]
[emerg] [<f95c1a7e>] map_private_extent_buffer+0x8e/0x170 [btrfs]
[emerg] [<f95c3dbf>] set_extent_dirty+0x3f/0x50 [btrfs]
[emerg] [<f95a73d3>] insert_with_overflow+0x33/0x130 [btrfs]
[emerg] [<f95a7982>] btrfs_insert_dir_item+0x172/0x240 [btrfs]
[emerg] [<f95afb77>] btrfs_add_link+0x87/0x120 [btrfs]
[emerg] [<f95afc31>] btrfs_add_nondir+0x21/0x50 [btrfs]
[emerg] [<f95b57e8>] btrfs_create+0x1c8/0x2b0 [btrfs]
[emerg] [<c018626a>] vfs_create+0xaa/0xf0
[emerg] [<c01888f9>] open_namei+0x559/0x5d0
[emerg] [<c017bdfe>] do_filp_open+0x2e/0x60
[emerg] [<c017bae0>] get_unused_fd_flags+0xb0/0xd0
[emerg] [<c017be7c>] do_sys_open+0x4c/0xf0
[emerg] [<c017bf5c>] sys_open+0x1c/0x20
[emerg] [<c01042ee>] sysenter_past_esp+0x5f/0x85
[emerg] =======================
[emerg] Code: 0c c1 e0 05 03 05 80 6e 64 c0 c3 0f 0b eb fe 8d 74 26 00 8d bc 27 00 00 00 00 57 89 cf 56 53 89 c3 89 e0 25 00 e0 ff ff ff 40 14 <8b> 03 c1 e8 1e 69 c0 80 03 00 00 05 80 a0 5a c0 2b 80 4c 03 00
[emerg] EIP: [<c0120e51>] kmap_atomic_prot+0x11/0xa0 SS:ESP 0068:f340da84
[warning] ---[ end trace e9be59e282d09030 ]---


Some more. Don't know what triggers these.

[warning] btrfs csum failed ino 270 off 13099008 csum 2441591740 private 3195497598
[warning] btrfs csum failed ino 270 off 13099008 csum 2441591740 private 3195497598
[warning] btrfs csum failed ino 265 off 6197248 csum 2251659755 private 2851584110
[warning] btrfs csum failed ino 265 off 6197248 csum 2251659755 private 2851584110
[warning] bad mapping eb start 35897344 len 4096, wanted 2860057669 4
[warning] WARNING: at /home/cocobo/btrfs_dev/kernel/extent_io.c:3225 map_private_extent_buffer()
[warning] Pid: 7388, comm: btrfs-async-sub Tainted: P        2.6.24-dirty #9
[warning] [<f9663b16>] map_private_extent_buffer+0x126/0x170 [btrfs]
[warning] [<f965c10d>] btrfs_item_offset+0x5d/0xe0 [btrfs]
[warning] [<f963b436>] leaf_space_used+0x96/0xe0 [btrfs]
[warning] [<f963b4c2>] btrfs_leaf_free_space+0x42/0xa0 [btrfs]
[warning] [<f963cc6a>] push_leaf_right+0x8a/0x650 [btrfs]
[warning] [<f9665dbf>] set_extent_dirty+0x3f/0x50 [btrfs]
[warning] [<f9663a7e>] map_private_extent_buffer+0x8e/0x170 [btrfs]
[warning] [<f964b15d>] verify_parent_transid+0x6d/0x1a0 [btrfs]
[warning] [<f9669b50>] alloc_extent_buffer+0x210/0x2e0 [btrfs]
[warning] [<f963d694>] split_leaf+0x84/0x940 [btrfs]
[warning] [<f963b436>] leaf_space_used+0x96/0xe0 [btrfs]
[warning] [<f963b4c2>] btrfs_leaf_free_space+0x42/0xa0 [btrfs]
[warning] [<f96405f4>] btrfs_search_slot+0xb04/0xec0 [btrfs]
[warning] [<c02d663a>] cfq_insert_request+0x2a/0x380
[warning] [<f9663a7e>] map_private_extent_buffer+0x8e/0x170 [btrfs]
[warning] [<f965c0a0>] btrfs_item_size+0xd0/0xe0 [btrfs]
[warning] [<f9640a2d>] btrfs_insert_empty_items+0x7d/0x420 [btrfs]
[warning] [<f965c0a0>] btrfs_item_size+0xd0/0xe0 [btrfs]
[warning] [<f964a5c9>] btrfs_csum_file_blocks+0x4a9/0x750 [btrfs]
[warning] [<f966be64>] btrfs_map_bio+0x134/0x220 [btrfs]
[warning] [<f9653cca>] __btrfs_submit_bio_hook+0x7a/0xd0 [btrfs]
[warning] [<f964b8b0>] btrfs_async_submit_work+0x60/0x90 [btrfs]
[warning] [<f964b850>] btrfs_async_submit_work+0x0/0x90 [btrfs]
[warning] [<c013a039>] run_workqueue+0x79/0x100
[warning] [<c013aa60>] worker_thread+0x0/0x100
[warning] [<c013ab04>] worker_thread+0xa4/0x100
[warning] [<c013de50>] autoremove_wake_function+0x0/0x50
[warning] [<c013aa60>] worker_thread+0x0/0x100
[warning] [<c013db82>] kthread+0x42/0x70
[warning] [<c013db40>] kthread+0x0/0x70
[warning] [<c0104f0f>] kernel_thread_helper+0x7/0x18
[warning] =======================
[alert] BUG: unable to handle kernel NULL pointer dereference at virtual address 00000000
[alert] printing eip: c0120e51 *pde = 00000000
[emerg] Oops: 0000 [#1] PREEMPT SMP
[warning] Modules linked in: btrfs crc32c libcrc32c aes_i586 aes_generic nvidia(P) fan gspca button k8temp snd_intel8x0
[warning]
[warning] Pid: 7388, comm: btrfs-async-sub Tainted: P        (2.6.24-dirty #9)
[warning] EIP: 0060:[<c0120e51>] EFLAGS: 00010202 CPU: 0
[warning] EIP is at kmap_atomic_prot+0x11/0xa0
[warning] EAX: f512e000 EBX: 00000000 ECX: 00000163 EDX: 00000004
[warning] ESI: 00000000 EDI: 00000163 EBP: e53ad380 ESP: f512fb68
[warning] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
[emerg] Process btrfs-async-sub (pid: 7388, ti=f512e000 task=f6adcff0 task.ti=f512e000)
[emerg] Stack: f3d53154 00000000 f3d53144 f9663a7e f9671135 f9670938 00000c99 f966fab9
[emerg] aa790445 00000004 00000000 00001000 aa790434 e53ad380 aa790445 00000000
[emerg] f965c10d f512fbd0 f512fbcc f512fbc8 f512fbc4 00000004 de91d27e 00001000
[emerg] Call Trace:
[emerg] [<f9663a7e>] map_private_extent_buffer+0x8e/0x170 [btrfs]
[emerg] [<f965c10d>] btrfs_item_offset+0x5d/0xe0 [btrfs]
[emerg] [<f963b436>] leaf_space_used+0x96/0xe0 [btrfs]
[emerg] [<f963b4c2>] btrfs_leaf_free_space+0x42/0xa0 [btrfs]
[emerg] [<f963cc6a>] push_leaf_right+0x8a/0x650 [btrfs]
[emerg] [<f9665dbf>] set_extent_dirty+0x3f/0x50 [btrfs]
[emerg] [<f9663a7e>] map_private_extent_buffer+0x8e/0x170 [btrfs]
[emerg] [<f964b15d>] verify_parent_transid+0x6d/0x1a0 [btrfs]
[emerg] [<f9669b50>] alloc_extent_buffer+0x210/0x2e0 [btrfs]
[emerg] [<f963d694>] split_leaf+0x84/0x940 [btrfs]
[emerg] [<f963b436>] leaf_space_used+0x96/0xe0 [btrfs]
[emerg] [<f963b4c2>] btrfs_leaf_free_space+0x42/0xa0 [btrfs]
[emerg] [<f96405f4>] btrfs_search_slot+0xb04/0xec0 [btrfs]
[emerg] [<c02d663a>] cfq_insert_request+0x2a/0x380
[emerg] [<f9663a7e>] map_private_extent_buffer+0x8e/0x170 [btrfs]
[emerg] [<f965c0a0>] btrfs_item_size+0xd0/0xe0 [btrfs]
[emerg] [<f9640a2d>] btrfs_insert_empty_items+0x7d/0x420 [btrfs]
[emerg] [<f965c0a0>] btrfs_item_size+0xd0/0xe0 [btrfs]
[emerg] [<f964a5c9>] btrfs_csum_file_blocks+0x4a9/0x750 [btrfs]
[emerg] [<f966be64>] btrfs_map_bio+0x134/0x220 [btrfs]
[emerg] [<f9653cca>] __btrfs_submit_bio_hook+0x7a/0xd0 [btrfs]
[emerg] [<f964b8b0>] btrfs_async_submit_work+0x60/0x90 [btrfs]
[emerg] [<f964b850>] btrfs_async_submit_work+0x0/0x90 [btrfs]
[emerg] [<c013a039>] run_workqueue+0x79/0x100
[emerg] [<c013aa60>] worker_thread+0x0/0x100
[emerg] [<c013ab04>] worker_thread+0xa4/0x100
[emerg] [<c013de50>] autoremove_wake_function+0x0/0x50
[emerg] [<c013aa60>] worker_thread+0x0/0x100
[emerg] [<c013db82>] kthread+0x42/0x70
[emerg] [<c013db40>] kthread+0x0/0x70
[emerg] [<c0104f0f>] kernel_thread_helper+0x7/0x18
[emerg] =======================
[emerg] Code: 0c c1 e0 05 03 05 80 6e 64 c0 c3 0f 0b eb fe 8d 74 26 00 8d bc 27 00 00 00 00 57 89 cf 56 53 89 c3 89 e0 25 00 e0 ff ff ff 40 14 <8b> 03 c1 e8 1e 69 c0 80 03 00 00 05 80 a0 5a c0 2b 80 4c 03 00
[emerg] EIP: [<c0120e51>] kmap_atomic_prot+0x11/0xa0 SS:ESP 0068:f512fb68
[warning] ---[ end trace 3b8b8f59b04f8cf6 ]---


[warning] WARNING: at /home/cocobo/btrfs_dev/kernel/extent-tree.c:2168 walk_down_tree()
[warning] Pid: 18352, comm: btrfs-transacti Tainted: P        2.6.24-dirty #9
[warning] [<f960eda1>] walk_down_tree+0x621/0x740 [btrfs]
[warning] [<f962ad18>] read_extent_buffer+0xb8/0x190 [btrfs]
[warning] [<f9622502>] btrfs_node_key+0x62/0x90 [btrfs]
[warning] [<f960ef4e>] btrfs_drop_snapshot+0x8e/0x1c0 [btrfs]
[warning] [<f96162ca>] drop_dirty_roots+0xea/0x200 [btrfs]
[warning] [<f9929000>] _nv006851rm+0xb4c/0x2674 [nvidia]
[warning] [<f9616da0>] btrfs_transaction_cleaner+0x0/0xe0 [btrfs]
[warning] [<f9616450>] btrfs_clean_old_snapshots+0x70/0x90 [btrfs]
[warning] [<f9616de4>] btrfs_transaction_cleaner+0x44/0xe0 [btrfs]
[warning] [<c013a039>] run_workqueue+0x79/0x100
[warning] [<c013aa60>] worker_thread+0x0/0x100
[warning] [<c013ab04>] worker_thread+0xa4/0x100
[warning] [<c013de50>] autoremove_wake_function+0x0/0x50
[warning] [<c013aa60>] worker_thread+0x0/0x100
[warning] [<c013db82>] kthread+0x42/0x70
[warning] [<c013db40>] kthread+0x0/0x70
[warning] [<c0104f0f>] kernel_thread_helper+0x7/0x18
[warning] =======================
[alert] BUG: unable to handle kernel paging request at virtual address 01b3401b
[alert] printing eip: f960e9a2 *pde = 00000000
[emerg] Oops: 0000 [#1] PREEMPT SMP
[warning] Modules linked in: btrfs crc32c libcrc32c aes_i586 aes_generic nvidia(P) fan k8temp gspca button snd_intel8x0
[warning]
[warning] Pid: 18352, comm: btrfs-transacti Tainted: P        (2.6.24-dirty #9)
[warning] EIP: 0060:[<f960e9a2>] EFLAGS: 00010286 CPU: 1
[warning] EIP is at walk_down_tree+0x222/0x740 [btrfs]
[warning] EAX: 000000da EBX: 000000da ECX: f2bb7f14 EDX: 00000003
[warning] ESI: 00000003 EDI: 01b33fff EBP: c5b2a888 ESP: f2bb7e58
[warning] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
[emerg] Process btrfs-transacti (pid: 18352, ti=f2bb6000 task=f4bccff0 task.ti=f2bb6000)
[emerg] Stack: f9637d7f f9636d88 00000878 f96366b4 f2bb7eb8 f591102c 00000011 c1a1c000
[emerg] 00000011 f962ad18 00000d93 c2614154 f7765720 c58eb3a8 f2b20800 f1e5b118
[emerg] 00001000 00000000 02952000 00000000 00000026 00000000 00000000 f9622502
[emerg] Call Trace:
[emerg] [<f962ad18>] read_extent_buffer+0xb8/0x190 [btrfs]
[emerg] [<f9622502>] btrfs_node_key+0x62/0x90 [btrfs]
[emerg] [<f960ef4e>] btrfs_drop_snapshot+0x8e/0x1c0 [btrfs]
[emerg] [<f96162ca>] drop_dirty_roots+0xea/0x200 [btrfs]
[emerg] [<f9929000>] _nv006851rm+0xb4c/0x2674 [nvidia]
[emerg] [<f9616da0>] btrfs_transaction_cleaner+0x0/0xe0 [btrfs]
[emerg] [<f9616450>] btrfs_clean_old_snapshots+0x70/0x90 [btrfs]
[emerg] [<f9616de4>] btrfs_transaction_cleaner+0x44/0xe0 [btrfs]
[emerg] [<c013a039>] run_workqueue+0x79/0x100
[emerg] [<c013aa60>] worker_thread+0x0/0x100
[emerg] [<c013ab04>] worker_thread+0xa4/0x100
[emerg] [<c013de50>] autoremove_wake_function+0x0/0x50
[emerg] [<c013aa60>] worker_thread+0x0/0x100
[emerg] [<c013db82>] kthread+0x42/0x70
[emerg] [<c013db40>] kthread+0x0/0x70
[emerg] [<c0104f0f>] kernel_thread_helper+0x7/0x18
[emerg] =======================
[emerg] Code: c7 44 9a 20 00 00 00 00 8b 4c 24 78 8b 01 85 c0 0f 88 71 fe ff ff 83 f8 07 0f 8f e2 03 00 00 8b 54 24 34 8b 3c 82 ba 03 00 00 00 <8b> 47 1c e8 36 25 b1 c6 ba 03 00 00 00 0f b6 58 64 e8 98 25 b1
[emerg] EIP: [<f960e9a2>] walk_down_tree+0x222/0x740 [btrfs] SS:ESP 0068:f2bb7e58
[warning] ---[ end trace ef2a94b0c406c581 ]---

Re: kernel BUG at volumes.c:1705

[Chris Mason]

On Sat, 2008-06-21 at 11:35 +0200, Thomas Lindroth wrote:
> Chris Mason wrote:
> > So, is the bittorrent program run every time you see the memory
> > corruption?
> > 
> > Could you please send along a few of the most common errors you see?
> 
> 
> It's possible to recreate it with mv, cp and other tools without having
> the torrent program running.
> 
> I've found a way to trigger the problem every time. If I copy a folder
> with music videos to a freshly created FS it will always fail on the
> same file when about 5.6G of data has been copied. The FS is 30G and
> there is nothing special with that file. If I only copy that file to a
> fresh FS everything works. I've also tried to run
> dd if=/dev/zero of=test to a fresh FS and I was able to write 15G, but I
> was not able to remove that file.
> 

Something in your config is much less stable than my test boxes here.
Focusing on the dd test, it looks like random corruption in the FS
metadata.  I see you've got gentoo i386 and a 2.6.24 kernel, are there
any other patches in that kernel?

-chris

Re: kernel BUG at volumes.c:1705

[Joe Peterson]

Chris Mason wrote:
> Something in your config is much less stable than my test boxes here.
> Focusing on the dd test, it looks like random corruption in the FS
> metadata.  I see you've got gentoo i386 and a 2.6.24 kernel, are there
> any other patches in that kernel?

I have used Gentoo's 2.4.26 with btrfs (0.15) for a while now - no
issues.  Are you using the official ebuilds?  Note that the latest
contains the 0.15 hotfix.

					-Joe

Re: kernel BUG at volumes.c:1705

[Thomas Lindroth]

Chris Mason wrote:
> On Sat, 2008-06-21 at 11:35 +0200, Thomas Lindroth wrote:
>> Chris Mason wrote:
>>> So, is the bittorrent program run every time you see the memory
>>> corruption?
>>>
>>> Could you please send along a few of the most common errors you see?
>>
>> It's possible to recreate it with mv, cp and other tools without having
>> the torrent program running.
>>
>> I've found a way to trigger the problem every time. If I copy a folder
>> with music videos to a freshly created FS it will always fail on the
>> same file when about 5.6G of data has been copied. The FS is 30G and
>> there is nothing special with that file. If I only copy that file to a
>> fresh FS everything works. I've also tried to run
>> dd if=/dev/zero of=test to a fresh FS and I was able to write 15G, but I
>> was not able to remove that file.
>>
> 
> Something in your config is much less stable than my test boxes here.
> Focusing on the dd test, it looks like random corruption in the FS
> metadata.  I see you've got gentoo i386 and a 2.6.24 kernel, are there
> any other patches in that kernel?
> 
> -chris

The only thing that differs from a vanilla 2.6.24 kernel is a different
drivers/video/logo/logo_linux_clut224.ppm file. That should really not
affect anything.

Here is the .config I use: http://www.cyd.liu.se/~tholi945/config

I followed lavajoe's advice and tried the latest portage ebuild. It did
contain the hotfixes but it had the same problem as my own checkout.

I've tried running in UP mode by issuing
"echo 0 > /sys/devices/system/cpu/cpu1/online" but it made no difference.

This is probably not compiler related but to make sure I'm using gcc 4.1.2