* Fwd: rbd kernel module fail [not found] <CAHULbCn+bdCHY3jm7Co+p3edQN5+wN6ACr_B8E1wwgFZ3cPpMg@mail.gmail.com> @ 2012-11-13 11:54 ` ruslan usifov 2012-11-13 13:19 ` Alex Elder 0 siblings, 1 reply; 4+ messages in thread From: ruslan usifov @ 2012-11-13 11:54 UTC (permalink / raw) To: ceph-devel Hello I test ceph cluster on VmWare machines (3 nodes in cluster) to make rbd scalable block device, and have troubles when try to map rbd image to device, i got follow message in kernel.log Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319319] ------------[ cut here ]------------ Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319437] kernel BUG at /build/buildd/linux-3.2.0/fs/sysfs/group.c:65! Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319583] invalid opcode: 0000 [#1] SMP. Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319756] CPU 0. Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319814] Modules linked in: rbd libceph ppdev vmw_balloon psmouse vmwgfx serio_raw ttm drm parport_pc i2c_piix Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.320989]. Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321063] Pid: 1860, comm: rbd Tainted: G W 3.2.0-32-generic #51-Ubuntu VMware, Inc. VMware Virtual P Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321426] RIP: 0010:[<ffffffff811efbc8>] [<ffffffff811efbc8>] internal_create_group+0xf8/0x110 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321659] RSP: 0018:ffff8800135f1cc8 EFLAGS: 00010246 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321785] RAX: 00000000ffffffef RBX: ffff88001ab8e078 RCX: 0000000000014a7b Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321937] RDX: ffffffff81c34b00 RSI: 0000000000000000 RDI: ffff88001ab8e078 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.322090] RBP: ffff8800135f1d08 R08: ffffea0000718480 R09: ffffffff813f3499 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.322241] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff81c34b00 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.322394] R13: ffff88001ab8e068 R14: 0000000000000000 R15: ffff88001ab8e000 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323204] FS: 00007f8bcf3aa780(0000) GS:ffff88001fc00000(0000) knlGS:0000000000000000 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323398] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323540] CR2: 00007f084b07f000 CR3: 000000001425c000 CR4: 00000000000006f0 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323720] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323891] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324047] Process rbd (pid: 1860, threadinfo ffff8800135f0000, task ffff880014275c00) Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324237] Stack: Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324318] ffff8800135f1d28 ffffffff813f34a9 0000000000000100 ffff880000000010 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324629] ffff88001ab8e000 ffff88001f270eb0 ffff88001ab8e068 ffff88001ab8e00c Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324941] ffff8800135f1d18 ffffffff811efc13 ffff8800135f1d28 ffffffff810feaf4 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325253] Call Trace: Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325344] [<ffffffff813f34a9>] ? device_add+0x89/0x3e0 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325476] [<ffffffff811efc13>] sysfs_create_group+0x13/0x20 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325621] [<ffffffff810feaf4>] blk_trace_init_sysfs+0x14/0x20 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325762] [<ffffffff812f5660>] blk_register_queue+0x40/0x110 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325901] [<ffffffff812fbacc>] add_disk+0xbc/0x230 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326028] [<ffffffffa01950a1>] rbd_init_disk+0x1b1/0x1d0 [rbd] Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326170] [<ffffffffa0195455>] rbd_add+0x285/0x4e0 [rbd] Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326304] [<ffffffff81157ce6>] ? alloc_pages_current+0xb6/0x120 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326447] [<ffffffff813f4727>] bus_attr_store+0x27/0x30 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326579] [<ffffffff811ec24f>] sysfs_write_file+0xef/0x170 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326715] [<ffffffff81178113>] vfs_write+0xb3/0x180 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326842] [<ffffffff8117843a>] sys_write+0x4a/0x90 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326971] [<ffffffff81663442>] system_call_fastpath+0x16/0x1b Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.327110] Code: c3 66 90 89 45 c8 e8 b8 d7 ff ff 8b 45 c8 eb df 0f 1f 00 4c 8b 6b 30 4c 89 6d d8 e9 77 ff ff ff Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.329592] RIP [<ffffffff811efbc8>] internal_create_group+0xf8/0x110 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.329773] RSP <ffff8800135f1cc8> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.329912]---[ end trace 2d22165b460e6596 ]--- Also i see different log message in kernel.log Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318832] ------------[ cut here ]------------ Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318862] WARNING: at /build/buildd/linux-3.2.0/fs/sysfs/dir.c:481 sysfs_add_one+0xc0/0xf0() Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318867] Hardware name: VMware Virtual Platform Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318868] sysfs: cannot create duplicate filename '/devices/virtual/block/rbd0' Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318871] Modules linked in: rbd libceph ppdev vmw_balloon psmouse vmwgfx serio_raw ttm drm parport_pc i2c_piix Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318895] Pid: 1860, comm: rbd Not tainted 3.2.0-32-generic #51-Ubuntu Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318897] Call Trace: Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318910] [<ffffffff81066e2f>] warn_slowpath_common+0x7f/0xc0 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318913] [<ffffffff81066f26>] warn_slowpath_fmt+0x46/0x50 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318915] [<ffffffff811edf60>] sysfs_add_one+0xc0/0xf0 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318917] [<ffffffff811ee007>] create_dir+0x77/0xd0 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318926] [<ffffffff81192142>] ? inode_init_always+0x102/0x1c0 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318928] [<ffffffff811ee10d>] sysfs_create_dir+0x7d/0xc0 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318940] [<ffffffff8130d7c1>] kobject_add_internal+0xb1/0x200 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318943] [<ffffffff8130dc87>] kobject_add+0x67/0xc0 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318945] [<ffffffff8130d6fa>] ? kobject_get+0x1a/0x30 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318958] [<ffffffff813f351d>] device_add+0xfd/0x3e0 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318963] [<ffffffff812fb8d1>] register_disk+0x41/0x180 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318965] [<ffffffff812fbac4>] add_disk+0xb4/0x230 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318969] [<ffffffffa01950a1>] rbd_init_disk+0x1b1/0x1d0 [rbd] Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318971] [<ffffffffa0195455>] rbd_add+0x285/0x4e0 [rbd] Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318980] [<ffffffff81157ce6>] ? alloc_pages_current+0xb6/0x120 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318983] [<ffffffff813f4727>] bus_attr_store+0x27/0x30 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318985] [<ffffffff811ec24f>] sysfs_write_file+0xef/0x170 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318989] [<ffffffff81178113>] vfs_write+0xb3/0x180 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318991] [<ffffffff8117843a>] sys_write+0x4a/0x90 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319002] [<ffffffff81663442>] system_call_fastpath+0x16/0x1b Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319004] ---[ end trace 2d22165b460e6595 ]--- Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319006] kobject_add_internal failed for rbd0 with -EEXIST, don't try to register things with the same name in Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319260] Pid: 1860, comm: rbd Tainted: G W 3.2.0-32-generic #51-Ubuntu Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319261] Call Trace: Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319263] [<ffffffff8130d865>] kobject_add_internal+0x155/0x200 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319266] [<ffffffff8130dc87>] kobject_add+0x67/0xc0 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319268] [<ffffffff8130d6fa>] ? kobject_get+0x1a/0x30 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319270] [<ffffffff813f351d>] device_add+0xfd/0x3e0 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319272] [<ffffffff812fb8d1>] register_disk+0x41/0x180 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319274] [<ffffffff812fbac4>] add_disk+0xb4/0x230 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319277] [<ffffffffa01950a1>] rbd_init_disk+0x1b1/0x1d0 [rbd] Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319279] [<ffffffffa0195455>] rbd_add+0x285/0x4e0 [rbd] Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319281] [<ffffffff81157ce6>] ? alloc_pages_current+0xb6/0x120 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319283] [<ffffffff813f4727>] bus_attr_store+0x27/0x30 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319285] [<ffffffff811ec24f>] sysfs_write_file+0xef/0x170 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319287] [<ffffffff81178113>] vfs_write+0xb3/0x180 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319289] [<ffffffff8117843a>] sys_write+0x4a/0x90 Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319291] [<ffffffff81663442>] system_call_fastpath+0x16/0x1b I use ubuntu 12.0.4 (Linux ceph-precie-64-02 3.2.0-32-generic #51-Ubuntu SMP Wed Sep 26 21:33:09 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux), and laster stable ceph release from http://ceph.com/debian/, also i use pacemaker + corosync with follow conf: node ceph-precie-64-01 node ceph-precie-64-02 node ceph-precie-64-03 primitive samba_fs ocf:heartbeat:Filesystem \ params device="-U cb4d3dda-92e9-4bd8-9fbc-2940c096e8ec" directory="/mnt" fstype="ext4" primitive samba_rbd ocf:ceph:rbd \ params name="samba" group samba samba_rbd samba_fs property $id="cib-bootstrap-options" \ dc-version="1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c" \ cluster-infrastructure="openais" \ expected-quorum-votes="3" \ stonith-enabled="false" \ no-quorum-policy="stop" \ last-lrm-refresh="1352806660" ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Fwd: rbd kernel module fail 2012-11-13 11:54 ` Fwd: rbd kernel module fail ruslan usifov @ 2012-11-13 13:19 ` Alex Elder 2012-11-13 13:48 ` ruslan usifov 0 siblings, 1 reply; 4+ messages in thread From: Alex Elder @ 2012-11-13 13:19 UTC (permalink / raw) To: ruslan usifov; +Cc: ceph-devel On 11/13/2012 05:54 AM, ruslan usifov wrote: > Hello > > I test ceph cluster on VmWare machines (3 nodes in cluster) to make > rbd scalable block device, and have troubles when try to map rbd image > to device, i got follow message in kernel.log I haven't looked into this really yet, but this is a relatively old version of rbd. Current rbd does not use /dev/rbd0 (which is evidently being reused in this case, or at least that's why the error occurred). I'll look a little closer later. I would be very interested to know if doing the same thing with current code exposes a similar problem, or whether you are hitting a problem that has been fixed. -Alex > > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319319] ------------[ > cut here ]------------ > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319437] kernel BUG at > /build/buildd/linux-3.2.0/fs/sysfs/group.c:65! > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319583] invalid > opcode: 0000 [#1] SMP. > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319756] CPU 0. > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319814] Modules > linked in: rbd libceph ppdev vmw_balloon psmouse vmwgfx serio_raw ttm > drm parport_pc i2c_piix > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.320989]. > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321063] Pid: 1860, > comm: rbd Tainted: G W 3.2.0-32-generic #51-Ubuntu VMware, > Inc. VMware Virtual P > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321426] RIP: > 0010:[<ffffffff811efbc8>] [<ffffffff811efbc8>] > internal_create_group+0xf8/0x110 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321659] RSP: > 0018:ffff8800135f1cc8 EFLAGS: 00010246 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321785] RAX: > 00000000ffffffef RBX: ffff88001ab8e078 RCX: 0000000000014a7b > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321937] RDX: > ffffffff81c34b00 RSI: 0000000000000000 RDI: ffff88001ab8e078 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.322090] RBP: > ffff8800135f1d08 R08: ffffea0000718480 R09: ffffffff813f3499 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.322241] R10: > 0000000000000000 R11: 0000000000000000 R12: ffffffff81c34b00 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.322394] R13: > ffff88001ab8e068 R14: 0000000000000000 R15: ffff88001ab8e000 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323204] FS: > 00007f8bcf3aa780(0000) GS:ffff88001fc00000(0000) > knlGS:0000000000000000 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323398] CS: 0010 DS: > 0000 ES: 0000 CR0: 0000000080050033 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323540] CR2: > 00007f084b07f000 CR3: 000000001425c000 CR4: 00000000000006f0 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323720] DR0: > 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323891] DR3: > 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324047] Process rbd > (pid: 1860, threadinfo ffff8800135f0000, task ffff880014275c00) > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324237] Stack: > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324318] > ffff8800135f1d28 ffffffff813f34a9 0000000000000100 ffff880000000010 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324629] > ffff88001ab8e000 ffff88001f270eb0 ffff88001ab8e068 ffff88001ab8e00c > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324941] > ffff8800135f1d18 ffffffff811efc13 ffff8800135f1d28 ffffffff810feaf4 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325253] Call Trace: > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325344] > [<ffffffff813f34a9>] ? device_add+0x89/0x3e0 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325476] > [<ffffffff811efc13>] sysfs_create_group+0x13/0x20 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325621] > [<ffffffff810feaf4>] blk_trace_init_sysfs+0x14/0x20 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325762] > [<ffffffff812f5660>] blk_register_queue+0x40/0x110 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325901] > [<ffffffff812fbacc>] add_disk+0xbc/0x230 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326028] > [<ffffffffa01950a1>] rbd_init_disk+0x1b1/0x1d0 [rbd] > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326170] > [<ffffffffa0195455>] rbd_add+0x285/0x4e0 [rbd] > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326304] > [<ffffffff81157ce6>] ? alloc_pages_current+0xb6/0x120 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326447] > [<ffffffff813f4727>] bus_attr_store+0x27/0x30 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326579] > [<ffffffff811ec24f>] sysfs_write_file+0xef/0x170 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326715] > [<ffffffff81178113>] vfs_write+0xb3/0x180 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326842] > [<ffffffff8117843a>] sys_write+0x4a/0x90 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326971] > [<ffffffff81663442>] system_call_fastpath+0x16/0x1b > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.327110] Code: c3 66 > 90 89 45 c8 e8 b8 d7 ff ff 8b 45 c8 eb df 0f 1f 00 4c 8b 6b 30 4c 89 > 6d d8 e9 77 ff ff ff > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.329592] RIP > [<ffffffff811efbc8>] internal_create_group+0xf8/0x110 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.329773] RSP <ffff8800135f1cc8> > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.329912]---[ end trace > 2d22165b460e6596 ]--- > > > Also i see different log message in kernel.log > > > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318832] ------------[ > cut here ]------------ > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318862] WARNING: at > /build/buildd/linux-3.2.0/fs/sysfs/dir.c:481 sysfs_add_one+0xc0/0xf0() > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318867] Hardware > name: VMware Virtual Platform > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318868] sysfs: cannot > create duplicate filename '/devices/virtual/block/rbd0' > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318871] Modules > linked in: rbd libceph ppdev vmw_balloon psmouse vmwgfx serio_raw ttm > drm parport_pc i2c_piix > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318895] Pid: 1860, > comm: rbd Not tainted 3.2.0-32-generic #51-Ubuntu > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318897] Call Trace: > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318910] > [<ffffffff81066e2f>] warn_slowpath_common+0x7f/0xc0 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318913] > [<ffffffff81066f26>] warn_slowpath_fmt+0x46/0x50 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318915] > [<ffffffff811edf60>] sysfs_add_one+0xc0/0xf0 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318917] > [<ffffffff811ee007>] create_dir+0x77/0xd0 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318926] > [<ffffffff81192142>] ? inode_init_always+0x102/0x1c0 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318928] > [<ffffffff811ee10d>] sysfs_create_dir+0x7d/0xc0 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318940] > [<ffffffff8130d7c1>] kobject_add_internal+0xb1/0x200 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318943] > [<ffffffff8130dc87>] kobject_add+0x67/0xc0 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318945] > [<ffffffff8130d6fa>] ? kobject_get+0x1a/0x30 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318958] > [<ffffffff813f351d>] device_add+0xfd/0x3e0 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318963] > [<ffffffff812fb8d1>] register_disk+0x41/0x180 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318965] > [<ffffffff812fbac4>] add_disk+0xb4/0x230 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318969] > [<ffffffffa01950a1>] rbd_init_disk+0x1b1/0x1d0 [rbd] > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318971] > [<ffffffffa0195455>] rbd_add+0x285/0x4e0 [rbd] > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318980] > [<ffffffff81157ce6>] ? alloc_pages_current+0xb6/0x120 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318983] > [<ffffffff813f4727>] bus_attr_store+0x27/0x30 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318985] > [<ffffffff811ec24f>] sysfs_write_file+0xef/0x170 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318989] > [<ffffffff81178113>] vfs_write+0xb3/0x180 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318991] > [<ffffffff8117843a>] sys_write+0x4a/0x90 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319002] > [<ffffffff81663442>] system_call_fastpath+0x16/0x1b > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319004] ---[ end > trace 2d22165b460e6595 ]--- > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319006] > kobject_add_internal failed for rbd0 with -EEXIST, don't try to > register things with the same name in > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319260] Pid: 1860, > comm: rbd Tainted: G W 3.2.0-32-generic #51-Ubuntu > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319261] Call Trace: > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319263] > [<ffffffff8130d865>] kobject_add_internal+0x155/0x200 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319266] > [<ffffffff8130dc87>] kobject_add+0x67/0xc0 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319268] > [<ffffffff8130d6fa>] ? kobject_get+0x1a/0x30 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319270] > [<ffffffff813f351d>] device_add+0xfd/0x3e0 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319272] > [<ffffffff812fb8d1>] register_disk+0x41/0x180 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319274] > [<ffffffff812fbac4>] add_disk+0xb4/0x230 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319277] > [<ffffffffa01950a1>] rbd_init_disk+0x1b1/0x1d0 [rbd] > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319279] > [<ffffffffa0195455>] rbd_add+0x285/0x4e0 [rbd] > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319281] > [<ffffffff81157ce6>] ? alloc_pages_current+0xb6/0x120 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319283] > [<ffffffff813f4727>] bus_attr_store+0x27/0x30 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319285] > [<ffffffff811ec24f>] sysfs_write_file+0xef/0x170 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319287] > [<ffffffff81178113>] vfs_write+0xb3/0x180 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319289] > [<ffffffff8117843a>] sys_write+0x4a/0x90 > Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319291] > [<ffffffff81663442>] system_call_fastpath+0x16/0x1b > > > I use ubuntu 12.0.4 (Linux ceph-precie-64-02 3.2.0-32-generic > #51-Ubuntu SMP Wed Sep 26 21:33:09 UTC 2012 x86_64 x86_64 x86_64 > GNU/Linux), and laster stable ceph release from > http://ceph.com/debian/, also i use pacemaker + corosync with follow > conf: > > node ceph-precie-64-01 > node ceph-precie-64-02 > node ceph-precie-64-03 > primitive samba_fs ocf:heartbeat:Filesystem \ > params device="-U cb4d3dda-92e9-4bd8-9fbc-2940c096e8ec" > directory="/mnt" fstype="ext4" > primitive samba_rbd ocf:ceph:rbd \ > params name="samba" > group samba samba_rbd samba_fs > property $id="cib-bootstrap-options" \ > dc-version="1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c" \ > cluster-infrastructure="openais" \ > expected-quorum-votes="3" \ > stonith-enabled="false" \ > no-quorum-policy="stop" \ > last-lrm-refresh="1352806660" > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Fwd: rbd kernel module fail 2012-11-13 13:19 ` Alex Elder @ 2012-11-13 13:48 ` ruslan usifov 2012-11-13 16:56 ` Alex Elder 0 siblings, 1 reply; 4+ messages in thread From: ruslan usifov @ 2012-11-13 13:48 UTC (permalink / raw) To: Alex Elder; +Cc: ceph-devel How can i compile current version of rbd module? Now i use rbd module that goes with standart lunux kernel in ubuntu 12.04 2012/11/13 Alex Elder <elder@inktank.com>: > On 11/13/2012 05:54 AM, ruslan usifov wrote: >> Hello >> >> I test ceph cluster on VmWare machines (3 nodes in cluster) to make >> rbd scalable block device, and have troubles when try to map rbd image >> to device, i got follow message in kernel.log > > I haven't looked into this really yet, but this is a relatively > old version of rbd. Current rbd does not use /dev/rbd0 (which > is evidently being reused in this case, or at least that's why > the error occurred). > > I'll look a little closer later. I would be very interested > to know if doing the same thing with current code exposes a > similar problem, or whether you are hitting a problem that > has been fixed. > > -Alex > >> >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319319] ------------[ >> cut here ]------------ >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319437] kernel BUG at >> /build/buildd/linux-3.2.0/fs/sysfs/group.c:65! >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319583] invalid >> opcode: 0000 [#1] SMP. >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319756] CPU 0. >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319814] Modules >> linked in: rbd libceph ppdev vmw_balloon psmouse vmwgfx serio_raw ttm >> drm parport_pc i2c_piix >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.320989]. >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321063] Pid: 1860, >> comm: rbd Tainted: G W 3.2.0-32-generic #51-Ubuntu VMware, >> Inc. VMware Virtual P >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321426] RIP: >> 0010:[<ffffffff811efbc8>] [<ffffffff811efbc8>] >> internal_create_group+0xf8/0x110 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321659] RSP: >> 0018:ffff8800135f1cc8 EFLAGS: 00010246 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321785] RAX: >> 00000000ffffffef RBX: ffff88001ab8e078 RCX: 0000000000014a7b >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321937] RDX: >> ffffffff81c34b00 RSI: 0000000000000000 RDI: ffff88001ab8e078 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.322090] RBP: >> ffff8800135f1d08 R08: ffffea0000718480 R09: ffffffff813f3499 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.322241] R10: >> 0000000000000000 R11: 0000000000000000 R12: ffffffff81c34b00 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.322394] R13: >> ffff88001ab8e068 R14: 0000000000000000 R15: ffff88001ab8e000 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323204] FS: >> 00007f8bcf3aa780(0000) GS:ffff88001fc00000(0000) >> knlGS:0000000000000000 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323398] CS: 0010 DS: >> 0000 ES: 0000 CR0: 0000000080050033 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323540] CR2: >> 00007f084b07f000 CR3: 000000001425c000 CR4: 00000000000006f0 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323720] DR0: >> 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323891] DR3: >> 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324047] Process rbd >> (pid: 1860, threadinfo ffff8800135f0000, task ffff880014275c00) >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324237] Stack: >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324318] >> ffff8800135f1d28 ffffffff813f34a9 0000000000000100 ffff880000000010 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324629] >> ffff88001ab8e000 ffff88001f270eb0 ffff88001ab8e068 ffff88001ab8e00c >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324941] >> ffff8800135f1d18 ffffffff811efc13 ffff8800135f1d28 ffffffff810feaf4 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325253] Call Trace: >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325344] >> [<ffffffff813f34a9>] ? device_add+0x89/0x3e0 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325476] >> [<ffffffff811efc13>] sysfs_create_group+0x13/0x20 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325621] >> [<ffffffff810feaf4>] blk_trace_init_sysfs+0x14/0x20 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325762] >> [<ffffffff812f5660>] blk_register_queue+0x40/0x110 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325901] >> [<ffffffff812fbacc>] add_disk+0xbc/0x230 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326028] >> [<ffffffffa01950a1>] rbd_init_disk+0x1b1/0x1d0 [rbd] >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326170] >> [<ffffffffa0195455>] rbd_add+0x285/0x4e0 [rbd] >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326304] >> [<ffffffff81157ce6>] ? alloc_pages_current+0xb6/0x120 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326447] >> [<ffffffff813f4727>] bus_attr_store+0x27/0x30 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326579] >> [<ffffffff811ec24f>] sysfs_write_file+0xef/0x170 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326715] >> [<ffffffff81178113>] vfs_write+0xb3/0x180 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326842] >> [<ffffffff8117843a>] sys_write+0x4a/0x90 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326971] >> [<ffffffff81663442>] system_call_fastpath+0x16/0x1b >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.327110] Code: c3 66 >> 90 89 45 c8 e8 b8 d7 ff ff 8b 45 c8 eb df 0f 1f 00 4c 8b 6b 30 4c 89 >> 6d d8 e9 77 ff ff ff >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.329592] RIP >> [<ffffffff811efbc8>] internal_create_group+0xf8/0x110 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.329773] RSP <ffff8800135f1cc8> >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.329912]---[ end trace >> 2d22165b460e6596 ]--- >> >> >> Also i see different log message in kernel.log >> >> >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318832] ------------[ >> cut here ]------------ >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318862] WARNING: at >> /build/buildd/linux-3.2.0/fs/sysfs/dir.c:481 sysfs_add_one+0xc0/0xf0() >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318867] Hardware >> name: VMware Virtual Platform >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318868] sysfs: cannot >> create duplicate filename '/devices/virtual/block/rbd0' >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318871] Modules >> linked in: rbd libceph ppdev vmw_balloon psmouse vmwgfx serio_raw ttm >> drm parport_pc i2c_piix >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318895] Pid: 1860, >> comm: rbd Not tainted 3.2.0-32-generic #51-Ubuntu >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318897] Call Trace: >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318910] >> [<ffffffff81066e2f>] warn_slowpath_common+0x7f/0xc0 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318913] >> [<ffffffff81066f26>] warn_slowpath_fmt+0x46/0x50 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318915] >> [<ffffffff811edf60>] sysfs_add_one+0xc0/0xf0 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318917] >> [<ffffffff811ee007>] create_dir+0x77/0xd0 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318926] >> [<ffffffff81192142>] ? inode_init_always+0x102/0x1c0 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318928] >> [<ffffffff811ee10d>] sysfs_create_dir+0x7d/0xc0 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318940] >> [<ffffffff8130d7c1>] kobject_add_internal+0xb1/0x200 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318943] >> [<ffffffff8130dc87>] kobject_add+0x67/0xc0 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318945] >> [<ffffffff8130d6fa>] ? kobject_get+0x1a/0x30 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318958] >> [<ffffffff813f351d>] device_add+0xfd/0x3e0 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318963] >> [<ffffffff812fb8d1>] register_disk+0x41/0x180 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318965] >> [<ffffffff812fbac4>] add_disk+0xb4/0x230 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318969] >> [<ffffffffa01950a1>] rbd_init_disk+0x1b1/0x1d0 [rbd] >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318971] >> [<ffffffffa0195455>] rbd_add+0x285/0x4e0 [rbd] >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318980] >> [<ffffffff81157ce6>] ? alloc_pages_current+0xb6/0x120 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318983] >> [<ffffffff813f4727>] bus_attr_store+0x27/0x30 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318985] >> [<ffffffff811ec24f>] sysfs_write_file+0xef/0x170 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318989] >> [<ffffffff81178113>] vfs_write+0xb3/0x180 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318991] >> [<ffffffff8117843a>] sys_write+0x4a/0x90 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319002] >> [<ffffffff81663442>] system_call_fastpath+0x16/0x1b >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319004] ---[ end >> trace 2d22165b460e6595 ]--- >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319006] >> kobject_add_internal failed for rbd0 with -EEXIST, don't try to >> register things with the same name in >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319260] Pid: 1860, >> comm: rbd Tainted: G W 3.2.0-32-generic #51-Ubuntu >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319261] Call Trace: >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319263] >> [<ffffffff8130d865>] kobject_add_internal+0x155/0x200 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319266] >> [<ffffffff8130dc87>] kobject_add+0x67/0xc0 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319268] >> [<ffffffff8130d6fa>] ? kobject_get+0x1a/0x30 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319270] >> [<ffffffff813f351d>] device_add+0xfd/0x3e0 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319272] >> [<ffffffff812fb8d1>] register_disk+0x41/0x180 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319274] >> [<ffffffff812fbac4>] add_disk+0xb4/0x230 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319277] >> [<ffffffffa01950a1>] rbd_init_disk+0x1b1/0x1d0 [rbd] >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319279] >> [<ffffffffa0195455>] rbd_add+0x285/0x4e0 [rbd] >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319281] >> [<ffffffff81157ce6>] ? alloc_pages_current+0xb6/0x120 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319283] >> [<ffffffff813f4727>] bus_attr_store+0x27/0x30 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319285] >> [<ffffffff811ec24f>] sysfs_write_file+0xef/0x170 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319287] >> [<ffffffff81178113>] vfs_write+0xb3/0x180 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319289] >> [<ffffffff8117843a>] sys_write+0x4a/0x90 >> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319291] >> [<ffffffff81663442>] system_call_fastpath+0x16/0x1b >> >> >> I use ubuntu 12.0.4 (Linux ceph-precie-64-02 3.2.0-32-generic >> #51-Ubuntu SMP Wed Sep 26 21:33:09 UTC 2012 x86_64 x86_64 x86_64 >> GNU/Linux), and laster stable ceph release from >> http://ceph.com/debian/, also i use pacemaker + corosync with follow >> conf: >> >> node ceph-precie-64-01 >> node ceph-precie-64-02 >> node ceph-precie-64-03 >> primitive samba_fs ocf:heartbeat:Filesystem \ >> params device="-U cb4d3dda-92e9-4bd8-9fbc-2940c096e8ec" >> directory="/mnt" fstype="ext4" >> primitive samba_rbd ocf:ceph:rbd \ >> params name="samba" >> group samba samba_rbd samba_fs >> property $id="cib-bootstrap-options" \ >> dc-version="1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c" \ >> cluster-infrastructure="openais" \ >> expected-quorum-votes="3" \ >> stonith-enabled="false" \ >> no-quorum-policy="stop" \ >> last-lrm-refresh="1352806660" >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Fwd: rbd kernel module fail 2012-11-13 13:48 ` ruslan usifov @ 2012-11-13 16:56 ` Alex Elder 0 siblings, 0 replies; 4+ messages in thread From: Alex Elder @ 2012-11-13 16:56 UTC (permalink / raw) To: ruslan usifov; +Cc: ceph-devel On 11/13/2012 07:48 AM, ruslan usifov wrote: > How can i compile current version of rbd module? Now i use rbd module > that goes with standart lunux kernel in ubuntu 12.04 The Ubuntu kernel team builds mainline kernel packages. Perhaps you could try that. http://kernel.ubuntu.com/~kernel-ppa/mainline/ -Alex > 2012/11/13 Alex Elder <elder@inktank.com>: >> On 11/13/2012 05:54 AM, ruslan usifov wrote: >>> Hello >>> >>> I test ceph cluster on VmWare machines (3 nodes in cluster) to make >>> rbd scalable block device, and have troubles when try to map rbd image >>> to device, i got follow message in kernel.log >> >> I haven't looked into this really yet, but this is a relatively >> old version of rbd. Current rbd does not use /dev/rbd0 (which >> is evidently being reused in this case, or at least that's why >> the error occurred). >> >> I'll look a little closer later. I would be very interested >> to know if doing the same thing with current code exposes a >> similar problem, or whether you are hitting a problem that >> has been fixed. >> >> -Alex >> >>> >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319319] ------------[ >>> cut here ]------------ >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319437] kernel BUG at >>> /build/buildd/linux-3.2.0/fs/sysfs/group.c:65! >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319583] invalid >>> opcode: 0000 [#1] SMP. >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319756] CPU 0. >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319814] Modules >>> linked in: rbd libceph ppdev vmw_balloon psmouse vmwgfx serio_raw ttm >>> drm parport_pc i2c_piix >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.320989]. >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321063] Pid: 1860, >>> comm: rbd Tainted: G W 3.2.0-32-generic #51-Ubuntu VMware, >>> Inc. VMware Virtual P >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321426] RIP: >>> 0010:[<ffffffff811efbc8>] [<ffffffff811efbc8>] >>> internal_create_group+0xf8/0x110 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321659] RSP: >>> 0018:ffff8800135f1cc8 EFLAGS: 00010246 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321785] RAX: >>> 00000000ffffffef RBX: ffff88001ab8e078 RCX: 0000000000014a7b >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.321937] RDX: >>> ffffffff81c34b00 RSI: 0000000000000000 RDI: ffff88001ab8e078 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.322090] RBP: >>> ffff8800135f1d08 R08: ffffea0000718480 R09: ffffffff813f3499 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.322241] R10: >>> 0000000000000000 R11: 0000000000000000 R12: ffffffff81c34b00 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.322394] R13: >>> ffff88001ab8e068 R14: 0000000000000000 R15: ffff88001ab8e000 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323204] FS: >>> 00007f8bcf3aa780(0000) GS:ffff88001fc00000(0000) >>> knlGS:0000000000000000 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323398] CS: 0010 DS: >>> 0000 ES: 0000 CR0: 0000000080050033 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323540] CR2: >>> 00007f084b07f000 CR3: 000000001425c000 CR4: 00000000000006f0 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323720] DR0: >>> 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.323891] DR3: >>> 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324047] Process rbd >>> (pid: 1860, threadinfo ffff8800135f0000, task ffff880014275c00) >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324237] Stack: >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324318] >>> ffff8800135f1d28 ffffffff813f34a9 0000000000000100 ffff880000000010 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324629] >>> ffff88001ab8e000 ffff88001f270eb0 ffff88001ab8e068 ffff88001ab8e00c >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.324941] >>> ffff8800135f1d18 ffffffff811efc13 ffff8800135f1d28 ffffffff810feaf4 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325253] Call Trace: >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325344] >>> [<ffffffff813f34a9>] ? device_add+0x89/0x3e0 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325476] >>> [<ffffffff811efc13>] sysfs_create_group+0x13/0x20 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325621] >>> [<ffffffff810feaf4>] blk_trace_init_sysfs+0x14/0x20 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325762] >>> [<ffffffff812f5660>] blk_register_queue+0x40/0x110 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.325901] >>> [<ffffffff812fbacc>] add_disk+0xbc/0x230 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326028] >>> [<ffffffffa01950a1>] rbd_init_disk+0x1b1/0x1d0 [rbd] >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326170] >>> [<ffffffffa0195455>] rbd_add+0x285/0x4e0 [rbd] >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326304] >>> [<ffffffff81157ce6>] ? alloc_pages_current+0xb6/0x120 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326447] >>> [<ffffffff813f4727>] bus_attr_store+0x27/0x30 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326579] >>> [<ffffffff811ec24f>] sysfs_write_file+0xef/0x170 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326715] >>> [<ffffffff81178113>] vfs_write+0xb3/0x180 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326842] >>> [<ffffffff8117843a>] sys_write+0x4a/0x90 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.326971] >>> [<ffffffff81663442>] system_call_fastpath+0x16/0x1b >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.327110] Code: c3 66 >>> 90 89 45 c8 e8 b8 d7 ff ff 8b 45 c8 eb df 0f 1f 00 4c 8b 6b 30 4c 89 >>> 6d d8 e9 77 ff ff ff >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.329592] RIP >>> [<ffffffff811efbc8>] internal_create_group+0xf8/0x110 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.329773] RSP <ffff8800135f1cc8> >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.329912]---[ end trace >>> 2d22165b460e6596 ]--- >>> >>> >>> Also i see different log message in kernel.log >>> >>> >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318832] ------------[ >>> cut here ]------------ >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318862] WARNING: at >>> /build/buildd/linux-3.2.0/fs/sysfs/dir.c:481 sysfs_add_one+0xc0/0xf0() >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318867] Hardware >>> name: VMware Virtual Platform >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318868] sysfs: cannot >>> create duplicate filename '/devices/virtual/block/rbd0' >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318871] Modules >>> linked in: rbd libceph ppdev vmw_balloon psmouse vmwgfx serio_raw ttm >>> drm parport_pc i2c_piix >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318895] Pid: 1860, >>> comm: rbd Not tainted 3.2.0-32-generic #51-Ubuntu >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318897] Call Trace: >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318910] >>> [<ffffffff81066e2f>] warn_slowpath_common+0x7f/0xc0 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318913] >>> [<ffffffff81066f26>] warn_slowpath_fmt+0x46/0x50 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318915] >>> [<ffffffff811edf60>] sysfs_add_one+0xc0/0xf0 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318917] >>> [<ffffffff811ee007>] create_dir+0x77/0xd0 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318926] >>> [<ffffffff81192142>] ? inode_init_always+0x102/0x1c0 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318928] >>> [<ffffffff811ee10d>] sysfs_create_dir+0x7d/0xc0 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318940] >>> [<ffffffff8130d7c1>] kobject_add_internal+0xb1/0x200 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318943] >>> [<ffffffff8130dc87>] kobject_add+0x67/0xc0 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318945] >>> [<ffffffff8130d6fa>] ? kobject_get+0x1a/0x30 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318958] >>> [<ffffffff813f351d>] device_add+0xfd/0x3e0 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318963] >>> [<ffffffff812fb8d1>] register_disk+0x41/0x180 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318965] >>> [<ffffffff812fbac4>] add_disk+0xb4/0x230 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318969] >>> [<ffffffffa01950a1>] rbd_init_disk+0x1b1/0x1d0 [rbd] >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318971] >>> [<ffffffffa0195455>] rbd_add+0x285/0x4e0 [rbd] >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318980] >>> [<ffffffff81157ce6>] ? alloc_pages_current+0xb6/0x120 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318983] >>> [<ffffffff813f4727>] bus_attr_store+0x27/0x30 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318985] >>> [<ffffffff811ec24f>] sysfs_write_file+0xef/0x170 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318989] >>> [<ffffffff81178113>] vfs_write+0xb3/0x180 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.318991] >>> [<ffffffff8117843a>] sys_write+0x4a/0x90 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319002] >>> [<ffffffff81663442>] system_call_fastpath+0x16/0x1b >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319004] ---[ end >>> trace 2d22165b460e6595 ]--- >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319006] >>> kobject_add_internal failed for rbd0 with -EEXIST, don't try to >>> register things with the same name in >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319260] Pid: 1860, >>> comm: rbd Tainted: G W 3.2.0-32-generic #51-Ubuntu >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319261] Call Trace: >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319263] >>> [<ffffffff8130d865>] kobject_add_internal+0x155/0x200 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319266] >>> [<ffffffff8130dc87>] kobject_add+0x67/0xc0 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319268] >>> [<ffffffff8130d6fa>] ? kobject_get+0x1a/0x30 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319270] >>> [<ffffffff813f351d>] device_add+0xfd/0x3e0 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319272] >>> [<ffffffff812fb8d1>] register_disk+0x41/0x180 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319274] >>> [<ffffffff812fbac4>] add_disk+0xb4/0x230 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319277] >>> [<ffffffffa01950a1>] rbd_init_disk+0x1b1/0x1d0 [rbd] >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319279] >>> [<ffffffffa0195455>] rbd_add+0x285/0x4e0 [rbd] >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319281] >>> [<ffffffff81157ce6>] ? alloc_pages_current+0xb6/0x120 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319283] >>> [<ffffffff813f4727>] bus_attr_store+0x27/0x30 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319285] >>> [<ffffffff811ec24f>] sysfs_write_file+0xef/0x170 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319287] >>> [<ffffffff81178113>] vfs_write+0xb3/0x180 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319289] >>> [<ffffffff8117843a>] sys_write+0x4a/0x90 >>> Nov 13 15:32:47 ceph-precie-64-02 kernel: [ 188.319291] >>> [<ffffffff81663442>] system_call_fastpath+0x16/0x1b >>> >>> >>> I use ubuntu 12.0.4 (Linux ceph-precie-64-02 3.2.0-32-generic >>> #51-Ubuntu SMP Wed Sep 26 21:33:09 UTC 2012 x86_64 x86_64 x86_64 >>> GNU/Linux), and laster stable ceph release from >>> http://ceph.com/debian/, also i use pacemaker + corosync with follow >>> conf: >>> >>> node ceph-precie-64-01 >>> node ceph-precie-64-02 >>> node ceph-precie-64-03 >>> primitive samba_fs ocf:heartbeat:Filesystem \ >>> params device="-U cb4d3dda-92e9-4bd8-9fbc-2940c096e8ec" >>> directory="/mnt" fstype="ext4" >>> primitive samba_rbd ocf:ceph:rbd \ >>> params name="samba" >>> group samba samba_rbd samba_fs >>> property $id="cib-bootstrap-options" \ >>> dc-version="1.1.6-9971ebba4494012a93c03b40a2c58ec0eb60f50c" \ >>> cluster-infrastructure="openais" \ >>> expected-quorum-votes="3" \ >>> stonith-enabled="false" \ >>> no-quorum-policy="stop" \ >>> last-lrm-refresh="1352806660" >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >> ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2012-11-13 16:56 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <CAHULbCn+bdCHY3jm7Co+p3edQN5+wN6ACr_B8E1wwgFZ3cPpMg@mail.gmail.com>
2012-11-13 11:54 ` Fwd: rbd kernel module fail ruslan usifov
2012-11-13 13:19 ` Alex Elder
2012-11-13 13:48 ` ruslan usifov
2012-11-13 16:56 ` Alex Elder
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.