* It's not a bug of ceph, but @ 2011-04-13 10:57 Fyodor Ustinov 2011-04-13 14:44 ` Henry Chang 0 siblings, 1 reply; 6+ messages in thread From: Fyodor Ustinov @ 2011-04-13 10:57 UTC (permalink / raw) To: ceph-devel Hi! may be anyone seen this trouble? it's OSD on ubuntu 11.04 beta (sic!). ext2 (sic!!) and journal on tmpfs (sic!!! :) FS mounted by /dev/md0 /mfs ext2 rw,noatime,nodiratime,async,noexec,nodev,user_xattr 0 0 It's absolutely reproduced in any time. [48106.592387] ------------[ cut here ]------------ [48106.592478] kernel BUG at /build/buildd/linux-2.6.38/fs/ext2/balloc.c:1169! [48106.592595] invalid opcode: 0000 [#1] SMP [48106.592669] last sysfs file: /sys/devices/system/cpu/cpu3/cache/index2/shared_cpu_map [48106.592782] CPU 2 [48106.592820] Modules linked in: 8021q garp stp radeon ttm lp drm_kms_helper drm parport ghes bonding i2c_algo_bit i3200_edac video hed serio_raw edac_core shpchp raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov raid6_pq async_tx raid1 raid0 multipath e1000e linear btrfs pata_it8213 zlib_deflate libcrc32c [48106.593715] [48106.593749] Pid: 1860, comm: cosd Not tainted 2.6.38-8-server #41-Ubuntu Supermicro X7SBi/X7SBi [48106.593926] RIP: 0010:[<ffffffff811f07e8>] [<ffffffff811f07e8>] ext2_try_to_allocate_with_rsv+0x2c8/0x2d0 [48106.594092] RSP: 0018:ffff880126f31a68 EFLAGS: 00010246 [48106.594174] RAX: 0000000000000027 RBX: ffff880126e80c40 RCX: 000000000003ffff [48106.594273] RDX: 0000000000000000 RSI: 0000000000000082 RDI: 0000000000000246 [48106.594372] RBP: ffff880126f31af8 R08: 0000000000000034 R09: 000000000001558b [48106.594474] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000 [48106.594579] R13: ffff880125818400 R14: 0000000000000e37 R15: ffff8801254ea778 [48106.594682] FS: 00007fd1d1de2700(0000) GS:ffff8800cfd00000(0000) knlGS:0000000000000000 [48106.594808] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [48106.594889] CR2: 00007fd1c08b7000 CR3: 0000000126361000 CR4: 00000000000406e0 [48106.594991] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [48106.595099] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [48106.595200] Process cosd (pid: 1860, threadinfo ffff880126f30000, task ffff880123760000) [48106.595319] Stack: [48106.595359] ffff880126f31a98 ffff8801254ea750 ffff880126f31a98 ffffffff811934a7 [48106.595509] 0000000000000000 ffff880126e80c60 00000000071bffff ffff880126f31b78 [48106.595655] 00000000071b8000 0000000000000e37 ffff880125818400 0000000000000001 [48106.595801] Call Trace: [48106.595844] [<ffffffff811934a7>] ? __find_get_block+0x87/0xe0 [48106.595929] [<ffffffff811f0e83>] ext2_new_blocks+0x203/0x620 [48106.596008] [<ffffffff811f4f4d>] ? ext2_write_begin+0x3d/0x80 [48106.596096] [<ffffffff811f12c1>] ext2_new_block+0x21/0x30 [48106.596185] [<ffffffff811f99cd>] ext2_xattr_set2+0x2bd/0x430 [48106.596270] [<ffffffff8115538f>] ? __kmalloc+0xff/0x140 [48106.596353] [<ffffffff811fa2b0>] ext2_xattr_set+0x520/0x8e0 [48106.596449] [<ffffffff811fa8d9>] ext2_xattr_user_set+0x49/0x50 [48106.601478] [<ffffffff81186bad>] generic_setxattr+0x6d/0x80 [48106.601478] [<ffffffff811873b0>] __vfs_setxattr_noperm+0x50/0x1a0 [48106.601478] [<ffffffff8116fb08>] ? inode_permission+0xa8/0xd0 [48106.601478] [<ffffffff811875bc>] vfs_setxattr+0xbc/0xc0 [48106.601478] [<ffffffff81187682>] setxattr+0xc2/0x150 [48106.601478] [<ffffffff8116fd85>] ? putname+0x35/0x50 [48106.601478] [<ffffffff81172d02>] ? user_path_at+0x62/0xa0 [48106.601478] [<ffffffff8109c72b>] ? do_futex+0xbb/0x210 [48106.601478] [<ffffffff8109c8fb>] ? sys_futex+0x7b/0x180 [48106.650005] [<ffffffff811877a5>] sys_setxattr+0x95/0xb0 [48106.650005] [<ffffffff8100bfc2>] system_call_fastpath+0x16/0x1b [48106.650005] Code: 48 ff ff ff 48 8d 14 11 48 89 53 28 eb e6 48 63 45 8c 48 01 43 28 eb dc 48 81 c7 48 01 00 00 48 c7 c6 40 6b 62 81 e8 f8 f6 ff ff <0f> 0b 66 0f 1f 44 00 00 55 48 89 e5 41 54 53 0f 1f 44 00 00 4c [48106.650005] RIP [<ffffffff811f07e8>] ext2_try_to_allocate_with_rsv+0x2c8/0x2d0 [48106.650005] RSP <ffff880126f31a68> [48106.697951] ---[ end trace 7a9e6977fb082a2e ]--- ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: It's not a bug of ceph, but 2011-04-13 10:57 It's not a bug of ceph, but Fyodor Ustinov @ 2011-04-13 14:44 ` Henry Chang 2011-04-13 16:22 ` Sage Weil 0 siblings, 1 reply; 6+ messages in thread From: Henry Chang @ 2011-04-13 14:44 UTC (permalink / raw) To: Fyodor Ustinov; +Cc: ceph-devel Hi, 2011/4/13 Fyodor Ustinov <ufm@ufm.su>: > Hi! > > may be anyone seen this trouble? I have seen this when running OSD with ext3 and never seen it after switching to ext4. -- Henry ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: It's not a bug of ceph, but 2011-04-13 14:44 ` Henry Chang @ 2011-04-13 16:22 ` Sage Weil 2011-04-13 19:32 ` Fyodor Ustinov 0 siblings, 1 reply; 6+ messages in thread From: Sage Weil @ 2011-04-13 16:22 UTC (permalink / raw) To: Henry Chang; +Cc: Fyodor Ustinov, ceph-devel On Wed, 13 Apr 2011, Henry Chang wrote: > Hi, > > 2011/4/13 Fyodor Ustinov <ufm@ufm.su>: > > Hi! > > > > may be anyone seen this trouble? > > I have seen this when running OSD with ext3 and never seen it after > switching to ext4. Yeah. We've seen it on ext3 prior to 2.6.34 or so... ext3 on newer kernels (or ext4) seem to be okay. sage ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: It's not a bug of ceph, but 2011-04-13 16:22 ` Sage Weil @ 2011-04-13 19:32 ` Fyodor Ustinov 2011-04-14 0:09 ` Fyodor Ustinov 2011-04-14 17:08 ` Gregory Farnum 0 siblings, 2 replies; 6+ messages in thread From: Fyodor Ustinov @ 2011-04-13 19:32 UTC (permalink / raw) To: Sage Weil; +Cc: Henry Chang, ceph-devel On 04/13/2011 07:22 PM, Sage Weil wrote: > On Wed, 13 Apr 2011, Henry Chang wrote: >> Hi, >> >> 2011/4/13 Fyodor Ustinov<ufm@ufm.su>: >>> Hi! >>> >>> may be anyone seen this trouble? >> I have seen this when running OSD with ext3 and never seen it after >> switching to ext4. > Yeah. We've seen it on ext3 prior to 2.6.34 or so... ext3 on newer > kernels (or ext4) seem to be okay. > > sage On this kernel (2.6.38-8-server ubuntu 11.04) i have trouble with ext4 also. I mount fresh ceph partition on client, test by bonnie++, unmount, stop ceph cluster, umount osd ext4 partition and forced fsck this partition. Mount string on OSD: /dev/md127 /mfs auto rw,noatime,nodiratime,async,noexec,nodev,user_xattr 0 0 No any messages in kernel log. But: root@stc1:/home/ufm# umount /mfs root@stc1:/home/ufm# fsck /dev/md127 fsck from util-linux-ng 2.17.2 e2fsck 1.41.14 (22-Dec-2010) /dev/md127: clean, 29779/59834368 files, 3887821/239314496 blocks root@stc1:/home/ufm# fsck -f /dev/md127 fsck from util-linux-ng 2.17.2 e2fsck 1.41.14 (22-Dec-2010) Pass 1: Checking inodes, blocks, and sizes Inode 4456487, i_blocks is 8200, should be 8208. Fix<y>? yes Inode 4456499, i_blocks is 8208, should be 8200. Fix<y>? yes Inode 4456549, i_blocks is 24, should be 16. Fix<y>? yes Inode 4456579, i_blocks is 24, should be 16. Fix<y>? yes Inode 4456580, i_blocks is 8, should be 16. Fix<y>? yes Inode 4456591, i_blocks is 8, should be 16. Fix<y>? yes Inode 4456601, i_blocks is 24, should be 16. Fix<y>? /dev/md127: e2fsck canceled. /dev/md127: ***** FILE SYSTEM WAS MODIFIED ***** May be md. I test now without md. WBR, Fyodor. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: It's not a bug of ceph, but 2011-04-13 19:32 ` Fyodor Ustinov @ 2011-04-14 0:09 ` Fyodor Ustinov 2011-04-14 17:08 ` Gregory Farnum 1 sibling, 0 replies; 6+ messages in thread From: Fyodor Ustinov @ 2011-04-14 0:09 UTC (permalink / raw) To: Sage Weil; +Cc: Henry Chang, ceph-devel On 04/13/2011 10:32 PM, Fyodor Ustinov wrote: > > May be md. I test now without md. It's not md. Excellent. It seems that ubuntu 11.04 (or, may be, kernel 2.6.38?) do not have a working fs for ceph? WBR, Fyodor. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: It's not a bug of ceph, but 2011-04-13 19:32 ` Fyodor Ustinov 2011-04-14 0:09 ` Fyodor Ustinov @ 2011-04-14 17:08 ` Gregory Farnum 1 sibling, 0 replies; 6+ messages in thread From: Gregory Farnum @ 2011-04-14 17:08 UTC (permalink / raw) To: Fyodor Ustinov; +Cc: Sage Weil, Henry Chang, ceph-devel On Wednesday, April 13, 2011 at 12:32 PM, Fyodor Ustinov wrote: On this kernel (2.6.38-8-server ubuntu 11.04) i have trouble with ext4 also. > > I mount fresh ceph partition on client, test by bonnie++, unmount, stop > ceph cluster, umount osd ext4 partition and forced fsck this partition. > > Mount string on OSD: > > /dev/md127 /mfs auto rw,noatime,nodiratime,async,noexec,nodev,user_xattr 0 0 > > No any messages in kernel log. But: > > root@stc1:/home/ufm# umount /mfs > root@stc1:/home/ufm# fsck /dev/md127 > fsck from util-linux-ng 2.17.2 > e2fsck 1.41.14 (22-Dec-2010) > /dev/md127: clean, 29779/59834368 files, 3887821/239314496 blocks > > root@stc1:/home/ufm# fsck -f /dev/md127 > fsck from util-linux-ng 2.17.2 > e2fsck 1.41.14 (22-Dec-2010) > Pass 1: Checking inodes, blocks, and sizes > Inode 4456487, i_blocks is 8200, should be 8208. Fix<y>? yes > > Inode 4456499, i_blocks is 8208, should be 8200. Fix<y>? yes > > Inode 4456549, i_blocks is 24, should be 16. Fix<y>? yes > > Inode 4456579, i_blocks is 24, should be 16. Fix<y>? yes > > Inode 4456580, i_blocks is 8, should be 16. Fix<y>? yes > > Inode 4456591, i_blocks is 8, should be 16. Fix<y>? yes > > Inode 4456601, i_blocks is 24, should be 16. Fix<y>? /dev/md127: e2fsck > canceled. > > /dev/md127: ***** FILE SYSTEM WAS MODIFIED ***** > > May be md. I test now without md. So Ceph worked fine and you got no complaints until you ran fsck? That's odd but I don't think we've gotten any reports of data corruption or anything with Ceph on ext4. You should probably report this to the ext4 devs as Ceph doesn't do anything that should be able to break that stuff, but it does use some less-common code paths in a lot of filesystems. -Greg ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-04-14 17:08 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2011-04-13 10:57 It's not a bug of ceph, but Fyodor Ustinov 2011-04-13 14:44 ` Henry Chang 2011-04-13 16:22 ` Sage Weil 2011-04-13 19:32 ` Fyodor Ustinov 2011-04-14 0:09 ` Fyodor Ustinov 2011-04-14 17:08 ` Gregory Farnum
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.