From: Noah Watkins <jayhawk@cs.ucsc.edu>
To: ceph-devel@vger.kernel.org
Subject: Assert failure in FileStore::_do_transaction
Date: Tue, 06 Dec 2011 13:31:24 -0800 [thread overview]
Message-ID: <4EDE89AC.5030405@cs.ucsc.edu> (raw)
I got the following failure after restarting an OSD. There were no
issues prior to this, but the cluster had been stopped/restarted many
times. This is on ext4.
Thanks,
Noah
2011-12-06 11:58:11.552177 7ffbe5f89700 filestore(/mnt/vol1/osd) error
error 22: Invalid argument not handled
os/FileStore.cc: In function 'unsigned int
FileStore::_do_transaction(ObjectStore::Transaction&, uint64_t)', in
thread '7ffbe5f89700'
os/FileStore.cc: 2412: FAILED assert(0 == "unexpected error")
ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381)
1: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned
long)+0x56e) [0x6de19e]
2: (FileStore::do_transactions(std::list<ObjectStore::Transaction*,
std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x78)
[0x6e2678]
3: (FileStore::_do_op(FileStore::OpSequencer*)+0x176) [0x6bb306]
4: (ThreadPool::worker()+0xa28) [0x5bee78]
5: (ThreadPool::WorkThread::entry()+0xd) [0x565f7d]
6: (()+0x7971) [0x7ffbf0a0e971]
7: (clone()+0x6d) [0x7ffbef09992d]
ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381)
1: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned
long)+0x56e) [0x6de19e]
2: (FileStore::do_transactions(std::list<ObjectStore::Transaction*,
std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x78)
[0x6e2678]
3: (FileStore::_do_op(FileStore::OpSequencer*)+0x176) [0x6bb306]
4: (ThreadPool::worker()+0xa28) [0x5bee78]
5: (ThreadPool::WorkThread::entry()+0xd) [0x565f7d]
6: (()+0x7971) [0x7ffbf0a0e971]
7: (clone()+0x6d) [0x7ffbef09992d]
*** Caught signal (Aborted) **
in thread 7ffbe5f89700
ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381)
1: /usr/bin/ceph-osd() [0x6b1c84]
2: (()+0xfb40) [0x7ffbf0a16b40]
3: (gsignal()+0x35) [0x7ffbeefe6ba5]
4: (abort()+0x180) [0x7ffbeefea6b0]
5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7ffbef88a6bd]
6: (()+0xb9906) [0x7ffbef888906]
7: (()+0xb9933) [0x7ffbef888933]
8: (()+0xb9a3e) [0x7ffbef888a3e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char
const*)+0x39f) [0x5ccdbf]
10: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned
long)+0x56e) [0x6de19e]
11: (FileStore::do_transactions(std::list<ObjectStore::Transaction*,
std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x78)
[0x6e2678]
12: (FileStore::_do_op(FileStore::OpSequencer*)+0x176) [0x6bb306]
13: (ThreadPool::worker()+0xa28) [0x5bee78]
14: (ThreadPool::WorkThread::entry()+0xd) [0x565f7d]
15: (()+0x7971) [0x7ffbf0a0e971]
16: (clone()+0x6d) [0x7ffbef09992d]
2011-12-06 13:25:13.390151 7fc539d3f780 ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381), process ceph-osd, pid
1314
2011-12-06 13:25:13.459719 7fc539d3f780 filestore(/mnt/vol1/osd) mount
FIEMAP ioctl is supported
2011-12-06 13:25:13.459854 7fc539d3f780 filestore(/mnt/vol1/osd) mount
did NOT detect btrfs
2011-12-06 13:25:13.482119 7fc539d3f780 filestore(/mnt/vol1/osd) mount
found snaps <>
2011-12-06 13:25:13.494092 7fc539d3f780 filestore(/mnt/vol1/osd) mount:
enabling WRITEAHEAD journal mode: btrfs not detected
2011-12-06 13:25:13.972914 7fc539d3f780 journal _open
/mnt/vol2/osd.journal fd 14: 1048576000 bytes, block size 4096 bytes,
directio = 1
2011-12-06 13:25:14.013331 7fc539d3f780 journal read_entry 803999744 :
seq 54062 1002407 bytes
2011-12-06 13:25:14.014773 7fc539d3f780 journal read_entry 803999744 :
seq 54062 1002407 bytes
2011-12-06 13:25:14.048581 7fc539d3f780 filestore(/mnt/vol1/osd) error
error 22: Invalid argument not handled
os/FileStore.cc: In function 'unsigned int
FileStore::_do_transaction(ObjectStore::Transaction&, uint64_t)', in
thread '7fc539d3f780'
os/FileStore.cc: 2412: FAILED assert(0 == "unexpected error")
ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381)
1: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned
long)+0x56e) [0x6de19e]
2: (FileStore::do_transactions(std::list<ObjectStore::Transaction*,
std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x78)
[0x6e2678]
3: (JournalingObjectStore::journal_replay(unsigned long)+0x8e0) [0x6eab90]
4: (FileStore::mount()+0x2a68) [0x6cc128]
5: (OSD::convertfs(std::string const&, std::string const&)+0xc8)
[0x525d38]
6: (main()+0x2bed) [0x49e6ed]
7: (__libc_start_main()+0xfe) [0x7fc537ee3d8e]
8: /usr/bin/ceph-osd() [0x49b6d9]
ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381)
1: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned
long)+0x56e) [0x6de19e]
2: (FileStore::do_transactions(std::list<ObjectStore::Transaction*,
std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x78)
[0x6e2678]
3: (JournalingObjectStore::journal_replay(unsigned long)+0x8e0) [0x6eab90]
4: (FileStore::mount()+0x2a68) [0x6cc128]
5: (OSD::convertfs(std::string const&, std::string const&)+0xc8)
[0x525d38]
6: (main()+0x2bed) [0x49e6ed]
7: (__libc_start_main()+0xfe) [0x7fc537ee3d8e]
8: /usr/bin/ceph-osd() [0x49b6d9]
*** Caught signal (Aborted) **
in thread 7fc539d3f780
ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381)
1: /usr/bin/ceph-osd() [0x6b1c84]
2: (()+0xfb40) [0x7fc539928b40]
3: (gsignal()+0x35) [0x7fc537ef8ba5]
4: (abort()+0x180) [0x7fc537efc6b0]
5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7fc53879c6bd]
6: (()+0xb9906) [0x7fc53879a906]
7: (()+0xb9933) [0x7fc53879a933]
8: (()+0xb9a3e) [0x7fc53879aa3e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char
const*)+0x39f) [0x5ccdbf]
10: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned
long)+0x56e) [0x6de19e]
11: (FileStore::do_transactions(std::list<ObjectStore::Transaction*,
std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x78)
[0x6e2678]
12: (JournalingObjectStore::journal_replay(unsigned long)+0x8e0)
[0x6eab90]
13: (FileStore::mount()+0x2a68) [0x6cc128]
14: (OSD::convertfs(std::string const&, std::string const&)+0xc8)
[0x525d38]
15: (main()+0x2bed) [0x49e6ed]
16: (__libc_start_main()+0xfe) [0x7fc537ee3d8e]
17: /usr/bin/ceph-osd() [0x49b6d9]
nwatkins@piha:~$
reply other threads:[~2011-12-06 21:31 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4EDE89AC.5030405@cs.ucsc.edu \
--to=jayhawk@cs.ucsc.edu \
--cc=ceph-devel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.