From: Noah Watkins <jayhawk@cs.ucsc.edu>
To: ceph-devel@vger.kernel.org
Subject: Assert failure in FileStore::_do_transaction
Date: Tue, 06 Dec 2011 13:31:24 -0800 [thread overview]
Message-ID: <4EDE89AC.5030405@cs.ucsc.edu> (raw)
I got the following failure after restarting an OSD. There were no
issues prior to this, but the cluster had been stopped/restarted many
times. This is on ext4.
Thanks,
Noah
2011-12-06 11:58:11.552177 7ffbe5f89700 filestore(/mnt/vol1/osd) error
error 22: Invalid argument not handled
os/FileStore.cc: In function 'unsigned int
FileStore::_do_transaction(ObjectStore::Transaction&, uint64_t)', in
thread '7ffbe5f89700'
os/FileStore.cc: 2412: FAILED assert(0 == "unexpected error")
ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381)
1: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned
long)+0x56e) [0x6de19e]
2: (FileStore::do_transactions(std::list<ObjectStore::Transaction*,
std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x78)
[0x6e2678]
3: (FileStore::_do_op(FileStore::OpSequencer*)+0x176) [0x6bb306]
4: (ThreadPool::worker()+0xa28) [0x5bee78]
5: (ThreadPool::WorkThread::entry()+0xd) [0x565f7d]
6: (()+0x7971) [0x7ffbf0a0e971]
7: (clone()+0x6d) [0x7ffbef09992d]
ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381)
1: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned
long)+0x56e) [0x6de19e]
2: (FileStore::do_transactions(std::list<ObjectStore::Transaction*,
std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x78)
[0x6e2678]
3: (FileStore::_do_op(FileStore::OpSequencer*)+0x176) [0x6bb306]
4: (ThreadPool::worker()+0xa28) [0x5bee78]
5: (ThreadPool::WorkThread::entry()+0xd) [0x565f7d]
6: (()+0x7971) [0x7ffbf0a0e971]
7: (clone()+0x6d) [0x7ffbef09992d]
*** Caught signal (Aborted) **
in thread 7ffbe5f89700
ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381)
1: /usr/bin/ceph-osd() [0x6b1c84]
2: (()+0xfb40) [0x7ffbf0a16b40]
3: (gsignal()+0x35) [0x7ffbeefe6ba5]
4: (abort()+0x180) [0x7ffbeefea6b0]
5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7ffbef88a6bd]
6: (()+0xb9906) [0x7ffbef888906]
7: (()+0xb9933) [0x7ffbef888933]
8: (()+0xb9a3e) [0x7ffbef888a3e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char
const*)+0x39f) [0x5ccdbf]
10: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned
long)+0x56e) [0x6de19e]
11: (FileStore::do_transactions(std::list<ObjectStore::Transaction*,
std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x78)
[0x6e2678]
12: (FileStore::_do_op(FileStore::OpSequencer*)+0x176) [0x6bb306]
13: (ThreadPool::worker()+0xa28) [0x5bee78]
14: (ThreadPool::WorkThread::entry()+0xd) [0x565f7d]
15: (()+0x7971) [0x7ffbf0a0e971]
16: (clone()+0x6d) [0x7ffbef09992d]
2011-12-06 13:25:13.390151 7fc539d3f780 ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381), process ceph-osd, pid
1314
2011-12-06 13:25:13.459719 7fc539d3f780 filestore(/mnt/vol1/osd) mount
FIEMAP ioctl is supported
2011-12-06 13:25:13.459854 7fc539d3f780 filestore(/mnt/vol1/osd) mount
did NOT detect btrfs
2011-12-06 13:25:13.482119 7fc539d3f780 filestore(/mnt/vol1/osd) mount
found snaps <>
2011-12-06 13:25:13.494092 7fc539d3f780 filestore(/mnt/vol1/osd) mount:
enabling WRITEAHEAD journal mode: btrfs not detected
2011-12-06 13:25:13.972914 7fc539d3f780 journal _open
/mnt/vol2/osd.journal fd 14: 1048576000 bytes, block size 4096 bytes,
directio = 1
2011-12-06 13:25:14.013331 7fc539d3f780 journal read_entry 803999744 :
seq 54062 1002407 bytes
2011-12-06 13:25:14.014773 7fc539d3f780 journal read_entry 803999744 :
seq 54062 1002407 bytes
2011-12-06 13:25:14.048581 7fc539d3f780 filestore(/mnt/vol1/osd) error
error 22: Invalid argument not handled
os/FileStore.cc: In function 'unsigned int
FileStore::_do_transaction(ObjectStore::Transaction&, uint64_t)', in
thread '7fc539d3f780'
os/FileStore.cc: 2412: FAILED assert(0 == "unexpected error")
ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381)
1: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned
long)+0x56e) [0x6de19e]
2: (FileStore::do_transactions(std::list<ObjectStore::Transaction*,
std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x78)
[0x6e2678]
3: (JournalingObjectStore::journal_replay(unsigned long)+0x8e0) [0x6eab90]
4: (FileStore::mount()+0x2a68) [0x6cc128]
5: (OSD::convertfs(std::string const&, std::string const&)+0xc8)
[0x525d38]
6: (main()+0x2bed) [0x49e6ed]
7: (__libc_start_main()+0xfe) [0x7fc537ee3d8e]
8: /usr/bin/ceph-osd() [0x49b6d9]
ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381)
1: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned
long)+0x56e) [0x6de19e]
2: (FileStore::do_transactions(std::list<ObjectStore::Transaction*,
std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x78)
[0x6e2678]
3: (JournalingObjectStore::journal_replay(unsigned long)+0x8e0) [0x6eab90]
4: (FileStore::mount()+0x2a68) [0x6cc128]
5: (OSD::convertfs(std::string const&, std::string const&)+0xc8)
[0x525d38]
6: (main()+0x2bed) [0x49e6ed]
7: (__libc_start_main()+0xfe) [0x7fc537ee3d8e]
8: /usr/bin/ceph-osd() [0x49b6d9]
*** Caught signal (Aborted) **
in thread 7fc539d3f780
ceph version 0.38-259-gd4aef20
(commit:d4aef20210d43e25eefe945009e6f77d5b045381)
1: /usr/bin/ceph-osd() [0x6b1c84]
2: (()+0xfb40) [0x7fc539928b40]
3: (gsignal()+0x35) [0x7fc537ef8ba5]
4: (abort()+0x180) [0x7fc537efc6b0]
5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7fc53879c6bd]
6: (()+0xb9906) [0x7fc53879a906]
7: (()+0xb9933) [0x7fc53879a933]
8: (()+0xb9a3e) [0x7fc53879aa3e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char
const*)+0x39f) [0x5ccdbf]
10: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned
long)+0x56e) [0x6de19e]
11: (FileStore::do_transactions(std::list<ObjectStore::Transaction*,
std::allocator<ObjectStore::Transaction*> >&, unsigned long)+0x78)
[0x6e2678]
12: (JournalingObjectStore::journal_replay(unsigned long)+0x8e0)
[0x6eab90]
13: (FileStore::mount()+0x2a68) [0x6cc128]
14: (OSD::convertfs(std::string const&, std::string const&)+0xc8)
[0x525d38]
15: (main()+0x2bed) [0x49e6ed]
16: (__libc_start_main()+0xfe) [0x7fc537ee3d8e]
17: /usr/bin/ceph-osd() [0x49b6d9]
nwatkins@piha:~$
reply other threads:[~2011-12-06 21:31 UTC|newest]
Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4EDE89AC.5030405@cs.ucsc.edu \
--to=jayhawk@cs.ucsc.edu \
--cc=ceph-devel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox