From: norbi <norbi@rocknob.de>
To: ceph-devel@vger.kernel.org
Subject: Re: v0.56 released
Date: Thu, 03 Jan 2013 09:28:00 +0100 [thread overview]
Message-ID: <50E54110.8090203@rocknob.de> (raw)
In-Reply-To: <alpine.DEB.2.00.1212311721310.1906@cobra.newdream.net>
Hi List,
after upgrading from 0.55.1 to 0.56 some MONs are crashing during the
upgrade.
I have 3 MONs with 0.55.1, mon.a, mon.b. and mon.c
So now i am upgrading mon.a to 0.56, i restarted mon.a and see that
mon.c is crashed... so i restarted mon.c and see, now mon.b is crashed,
after restart all mons are running ?
The Log from mon.b
-7> 2013-01-03 09:09:02.011229 7fc4d1d00700 -1 mon/PaxosService.cc:
In function 'void PaxosService::propose_pending()' thread 7fc4d1d00700
time 2013-01-03 09:09:01.900100
mon/PaxosService.cc: 110: FAILED assert(have_pending)
ceph version 0.55.1 (8e25c8d984f9258644389a18997ec6bdef8e056b)
1: /usr/local/bin/ceph-mon() [0x4a6e94]
2: (MDSMonitor::tick()+0x1a45) [0x4e1245]
3: (MDSMonitor::on_active()+0x1f) [0x4d67ef]
4: (PaxosService::_active()+0x245) [0x4a7a95]
5: (Context::complete(int)+0xa) [0x48bbda]
6: (finish_contexts(CephContext*, std::list<Context*,
std::allocator<Context*> >&, int)+0x122) [0x496d72]
7: (Monitor::recovered_leader(int)+0x378) [0x478ed8]
8: (Paxos::handle_last(MMonPaxos*)+0xb19) [0x4a3919]
9: (Paxos::dispatch(PaxosServiceMessage*)+0x27b) [0x4a40fb]
10: (Monitor::_ms_dispatch(Message*)+0x1298) [0x48ae78]
11: (Monitor::ms_dispatch(Message*)+0x32) [0x49a932]
12: (DispatchQueue::entry()+0x2d9) [0x620c19]
13: (DispatchQueue::DispatchThread::entry()+0xd) [0x5c3a8d]
14: (()+0x7851) [0x7fc4d65e6851]
15: (clone()+0x6d) [0x7fc4d4df011d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is
needed to interpret this.
-6> 2013-01-03 09:09:02.044710 7fc4cf7e9700 1 --
46.252.23.110:6789/0 >> :/0 pipe(0x477e540 sd=26 :6789 pgs=0 cs=0
l=0).accept sd=26
-5> 2013-01-03 09:09:02.219117 7fc4cf4e6700 1 --
46.252.23.110:6789/0 >> :/0 pipe(0x4778480 sd=28 :6789 pgs=0 cs=0
l=0).accept sd=28
-4> 2013-01-03 09:09:02.462884 7fc4cf3e5700 1 --
46.252.23.110:6789/0 >> :/0 pipe(0x4718240 sd=29 :6789 pgs=0 cs=0
l=0).accept sd=29
-3> 2013-01-03 09:09:02.848348 7fc4cfcee700 1 --
46.252.23.110:6789/0 >> :/0 pipe(0x4718000 sd=30 :6789 pgs=0 cs=0
l=0).accept sd=30
-2> 2013-01-03 09:09:02.924980 7fc4ceddf700 2 --
46.252.23.110:6789/0 >> 80.67.16.129:6800/31582 pipe(0x471a640 sd=17
:6789 pgs=22 cs=1 l=1).reader couldn't read tag, Success
-1> 2013-01-03 09:09:02.925020 7fc4ceddf700 2 --
46.252.23.110:6789/0 >> 80.67.16.129:6800/31582 pipe(0x471a640 sd=17
:6789 pgs=22 cs=1 l=1).fault 0: Success
--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
0/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 hadoop
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 100000
max_new 1000
log_file /var/log/ceph/mon.b.log
--- end dump of recent events ---
2013-01-03 09:09:03.039368 7fc4d1d00700 -1 *** Caught signal (Aborted) **
in thread 7fc4d1d00700
ceph version 0.55.1 (8e25c8d984f9258644389a18997ec6bdef8e056b)
1: /usr/local/bin/ceph-mon() [0x537729]
2: (()+0xf500) [0x7fc4d65ee500]
3: (gsignal()+0x35) [0x7fc4d4d3a8a5]
4: (abort()+0x175) [0x7fc4d4d3c085]
5: (__gnu_cxx::__verbose_terminate_handler()+0x12d) [0x7fc4d55f3a5d]
6: (()+0xbcbe6) [0x7fc4d55f1be6]
7: (()+0xbcc13) [0x7fc4d55f1c13]
8: (()+0xbcd0e) [0x7fc4d55f1d0e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char
const*)+0x7c9) [0x5cfe39]
10: /usr/local/bin/ceph-mon() [0x4a6e94]
11: (MDSMonitor::tick()+0x1a45) [0x4e1245]
12: (MDSMonitor::on_active()+0x1f) [0x4d67ef]
13: (PaxosService::_active()+0x245) [0x4a7a95]
14: (Context::complete(int)+0xa) [0x48bbda]
15: (finish_contexts(CephContext*, std::list<Context*,
std::allocator<Context*> >&, int)+0x122) [0x496d72]
16: (Monitor::recovered_leader(int)+0x378) [0x478ed8]
17: (Paxos::handle_last(MMonPaxos*)+0xb19) [0x4a3919]
18: (Paxos::dispatch(PaxosServiceMessage*)+0x27b) [0x4a40fb]
19: (Monitor::_ms_dispatch(Message*)+0x1298) [0x48ae78]
20: (Monitor::ms_dispatch(Message*)+0x32) [0x49a932]
21: (DispatchQueue::entry()+0x2d9) [0x620c19]
22: (DispatchQueue::DispatchThread::entry()+0xd) [0x5c3a8d]
23: (()+0x7851) [0x7fc4d65e6851]
24: (clone()+0x6d) [0x7fc4d4df011d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is
needed to interpret this.
--- begin dump of recent events ---
-1> 2013-01-03 09:09:03.039368 7fc4d1d00700 -1 *** Caught signal
(Aborted) **
in thread 7fc4d1d00700
ceph version 0.55.1 (8e25c8d984f9258644389a18997ec6bdef8e056b)
1: /usr/local/bin/ceph-mon() [0x537729]
2: (()+0xf500) [0x7fc4d65ee500]
3: (gsignal()+0x35) [0x7fc4d4d3a8a5]
4: (abort()+0x175) [0x7fc4d4d3c085]
5: (__gnu_cxx::__verbose_terminate_handler()+0x12d) [0x7fc4d55f3a5d]
6: (()+0xbcbe6) [0x7fc4d55f1be6]
7: (()+0xbcc13) [0x7fc4d55f1c13]
8: (()+0xbcd0e) [0x7fc4d55f1d0e]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char
const*)+0x7c9) [0x5cfe39]
10: /usr/local/bin/ceph-mon() [0x4a6e94]
11: (MDSMonitor::tick()+0x1a45) [0x4e1245]
12: (MDSMonitor::on_active()+0x1f) [0x4d67ef]
13: (PaxosService::_active()+0x245) [0x4a7a95]
14: (Context::complete(int)+0xa) [0x48bbda]
15: (finish_contexts(CephContext*, std::list<Context*,
std::allocator<Context*> >&, int)+0x122) [0x496d72]
16: (Monitor::recovered_leader(int)+0x378) [0x478ed8]
17: (Paxos::handle_last(MMonPaxos*)+0xb19) [0x4a3919]
18: (Paxos::dispatch(PaxosServiceMessage*)+0x27b) [0x4a40fb]
19: (Monitor::_ms_dispatch(Message*)+0x1298) [0x48ae78]
20: (Monitor::ms_dispatch(Message*)+0x32) [0x49a932]
21: (DispatchQueue::entry()+0x2d9) [0x620c19]
22: (DispatchQueue::DispatchThread::entry()+0xd) [0x5c3a8d]
23: (()+0x7851) [0x7fc4d65e6851]
24: (clone()+0x6d) [0x7fc4d4df011d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is
needed to interpret this.
--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
0/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 hadoop
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 100000
max_new 1000
log_file /var/log/ceph/mon.b.log
--- end dump of recent events ---
Norbert
On 01.01.2013 07:02, Sage Weil wrote:
> We're bringing in the new year with a new release, v0.56, which will form
> the basis of the next stable series "bobtail." There is little in the way
> of new functionality since v0.55, as we've been focusing primarily on
> stability, performance, and upgradability from the previous argonaut
> stable series (v0.48.x). If you are a current argonaut user, you can
> either upgrade now, or watch the Inktank blog for the bobtail announcement
> after some additional testing has been completed. If you are a v0.55 or
> v0.55.1 user, we recommend upgrading now.
>
> Notable changes since v0.55 include:
>
> * librbd: fixes for read-only pools for image cloning
> * osd: fix for mixing argonaut and post-v0.54 OSDs
> * osd: some recovery tuning
> * osd: fix for several scrub, recovery, and watch/notify races/bugs
> * osd: fix pool_stat_t backwawrd compatibility with pre-v0.41 clients
> * osd: experimental split support
> * mkcephfs: misc fixes for fs initialization, mounting
> * radosgw: usage and op logs off by default
> * radosgw: keystone authentication off by default
> * upstart: only enabled with 'upstart' file exists in daemon data
> directory
> * mount.fuse.ceph: allow mounting of ceph-fuse via /etc/fstab
> * config: always complain about config parsing errors
> * mon: fixed memory leaks, misc bugs
> * mds: many misc fixes
>
> Notable changes since v0.48.2 ("argonaut"):
>
> * auth: authentication is now on by default; see release notes!
> * osd: improved threading, small io performance
> * osd: deep scrubbing (verify object data)
> * osd: chunky scrubs (more efficient)
> * osd: improved performance during recovery
> * librbd: cloning support
> * librbd: fine-grained striping support
> * librbd: better caching
> * radosgw: improved Swift and S3 API coverage (POST, multi-object delete,
> striping)
> * radosgw: OpenStack Keystone integration
> * radosgw: efficient usage stats aggregation (for billing)
> * crush: improvements in distribution (still off by default; see CRUSH
> tunables)
> * ceph-fuse, mds: general stability improvements
> * release RPMs for OpenSUSE, SLES, Fedora, RHEL, CentOS
> * tons and bug fixes and small improvements across the board
>
> If you are upgrading from v0.55, there are no special upgrade
> instructions. If you are upgrading from an older version, please read the
> release notes. Authentication is now enabled by default, and if you do not
> adjust your ceph.conf accordingly before upgrading the system will not
> come up by itself.
>
> You can get this release from the usual locations:
>
> * Git at git://github.com/ceph/ceph.git
> * Tarball at http://ceph.com/download/ceph-0.56.tar.gz
> * For Debian/Ubuntu packages, see http://ceph.com/docs/master/install/debian
> * For RPMs, see http://ceph.com/docs/master/install/rpm
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
next prev parent reply other threads:[~2013-01-03 8:28 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-01-01 6:02 v0.56 released Sage Weil
2013-01-02 2:24 ` Dennis Jacobfeuerborn
2013-01-02 2:28 ` Mark Nelson
2013-01-02 9:45 ` Sébastien Han
2013-01-02 17:35 ` Sage Weil
2013-01-02 18:03 ` Sébastien Han
2013-01-02 18:11 ` Sage Weil
2013-01-02 18:14 ` Sébastien Han
2013-01-02 20:27 ` Dennis Jacobfeuerborn
2013-01-02 20:46 ` Sage Weil
2013-01-03 4:00 ` Dennis Jacobfeuerborn
2013-01-03 5:49 ` Dan Mick
2013-01-03 8:28 ` norbi [this message]
2013-01-03 10:44 ` Joao Eduardo Luis
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=50E54110.8090203@rocknob.de \
--to=norbi@rocknob.de \
--cc=ceph-devel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox