From: "Piotr Dałek" <branch@predictor.org.pl>
To: Somnath Roy <Somnath.Roy@sandisk.com>,
"ceph-users@lists.ceph.com" <ceph-users@lists.ceph.com>,
"ceph-devel@vger.kernel.org" <ceph-devel@vger.kernel.org>
Subject: Re: OSDs are flapping and marked down wrongly
Date: Mon, 17 Oct 2016 10:19:38 +0200 [thread overview]
Message-ID: <20161017081938.GB28088@predictor> (raw)
In-Reply-To: <BY2PR02MB396DD0B2B7541A1DB23123CF4D00@BY2PR02MB396.namprd02.prod.outlook.com>
On Mon, Oct 17, 2016 at 08:06:19AM +0000, Somnath Roy wrote:
> Thanks Piotr, Wido for quick response.
>
> @Wido , yes, I thought of trying with those values but I am seeing in the log messages at least 7 osds are reporting failure , so, didn't try. BTW, I found default mon_osd_min_down_reporters is 2 , not 1 and latest master is not having mon_osd_min_down_reports anymore. Not sure what it is replaced with..
>
> @Piotr , yes, your PR really helps , thanks ! Regarding each messenger needs to respond to HB is confusing, I know each thread has a HB timeout value and beyond which it will crash with suicide timeout , are you talking about that ?
Not really, as I wrote previously - if you keep filling up the pipeline,
OSDs will fail to respond for heartbeats because they won't process them at
all or will process them, but the output pipeline will be so full that the
response won't get to the recipient in time.
Suicide timeouts occur when disk threads fail to process ops in reasonable
amount of time (hence the name: "suicide").
--
Piotr Dałek
branch@predictor.org.pl
http://blog.predictor.org.pl
next prev parent reply other threads:[~2016-10-17 8:18 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-10-17 7:16 OSDs are flapping and marked down wrongly Somnath Roy
[not found] ` <BY2PR02MB3964B80170065D0141B7931F4D00-USF8g7QUirCbDkdw+1LTknlDjJuWSFo1XA4E9RH9d+qIuWR1G4zioA@public.gmane.org>
2016-10-17 7:23 ` Wido den Hollander
2016-10-17 9:13 ` Wei Jin
2016-10-17 17:14 ` [ceph-users] " Somnath Roy
2016-10-17 7:51 ` Piotr Dałek
2016-10-17 8:06 ` Somnath Roy
2016-10-17 8:19 ` Piotr Dałek [this message]
-- strict thread matches above, loose matches on Subject: below --
2016-10-17 8:24 Pavan Rallabhandi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20161017081938.GB28088@predictor \
--to=branch@predictor.org.pl \
--cc=Somnath.Roy@sandisk.com \
--cc=ceph-devel@vger.kernel.org \
--cc=ceph-users@lists.ceph.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.