Re: SQ overflow seen running isert traffic with high block sizes

public inbox for linux-rdma@vger.kernel.org
 help / color / mirror / Atom feed

From: Potnuri Bharat Teja <bharat-ut6Up61K2wZBDgjK7y7TUQ@public.gmane.org>
To: "Amrani, Ram" <Ram.Amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Cc: "linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org"
	<linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	"Elior,
	Ariel" <Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Subject: Re: SQ overflow seen running isert traffic with high block sizes
Date: Wed, 28 Jun 2017 16:05:07 +0530	[thread overview]
Message-ID: <20170628103505.GA27517@chelsio.com> (raw)
In-Reply-To: <BN3PR07MB25784033E7FCD062FA0A7855F8DD0-EldUQEzkDQfpW3VS/XPqkOFPX92sqiQdvxpqHgZTriW3zl9H0oFU5g@public.gmane.org>

Here is some more discussion regarding SQ overflow.
https://www.spinics.net/lists/linux-rdma/msg47635.html

Current SQ overflow handling is post error handling to keep the IO running and 
We still see SQ post failures filling the dmesg.
We are still working on fixing SQ overflow.

Thanks,
Bharat.

On Wednesday, June 06/28/17, 2017 at 14:55:45 +0530, Amrani, Ram wrote:
>    We are hitting SQ overflow on iSER target side with high block sizes over
>    RoCE
>    (see dmesg output below).
> 
>    We are using Q-Logic/Cavium NIC with a capability of 4 sges.
> 
>    Following the thread "SQ overflow seen running isert traffic" [1], I was
>    wondering
>    if someone is working on SQ accounting, or more graceful handling of
>    overflow,
>    as the messages are printed over and over.
> 
>    Dmesg output:
>    2017-06-06T09:23:28.824234+05:30 SLES12SP3Beta3 kernel: [65057.799615]
>    isert: isert_rdma_rw_ctx_post: Cmd: ffff880f83cb2cb0 failed to post RDMA
>    res
>    2017-06-06T09:23:29.500095+05:30 SLES12SP3Beta3 kernel: [65058.475858]
>    isert: isert_rdma_rw_ctx_post: Cmd: ffff880ec53ec020 failed to post RDMA
>    res
>    2017-06-06T09:23:29.560085+05:30 SLES12SP3Beta3 kernel: [65058.533787]
>    isert: isert_rdma_rw_ctx_post: Cmd: ffff880ec622ae08 failed to post RDMA
>    res
>    2017-06-06T09:23:29.984209+05:30 SLES12SP3Beta3 kernel: [65058.958509]
>    isert: isert_rdma_rw_ctx_post: Cmd: ffff880ff08e6bb0 failed to post RDMA
>    res
>    2017-06-06T09:23:30.056098+05:30 SLES12SP3Beta3 kernel: [65059.032182]
>    isert: isert_rdma_rw_ctx_post: Cmd: ffff880fa6761138 failed to post RDMA
>    res
>    2017-06-06T09:23:30.288152+05:30 SLES12SP3Beta3 kernel: [65059.262748]
>    isert: isert_rdma_rw_ctx_post: Cmd: ffff880ec3caf668 failed to post RDMA
>    res
>    2017-06-06T09:23:30.444068+05:30 SLES12SP3Beta3 kernel: [65059.421071]
>    isert: isert_rdma_rw_ctx_post: Cmd: ffff880f2186cc30 failed to post RDMA
>    res
>    2017-06-06T09:23:30.532135+05:30 SLES12SP3Beta3 kernel: [65059.505380]
>    isert: isert_rdma_rw_ctx_post: Cmd: ffff880ec6429bf0 failed to post RDMA
>    res
>    2017-06-06T09:23:30.672098+05:30 SLES12SP3Beta3 kernel: [65059.645585]
>    isert: isert_rdma_rw_ctx_post: Cmd: ffff880fa5526bb0 failed to post RDMA
>    res
>    2017-06-06T09:23:30.852121+05:30 SLES12SP3Beta3 kernel: [65059.828072]
>    isert: isert_rdma_rw_ctx_post: Cmd: ffff880f5c8a89d8 failed to post RDMA
>    res
>    2017-06-06T09:23:31.464125+05:30 SLES12SP3Beta3 kernel: [65060.440092]
>    isert: isert_rdma_rw_ctx_post: Cmd: ffff880ffefdf918 failed to post RDMA
>    res
>    2017-06-06T09:23:31.576074+05:30 SLES12SP3Beta3 kernel: [65060.550314]
>    isert: isert_rdma_rw_ctx_post: Cmd: ffff880f83222350 failed to post RDMA
>    res
>    2017-06-06T09:24:30.532064+05:30 SLES12SP3Beta3 kernel: [65119.503466]
>    ABORT_TASK: Found referenced iSCSI task_tag: 103
>    2017-06-06T09:24:30.532079+05:30 SLES12SP3Beta3 kernel: [65119.503468]
>    ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST for ref_tag: 103
>    2017-06-06T09:24:31.428084+05:30 SLES12SP3Beta3 kernel: [65120.399433]
>    ABORT_TASK: Found referenced iSCSI task_tag: 101
>    2017-06-06T09:24:31.428101+05:30 SLES12SP3Beta3 kernel: [65120.399436]
>    ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST for ref_tag: 101
>    2017-06-06T09:24:31.556053+05:30 SLES12SP3Beta3 kernel: [65120.527461]
>    ABORT_TASK: Found referenced iSCSI task_tag: 119
>    2017-06-06T09:24:31.556060+05:30 SLES12SP3Beta3 kernel: [65120.527465]
>    ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST for ref_tag: 119
>    2017-06-06T09:24:31.556061+05:30 SLES12SP3Beta3 kernel: [65120.527468]
>    ABORT_TASK: Found referenced iSCSI task_tag: 43
>    2017-06-06T09:24:31.556062+05:30 SLES12SP3Beta3 kernel: [65120.527469]
>    ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST for ref_tag: 43
>    2017-06-06T09:24:31.556063+05:30 SLES12SP3Beta3 kernel: [65120.527470]
>    ABORT_TASK: Found referenced iSCSI task_tag: 79
>    2017-06-06T09:24:31.556064+05:30 SLES12SP3Beta3 kernel: [65120.527471]
>    ABORT_TASK: Found referenced iSCSI task_tag: 71
>    2017-06-06T09:24:31.556066+05:30 SLES12SP3Beta3 kernel: [65120.527472]
>    ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST for ref_tag: 79
>    2017-06-06T09:24:31.556067+05:30 SLES12SP3Beta3 kernel: [65120.527472]
>    ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST for ref_tag: 71
>    2017-06-06T09:24:31.556068+05:30 SLES12SP3Beta3 kernel: [65120.527506]
>    ABORT_TASK: Found referenced iSCSI task_tag: 122
>    2017-06-06T09:24:31.556068+05:30 SLES12SP3Beta3 kernel: [65120.527508]
>    ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST for ref_tag: 122
>    2017-06-06T09:24:32.452073+05:30 SLES12SP3Beta3 kernel: [65121.423425]
>    ABORT_TASK: Found referenced iSCSI task_tag: 58
>    2017-06-06T09:24:32.452080+05:30 SLES12SP3Beta3 kernel: [65121.423427]
>    ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST for ref_tag: 58
>    2017-06-06T09:24:32.516054+05:30 SLES12SP3Beta3 kernel: [65121.487380]
>    ABORT_TASK: Found referenced iSCSI task_tag: 100
>    2017-06-06T09:24:32.516061+05:30 SLES12SP3Beta3 kernel: [65121.487382]
>    ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST for ref_tag: 100
>    2017-06-06T09:24:32.584031+05:30 SLES12SP3Beta3 kernel: [65121.555374]
>    ABORT_TASK: Found referenced iSCSI task_tag: 52
>    2017-06-06T09:24:32.584041+05:30 SLES12SP3Beta3 kernel: [65121.555376]
>    ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST for ref_tag: 52
>    2017-06-06T09:24:33.412057+05:30 SLES12SP3Beta3 kernel: [65122.383341]
>    ABORT_TASK: Found referenced iSCSI task_tag: 43
>    2017-06-06T09:24:33.412065+05:30 SLES12SP3Beta3 kernel: [65122.383376]
>    ABORT_TASK: Sending TMR_FUNCTION_COMPLETE for ref_tag: 43
>    2017-06-06T09:24:33.476061+05:30 SLES12SP3Beta3 kernel: [65122.447354]
>    ABORT_TASK: Found referenced iSCSI task_tag: 63
>    2017-06-06T09:24:33.476070+05:30 SLES12SP3Beta3 kernel: [65122.447360]
>    ABORT_TASK: Sending TMR_TASK_DOES_NOT_EXIST for ref_tag: 63
> 
>    [1] [1]https://patchwork.kernel.org/patch/9633675/
> 
>    Thanks,
>    Ram
>    --
>    To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>    the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>    More majordomo info at  [2]http://vger.kernel.org/majordomo-info.html
> 
> References
> 
>    Visible links
>    1. https://patchwork.kernel.org/patch/9633675/
>    2. http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

next prev parent reply	other threads:[~2017-06-28 10:35 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-06-28  9:25 SQ overflow seen running isert traffic with high block sizes Amrani, Ram
     [not found] ` <BN3PR07MB25784033E7FCD062FA0A7855F8DD0-EldUQEzkDQfpW3VS/XPqkOFPX92sqiQdvxpqHgZTriW3zl9H0oFU5g@public.gmane.org>
2017-06-28 10:35   ` Potnuri Bharat Teja [this message]
     [not found]     ` <20170628103505.GA27517-ut6Up61K2wZBDgjK7y7TUQ@public.gmane.org>
2017-06-28 11:29       ` Amrani, Ram
2017-06-28 10:39 ` Sagi Grimberg
2017-06-28 11:32   ` Amrani, Ram
     [not found]     ` <BN3PR07MB25786338EADC77A369A6D493F8DD0-EldUQEzkDQfpW3VS/XPqkOFPX92sqiQdvxpqHgZTriW3zl9H0oFU5g@public.gmane.org>
2017-07-13 18:29       ` Nicholas A. Bellinger
2017-07-17  9:26         ` Amrani, Ram
     [not found]           ` <BN3PR07MB2578E6561CC669922A322245F8A00-EldUQEzkDQfpW3VS/XPqkOFPX92sqiQdvxpqHgZTriW3zl9H0oFU5g@public.gmane.org>
2017-10-06 22:40             ` Shiraz Saleem
     [not found]               ` <20171006224025.GA23364-GOXS9JX10wfOxmVO0tvppfooFf0ArEBIu+b9c/7xato@public.gmane.org>
2018-01-15  4:56                 ` Nicholas A. Bellinger
     [not found]                   ` <1515992195.24576.156.camel-XoQW25Eq2zs8TOCF0fvnoXxStJ4P+DSV@public.gmane.org>
2018-01-15 10:12                     ` Kalderon, Michal
     [not found]                       ` <CY1PR0701MB2012E53C69D1CE3E16BA320B88EB0-UpKza+2NMNLHMJvQ0dyT705OhdzP3rhOnBOFsp37pqbUKgpGm//BTAC/G2K4zDHf@public.gmane.org>
2018-01-15 15:22                         ` Shiraz Saleem
2018-01-18  9:58                           ` Nicholas A. Bellinger
2018-01-18 17:53                             ` Potnuri Bharat Teja
     [not found]                               ` <20180118175316.GA11338-ut6Up61K2wZBDgjK7y7TUQ@public.gmane.org>
2018-01-24  7:25                                 ` Nicholas A. Bellinger
2018-01-24 12:21                                   ` Potnuri Bharat Teja
     [not found]                               ` <1516778717.24576.319.came l@haakon3.daterainc.com>
     [not found]                                 ` <1516778717.24576.319.camel-XoQW25Eq2zs8TOCF0fvnoXxStJ4P+DSV@public.gmane.org>
2018-01-24 16:03                                   ` Steve Wise
2018-01-19 19:33                             ` Kalderon, Michal
2018-01-24  7:55                               ` Nicholas A. Bellinger
2018-01-24  8:09                                 ` Kalderon, Michal
2018-01-29 19:20                                   ` Sagi Grimberg
     [not found]                                 ` <1516780534.24576.335.camel-XoQW25Eq2zs8TOCF0fvnoXxStJ4P+DSV@public.gmane.org>
2018-01-29 19:17                                   ` Sagi Grimberg
     [not found]                                     ` <55569d98-7f8c-7414-ab03-e52e2bfc518b-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
2018-01-30 16:30                                       ` Shiraz Saleem
     [not found]                             ` <1516269522.24576.274.camel-XoQW25Eq2zs8TOCF0fvnoXxStJ4P+DSV@public.gmane.org>
2018-01-22 17:49                               ` Saleem, Shiraz
2018-01-24  8:01                                 ` Nicholas A. Bellinger
2018-01-26 18:52                                   ` Shiraz Saleem
2018-01-29 19:36                                   ` Sagi Grimberg

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170628103505.GA27517@chelsio.com \
    --to=bharat-ut6up61k2wzbdgjk7y7tuq@public.gmane.org \
    --cc=Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org \
    --cc=Ram.Amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org \
    --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Be sure your reply has a Subject: header at the top and a blank line before the message body.

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox