All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andrew Donnellan <andrew.donnellan-8fk3Idey6ehBDgjK7y7TUQ@public.gmane.org>
To: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>,
	Bart Van Assche
	<bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org,
	axboe-b10kYP2dOMg@public.gmane.org,
	linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linuxppc-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
Subject: Re: Re: [PATCH 03/13] irq_poll: fold irq_poll_sched_prep into irq_poll_sched
Date: Wed, 20 Jan 2016 18:02:30 +1100	[thread overview]
Message-ID: <569F3106.7000205@au1.ibm.com> (raw)
In-Reply-To: <20151230094253.GB12904-jcswGhMUV9g@public.gmane.org>

On 30/12/15 20:42, Christoph Hellwig wrote:
> On Tue, Dec 29, 2015 at 10:54:18AM +0100, Bart Van Assche wrote:
>> After having applied these changes the SRP initiator didn't receive any
>> RDMA completions anymore. I could remedy that by changing
>> "!test_and_set_bit()" into "test_and_set_bit()":
>
> Yes.  I actually had this bug earlier, fixed it and managed to get
> it back during a rebase, d'oh.

I'm hitting an issue on a ppc64le box running linux-next, which 
according to git bisect is caused by this patch.

It looks like I might be hitting a dodgy error path as well, as we seem 
to be trying to execute data.

Any ideas?


Andrew

---

Sent SIGTERM to all processes
Sent SIGKILL to all processes
  -> smp_release_cpus()
spinning_secondaries = 47
  <- smp_release_cpus()
  <- setup_system()
sr 0:0:1:0: tag#0 Resetting device
ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 t0
ata1.00: cmd a0/00:00:00:00:00/00:00:00:00:00/a0 tag 3
          Test Unit Ready 00 00 00 00 00 00res 
40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
ata1.00: status: { DRDY }
ata1: translated ATA stat/err 0xd1/00 to SCSI SK/ASC/ASCQ 0xb/47/00
sd 1:2:0:0: tag#0 Resetting device
ata1.00: failed to set xfermode (err_mask=0x4)
ipr 0003:04:00.0: Timed out waiting for aborted commands
ipr 0003:04:00.0: Adapter being reset as a result of error recovery.
ata1.00: failed to set xfermode (err_mask=0x4)
ata1.00: failed to set xfermode (err_mask=0x4)
ipr 0001:04:00.0: Adapter being reset as a result of error recovery.
cpu 0x0: Vector: e40 (Emulation Assist) at [c000000000daf2e0]
     pc: c000000000e51ae8: dump_list_lock+0x0/0x4
     lr: c0000000000f46e4: __wake_up_common+0x84/0xf0
     sp: c000000000daf560
    msr: 9000000102089033
   current = 0xc000000000d6f500
   paca    = 0xc00000000fe00000   softe: 0        irq_happened: 0x01
     pid   = 0, comm = swapper/0
Linux version 4.4.0-next-20160118 (ajd@ka1) (gcc version 5.2.1 20150930 
(GCC) ) #13 SMP Tue Jan 19 12:04:19 AEDT 2016
enter ? for help
[link register   ] c0000000000f46e4 __wake_up_common+0x84/0xf0
[c000000000daf560] c000000000da1100 pps_cdev_fops+0xc8/0x100 (unreliable)
[c000000000daf5c0] c0000000000f5264 complete+0x54/0x90
[c000000000daf600] c00000000061f44c ata_qc_complete_internal+0x1c/0x30
[c000000000daf620] c000000000622828 __ata_qc_complete+0xb8/0x190
[c000000000daf660] c0000000005ef6e4 ipr_sata_eh_done+0x64/0x80
[c000000000daf680] c0000000005ef530 ipr_fail_all_ops+0x100/0x250
[c000000000daf740] c0000000005ffbf8 ipr_reset_restore_cfg_space+0x98/0x230
[c000000000daf7b0] c0000000005ed500 ipr_reset_ioa_job+0x80/0xf0
[c000000000daf7e0] c0000000005ebfac ipr_reset_timer_done+0xac/0xe0
[c000000000daf820] c00000000011eae4 call_timer_fn+0x54/0x180
[c000000000daf8b0] c00000000011ef2c run_timer_softirq+0x2ec/0x3a0
[c000000000daf980] c0000000000a4ee8 __do_softirq+0x188/0x3b0
[c000000000dafa70] c0000000000a5358 irq_exit+0xc8/0x100
[c000000000dafa90] c00000000001d894 timer_interrupt+0xa4/0xe0
[c000000000dafac0] c000000000002750 decrementer_common+0x150/0x180
--- Exception: 901 (Decrementer) at c000000000010364 
arch_local_irq_restore+0x74/0x90
[c000000000dafdb0] c000000000dac000 init_thread_union+0x0/0x4000 
(unreliable)
[c000000000dafdd0] c000000000016be8 arch_cpu_idle+0x108/0x160
[c000000000dafe00] c0000000000f5594 default_idle_call+0x44/0x80
[c000000000dafe20] c0000000000f5a48 cpu_startup_entry+0x3d8/0x450
[c000000000dafee0] c00000000000bbe4 rest_init+0xa4/0xc0
[c000000000daff00] c000000000c14014 start_kernel+0x524/0x540
[c000000000daff90] c000000000008c60 start_here_common+0x20/0xa0
0:mon>

-- 
Andrew Donnellan              Software Engineer, OzLabs
andrew.donnellan-8fk3Idey6ehBDgjK7y7TUQ@public.gmane.org  Australia Development Lab, Canberra
+61 2 6201 8874 (work)        IBM Australia Limited

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

WARNING: multiple messages have this Message-ID (diff)
From: Andrew Donnellan <andrew.donnellan@au1.ibm.com>
To: Christoph Hellwig <hch@lst.de>,
	Bart Van Assche <bart.vanassche@sandisk.com>
Cc: linux-rdma@vger.kernel.org, sagig@dev.mellanox.co.il,
	axboe@fb.com, linux-scsi@vger.kernel.org,
	linux-kernel@vger.kernel.org, linuxppc@lists.ozlabs.org
Subject: Re: Re: [PATCH 03/13] irq_poll: fold irq_poll_sched_prep into irq_poll_sched
Date: Wed, 20 Jan 2016 18:02:30 +1100	[thread overview]
Message-ID: <569F3106.7000205@au1.ibm.com> (raw)
In-Reply-To: <20151230094253.GB12904@lst.de>

On 30/12/15 20:42, Christoph Hellwig wrote:
> On Tue, Dec 29, 2015 at 10:54:18AM +0100, Bart Van Assche wrote:
>> After having applied these changes the SRP initiator didn't receive any
>> RDMA completions anymore. I could remedy that by changing
>> "!test_and_set_bit()" into "test_and_set_bit()":
>
> Yes.  I actually had this bug earlier, fixed it and managed to get
> it back during a rebase, d'oh.

I'm hitting an issue on a ppc64le box running linux-next, which 
according to git bisect is caused by this patch.

It looks like I might be hitting a dodgy error path as well, as we seem 
to be trying to execute data.

Any ideas?


Andrew

---

Sent SIGTERM to all processes
Sent SIGKILL to all processes
  -> smp_release_cpus()
spinning_secondaries = 47
  <- smp_release_cpus()
  <- setup_system()
sr 0:0:1:0: tag#0 Resetting device
ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 t0
ata1.00: cmd a0/00:00:00:00:00/00:00:00:00:00/a0 tag 3
          Test Unit Ready 00 00 00 00 00 00res 
40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
ata1.00: status: { DRDY }
ata1: translated ATA stat/err 0xd1/00 to SCSI SK/ASC/ASCQ 0xb/47/00
sd 1:2:0:0: tag#0 Resetting device
ata1.00: failed to set xfermode (err_mask=0x4)
ipr 0003:04:00.0: Timed out waiting for aborted commands
ipr 0003:04:00.0: Adapter being reset as a result of error recovery.
ata1.00: failed to set xfermode (err_mask=0x4)
ata1.00: failed to set xfermode (err_mask=0x4)
ipr 0001:04:00.0: Adapter being reset as a result of error recovery.
cpu 0x0: Vector: e40 (Emulation Assist) at [c000000000daf2e0]
     pc: c000000000e51ae8: dump_list_lock+0x0/0x4
     lr: c0000000000f46e4: __wake_up_common+0x84/0xf0
     sp: c000000000daf560
    msr: 9000000102089033
   current = 0xc000000000d6f500
   paca    = 0xc00000000fe00000   softe: 0        irq_happened: 0x01
     pid   = 0, comm = swapper/0
Linux version 4.4.0-next-20160118 (ajd@ka1) (gcc version 5.2.1 20150930 
(GCC) ) #13 SMP Tue Jan 19 12:04:19 AEDT 2016
enter ? for help
[link register   ] c0000000000f46e4 __wake_up_common+0x84/0xf0
[c000000000daf560] c000000000da1100 pps_cdev_fops+0xc8/0x100 (unreliable)
[c000000000daf5c0] c0000000000f5264 complete+0x54/0x90
[c000000000daf600] c00000000061f44c ata_qc_complete_internal+0x1c/0x30
[c000000000daf620] c000000000622828 __ata_qc_complete+0xb8/0x190
[c000000000daf660] c0000000005ef6e4 ipr_sata_eh_done+0x64/0x80
[c000000000daf680] c0000000005ef530 ipr_fail_all_ops+0x100/0x250
[c000000000daf740] c0000000005ffbf8 ipr_reset_restore_cfg_space+0x98/0x230
[c000000000daf7b0] c0000000005ed500 ipr_reset_ioa_job+0x80/0xf0
[c000000000daf7e0] c0000000005ebfac ipr_reset_timer_done+0xac/0xe0
[c000000000daf820] c00000000011eae4 call_timer_fn+0x54/0x180
[c000000000daf8b0] c00000000011ef2c run_timer_softirq+0x2ec/0x3a0
[c000000000daf980] c0000000000a4ee8 __do_softirq+0x188/0x3b0
[c000000000dafa70] c0000000000a5358 irq_exit+0xc8/0x100
[c000000000dafa90] c00000000001d894 timer_interrupt+0xa4/0xe0
[c000000000dafac0] c000000000002750 decrementer_common+0x150/0x180
--- Exception: 901 (Decrementer) at c000000000010364 
arch_local_irq_restore+0x74/0x90
[c000000000dafdb0] c000000000dac000 init_thread_union+0x0/0x4000 
(unreliable)
[c000000000dafdd0] c000000000016be8 arch_cpu_idle+0x108/0x160
[c000000000dafe00] c0000000000f5594 default_idle_call+0x44/0x80
[c000000000dafe20] c0000000000f5a48 cpu_startup_entry+0x3d8/0x450
[c000000000dafee0] c00000000000bbe4 rest_init+0xa4/0xc0
[c000000000daff00] c000000000c14014 start_kernel+0x524/0x540
[c000000000daff90] c000000000008c60 start_here_common+0x20/0xa0
0:mon>

-- 
Andrew Donnellan              Software Engineer, OzLabs
andrew.donnellan@au1.ibm.com  Australia Development Lab, Canberra
+61 2 6201 8874 (work)        IBM Australia Limited

  parent reply	other threads:[~2016-01-20  7:02 UTC|newest]

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-12-07 20:51 completion queue abstraction V2 Christoph Hellwig
2015-12-07 20:51 ` Christoph Hellwig
2015-12-07 20:51 ` [PATCH 01/13] irq_poll: make blk-iopoll available outside the block layer Christoph Hellwig
2015-12-10 18:41   ` Bart Van Assche
2015-12-10 18:41     ` Bart Van Assche
2015-12-07 20:51 ` [PATCH 02/13] irq_poll: don't disable new irq_poll instances Christoph Hellwig
2015-12-10 18:41   ` Bart Van Assche
2015-12-10 18:41     ` Bart Van Assche
2015-12-07 20:51 ` [PATCH 03/13] irq_poll: fold irq_poll_sched_prep into irq_poll_sched Christoph Hellwig
2015-12-10 18:41   ` Bart Van Assche
2015-12-10 18:41     ` Bart Van Assche
     [not found]   ` <1449521512-22921-4-git-send-email-hch-jcswGhMUV9g@public.gmane.org>
2015-12-29  9:54     ` Bart Van Assche
2015-12-29  9:54       ` Bart Van Assche
     [not found]       ` <5682584A.5030708-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
2015-12-30  9:42         ` Christoph Hellwig
2015-12-30  9:42           ` Christoph Hellwig
     [not found]           ` <20151230094253.GB12904-jcswGhMUV9g@public.gmane.org>
2016-01-20  7:02             ` Andrew Donnellan [this message]
2016-01-20  7:02               ` Andrew Donnellan
     [not found]               ` <569F3106.7000205-8fk3Idey6ehBDgjK7y7TUQ@public.gmane.org>
2016-01-20  7:15                 ` Andrew Donnellan
2016-01-20  7:15                   ` Andrew Donnellan
2015-12-07 20:51 ` [PATCH 05/13] irq_poll: mark __irq_poll_complete static Christoph Hellwig
2015-12-10 18:42   ` Bart Van Assche
2015-12-10 18:42     ` Bart Van Assche
2015-12-07 20:51 ` [PATCH 06/13] irq_poll: remove unused data and max fields Christoph Hellwig
     [not found]   ` <1449521512-22921-7-git-send-email-hch-jcswGhMUV9g@public.gmane.org>
2015-12-10 18:42     ` Bart Van Assche
2015-12-10 18:42       ` Bart Van Assche
2015-12-07 20:51 ` [PATCH 07/13] IB: add a proper completion queue abstraction Christoph Hellwig
2015-12-10 18:42   ` Bart Van Assche
2015-12-10 18:42     ` Bart Van Assche
2015-12-11 14:17     ` Christoph Hellwig
     [not found]   ` <1449521512-22921-8-git-send-email-hch-jcswGhMUV9g@public.gmane.org>
2016-01-15 13:54     ` Parav Pandit
2016-01-15 13:54       ` Parav Pandit
2016-01-17  9:24       ` Sagi Grimberg
     [not found]         ` <569B5DE3.1010908-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2016-01-17 11:01           ` Parav Pandit
2016-01-17 11:01             ` Parav Pandit
2016-01-17 11:06             ` Sagi Grimberg
2016-01-17 11:09               ` Parav Pandit
2015-12-07 20:51 ` [PATCH 08/13] IB/srpt: chain RDMA READ/WRITE requests Christoph Hellwig
2015-12-10 18:42   ` Bart Van Assche
2015-12-10 18:42     ` Bart Van Assche
2015-12-29  9:58   ` Bart Van Assche
2015-12-29  9:58     ` Bart Van Assche
     [not found]     ` <56825940.5070404-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
2015-12-30  9:43       ` Christoph Hellwig
2015-12-30  9:43         ` Christoph Hellwig
2015-12-07 20:51 ` [PATCH 09/13] IB/srpt: use the new CQ API Christoph Hellwig
2015-12-10 18:42   ` Bart Van Assche
2015-12-10 18:42     ` Bart Van Assche
2015-12-07 20:51 ` [PATCH 10/13] IB/srp: " Christoph Hellwig
2015-12-10 18:42   ` Bart Van Assche
2015-12-10 18:42     ` Bart Van Assche
2015-12-11 14:22     ` Christoph Hellwig
     [not found]       ` <20151211142220.GB20201-jcswGhMUV9g@public.gmane.org>
2015-12-11 17:59         ` Doug Ledford
2015-12-11 17:59           ` Doug Ledford
2015-12-12  8:08           ` Christoph Hellwig
     [not found]             ` <20151212080833.GA32638-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2015-12-13 10:26               ` Sagi Grimberg
2015-12-13 10:26                 ` Sagi Grimberg
2015-12-14 16:26                 ` Doug Ledford
2015-12-07 20:51 ` [PATCH 11/13] IB/iser: Use a dedicated descriptor for login Christoph Hellwig
2015-12-07 20:51 ` [PATCH 12/13] IB/iser: Use helper for container_of Christoph Hellwig
2015-12-07 20:51 ` [PATCH 13/13] IB/iser: Convert to CQ abstraction Christoph Hellwig
     [not found] ` <1449521512-22921-1-git-send-email-hch-jcswGhMUV9g@public.gmane.org>
2015-12-07 20:51   ` [PATCH 04/13] irq_poll: fold irq_poll_disable_pending into irq_poll_softirq Christoph Hellwig
2015-12-07 20:51     ` Christoph Hellwig
2015-12-10 18:41     ` Bart Van Assche
2015-12-10 18:41       ` Bart Van Assche
2015-12-13 10:25   ` completion queue abstraction V2 Sagi Grimberg
2015-12-13 10:25     ` Sagi Grimberg
2015-12-23 19:44 ` Doug Ledford
2015-12-29  9:51 ` Bart Van Assche
2015-12-29  9:51   ` Bart Van Assche

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=569F3106.7000205@au1.ibm.com \
    --to=andrew.donnellan-8fk3idey6ehbdgjk7y7tuq@public.gmane.org \
    --cc=axboe-b10kYP2dOMg@public.gmane.org \
    --cc=bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org \
    --cc=hch-jcswGhMUV9g@public.gmane.org \
    --cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linux-scsi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linuxppc-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org \
    --cc=sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.