custom ide driver causes "Badness in = smp_call

From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Tue, 23 Aug 2005 15:02:17 +0100 (BST) Received: from rwcrmhc12.comcast.net ([IPv6:::ffff:204.127.198.43]:37367 "EHLO rwcrmhc12.comcast.net") by linux-mips.org with ESMTP id ; Tue, 23 Aug 2005 15:01:59 +0100 Received: from ba3pi (pcp0010731669pcs.howard01.md.comcast.net[69.243.71.130]) by comcast.net (rwcrmhc12) with SMTP id <2005082314071501400q9m8ke>; Tue, 23 Aug 2005 14:07:16 +0000 From: "Bryan Althouse" To: Subject: custom ide driver causes "Badness in smp_call_function" Date: Tue, 23 Aug 2005 10:07:02 -0400 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_008F_01C5A7CA.68D6B0E0" X-Mailer: Microsoft Office Outlook, Build 11.0.6353 Thread-Index: AcWn6+8Url8jVLf9TUGrSIXTM5IkDQ== X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Message-Id: <20050823140159Z8225393-3678+7283@linux-mips.org> Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8790 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: bryan.althouse@3phoenix.com Precedence: bulk X-list: linux-mips This is a multi-part message in MIME format. ------=_NextPart_000_008F_01C5A7CA.68D6B0E0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Hi all, I have added a compact flash disk onto the local bus of an rm9224 mips processor. We are using an FPGA as an IDE host adaptor. Our driver works great when the kernel is compiled for a single processor. But if SMP support is enabled, the kernel dies with: . Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2 Ide: Assuming 50MHz system bus speed for PIO modes; override with idebuss=xx Badness in smp_call_function at arch/mips/kernel/smp.c:149 Call Trace: [] smp_call_function+0x1f8/0x200 [] schedule+0x950/0xa08 [] local_rm9k_perfcounter_irq_startup+0x0/0x68 [] rm9k_perfcounter_irq_startup+0x48/0x68 [] probe_irq_on+0x2b8/0x2e8 [] del_singleshot_timer_sync+0x38/0x50 [] try_to_identify+0x180/0x190 [] process_timeout+0x0/0x8 [] do_probe+0x14c/0x338 [] do_probe+0x5c/0x338 [] wait_hwif_ready+0x178/0x180 [] probe_hwif+0x508/0x6e8 [] probe_hwif+0x1a0/0x6e8 [] probe_hwif_init_with_fixup+0x1c/0xe8 [] ide_3P_init+0xb4/0x118 [] ide_3P_init+0x7c/0x118 [] idr_get_new+0x18/0x50 etc. Does anyone know what sort of bug could cause problems with SMP, but would work fine otherwise? I could supply my driver code if anyone is interested. Thanks! Bryan ------=_NextPart_000_008F_01C5A7CA.68D6B0E0 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable custom ide driver causes "Badness in = smp_call_function"

Hi = all,

I have added a compact flash = disk onto the local bus of an rm9224 mips = processor. We are using an FPGA as an IDE host adaptor. Our driver works great when the kernel is compiled for = a single processor. But if = SMP support is enabled, the kernel dies with: =

&nbs= p; = …

Uniform Multi-Platform E-IDE driver Revision: = 7.00alpha2

Ide: Assuming 50MHz system bus speed for PIO modes; = override with idebuss=3Dxx

Badness in smp_call_function at = arch/mips/kernel/smp.c:149

Call = Trace:

[<ffffffff8010aff8>] = smp_call_function+0x1f8/0x200

[<ffffffff8032adb0>] = schedule+0x950/0xa08

[<ffffffff8010b9f8>] = local_rm9k_perfcounter_irq_startup+0x0/0x68

[<ffffffff8010baa8>] = rm9k_perfcounter_irq_startup+0x48/0x68

[<ffffffff801658b8>] = probe_irq_on+0x2b8/0x2e8

[<ffffffff80146c00>] = del_singleshot_timer_sync+0x38/0x50

[<ffffffff8028e510>] = try_to_identify+0x180/0x190

[<ffffffff80147d88>] = process_timeout+0x0/0x8

[<ffffffff8028e66c>] = do_probe+0x14c/0x338

[<ffffffff8028e57c>] = do_probe+0x5c/0x338

[<ffffffff8028ec40>] = wait_hwif_ready+0x178/0x180

[<ffffffff8028f230>] = probe_hwif+0x508/0x6e8

[<ffffffff8028eec8>] = probe_hwif+0x1a0/0x6e8

[<ffffffff8028f42c>] = probe_hwif_init_with_fixup+0x1c/0xe8

[<ffffffff8029435c>] = ide_3P_init+0xb4/0x118

[<ffffffff80294324>] = ide_3P_init+0x7c/0x118

[<ffffffff8023d100>] = idr_get_new+0x18/0x50

etc…

Does = anyone know what sort of bug could cause problems with SMP, but would = work fine otherwise? I could supply my driver code if anyone = is interested. = Thanks!

Bryan

------=_NextPart_000_008F_01C5A7CA.68D6B0E0-- From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Wed, 24 Aug 2005 16:19:43 +0100 (BST) Received: from extgw-uk.mips.com ([IPv6:::ffff:62.254.210.129]:61212 "EHLO bacchus.net.dhis.org") by linux-mips.org with ESMTP id ; Wed, 24 Aug 2005 16:19:27 +0100 Received: from dea.linux-mips.net (localhost.localdomain [127.0.0.1]) by bacchus.net.dhis.org (8.13.4/8.13.1) with ESMTP id j7OFOjkg011953; Wed, 24 Aug 2005 16:24:45 +0100 Received: (from ralf@localhost) by dea.linux-mips.net (8.13.4/8.13.4/Submit) id j7OFOjbo011952; Wed, 24 Aug 2005 16:24:45 +0100 Date: Wed, 24 Aug 2005 16:24:44 +0100 From: Ralf Baechle To: Bryan Althouse Cc: linux-mips@linux-mips.org Subject: Re: custom ide driver causes "Badness in smp_call_function" Message-ID: <20050824152444.GE2783@linux-mips.org> References: <20050823140159Z8225393-3678+7283@linux-mips.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050823140159Z8225393-3678+7283@linux-mips.org> User-Agent: Mutt/1.4.2.1i Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8793 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: ralf@linux-mips.org Precedence: bulk X-list: linux-mips On Tue, Aug 23, 2005 at 10:07:02AM -0400, Bryan Althouse wrote: > I have added a compact flash disk onto the local bus of an rm9224 mips > processor. We are using an FPGA as an IDE host adaptor. Our driver works > great when the kernel is compiled for a single processor. But if SMP > support is enabled, the kernel dies with: > . > Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2 > Ide: Assuming 50MHz system bus speed for PIO modes; override with > idebuss=xx > Badness in smp_call_function at arch/mips/kernel/smp.c:149 That's just a rather drastically looking warning, btw. > Call Trace: > [] smp_call_function+0x1f8/0x200 > [] schedule+0x950/0xa08 > [] local_rm9k_perfcounter_irq_startup+0x0/0x68 > [] rm9k_perfcounter_irq_startup+0x48/0x68 > [] probe_irq_on+0x2b8/0x2e8 > [] del_singleshot_timer_sync+0x38/0x50 > [] try_to_identify+0x180/0x190 > [] process_timeout+0x0/0x8 > [] do_probe+0x14c/0x338 > [] do_probe+0x5c/0x338 > [] wait_hwif_ready+0x178/0x180 > [] probe_hwif+0x508/0x6e8 > [] probe_hwif+0x1a0/0x6e8 > [] probe_hwif_init_with_fixup+0x1c/0xe8 > [] ide_3P_init+0xb4/0x118 > [] ide_3P_init+0x7c/0x118 > [] idr_get_new+0x18/0x50 > etc. > > Does anyone know what sort of bug could cause problems with SMP, but would > work fine otherwise? I could supply my driver code if anyone is interested. > Thanks! Your driver is probably fine. The problem is that doing PIO may result in cache aliases and that requires a cache flush. Normally that's not terribly hard to do - but in your case an SMP cacheflush is needed which in turn requires smp_call_function to be used and that one again may deadlock if called with interrupts disabled. See also include/asm-mips/mach-generic/ide.h. So until we have a better implementation I suggest try to avoid the use of ide_inb() etc. with interrupts disabled. Ralf From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Thu, 25 Aug 2005 16:21:50 +0100 (BST) Received: from rwcrmhc11.comcast.net ([IPv6:::ffff:204.127.198.35]:22430 "EHLO rwcrmhc11.comcast.net") by linux-mips.org with ESMTP id ; Thu, 25 Aug 2005 16:21:31 +0100 Received: from ba3pi (pcp0010731669pcs.howard01.md.comcast.net[69.243.71.130]) by comcast.net (rwcrmhc11) with SMTP id <200508251526590130063f7oe>; Thu, 25 Aug 2005 15:26:59 +0000 From: "Bryan Althouse" To: "'Ralf Baechle'" Cc: Subject: RE: custom ide driver causes "Badness in smp_call_function" Date: Thu, 25 Aug 2005 11:26:58 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.6353 Thread-Index: AcWov/o2tje51yNZRMqC6qvo2vU/0gAyUQrg X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 In-Reply-To: <20050824152444.GE2783@linux-mips.org> Message-Id: <20050825152131Z8225298-3678+7482@linux-mips.org> Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8803 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: bryan.althouse@3phoenix.com Precedence: bulk X-list: linux-mips Ralf, Thank you for your help. I'm doing MMIO, not PIO, but it looks like your assessment is still valid. I've been searching for places where ide MMIO is performed with interrupts disabled. I got excited when I found these lines in probe_hwif() of ide-probe.c: irqd = hwif->irq; if (irqd) disable_irq(hwif->irq); I was not initializing hwif->irq in my driver, so probably the interrupts were being disabled here, and subsequent lines were causing the SMP badness. I added the line "hwif->irq = 0" to my driver. Interrupts are no longer disabled here, but still I get the SMP badness. I'll keep looking for other places where the interrupts might be disabled. Does anyone know if the mips/swarm.c driver has this problem with SMP? Thanks! Bryan From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Thu, 25 Aug 2005 16:37:49 +0100 (BST) Received: from extgw-uk.mips.com ([IPv6:::ffff:62.254.210.129]:31517 "EHLO bacchus.net.dhis.org") by linux-mips.org with ESMTP id ; Thu, 25 Aug 2005 16:37:28 +0100 Received: from dea.linux-mips.net (localhost.localdomain [127.0.0.1]) by bacchus.net.dhis.org (8.13.4/8.13.1) with ESMTP id j7PFgnBe009413; Thu, 25 Aug 2005 16:42:49 +0100 Received: (from ralf@localhost) by dea.linux-mips.net (8.13.4/8.13.4/Submit) id j7PFgnco009412; Thu, 25 Aug 2005 16:42:49 +0100 Date: Thu, 25 Aug 2005 16:42:49 +0100 From: Ralf Baechle To: Bryan Althouse Cc: linux-mips@linux-mips.org Subject: Re: custom ide driver causes "Badness in smp_call_function" Message-ID: <20050825154249.GC2731@linux-mips.org> References: <20050824152444.GE2783@linux-mips.org> <20050825152131Z8225298-3678+7482@linux-mips.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050825152131Z8225298-3678+7482@linux-mips.org> User-Agent: Mutt/1.4.2.1i Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8805 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: ralf@linux-mips.org Precedence: bulk X-list: linux-mips On Thu, Aug 25, 2005 at 11:26:58AM -0400, Bryan Althouse wrote: > From: "Bryan Althouse" > To: "'Ralf Baechle'" > Cc: > Subject: RE: custom ide driver causes "Badness in smp_call_function" > Date: Thu, 25 Aug 2005 11:26:58 -0400 > Content-Type: text/plain; > charset="us-ascii" > > Ralf, > > Thank you for your help. > I'm doing MMIO, not PIO, but it looks like your assessment is still valid. ... which still is programmed io ... > I've been searching for places where ide MMIO is performed with interrupts > disabled. I got excited when I found these lines in probe_hwif() of > ide-probe.c: > > irqd = hwif->irq; > if (irqd) > disable_irq(hwif->irq); > > I was not initializing hwif->irq in my driver, so probably the interrupts > were being disabled here, and subsequent lines were causing the SMP badness. > I added the line "hwif->irq = 0" to my driver. Interrupts are no longer > disabled here, but still I get the SMP badness. I'll keep looking for other > places where the interrupts might be disabled. > > Does anyone know if the mips/swarm.c driver has this problem with SMP? > Thanks! No, SB1 has sane caches. Ralf From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Thu, 25 Aug 2005 22:12:33 +0100 (BST) Received: from rwcrmhc12.comcast.net ([IPv6:::ffff:204.127.198.43]:14753 "EHLO rwcrmhc12.comcast.net") by linux-mips.org with ESMTP id ; Thu, 25 Aug 2005 22:12:18 +0100 Received: from ba3pi (pcp0010731669pcs.howard01.md.comcast.net[69.243.71.130]) by comcast.net (rwcrmhc12) with SMTP id <2005082521174901400kkekge>; Thu, 25 Aug 2005 21:17:50 +0000 From: "Bryan Althouse" To: "'Ralf Baechle'" Cc: Subject: RE: custom ide driver causes "Badness in smp_call_function" Date: Thu, 25 Aug 2005 17:17:48 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.6353 Thread-Index: AcWpi6yG5OnXfWlpRq2rlFKyrGwD5AALSfIg In-Reply-To: <20050825154249.GC2731@linux-mips.org> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Message-Id: <20050825211218Z8225471-3678+7505@linux-mips.org> Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8807 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: bryan.althouse@3phoenix.com Precedence: bulk X-list: linux-mips Here are some observations... If I change the line "hwif->irq = 0" in my driver to "hwif->irq = 5", my SMP kernel no longer experiences SMP badness. Instead, I get many lines like "hda: lost interrupt", and the drive is not usable. If I compile the kernel without SMP, the drive works properly as before. I tried irq = 5 because I noticed that /proc/interrupts indicated that ide0 was being probed at 5. With the SMP kernel, /proc/interrupts shows a count of 0 for ide0. My non-SMP kernel shows a count that increments when the drive is being used (as expected). Bryan From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Fri, 26 Aug 2005 15:05:32 +0100 (BST) Received: from extgw-uk.mips.com ([IPv6:::ffff:62.254.210.129]:25607 "EHLO bacchus.net.dhis.org") by linux-mips.org with ESMTP id ; Fri, 26 Aug 2005 15:05:16 +0100 Received: from dea.linux-mips.net (localhost.localdomain [127.0.0.1]) by bacchus.net.dhis.org (8.13.4/8.13.1) with ESMTP id j7QEAm33010523; Fri, 26 Aug 2005 15:10:48 +0100 Received: (from ralf@localhost) by dea.linux-mips.net (8.13.4/8.13.4/Submit) id j7QEAl0w010522; Fri, 26 Aug 2005 15:10:47 +0100 Date: Fri, 26 Aug 2005 15:10:47 +0100 From: Ralf Baechle To: Bryan Althouse Cc: linux-mips@linux-mips.org Subject: Re: custom ide driver causes "Badness in smp_call_function" Message-ID: <20050826141047.GA8777@linux-mips.org> References: <20050825154249.GC2731@linux-mips.org> <20050825211218Z8225471-3678+7505@linux-mips.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050825211218Z8225471-3678+7505@linux-mips.org> User-Agent: Mutt/1.4.2.1i Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8814 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: ralf@linux-mips.org Precedence: bulk X-list: linux-mips On Thu, Aug 25, 2005 at 05:17:48PM -0400, Bryan Althouse wrote: > If I change the line "hwif->irq = 0" in my driver to "hwif->irq = 5", my SMP > kernel no longer experiences SMP badness. Instead, I get many lines like > "hda: lost interrupt", and the drive is not usable. If I compile the kernel > without SMP, the drive works properly as before. I tried irq = 5 because I > noticed that /proc/interrupts indicated that ide0 was being probed at 5. > With the SMP kernel, /proc/interrupts shows a count of 0 for ide0. My > non-SMP kernel shows a count that increments when the drive is being used > (as expected). Try this patch below and let me know. I would also like to ask those people who used to suffer from aliases with IDE PIO to try this patch. Ralf Index: include/asm-mips/mach-generic/ide.h =================================================================== RCS file: /home/cvs/linux/include/asm-mips/mach-generic/ide.h,v retrieving revision 1.9 diff -u -r1.9 ide.h --- include/asm-mips/mach-generic/ide.h 19 Apr 2005 12:26:59 -0000 1.9 +++ include/asm-mips/mach-generic/ide.h 26 Aug 2005 14:04:38 -0000 @@ -19,6 +19,7 @@ #include #include #include +#include #ifndef MAX_HWIFS # ifdef CONFIG_BLK_DEV_IDEPCI @@ -105,12 +106,14 @@ /* MIPS port and memory-mapped I/O string operations. */ -static inline void __ide_flush_dcache_range(unsigned long addr, unsigned long size) +static inline void __ide_flush_dcache_range(unsigned long addr, + unsigned long size) { - if (cpu_has_dc_aliases) { - unsigned long end = addr + size; - for (; addr < end; addr += PAGE_SIZE) - flush_dcache_page(virt_to_page(addr)); + unsigned long end = addr + size; + + while (addr < end) { + SetPageDcacheDirty(virt_to_page(addr)); + addr += PAGE_SIZE; } } From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Fri, 26 Aug 2005 15:53:18 +0100 (BST) Received: from rwcrmhc14.comcast.net ([IPv6:::ffff:204.127.198.54]:33190 "EHLO rwcrmhc12.comcast.net") by linux-mips.org with ESMTP id ; Fri, 26 Aug 2005 15:53:03 +0100 Received: from ba3pi (pcp0010731669pcs.howard01.md.comcast.net[69.243.71.130]) by comcast.net (rwcrmhc14) with SMTP id <2005082614584001400hfrvde>; Fri, 26 Aug 2005 14:58:41 +0000 From: "Bryan Althouse" To: "'Ralf Baechle'" Cc: Subject: RE: custom ide driver causes "Badness in smp_call_function" Date: Fri, 26 Aug 2005 10:58:40 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.6353 Thread-Index: AcWqR/nIHNFE/tghSraa2RLTN9Py/gABmxaA In-Reply-To: <20050826141047.GA8777@linux-mips.org> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Message-Id: <20050826145303Z8224974-3678+7581@linux-mips.org> Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8817 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: bryan.althouse@3phoenix.com Precedence: bulk X-list: linux-mips Ralf, The patch doesn't seem to make any difference. :( Bryan From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Fri, 26 Aug 2005 16:13:24 +0100 (BST) Received: from clock-tower.bc.nu ([IPv6:::ffff:81.2.110.250]:48055 "EHLO lxorguk.ukuu.org.uk") by linux-mips.org with ESMTP id ; Fri, 26 Aug 2005 16:13:06 +0100 Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by lxorguk.ukuu.org.uk (8.13.4/8.13.4) with ESMTP id j7QFlX6Q007340; Fri, 26 Aug 2005 16:47:34 +0100 Received: (from alan@localhost) by localhost.localdomain (8.13.4/8.13.4/Submit) id j7QFlXad007339; Fri, 26 Aug 2005 16:47:33 +0100 X-Authentication-Warning: localhost.localdomain: alan set sender to alan@lxorguk.ukuu.org.uk using -f Subject: RE: custom ide driver causes "Badness in smp_call_function" From: Alan Cox To: Bryan Althouse Cc: linux-mips@linux-mips.org, "'Ralf Baechle'" In-Reply-To: <20050826145303Z8224974-3678+7581@linux-mips.org> References: <20050826145303Z8224974-3678+7581@linux-mips.org> Content-Type: text/plain Content-Transfer-Encoding: 7bit Date: Fri, 26 Aug 2005 16:47:24 +0100 Message-Id: <1125071244.7298.2.camel@localhost.localdomain> Mime-Version: 1.0 X-Mailer: Evolution 2.2.2 (2.2.2-5) Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8818 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: alan@lxorguk.ukuu.org.uk Precedence: bulk X-list: linux-mips On Gwe, 2005-08-26 at 10:58 -0400, Bryan Althouse wrote: > Ralf, > > The patch doesn't seem to make any difference. :( Assuming your hardware is sane another approach might be to force drive->unmask = 1. That will mean that PIO mode is running with interrupts enabled which should avoid the problem. Add a .fixup handler to your driver (assuming you are using a recent 2.6.x) and in the handler do something like this: +void ide_unmask_interrupts(ide_hwif_t *hwif) +{ + int i; + for (i = 0; i < 2; i++) { + ide_drive_t *drive = &hwif->drives[i]; + if(drive->present) + drive->unmask = 1; + } +} hopefully that will be early enough. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Fri, 26 Aug 2005 17:23:18 +0100 (BST) Received: from extgw-uk.mips.com ([IPv6:::ffff:62.254.210.129]:41483 "EHLO bacchus.net.dhis.org") by linux-mips.org with ESMTP id ; Fri, 26 Aug 2005 17:23:00 +0100 Received: from dea.linux-mips.net (localhost.localdomain [127.0.0.1]) by bacchus.net.dhis.org (8.13.4/8.13.1) with ESMTP id j7QGSX9m015302; Fri, 26 Aug 2005 17:28:33 +0100 Received: (from ralf@localhost) by dea.linux-mips.net (8.13.4/8.13.4/Submit) id j7QGSWrR015301; Fri, 26 Aug 2005 17:28:32 +0100 Date: Fri, 26 Aug 2005 17:28:32 +0100 From: Ralf Baechle To: Bryan Althouse Cc: linux-mips@linux-mips.org Subject: Re: custom ide driver causes "Badness in smp_call_function" Message-ID: <20050826162832.GB2712@linux-mips.org> References: <20050826141047.GA8777@linux-mips.org> <20050826145303Z8224974-3678+7581@linux-mips.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050826145303Z8224974-3678+7581@linux-mips.org> User-Agent: Mutt/1.4.2.1i Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8823 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: ralf@linux-mips.org Precedence: bulk X-list: linux-mips On Fri, Aug 26, 2005 at 10:58:40AM -0400, Bryan Althouse wrote: > The patch doesn't seem to make any difference. :( To clarify, the patch wasn't meant to resolve your interrupt problems but the SMP cacheflush issues only. Ralf From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Fri, 26 Aug 2005 17:31:08 +0100 (BST) Received: from rwcrmhc14.comcast.net ([IPv6:::ffff:204.127.198.54]:47248 "EHLO rwcrmhc12.comcast.net") by linux-mips.org with ESMTP id ; Fri, 26 Aug 2005 17:30:50 +0100 Received: from ba3pi (pcp0010731669pcs.howard01.md.comcast.net[69.243.71.130]) by comcast.net (rwcrmhc14) with SMTP id <2005082616362701400hfmq0e>; Fri, 26 Aug 2005 16:36:27 +0000 From: "Bryan Althouse" To: "'Ralf Baechle'" Cc: Subject: RE: custom ide driver causes "Badness in smp_call_function" Date: Fri, 26 Aug 2005 12:36:26 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.6353 Thread-Index: AcWqWzeC/+HCp/FoTVCO9YZvOG8WzwAAFvxA In-Reply-To: <20050826162832.GB2712@linux-mips.org> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Message-Id: <20050826163050Z8224979-3678+7593@linux-mips.org> Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8824 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: bryan.althouse@3phoenix.com Precedence: bulk X-list: linux-mips I still get the SMP badness when hwif->irq is set to 0 in my driver. Bryan -----Original Message----- From: Ralf Baechle [mailto:ralf@linux-mips.org] Sent: Friday, August 26, 2005 12:29 PM To: Bryan Althouse Cc: linux-mips@linux-mips.org Subject: Re: custom ide driver causes "Badness in smp_call_function" On Fri, Aug 26, 2005 at 10:58:40AM -0400, Bryan Althouse wrote: > The patch doesn't seem to make any difference. :( To clarify, the patch wasn't meant to resolve your interrupt problems but the SMP cacheflush issues only. Ralf From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Fri, 26 Aug 2005 17:36:02 +0100 (BST) Received: from extgw-uk.mips.com ([IPv6:::ffff:62.254.210.129]:34572 "EHLO bacchus.net.dhis.org") by linux-mips.org with ESMTP id ; Fri, 26 Aug 2005 17:35:43 +0100 Received: from dea.linux-mips.net (localhost.localdomain [127.0.0.1]) by bacchus.net.dhis.org (8.13.4/8.13.1) with ESMTP id j7QGfGRT015776; Fri, 26 Aug 2005 17:41:16 +0100 Received: (from ralf@localhost) by dea.linux-mips.net (8.13.4/8.13.4/Submit) id j7QGfFHh015775; Fri, 26 Aug 2005 17:41:15 +0100 Date: Fri, 26 Aug 2005 17:41:15 +0100 From: Ralf Baechle To: Bryan Althouse Cc: linux-mips@linux-mips.org Subject: Re: custom ide driver causes "Badness in smp_call_function" Message-ID: <20050826164115.GC2712@linux-mips.org> References: <20050826162832.GB2712@linux-mips.org> <20050826163050Z8224979-3678+7593@linux-mips.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050826163050Z8224979-3678+7593@linux-mips.org> User-Agent: Mutt/1.4.2.1i Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8826 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: ralf@linux-mips.org Precedence: bulk X-list: linux-mips On Fri, Aug 26, 2005 at 12:36:26PM -0400, Bryan Althouse wrote: > I still get the SMP badness when hwif->irq is set to 0 in my driver. Interesting. That implies smp_call_function is called through a different path on your system that on mine here. Doesn't invalidate the patch, just means some extra debugging work for you ;-) Ralf From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Fri, 26 Aug 2005 19:00:26 +0100 (BST) Received: from rwcrmhc11.comcast.net ([IPv6:::ffff:204.127.198.35]:30854 "EHLO rwcrmhc11.comcast.net") by linux-mips.org with ESMTP id ; Fri, 26 Aug 2005 19:00:08 +0100 Received: from ba3pi (pcp0010731669pcs.howard01.md.comcast.net[69.243.71.130]) by comcast.net (rwcrmhc11) with SMTP id <200508261805440130060joce>; Fri, 26 Aug 2005 18:05:44 +0000 From: "Bryan Althouse" Cc: Subject: RE: custom ide driver causes "Badness in smp_call_function" Date: Fri, 26 Aug 2005 14:05:43 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.6353 Thread-Index: AcWqUXV7nhNPHZleS2iU5erhy9yRrQAFfx5g In-Reply-To: <1125071244.7298.2.camel@localhost.localdomain> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Message-Id: <20050826180008Z8224984-3678+7604@linux-mips.org> To: unlisted-recipients:; (no To-header on input) Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8827 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: bryan.althouse@3phoenix.com Precedence: bulk X-list: linux-mips Alan, Thanks for your suggestion. I'm not sure how to write a .fixup handler. I did some Googling, but got nowhere. I looked through drivers/ide to see what drive->unmask was doing. I found this in ide-io.c: if (drive->unmask) local_irq_enable(); And this in ide-taskfile.c: if (!drive->unmask) local_irq_disable(); I modified both of these files so that execution would be as if unmask = 1. This resulted in no change of behavior. Bryan -----Original Message----- From: Alan Cox [mailto:alan@lxorguk.ukuu.org.uk] Sent: Friday, August 26, 2005 11:47 AM To: Bryan Althouse Cc: linux-mips@linux-mips.org; 'Ralf Baechle' Subject: RE: custom ide driver causes "Badness in smp_call_function" On Gwe, 2005-08-26 at 10:58 -0400, Bryan Althouse wrote: > Ralf, > > The patch doesn't seem to make any difference. :( Assuming your hardware is sane another approach might be to force drive->unmask = 1. That will mean that PIO mode is running with interrupts enabled which should avoid the problem. Add a .fixup handler to your driver (assuming you are using a recent 2.6.x) and in the handler do something like this: +void ide_unmask_interrupts(ide_hwif_t *hwif) +{ + int i; + for (i = 0; i < 2; i++) { + ide_drive_t *drive = &hwif->drives[i]; + if(drive->present) + drive->unmask = 1; + } +} hopefully that will be early enough. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Fri, 26 Aug 2005 19:12:20 +0100 (BST) Received: from laf31-5-82-235-130-100.fbx.proxad.net ([IPv6:::ffff:82.235.130.100]:23548 "EHLO lexbox.fr") by linux-mips.org with ESMTP id ; Fri, 26 Aug 2005 19:12:01 +0100 Received: from mail pickup service by lexbox.fr with Microsoft SMTPSVC; Fri, 26 Aug 2005 20:16:10 +0200 From: "Bryan Althouse" Cc: Subject: RE: custom ide driver causes "Badness in smp_call_function" Message-ID: <000501c5aa6a$3beadc30$0300a8c0@intra.lexbox.org> Date: Fri, 26 Aug 2005 20:16:09 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.6353 thread-index: AcWqUXV7nhNPHZleS2iU5erhy9yRrQAFfx5g In-Reply-To: <1125071244.7298.2.camel@localhost.localdomain> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.181 To: , , "IMB Recipient 1" X-archive-position: 8827 X-ecartis-version: Ecartis v1.0.0 Content-Class: urn:content-classes:message Importance: normal Priority: normal X-original-sender: bryan.althouse@3phoenix.com Precedence: bulk X-list: linux-mips X-OriginalArrivalTime: 26 Aug 2005 18:16:10.0343 (UTC) FILETIME=[3C372770:01C5AA6A] Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8828 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: bryan.althouse@3phoenix.com Precedence: bulk X-list: linux-mips Alan, Thanks for your suggestion. I'm not sure how to write a .fixup handler. I did some Googling, but got nowhere. I looked through drivers/ide to see what drive->unmask was doing. I found this in ide-io.c: if (drive->unmask) local_irq_enable(); And this in ide-taskfile.c: if (!drive->unmask) local_irq_disable(); I modified both of these files so that execution would be as if unmask = 1. This resulted in no change of behavior. Bryan -----Original Message----- From: Alan Cox [mailto:alan@lxorguk.ukuu.org.uk] Sent: Friday, August 26, 2005 11:47 AM To: Bryan Althouse Cc: linux-mips@linux-mips.org; 'Ralf Baechle' Subject: RE: custom ide driver causes "Badness in smp_call_function" On Gwe, 2005-08-26 at 10:58 -0400, Bryan Althouse wrote: > Ralf, > > The patch doesn't seem to make any difference. :( Assuming your hardware is sane another approach might be to force drive->unmask = 1. That will mean that PIO mode is running with interrupts enabled which should avoid the problem. Add a .fixup handler to your driver (assuming you are using a recent 2.6.x) and in the handler do something like this: +void ide_unmask_interrupts(ide_hwif_t *hwif) +{ + int i; + for (i = 0; i < 2; i++) { + ide_drive_t *drive = &hwif->drives[i]; + if(drive->present) + drive->unmask = 1; + } +} hopefully that will be early enough. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Thu, 01 Sep 2005 17:10:40 +0100 (BST) Received: from rwcrmhc14.comcast.net ([IPv6:::ffff:204.127.198.54]:40396 "EHLO rwcrmhc12.comcast.net") by linux-mips.org with ESMTP id ; Thu, 1 Sep 2005 17:10:23 +0100 Received: from ba3pi (wsip-70-184-242-86.dc.dc.cox.net[70.184.242.86]) by comcast.net (rwcrmhc14) with SMTP id <2005090116163801400e9lt7e>; Thu, 1 Sep 2005 16:16:39 +0000 From: "Bryan Althouse" To: "'Ralf Baechle'" Cc: Subject: RE: custom ide driver causes "Badness in smp_call_function" Date: Thu, 1 Sep 2005 12:15:31 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.6353 Thread-Index: AcWqR/nIHNFE/tghSraa2RLTN9Py/gEUAOVw In-Reply-To: <20050826141047.GA8777@linux-mips.org> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Message-Id: <20050901161023Z8225398-3678+8096@linux-mips.org> Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8858 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: bryan.althouse@3phoenix.com Precedence: bulk X-list: linux-mips Ralf, I've done more testing with your patch. When I use it with my non-SMP kernel, disk access will cause a panic with a cache error message. Is this patch only intended for SMP, or is this a legitimate problem? Bryan Index: include/asm-mips/mach-generic/ide.h =================================================================== RCS file: /home/cvs/linux/include/asm-mips/mach-generic/ide.h,v retrieving revision 1.9 diff -u -r1.9 ide.h --- include/asm-mips/mach-generic/ide.h 19 Apr 2005 12:26:59 -0000 1.9 +++ include/asm-mips/mach-generic/ide.h 26 Aug 2005 14:04:38 -0000 @@ -19,6 +19,7 @@ #include #include #include +#include #ifndef MAX_HWIFS # ifdef CONFIG_BLK_DEV_IDEPCI @@ -105,12 +106,14 @@ /* MIPS port and memory-mapped I/O string operations. */ -static inline void __ide_flush_dcache_range(unsigned long addr, unsigned long size) +static inline void __ide_flush_dcache_range(unsigned long addr, + unsigned long size) { - if (cpu_has_dc_aliases) { - unsigned long end = addr + size; - for (; addr < end; addr += PAGE_SIZE) - flush_dcache_page(virt_to_page(addr)); + unsigned long end = addr + size; + + while (addr < end) { + SetPageDcacheDirty(virt_to_page(addr)); + addr += PAGE_SIZE; } } From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Thu, 01 Sep 2005 17:46:43 +0100 (BST) Received: from extgw-uk.mips.com ([IPv6:::ffff:62.254.210.129]:6670 "EHLO bacchus.net.dhis.org") by linux-mips.org with ESMTP id ; Thu, 1 Sep 2005 17:46:27 +0100 Received: from dea.linux-mips.net (localhost.localdomain [127.0.0.1]) by bacchus.net.dhis.org (8.13.4/8.13.1) with ESMTP id j81GqakJ017648; Thu, 1 Sep 2005 17:52:36 +0100 Received: (from ralf@localhost) by dea.linux-mips.net (8.13.4/8.13.4/Submit) id j81GqZiN017647; Thu, 1 Sep 2005 17:52:35 +0100 Date: Thu, 1 Sep 2005 17:52:35 +0100 From: Ralf Baechle DL5RB To: Bryan Althouse Cc: linux-mips@linux-mips.org Subject: Re: custom ide driver causes "Badness in smp_call_function" Message-ID: <20050901165235.GI2876@linux-mips.org> References: <20050826141047.GA8777@linux-mips.org> <20050901161023Z8225398-3678+8096@linux-mips.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20050901161023Z8225398-3678+8096@linux-mips.org> User-Agent: Mutt/1.4.2.1i Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8859 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: ralf@linux-mips.org Precedence: bulk X-list: linux-mips On Thu, Sep 01, 2005 at 12:15:31PM -0400, Bryan Althouse wrote: > I've done more testing with your patch. When I use it with my non-SMP > kernel, disk access will cause a panic with a cache error message. Is this > patch only intended for SMP, or is this a legitimate problem? There is a real problem but it only hits on SMP. (The implementation is a hack though, it makes assumptions about the IDE code's internal workings) Ralf From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Fri, 02 Sep 2005 13:41:24 +0100 (BST) Received: from topsns.toshiba-tops.co.jp ([IPv6:::ffff:202.230.225.5]:43782 "HELO topsns.toshiba-tops.co.jp") by linux-mips.org with SMTP id ; Fri, 2 Sep 2005 13:41:01 +0100 Received: from inside-ms1.toshiba-tops.co.jp by topsns.toshiba-tops.co.jp via smtpd (for mail.linux-mips.org [62.254.210.162]) with SMTP; 2 Sep 2005 12:49:03 UT Received: from topsms.toshiba-tops.co.jp (localhost.localdomain [127.0.0.1]) by localhost.toshiba-tops.co.jp (Postfix) with ESMTP id B27D61FE03; Fri, 2 Sep 2005 21:48:59 +0900 (JST) Received: from srd2sd.toshiba-tops.co.jp (gw-chiba7.toshiba-tops.co.jp [172.17.244.27]) by topsms.toshiba-tops.co.jp (Postfix) with ESMTP id 9E5D41797B; Fri, 2 Sep 2005 21:48:59 +0900 (JST) Received: from localhost (fragile [172.17.28.65]) by srd2sd.toshiba-tops.co.jp (8.12.10/8.12.10) with ESMTP id j82Cmxoj003054; Fri, 2 Sep 2005 21:48:59 +0900 (JST) (envelope-from anemo@mba.ocn.ne.jp) Date: Fri, 02 Sep 2005 21:48:59 +0900 (JST) Message-Id: <20050902.214859.122594668.nemoto@toshiba-tops.co.jp> To: ralf@linux-mips.org Cc: linux-mips@linux-mips.org Subject: Re: custom ide driver causes "Badness in smp_call_function" From: Atsushi Nemoto In-Reply-To: <20050826141047.GA8777@linux-mips.org> References: <20050825154249.GC2731@linux-mips.org> <20050825211218Z8225471-3678+7505@linux-mips.org> <20050826141047.GA8777@linux-mips.org> X-Fingerprint: 6ACA 1623 39BD 9A94 9B1A B746 CA77 FE94 2874 D52F X-Pgp-Public-Key: http://wwwkeys.pgp.net/pks/lookup?op=get&search=0x2874D52F X-Mailer: Mew version 3.3 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 8864 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: anemo@mba.ocn.ne.jp Precedence: bulk X-list: linux-mips >>>>> On Fri, 26 Aug 2005 15:10:47 +0100, Ralf Baechle said: ralf> Try this patch below and let me know. I would also like to ask ralf> those people who used to suffer from aliases with IDE PIO to try ralf> this patch. I'm using CPU which suffer from dcache aliasing. I tried this patch for IDE PIO and it seems work fine too. Maybe Cobalt user can try it. Anyone? :-) --- Atsushi Nemoto From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Bryan Althouse" Subject: custom ide driver causes "Badness in smp_call_function" Date: Tue, 23 Aug 2005 10:07:02 -0400 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_008F_01C5A7CA.68D6B0E0" Message-ID: <20050823140159Z8225393-3678+7283@linux-mips.org> Return-Path: Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org To: linux-mips@linux-mips.org Message-ID: <20050823140702.xE43CfVdFFFJtbRLCecmGwTvUA_Kgyis_HYWvLOP2Uk@z> This is a multi-part message in MIME format. ------=_NextPart_000_008F_01C5A7CA.68D6B0E0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Hi all, I have added a compact flash disk onto the local bus of an rm9224 mips processor. We are using an FPGA as an IDE host adaptor. Our driver works great when the kernel is compiled for a single processor. But if SMP support is enabled, the kernel dies with: . Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2 Ide: Assuming 50MHz system bus speed for PIO modes; override with idebuss=xx Badness in smp_call_function at arch/mips/kernel/smp.c:149 Call Trace: [] smp_call_function+0x1f8/0x200 [] schedule+0x950/0xa08 [] local_rm9k_perfcounter_irq_startup+0x0/0x68 [] rm9k_perfcounter_irq_startup+0x48/0x68 [] probe_irq_on+0x2b8/0x2e8 [] del_singleshot_timer_sync+0x38/0x50 [] try_to_identify+0x180/0x190 [] process_timeout+0x0/0x8 [] do_probe+0x14c/0x338 [] do_probe+0x5c/0x338 [] wait_hwif_ready+0x178/0x180 [] probe_hwif+0x508/0x6e8 [] probe_hwif+0x1a0/0x6e8 [] probe_hwif_init_with_fixup+0x1c/0xe8 [] ide_3P_init+0xb4/0x118 [] ide_3P_init+0x7c/0x118 [] idr_get_new+0x18/0x50 etc. Does anyone know what sort of bug could cause problems with SMP, but would work fine otherwise? I could supply my driver code if anyone is interested. Thanks! Bryan ------=_NextPart_000_008F_01C5A7CA.68D6B0E0 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable custom ide driver causes "Badness in = smp_call_function"

Hi = all,

&nbs= p; = …

Uniform Multi-Platform E-IDE driver Revision: = 7.00alpha2

Ide: Assuming 50MHz system bus speed for PIO modes; = override with idebuss=3Dxx

Badness in smp_call_function at = arch/mips/kernel/smp.c:149

Call = Trace:

[<ffffffff8010aff8>] = smp_call_function+0x1f8/0x200

[<ffffffff8032adb0>] = schedule+0x950/0xa08

[<ffffffff8010b9f8>] = local_rm9k_perfcounter_irq_startup+0x0/0x68

[<ffffffff8010baa8>] = rm9k_perfcounter_irq_startup+0x48/0x68

[<ffffffff801658b8>] = probe_irq_on+0x2b8/0x2e8

[<ffffffff80146c00>] = del_singleshot_timer_sync+0x38/0x50

[<ffffffff8028e510>] = try_to_identify+0x180/0x190

[<ffffffff80147d88>] = process_timeout+0x0/0x8

[<ffffffff8028e66c>] = do_probe+0x14c/0x338

[<ffffffff8028e57c>] = do_probe+0x5c/0x338

[<ffffffff8028ec40>] = wait_hwif_ready+0x178/0x180

[<ffffffff8028f230>] = probe_hwif+0x508/0x6e8

[<ffffffff8028eec8>] = probe_hwif+0x1a0/0x6e8

[<ffffffff8028f42c>] = probe_hwif_init_with_fixup+0x1c/0xe8

[<ffffffff8029435c>] = ide_3P_init+0xb4/0x118

[<ffffffff80294324>] = ide_3P_init+0x7c/0x118

[<ffffffff8023d100>] = idr_get_new+0x18/0x50

etc…

Does = anyone know what sort of bug could cause problems with SMP, but would = work fine otherwise? I could supply my driver code if anyone = is interested. = Thanks!

Bryan

------=_NextPart_000_008F_01C5A7CA.68D6B0E0-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Bryan Althouse" Subject: RE: custom ide driver causes "Badness in smp_call_function" Date: Thu, 25 Aug 2005 11:26:58 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit In-Reply-To: <20050824152444.GE2783@linux-mips.org> Message-ID: <20050825152131Z8225298-3678+7482@linux-mips.org> Return-Path: Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org To: 'Ralf Baechle' Cc: linux-mips@linux-mips.org Message-ID: <20050825152658.kSWkoRguc0DsIEZAbjs1TCQrXRFSaU34hNdC83AhpJY@z> Ralf, Thank you for your help. I'm doing MMIO, not PIO, but it looks like your assessment is still valid. I've been searching for places where ide MMIO is performed with interrupts disabled. I got excited when I found these lines in probe_hwif() of ide-probe.c: irqd = hwif->irq; if (irqd) disable_irq(hwif->irq); I was not initializing hwif->irq in my driver, so probably the interrupts were being disabled here, and subsequent lines were causing the SMP badness. I added the line "hwif->irq = 0" to my driver. Interrupts are no longer disabled here, but still I get the SMP badness. I'll keep looking for other places where the interrupts might be disabled. Does anyone know if the mips/swarm.c driver has this problem with SMP? Thanks! Bryan From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Bryan Althouse" Subject: RE: custom ide driver causes "Badness in smp_call_function" Date: Thu, 25 Aug 2005 17:17:48 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit In-Reply-To: <20050825154249.GC2731@linux-mips.org> Message-ID: <20050825211218Z8225471-3678+7505@linux-mips.org> Return-Path: Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org To: 'Ralf Baechle' Cc: linux-mips@linux-mips.org Message-ID: <20050825211748.P8sgfR2udlzrJH6EDR21v8RUV3_97mRqUNVHikhJPZ0@z> Here are some observations... If I change the line "hwif->irq = 0" in my driver to "hwif->irq = 5", my SMP kernel no longer experiences SMP badness. Instead, I get many lines like "hda: lost interrupt", and the drive is not usable. If I compile the kernel without SMP, the drive works properly as before. I tried irq = 5 because I noticed that /proc/interrupts indicated that ide0 was being probed at 5. With the SMP kernel, /proc/interrupts shows a count of 0 for ide0. My non-SMP kernel shows a count that increments when the drive is being used (as expected). Bryan From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Bryan Althouse" Subject: RE: custom ide driver causes "Badness in smp_call_function" Date: Fri, 26 Aug 2005 10:58:40 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit In-Reply-To: <20050826141047.GA8777@linux-mips.org> Message-ID: <20050826145303Z8224974-3678+7581@linux-mips.org> Return-Path: Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org To: 'Ralf Baechle' Cc: linux-mips@linux-mips.org Message-ID: <20050826145840.SpPOHusx0mT332_pFTkDputcVOs9OnoqtOrDTxqxYAU@z> Ralf, The patch doesn't seem to make any difference. :( Bryan From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Bryan Althouse" Subject: RE: custom ide driver causes "Badness in smp_call_function" Date: Fri, 26 Aug 2005 12:36:26 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit In-Reply-To: <20050826162832.GB2712@linux-mips.org> Message-ID: <20050826163050Z8224979-3678+7593@linux-mips.org> Return-Path: Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org To: 'Ralf Baechle' Cc: linux-mips@linux-mips.org Message-ID: <20050826163626.lNeeH_yMN7W7bcKQo8T_zHyh1uDZvC1ekEt8tt40Nn4@z> I still get the SMP badness when hwif->irq is set to 0 in my driver. Bryan -----Original Message----- From: Ralf Baechle [mailto:ralf@linux-mips.org] Sent: Friday, August 26, 2005 12:29 PM To: Bryan Althouse Cc: linux-mips@linux-mips.org Subject: Re: custom ide driver causes "Badness in smp_call_function" On Fri, Aug 26, 2005 at 10:58:40AM -0400, Bryan Althouse wrote: > The patch doesn't seem to make any difference. :( To clarify, the patch wasn't meant to resolve your interrupt problems but the SMP cacheflush issues only. Ralf From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Bryan Althouse" Subject: RE: custom ide driver causes "Badness in smp_call_function" Date: Fri, 26 Aug 2005 14:05:43 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit In-Reply-To: <1125071244.7298.2.camel@localhost.localdomain> Message-ID: <20050826180008Z8224984-3678+7604@linux-mips.org> Return-Path: Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org Cc: linux-mips@linux-mips.org Message-ID: <20050826180543.MbbM84ueQUsFH7e47oLEHKDn_fPHgPNlmW5R7KqjfFw@z> Alan, Thanks for your suggestion. I'm not sure how to write a .fixup handler. I did some Googling, but got nowhere. I looked through drivers/ide to see what drive->unmask was doing. I found this in ide-io.c: if (drive->unmask) local_irq_enable(); And this in ide-taskfile.c: if (!drive->unmask) local_irq_disable(); I modified both of these files so that execution would be as if unmask = 1. This resulted in no change of behavior. Bryan -----Original Message----- From: Alan Cox [mailto:alan@lxorguk.ukuu.org.uk] Sent: Friday, August 26, 2005 11:47 AM To: Bryan Althouse Cc: linux-mips@linux-mips.org; 'Ralf Baechle' Subject: RE: custom ide driver causes "Badness in smp_call_function" On Gwe, 2005-08-26 at 10:58 -0400, Bryan Althouse wrote: > Ralf, > > The patch doesn't seem to make any difference. :( Assuming your hardware is sane another approach might be to force drive->unmask = 1. That will mean that PIO mode is running with interrupts enabled which should avoid the problem. Add a .fixup handler to your driver (assuming you are using a recent 2.6.x) and in the handler do something like this: +void ide_unmask_interrupts(ide_hwif_t *hwif) +{ + int i; + for (i = 0; i < 2; i++) { + ide_drive_t *drive = &hwif->drives[i]; + if(drive->present) + drive->unmask = 1; + } +} hopefully that will be early enough. From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Bryan Althouse" Subject: RE: custom ide driver causes "Badness in smp_call_function" Message-ID: <000501c5aa6a$3beadc30$0300a8c0@intra.lexbox.org> Date: Fri, 26 Aug 2005 20:16:09 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit In-Reply-To: <1125071244.7298.2.camel@localhost.localdomain> Content-Class: urn:content-classes:message Return-Path: Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org To: unlisted-recipients, no To-header on input, IMB Recipient 1 Cc: linux-mips@linux-mips.org Message-ID: <20050826181609.s_qYSUcpjLYxLzLgDBdszlVwPjUZBItgGo4ogThUwtk@z> Alan, Thanks for your suggestion. I'm not sure how to write a .fixup handler. I did some Googling, but got nowhere. I looked through drivers/ide to see what drive->unmask was doing. I found this in ide-io.c: if (drive->unmask) local_irq_enable(); And this in ide-taskfile.c: if (!drive->unmask) local_irq_disable(); I modified both of these files so that execution would be as if unmask = 1. This resulted in no change of behavior. Bryan -----Original Message----- From: Alan Cox [mailto:alan@lxorguk.ukuu.org.uk] Sent: Friday, August 26, 2005 11:47 AM To: Bryan Althouse Cc: linux-mips@linux-mips.org; 'Ralf Baechle' Subject: RE: custom ide driver causes "Badness in smp_call_function" On Gwe, 2005-08-26 at 10:58 -0400, Bryan Althouse wrote: > Ralf, > > The patch doesn't seem to make any difference. :( Assuming your hardware is sane another approach might be to force drive->unmask = 1. That will mean that PIO mode is running with interrupts enabled which should avoid the problem. Add a .fixup handler to your driver (assuming you are using a recent 2.6.x) and in the handler do something like this: +void ide_unmask_interrupts(ide_hwif_t *hwif) +{ + int i; + for (i = 0; i < 2; i++) { + ide_drive_t *drive = &hwif->drives[i]; + if(drive->present) + drive->unmask = 1; + } +} hopefully that will be early enough. From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Bryan Althouse" Subject: RE: custom ide driver causes "Badness in smp_call_function" Date: Thu, 1 Sep 2005 12:15:31 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit In-Reply-To: <20050826141047.GA8777@linux-mips.org> Message-ID: <20050901161023Z8225398-3678+8096@linux-mips.org> Return-Path: Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org To: 'Ralf Baechle' Cc: linux-mips@linux-mips.org Message-ID: <20050901161531.wNZg9n_XvXiYNWJycz_O91ZwKCI164wGQQg_BiesDtU@z> Ralf, I've done more testing with your patch. When I use it with my non-SMP kernel, disk access will cause a panic with a cache error message. Is this patch only intended for SMP, or is this a legitimate problem? Bryan Index: include/asm-mips/mach-generic/ide.h =================================================================== RCS file: /home/cvs/linux/include/asm-mips/mach-generic/ide.h,v retrieving revision 1.9 diff -u -r1.9 ide.h --- include/asm-mips/mach-generic/ide.h 19 Apr 2005 12:26:59 -0000 1.9 +++ include/asm-mips/mach-generic/ide.h 26 Aug 2005 14:04:38 -0000 @@ -19,6 +19,7 @@ #include #include #include +#include #ifndef MAX_HWIFS # ifdef CONFIG_BLK_DEV_IDEPCI @@ -105,12 +106,14 @@ /* MIPS port and memory-mapped I/O string operations. */ -static inline void __ide_flush_dcache_range(unsigned long addr, unsigned long size) +static inline void __ide_flush_dcache_range(unsigned long addr, + unsigned long size) { - if (cpu_has_dc_aliases) { - unsigned long end = addr + size; - for (; addr < end; addr += PAGE_SIZE) - flush_dcache_page(virt_to_page(addr)); + unsigned long end = addr + size; + + while (addr < end) { + SetPageDcacheDirty(virt_to_page(addr)); + addr += PAGE_SIZE; } }