strange segfaults with CompactFlash in "true-ide" mode on PXA270

strange segfaults with CompactFlash in "true-ide" mode on PXA270

[Juergen Schindele]

Dear developers,
we have a strange problem when booting from a Compact Flash card
as root-device. In one of 10 cases while booting an application from
root-filesystem dies with "Segmentation fault" without a visible error.
After booting the same application runs hundred times without error.
(we're talking about standard appl. like chat, pppd, hwclock and so on)

For debugging purposes we put printk's in kernels IDE driver 
and the problem disappeared. But replacing the printk by a usleep
or so (which is not a solution !) brings the problem back.
It seems that the application received faulty data from CF.
But how and why ?????????????????????????????

We tested with a custom PXA270 board with a CompactFlash in 
"true ide" on processor bus driven by "generic ide / pata_platform"
driver from linux-2.6.27.x.
(by the way linux-2.6.20.x showed the same problem).

Any ideas, hints, experiences, patches are very welcome :-)
--------------------------------------------------------------
J?rgen Schindele  
Software-Entwicklung

PSI NENTEC GmbH
Greschbachstr. 12
76229 Karlsruhe
Deutschland
Telefon: +49 721 94249-51
Telefax: +49 721 94249-10
E-Mail:  schindele at nentec.de
WEB:     www.nentec.de
 
Gesch?ftsf?hrung: Klaus Becker, Roland Knapp
Sitz der Gesellschaft: Karlsruhe
Handelsregister: Amtsgericht Mannheim HRB 107658
--------------------------------------------------------------

strange segfaults with CompactFlash in "true-ide" mode on PXA270

[Lothar Waßmann]

Hi,

> we have a strange problem when booting from a Compact Flash card
> as root-device. In one of 10 cases while booting an application from
> root-filesystem dies with "Segmentation fault" without a visible error.
> After booting the same application runs hundred times without error.
> (we're talking about standard appl. like chat, pppd, hwclock and so on)
> 
> For debugging purposes we put printk's in kernels IDE driver 
> and the problem disappeared. But replacing the printk by a usleep
> or so (which is not a solution !) brings the problem back.
> It seems that the application received faulty data from CF.
> But how and why ?????????????????????????????
> 
> We tested with a custom PXA270 board with a CompactFlash in 
> "true ide" on processor bus driven by "generic ide / pata_platform"
> driver from linux-2.6.27.x.
> (by the way linux-2.6.20.x showed the same problem).
> 
> Any ideas, hints, experiences, patches are very welcome :-)
> --------------------------------------------------------------
I guess that is the old cache problem that apparently was never really
fixed:
http://marc.info/?t=108537828400002&r=1&w=2


Lothar Wa?mann
-- 
___________________________________________________________

Ka-Ro electronics GmbH | Pascalstra?e 22 | D - 52076 Aachen
Phone: +49 2408 1402-0 | Fax: +49 2408 1402-10
Gesch?ftsf?hrer: Matthias Kaussen
Handelsregistereintrag: Amtsgericht Aachen, HRB 4996

www.karo-electronics.de | info at karo-electronics.de
___________________________________________________________

strange segfaults with CompactFlash in "true-ide" mode on PXA270

[Rabin Vincent]

On Wed, Jun 23, 2010 at 8:54 PM, Lothar Wa?mann <LW@karo-electronics.de> wrote:
>> we have a strange problem when booting from a Compact Flash card
>> as root-device. In one of 10 cases while booting an application from
>> root-filesystem dies with "Segmentation fault" without a visible error.
>> After booting the same application runs hundred times without error.
>> (we're talking about standard appl. like chat, pppd, hwclock and so on)
>>
>> For debugging purposes we put printk's in kernels IDE driver
>> and the problem disappeared. But replacing the printk by a usleep
>> or so (which is not a solution !) brings the problem back.
>> It seems that the application received faulty data from CF.
>> But how and why ?????????????????????????????
>>
>> We tested with a custom PXA270 board with a CompactFlash in
>> "true ide" on processor bus driven by "generic ide / pata_platform"
>> driver from linux-2.6.27.x.
>> (by the way linux-2.6.20.x showed the same problem).
>>
>> Any ideas, hints, experiences, patches are very welcome :-)
>> --------------------------------------------------------------
> I guess that is the old cache problem that apparently was never really
> fixed:
> http://marc.info/?t=108537828400002&r=1&w=2

Catalin recently posted some patches addressing the cache
problem:

http://thread.gmane.org/gmane.linux.ports.arm.kernel/82987

Rabin

strange segfaults with CompactFlash in "true-ide" mode on PXA270

[Juergen Schindele]

Am Mittwoch, 23. Juni 2010 schrieb Lothar Wa?mann:
> Hi,
> 
> > we have a strange problem when booting from a Compact Flash card
> > as root-device. In one of 10 cases while booting an application from
> > root-filesystem dies with "Segmentation fault" without a visible error.
> > After booting the same application runs hundred times without error.
> > (we're talking about standard appl. like chat, pppd, hwclock and so on)
> > 
> > For debugging purposes we put printk's in kernels IDE driver 
> > and the problem disappeared. But replacing the printk by a usleep
> > or so (which is not a solution !) brings the problem back.
> > It seems that the application received faulty data from CF.
> > But how and why ?????????????????????????????
> > 
> > We tested with a custom PXA270 board with a CompactFlash in 
> > "true ide" on processor bus driven by "generic ide / pata_platform"
> > driver from linux-2.6.27.x.
> > (by the way linux-2.6.20.x showed the same problem).
> > 
> > Any ideas, hints, experiences, patches are very welcome :-)
> > --------------------------------------------------------------
> I guess that is the old cache problem that apparently was never really
> fixed:
> http://marc.info/?t=108537828400002&r=1&w=2
Hello
this hint was very helpful for understanding the problem.
In the mail from Russell King at 
<http://marc.info/?l=linux-arm-kernel&m=108611676807909&w=2>
he said : .... the rule is if the CPU writes to a page cache page,
it must call flush_dcache_page afterwards to ensure cache coherency with user space".

so i made the following patch in drivers/ide/ide-taskfile.c
in the function ide_pio_sector() where blocks from disk are read
into a cached page to apply the rule from Russell. 
===================================================================
--- ide-taskfile.c
+++ ide-taskfile.c.new
@@ -278,6 +278,7 @@
                hwif->tp_ops->input_data(drive, rq, buf, SECTOR_SIZE);

        kunmap_atomic(buf, KM_BIO_SRC_IRQ);
+       flush_dcache_page(page);
 #ifdef CONFIG_HIGHMEM
        local_irq_restore(flags);
 #endif

And from there on the problem disappeared :-))
What are you think about ???
Isn't this missing for years in IDE PIO driver ???
--------------------------------------------------------------
J?rgen Schindele  
Software-Entwicklung

PSI NENTEC GmbH
Greschbachstr. 12
76229 Karlsruhe
Deutschland
Telefon: +49 721 94249-51
Telefax: +49 721 94249-10
E-Mail:  schindele at nentec.de
WEB:     www.nentec.de
 
Gesch?ftsf?hrung: Klaus Becker, Roland Knapp
Sitz der Gesellschaft: Karlsruhe
Handelsregister: Amtsgericht Mannheim HRB 107658
--------------------------------------------------------------

strange segfaults with CompactFlash in "true-ide" mode on PXA270

[Sergei Shtylyov]

Hello.

Juergen Schindele wrote:

>>> we have a strange problem when booting from a Compact Flash card
>>> as root-device. In one of 10 cases while booting an application from
>>> root-filesystem dies with "Segmentation fault" without a visible error.
>>> After booting the same application runs hundred times without error.
>>> (we're talking about standard appl. like chat, pppd, hwclock and so on)
>>>
>>> For debugging purposes we put printk's in kernels IDE driver 
>>> and the problem disappeared. But replacing the printk by a usleep
>>> or so (which is not a solution !) brings the problem back.
>>> It seems that the application received faulty data from CF.
>>> But how and why ?????????????????????????????
>>>
>>> We tested with a custom PXA270 board with a CompactFlash in 
>>> "true ide" on processor bus driven by "generic ide / pata_platform"
>>> driver from linux-2.6.27.x.
>>> (by the way linux-2.6.20.x showed the same problem).
>>>
>>> Any ideas, hints, experiences, patches are very welcome :-)
>>> --------------------------------------------------------------
>> I guess that is the old cache problem that apparently was never really
>> fixed:
>> http://marc.info/?t=108537828400002&r=1&w=2

> Hello
> this hint was very helpful for understanding the problem.
> In the mail from Russell King at 
> <http://marc.info/?l=linux-arm-kernel&m=108611676807909&w=2>
> he said : .... the rule is if the CPU writes to a page cache page,
> it must call flush_dcache_page afterwards to ensure cache coherency with user space".

> so i made the following patch in drivers/ide/ide-taskfile.c
> in the function ide_pio_sector() where blocks from disk are read
> into a cached page to apply the rule from Russell. 
> ===================================================================
> --- ide-taskfile.c
> +++ ide-taskfile.c.new
> @@ -278,6 +278,7 @@
>                 hwif->tp_ops->input_data(drive, rq, buf, SECTOR_SIZE);
> 
>         kunmap_atomic(buf, KM_BIO_SRC_IRQ);
> +       flush_dcache_page(page);
>  #ifdef CONFIG_HIGHMEM
>         local_irq_restore(flags);
>  #endif

> And from there on the problem disappeared :-))
> What are you think about ???

    Flushing is only needed for reads and shouldn't be done for the slab pages 
(judging on libata's code).

> Isn't this missing for years in IDE PIO driver ???

    It's actually missing from the ARM port of IDE. E.g. the MIPS port 
provides for that -- see arch/mips/include/asm/mach-generic/ide.h.

WBR, Sergei

strange segfaults with CompactFlash in "true-ide" mode on PXA270

[Catalin Marinas]

On Fri, 2010-06-25 at 08:11 +0100, Juergen Schindele wrote:
> Am Mittwoch, 23. Juni 2010 schrieb Lothar Wa?mann:
> > Hi,
> >
> > > we have a strange problem when booting from a Compact Flash card
> > > as root-device. In one of 10 cases while booting an application from
> > > root-filesystem dies with "Segmentation fault" without a visible error.
> > > After booting the same application runs hundred times without error.
> > > (we're talking about standard appl. like chat, pppd, hwclock and so on)
> > >
> > > For debugging purposes we put printk's in kernels IDE driver
> > > and the problem disappeared. But replacing the printk by a usleep
> > > or so (which is not a solution !) brings the problem back.
> > > It seems that the application received faulty data from CF.
> > > But how and why ?????????????????????????????
> > >
> > > We tested with a custom PXA270 board with a CompactFlash in
> > > "true ide" on processor bus driven by "generic ide / pata_platform"
> > > driver from linux-2.6.27.x.
> > > (by the way linux-2.6.20.x showed the same problem).
> > >
> > > Any ideas, hints, experiences, patches are very welcome :-)
> > > --------------------------------------------------------------
> > I guess that is the old cache problem that apparently was never really
> > fixed:
> > http://marc.info/?t=108537828400002&r=1&w=2
> Hello
> this hint was very helpful for understanding the problem.
> In the mail from Russell King at
> <http://marc.info/?l=linux-arm-kernel&m=108611676807909&w=2>
> he said : .... the rule is if the CPU writes to a page cache page,
> it must call flush_dcache_page afterwards to ensure cache coherency with user space".
> 
> so i made the following patch in drivers/ide/ide-taskfile.c
> in the function ide_pio_sector() where blocks from disk are read
> into a cached page to apply the rule from Russell.

That rule has been discussed on several occasions on LKML. So rather
than changing all the faulty drivers, you can try this:

http://article.gmane.org/gmane.linux.ports.arm.kernel/83413

-- 
Catalin