[parisc-linux] Pb with dump driver.

[parisc-linux] Pb with dump driver.

[Bruno Vidal]

[-- Attachment #1: Type: text/plain, Size: 955 bytes --]

	Hi
I have a problem with the dump driver. I was only testing it with
a "breakpoint" and "instruction fault", now i try the real life,
a good old "data page fault", but the problem is, it start the dump, 
but at the first try to write on the dump device if "sleep"
forever in "brw_kiovec". I'm pretty sure that it comes from
a spinlock deadlock. So I'm looking for someone who is able
to explain the "data page fault" function. 
For me it is the signal 15 in function handle_interruption() 
of traps.c. If it is the space 0 (kernel space), it call 
parisc_terminate(), that call dump() (my funtion). But in dump 
I use a buffer, and I use this buffer with "brw_kiovec". So I 
think that "brw_kiovec" do a page_fault somewhere, but because 
the previous page_fault fail, there is still a spinlock somewhere.
Can someone help on this issue ?

	Thanks.

-- 
	Vidal Bruno, (770-4271)
        SSD-HA Team, HP-UX & LINUX Support
	bruno_vidal@admin.france.hp.com

[-- Attachment #2: Card for Bruno Vidal --]
[-- Type: text/x-vcard, Size: 386 bytes --]

begin:vcard 
n:Bruno;VIDAL
tel;fax:01-69-82-60-14
tel;work:01-69-29-42-71
x-mozilla-html:TRUE
url:www.france.hp.com
org:Solution Center
version:2.1
email;internet:bruno_vidal@hp.com
title:Expert Logiciel Environnement Haute Disponibilité
adr;quoted-printable:;;HP France=0D=0A;Z.A de Courtaboeuf;1 Av. du Canada;91947 Les Ulis cedex;France
x-mozilla-cpt:;26208
fn:VIDAL Bruno
end:vcard

Re: [parisc-linux] Pb with dump driver.

[Bruno Vidal]

[-- Attachment #1: Type: text/plain, Size: 331 bytes --]

	Another thing that can produce this behavior is the interruption mask.
When calling the do_page_fault, what it the interruption mask ? I think it is 
high enought to mask SCSI interruption ? When is it reset back ?

	Thanks.

-- 
	Vidal Bruno, (770-4271)
        SSD-HA Team, HP-UX & LINUX Support
	bruno_vidal@admin.france.hp.com

[-- Attachment #2: Card for Bruno Vidal --]
[-- Type: text/x-vcard, Size: 386 bytes --]

begin:vcard 
n:Bruno;VIDAL
tel;fax:01-69-82-60-14
tel;work:01-69-29-42-71
x-mozilla-html:TRUE
url:www.france.hp.com
org:Solution Center
version:2.1
email;internet:bruno_vidal@hp.com
title:Expert Logiciel Environnement Haute Disponibilité
adr;quoted-printable:;;HP France=0D=0A;Z.A de Courtaboeuf;1 Av. du Canada;91947 Les Ulis cedex;France
x-mozilla-cpt:;26208
fn:VIDAL Bruno
end:vcard

Re: [parisc-linux] Pb with dump driver.

[Grant Grundler]

Bruno Vidal wrote:
> but at the first try to write on the dump device if "sleep"
> forever in "brw_kiovec". I'm pretty sure that it comes from
> a spinlock deadlock. So I'm looking for someone who is able
> to explain the "data page fault" function. 

Have you considered using IODC to write out the data
instead of using OS code?

It would avoid much of the ugliness of spinlocks still
held in the IO or VM code.

> For me it is the signal 15 in function handle_interruption() 
> of traps.c. If it is the space 0 (kernel space), it call 
> parisc_terminate(), that call dump() (my funtion). But in dump 
> I use a buffer, and I use this buffer with "brw_kiovec". So I 
> think that "brw_kiovec" do a page_fault somewhere, but because 
> the previous page_fault fail, there is still a spinlock somewhere.

If you TOC the system, you should see the code was spinning
on a ldcw instruction.  either IOAQ should point to the
function and the address referenced by ldcw should be
visible in System.map.

> Can someone help on this issue ?

Once we know either (offending code location or which
spinlock), we can start looking at code to figure out
what the problem is.


In the next mail:
| Another thing that can produce this behavior is the interruption mask.
| When calling the do_page_fault, what it the interruption mask ?

I believe PSW_I bit is off. EIEM is irrelevant at this point.
See entry.S where it calls handle_interruption.

| I think it is high enough to mask SCSI interruption?
| When is it reset back ?

See "do_cpu_irq_mask()" (arch/parisc/kernel/irq.c).
SCSI interrupt as an "External Interrupt" and remains masked in the EIEM
even after PSW_I bit is set. No interrupts from anything using the masked
EIR bit will be processed until we exit do_cpu_irq_mask(). If that
gets interrupted by a page fault or something, using that driver
is toast until we reset the EIEM registers of all cpus....another
reason to use IODC since it doesn't use interrupts for
processing.

thanks,
grant

Re: [parisc-linux] Pb with dump driver.

[Bruno Vidal]

[-- Attachment #1: Type: text/plain, Size: 737 bytes --]

	Hi
I finaly found where the code is waiting. It is wainting on the interruption
from the scsi drive (it is normal :-( ). So for several "good" reason, Grant is
totaly right, I can't trust the driver, the buffer, and several other things....
So I take the decision to modify now the code and create my own dump_iodc_write,
dump_iodc_read, dump_iodc_wait, dump_iodc_open, dump_iodc_seek (I don't think
I need other functions). It is lots of works (because I have to understand how it 
works). So I'll let you know when it will be available. Thanks a lot to grant that
point to the right direction, now let's go at work :-)

	Thanks.

-- 
	Vidal Bruno, (770-4271)
        SSD-HA Team, HP-UX & LINUX Support
	bruno_vidal@admin.france.hp.com

[-- Attachment #2: Card for Bruno Vidal --]
[-- Type: text/x-vcard, Size: 386 bytes --]

begin:vcard 
n:Bruno;VIDAL
tel;fax:01-69-82-60-14
tel;work:01-69-29-42-71
x-mozilla-html:TRUE
url:www.france.hp.com
org:Solution Center
version:2.1
email;internet:bruno_vidal@hp.com
title:Expert Logiciel Environnement Haute Disponibilité
adr;quoted-printable:;;HP France=0D=0A;Z.A de Courtaboeuf;1 Av. du Canada;91947 Les Ulis cedex;France
x-mozilla-cpt:;26208
fn:VIDAL Bruno
end:vcard

Re: [parisc-linux] Pb with dump driver.

[Grant Grundler]

Bruno Vidal wrote:
> So I take the decision to modify now the code and create my own
> dump_iodc_write, dump_iodc_read, dump_iodc_wait, dump_iodc_open,
> dump_iodc_seek (I don't think I need other functions).

You should be able to leverage most of the code from palo.
Note that palo runs in th CPU in real-mode - not virtual mode.
You might consider doing the same for the dump code and just
be *very* careful about which functions you call.

Oh, and the dump path might want to do two other things:
1) PDC_IO_RESET_CLEAR
2) switch linux console back to PDC console

I don't know if some of those things are already happening.
If I have time, I'll look later.

palo is in CVS on the same server (cvs.parisc-linux.org) as our kernel.
(ie "cvs co palo" instead of linux)

> Thanks a lot to grant that point to the right direction,
> now let's go at work :-)

hehe..very welcome...I'm very happy to see you doing it...it's been
on my wish list for a long time.

thanks,
grant

Re: [parisc-linux] Pb with dump driver.

[Matthew Wilcox]

On Fri, Feb 15, 2002 at 12:08:18PM -0700, Grant Grundler wrote:
> You should be able to leverage most of the code from palo.
> Note that palo runs in th CPU in real-mode - not virtual mode.
> You might consider doing the same for the dump code and just
> be *very* careful about which functions you call.

You have to do that anyway.  If a device has managed to DMA all over
the kernel text, you have to have a completely independent dump
mechanism.  I personally HATE the idea of a kernel dump driver and
think that MPE's solution (on a TOC, dump the ram after reset) was
more elegant and easy to prove correct.

Note that grant disagrees with me on this.

-- 
Revolutions do not require corporate support.

Re: [parisc-linux] Pb with dump driver.

[Grant Grundler]

Matthew Wilcox wrote:
> I personally HATE the idea of a kernel dump driver and
> think that MPE's solution (on a TOC, dump the ram after reset) was
> more elegant and easy to prove correct.
> 
> Note that grant disagrees with me on this.

I'm not religous about this. At this point, I'm happy for *any*
kind of a dump tool to debug kernel state when the system crashes.

grant