Re: smp_send_stop() and disable_local_APIC()

[ebiederm@xmission.com (Eric W. Biederman)]

"Matt D. Robinson" <yakker@alacritech.com> writes:

> It looks like around 2.3.30 or so, someone added the call
> disable_local_APIC() to smp_send_stop().  I'm not sure what the
> intention was, but I'm getting some strange behavior as a result
> based on some code I'm writing.
> 
> Basically, I'm doing the following ...
> 
>     panic()
>     {
>         /* do whatever you want, notifier list, etc. */
>         smp_send_stop();
>         write_system_memory();
>         /* then do whatever */
>     }
> 
> write_system_memory() does a write of all system memory pages to some
> block device.  It uses kiobufs as the way to get the pages to disk,
> doing brw_kiovec() on those pages (using either the IDE or SCSI
> driver to write the data).

IDE being less likely to hang than SCSI as it tends to use legacy isa
interrupt lines.
 
> The wierd behavior I see is that sometimes, smp_send_stop()
> being called causes the system to hang up (not every time). 

Doing event driver i/o after disabling the interrupt controller
hmm, I wonder why...

> If we don't call smp_send_stop() on those systems, everything works fine.
> This looks to be directly caused by the disabling of the APIC, which
> we may need to dump pages to local disk.  This only applies to some
> people's systems -- not everyone displays the same behavior.
> 
> I'm sure it's good to disable the APIC, but there's no clean way to
> wait on disabling the APIC until after I'm done writing pages out.
> 
> My questions are:
> 
> 1) Why was disable_local_APIC() added to stop_this_cpu()
>    and smp_send_stop()?  Completeness?
> 
> 2) Is there a better way around this to disable all the
>    other CPUs without disabling the APIC?
> 

I don't know what a good way is, since there is a kernel panic it
should only be something truly fatal.  Given that reusing anything
that hasn't been designed to run in that situation is playing with
fire.

Eric