Re: PCI PM: Restore standard config registers of all devices early

["Rafael J. Wysocki" <rjw@sisk.pl>]

On Monday 02 February 2009, Benjamin Herrenschmidt wrote:
> On Mon, 2009-02-02 at 14:15 -0800, Linus Torvalds wrote:
> > 
> > On Mon, 2 Feb 2009, Rafael J. Wysocki wrote:
> > > 
> > > I think it would be easier to make ACPI allow us to run AML with interrupts
> > > off.
> > 
> > Well, I'd agree, except I have this strong memory of us having known bugs 
> > with ACPI turning hard-interrupts on again. Similarly, it uses mutexes etc 
> > that simply don't work with interrupts off and/or may turn them on again 
> > thanks to scheduling.
> > 
> > "Fixing" that seems not very easy. ACPI has a bad habit of being _really_ 
> > hard to fix in this area.
> > 
> > I do agree that _if_ we can just fix ACPI, we wouldn't have these issues, 
> > and we should  just call it with interrupts disabled with our existing 
> > code. But my previous email was a "maybe we can do it like this" kind of 
> > thing, which might allow us to use ACPI with none of the irq-off issues.
> 
> Len, what's your take here ? How much of that stuff is burried deep and
> how much is nicely split in a "helper" layer we could more easily fix ?
> 
> I'm adding Ingo to the CC as he might have more ideas on how best to
> just make the mutexes work & not complain rather than touching ACPI
> itself... again, just like boot, might just be a matter of instructing
> the mutexes/lockdep to shut up and ignore in_atomic() in those "special"
> phases such as late suspend and early resume().
> 
> That would help me for something else that broke recently too ... I have
> a special "hook" in radeonfb that my arch calls to resume it -very-
> early (interrupts off, I haven't even re-enabled the L2 cache). This is
> very useful to help debugging problems at resume since without that you
> basically don't see a thing and we have no serial port on most of these
> machines.
> 
> However, that started breaking recently due to fb_set_suspend() calling
> into various bits of infrastructure that is no longer safe to call in
> atomic context.
> 
> Here too, in fact, those -would- be safe since it's mostly a matter of
> teaching things like mutex of kmalloc that we are not in standard
> SYSTEM_RUNNING state, and thus mutex can just pretty much ignore the
> problem of being in atomic state and kmalloc/gfp could automatically
> degrade to GFP_ATOMIC (*)
> 
> So it might just all be a matter of making might_sleep() shut up in
> late suspend/early resume, and possibly msleep() silently turn into
> mdelay or something like that. Just make sure we don't actually try to
> schedule (and possibly BUG_ON if we actually end up blocking on a mutex,
> we should not).
> 
> Len, do you think that would work with ACPI or it's more convoluted than
> that ?
> 
> (*) There are reasons to think that kmalloc/gfp should both silently
> turn into GFP_NOIO always while the suspend process is started, but
> that's somewhat a different subject. Rafael, did we ever act on that ?
> It's an old discussion we had but I don't know if we actually
> implemented anything.

We have the ->prepare(), ->complete() callbacks that, among other things,
can be used for allocating and freeing memory with GFP_KERNEL safely.

> IE. Without that, afaik, a driver that hasn't suspend yet might end up
> being blocked in some allocation somewhere due to an attempt to page
> things out on to an already sleeping device. That driver might be in
> such blockage while holding one of its internal mutexes or other thing
> that will cause it's own suspend routine later on to screw up. etc etc
> etc...

Yes, that's possible in theory, never observed in practice from what I can
tell.

> In general, best to avoid having to teach drivers that in suspend-land,
> non-atomic, allocations may block for ever. Better to make them all atomic
> magically.

Hm, atomic allocations may cause other problems to happen (ie. fail easily).

Thanks,
Rafael