Re: Reworking suspend-resume sequence (was: Re: PCI PM: Restore standard config registers of all devices early)

Ingo Molnar
2009-02-03T21:05:32Z

* Ingo Molnar <mingo@elte.hu> wrote:

[...]
> That principle works both for networking and for other IO transports - but
> we have little support for it yet. It would work really well for workloads
> where one physical device is shared by many CPUs.

Which is really what we have in _practice_ - only benchmarketing sets up one
physical device per CPU and avoids all the ugly consequences of
gathering/spreadig a channel of information from/to a single device to
multiple CPUs.

> (A lesser method that approximates this is the use of lots of
> submission/completion rings per device and their binding to cpus - but
> that can never really approach the number of CPUs really possible in a
> system.)

btw., more advanced device IRQ models was one of the thinking behind sparse
IRQ support: defining a really large NR_IRQS limit on x86 by default, on all
form factors, and making it really easy and cheap to have a _ton_ of IRQs in
a Linux system might give hw designers ideas to create such hardware.

/me dreams on ;-)

> And in this most advanced mode of MSI IRQs, and if MSI devices had the
> ability to direct IRQs to a specific CPU (they dont have that right now
> AFAICT), we'd run into the overhead scenarios you describe above, and your
> edge-triggered flow is the most performant one.

So i'd still like your tentative Signed-off-by for your patch - it's i think
not v2.6.29 material but if it stays problem free in testing we can try it
in v2.6.30. If it causes problem it will be clearly bisectable and clearly
revertable.

Maybe we could split it in two: and for MSI we could introduce a 'simpler
and faster' edge flow as well - and keep the legacy handler untouched. That
way it's low-risk in its entirety. (and avoids the MSI ->mask complication
as well.)

Ingo