A page fault causes exiting from WFI mode for all idling cores?

A page fault causes exiting from WFI mode for all idling cores?

[heechul Yun]

While I was investigating page fault handling performance on a quad
core Cortex-A9 ARM processor,?I noticed that whenever there is a
page fault, all idling cores?also?awakened---exiting from WFI instruction
on the idle loop?(and go back to WFI again).

Since the page fault generating program is running on a?single core,
it seems unnecessary to wake up?other idling cores (hence saving
power) as well.

Can anyone tell me if this, exiting from WFI on every page fault, is
correct behavior??If so, why?

Heechul

A page fault causes exiting from WFI mode for all idling cores?

[Catalin Marinas]

On 28 May 2011 12:59, heechul Yun <heechul@illinois.edu> wrote:
> While I was investigating page fault handling performance on a quad
> core Cortex-A9 ARM processor,?I noticed that whenever there is a
> page fault, all idling cores?also?awakened---exiting from WFI instruction
> on the idle loop?(and go back to WFI again).
>
> Since the page fault generating program is running on a?single core,
> it seems unnecessary to wake up?other idling cores (hence saving
> power) as well.
>
> Can anyone tell me if this, exiting from WFI on every page fault, is
> correct behavior??If so, why?

I don't think this is explicitly done by Linux, at least not with
Cortex-A9. For ARM11MPCore, we do some TLB operations which are
broadcast in software via IPI. Which kernel version are you using?

Also note that WFI is just a hint to the processor, so it could get
out of it at any time. Maybe the A9 just does this when it gets some
TLB ops from another CPU.

-- 
Catalin

A page fault causes exiting from WFI mode for all idling cores?

[Lorenzo Pieralisi]

On Wed, 2011-06-01 at 11:14 +0100, Catalin Marinas wrote:
> On 28 May 2011 12:59, heechul Yun <heechul@illinois.edu> wrote:
> > While I was investigating page fault handling performance on a quad
> > core Cortex-A9 ARM processor, I noticed that whenever there is a
> > page fault, all idling cores also awakened---exiting from WFI instruction
> > on the idle loop (and go back to WFI again).
> >
> > Since the page fault generating program is running on a single core,
> > it seems unnecessary to wake up other idling cores (hence saving
> > power) as well.
> >
> > Can anyone tell me if this, exiting from WFI on every page fault, is
> > correct behavior? If so, why?
> 
> I don't think this is explicitly done by Linux, at least not with
> Cortex-A9. For ARM11MPCore, we do some TLB operations which are
> broadcast in software via IPI. Which kernel version are you using?
> 
> Also note that WFI is just a hint to the processor, so it could get
> out of it at any time. Maybe the A9 just does this when it gets some
> TLB ops from another CPU.
> 

According to A9 TRM transition from standby to run mode can be triggered
by cp15 maintenance requests generated by another cpu, just to reiterate
Catalin's reply.

Lorenzo

A page fault causes exiting from WFI mode for all idling cores?

[heechul Yun]

On Wed, Jun 1, 2011 at 5:55 AM, Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
> On Wed, 2011-06-01 at 11:14 +0100, Catalin Marinas wrote:
>> On 28 May 2011 12:59, heechul Yun <heechul@illinois.edu> wrote:
>> > While I was investigating page fault handling performance on a quad
>> > core Cortex-A9 ARM processor, I noticed that whenever there is a
>> > page fault, all idling cores also awakened---exiting from WFI instruction
>> > on the idle loop (and go back to WFI again).
>> >
>> > Since the page fault generating program is running on a single core,
>> > it seems unnecessary to wake up other idling cores (hence saving
>> > power) as well.
>> >
>> > Can anyone tell me if this, exiting from WFI on every page fault, is
>> > correct behavior? If so, why?
>>
>> I don't think this is explicitly done by Linux, at least not with
>> Cortex-A9. For ARM11MPCore, we do some TLB operations which are
>> broadcast in software via IPI. Which kernel version are you using?

The kernel version is 2.6.36.
I checked the number of IPIs before/after the program and found there
were close to none IPIs during the run.

>>
>> Also note that WFI is just a hint to the processor, so it could get
>> out of it at any time. Maybe the A9 just does this when it gets some
>> TLB ops from another CPU.
>>
>
> According to A9 TRM transition from standby to run mode can be triggered
> by cp15 maintenance requests generated by another cpu, just to reiterate
> Catalin's reply.
>

You mean cache/tlb operations can automatically generate maintenance
request to other cores (not via IPIs) ?

> Lorenzo
>
>
>

A page fault causes exiting from WFI mode for all idling cores?

[Catalin Marinas]

On Thu, Jun 02, 2011 at 12:16:57AM +0100, heechul Yun wrote:
> On Wed, Jun 1, 2011 at 5:55 AM, Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
> > On Wed, 2011-06-01 at 11:14 +0100, Catalin Marinas wrote:
> >> On 28 May 2011 12:59, heechul Yun <heechul@illinois.edu> wrote:
> >> > While I was investigating page fault handling performance on a quad
> >> > core Cortex-A9 ARM processor, I noticed that whenever there is a
> >> > page fault, all idling cores also awakened---exiting from WFI instruction
> >> > on the idle loop (and go back to WFI again).
> >> >
> >> > Since the page fault generating program is running on a single core,
> >> > it seems unnecessary to wake up other idling cores (hence saving
> >> > power) as well.
> >> >
> >> > Can anyone tell me if this, exiting from WFI on every page fault, is
> >> > correct behavior? If so, why?
...
> >> Also note that WFI is just a hint to the processor, so it could get
> >> out of it at any time. Maybe the A9 just does this when it gets some
> >> TLB ops from another CPU.
> >>
> >
> > According to A9 TRM transition from standby to run mode can be triggered
> > by cp15 maintenance requests generated by another cpu, just to reiterate
> > Catalin's reply.
> 
> You mean cache/tlb operations can automatically generate maintenance
> request to other cores (not via IPIs) ?

Yes, starting with Cortex-A9. Without this feature we struggle a bit on
ARM11MPCore where we have to broadcast TLB ops via IPI and do some weird
tricks for cache maintenance (like read for ownership).

-- 
Catalin