Access to local APIC registers during an interrupt handler

Access to local APIC registers during an interrupt handler

[Robert Uhl]

Hi,

I wrote a small interrupt handler (for a 64 bit system) which is 
installed by a kernel module in the IDT. My handler just increments a 
global variable and then jumps to the original interrupt handler. So 
Linux does (hopefully ;-) not notice my handler. With sysfs I can read 
the value of these interrupt counter and know exactly how often a 
specific interrupt occured.
Of course on a multicore system the interrupts of all cores are counted 
together, but I want to separate between the cores. On newer CPUs I can 
use the instruction RDTSCP to get the CPU number in ECX, but on older 
CPUs it's unsupported.
So I had the idea to use the local APIC ID to check on which core my 
handler is executed, even though sometimes local APIC ID != core number, 
but the ID should be at least unique.
I get the address of the local APIC ID register at module init with

u64 lapic_idregister = (u64) fix_to_virt(FIX_APIC_BASE) + 0x20;

and use it in my interrupt handler (of course I push/pop all used 
registers):

movq (lapic_idregister), %rcx
movq (%rcx), %rcx

But on real hardware the last instruction seems to cause a page fault or 
something (SUSE with 2.6.37.6, Fedora with 2.6.38.6), the system simply 
reboots. Without this instruction, the handler is executed without any 
problems.
And in qemu with vanilla 2.6.37.6 and a buildroot system everything 
works fine!

I already had a look with qemu which instructions are executed at a 
local APIC timer interrupt (0xEF) until it writes 0x00 to the local APIC 
EOI register (same page), but I still can't figure out what's the 
problem with my code.

Maybe someone knows what is missing or if there is any other fast way to 
figure out on which core the handler is running?

Kind regards,
Robert

Re: Access to local APIC registers during an interrupt handler

[Mikael Pettersson]

Robert Uhl writes:
 > Hi,
 > 
 > I wrote a small interrupt handler (for a 64 bit system) which is 
 > installed by a kernel module in the IDT. My handler just increments a 
 > global variable and then jumps to the original interrupt handler. So 
 > Linux does (hopefully ;-) not notice my handler. With sysfs I can read 
 > the value of these interrupt counter and know exactly how often a 
 > specific interrupt occured.
 > Of course on a multicore system the interrupts of all cores are counted 
 > together, but I want to separate between the cores. On newer CPUs I can 
 > use the instruction RDTSCP to get the CPU number in ECX, but on older 
 > CPUs it's unsupported.
 > So I had the idea to use the local APIC ID to check on which core my 
 > handler is executed, even though sometimes local APIC ID != core number, 
 > but the ID should be at least unique.
 > I get the address of the local APIC ID register at module init with
 > 
 > u64 lapic_idregister = (u64) fix_to_virt(FIX_APIC_BASE) + 0x20;
 > 
 > and use it in my interrupt handler (of course I push/pop all used 
 > registers):
 > 
 > movq (lapic_idregister), %rcx
 > movq (%rcx), %rcx

You're doing a 64-bit load from a 32-bit lapic register.

 > But on real hardware the last instruction seems to cause a page fault or 
 > something (SUSE with 2.6.37.6, Fedora with 2.6.38.6), the system simply 
 > reboots. Without this instruction, the handler is executed without any 
 > problems.
 > And in qemu with vanilla 2.6.37.6 and a buildroot system everything 
 > works fine!

If changing the above movq (%rcx), %rcx to use movl instead makes it
work in real HW, then you've found an accepts-invalid bug in qemu.

Re: Access to local APIC registers during an interrupt handler

[Robert Uhl]

On 12.06.2011 13:01, Mikael Pettersson wrote:
>   >  movq (lapic_idregister), %rcx
>   >  movq (%rcx), %rcx
>
> You're doing a 64-bit load from a 32-bit lapic register.
>
> If changing the above movq (%rcx), %rcx to use movl instead makes it
> work in real HW, then you've found an accepts-invalid bug in qemu.

It works with movl, thank you very much!
I will send a bug report to the qemu team.