clockevents_shutdown vs pending interrupt

clockevents_shutdown vs pending interrupt

[Andriy Gapon]

Pardon if I am asking something obvious or silly...

tick_check_new_device() has the following code:

        if (tick_is_broadcast_device(curdev)) {
                clockevents_shutdown(curdev);
                curdev = NULL;
        }

and

void clockevents_shutdown(struct clock_event_device *dev)
{
        clockevents_set_mode(dev, CLOCK_EVT_MODE_SHUTDOWN);
        dev->next_event.tv64 = KTIME_MAX;
}

This is all done while interrupts are disabled on the current CPU.
But what if there is already a pending interrupt from the current source?
Is it possible that the timer interrupt would be processed by the device that
was put in the shutdown mode?

Some context: I am experiencing exactly the same symptoms as described here
http://thread.gmane.org/gmane.linux.kernel/1483297.  But I run a kernel where
that bug is fixed.  And my problem happens in a VM, so it's possible that there
are timing issues which are very unlikely on real hardware.


-- 
Andriy Gapon

Re: clockevents_shutdown vs pending interrupt

[Thomas Gleixner]

On Tue, 23 Jun 2015, Andriy Gapon wrote:
> Pardon if I am asking something obvious or silly...
> 
> tick_check_new_device() has the following code:
> 
>         if (tick_is_broadcast_device(curdev)) {
>                 clockevents_shutdown(curdev);
>                 curdev = NULL;
>         }
> 
> and
> 
> void clockevents_shutdown(struct clock_event_device *dev)
> {
>         clockevents_set_mode(dev, CLOCK_EVT_MODE_SHUTDOWN);
>         dev->next_event.tv64 = KTIME_MAX;
> }
> 
> This is all done while interrupts are disabled on the current CPU.
> But what if there is already a pending interrupt from the current source?
> Is it possible that the timer interrupt would be processed by the device that
> was put in the shutdown mode?
> 
> Some context: I am experiencing exactly the same symptoms as described here
> http://thread.gmane.org/gmane.linux.kernel/1483297.  But I run a kernel where
> that bug is fixed.  And my problem happens in a VM, so it's possible that there
> are timing issues which are very unlikely on real hardware.

Can you provide a full dmesg please?

Thanks,

	tglx

Re: clockevents_shutdown vs pending interrupt

[Andriy Gapon]

On 23/06/2015 13:15, Thomas Gleixner wrote:
> Can you provide a full dmesg please?

Thomas,

I've caught a couple of boot logs with different stack traces from unsuccessful
boot attempts and one from a successful attempt with exactly the same VM
configuration.
The logs are here:
https://people.freebsd.org/~avg/linux-boot-hang/

-- 
Andriy Gapon

Re: clockevents_shutdown vs pending interrupt

[Thomas Gleixner]

Andriy,

On Wed, 1 Jul 2015, Andriy Gapon wrote:
> I've caught a couple of boot logs with different stack traces from unsuccessful
> boot attempts and one from a successful attempt with exactly the same VM
> configuration.
> The logs are here:
> https://people.freebsd.org/~avg/linux-boot-hang/

I have to admit, that I'm thoroughly confused about that broadcast
check in the install path. Can you apply the debug patch below and
provide the output?

Thanks,

	tglx
---
diff --git a/kernel/time/tick-broadcast.c b/kernel/time/tick-broadcast.c
index d39f32cdd1b5..f1f921a49da9 100644
--- a/kernel/time/tick-broadcast.c
+++ b/kernel/time/tick-broadcast.c
@@ -100,6 +100,7 @@ void tick_install_broadcast_device(struct clock_event_device *dev)
 	if (cur)
 		cur->event_handler = clockevents_handle_noop;
 	tick_broadcast_device.evtdev = dev;
+	pr_err("Install broadcast device %p %s\n", dev, dev->name);
 	if (!cpumask_empty(tick_broadcast_mask))
 		tick_broadcast_start_periodic(dev);
 	/*
@@ -301,6 +302,13 @@ static void tick_handle_periodic_broadcast(struct clock_event_device *dev)
 	bool bc_local;
 
 	raw_spin_lock(&tick_broadcast_lock);
+	/* Handle spurious interrupt */
+	if (clockevent_state_shutdown(dev)) {
+		pr_err("Spurious broadcast event %p %s\n", dev, dev->name);
+		raw_spin_unlock(&tick_broadcast_lock);
+		return;
+	}
+
 	bc_local = tick_do_periodic_broadcast();
 
 	if (clockevent_state_oneshot(dev)) {
diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c
index 76446cb5dfe1..ecd439b2de7e 100644
--- a/kernel/time/tick-common.c
+++ b/kernel/time/tick-common.c
@@ -321,6 +321,7 @@ void tick_check_new_device(struct clock_event_device *newdev)
 	if (!try_module_get(newdev->owner))
 		return;
 
+	pr_err("Install per cpu tick device %p %s\n", newdev, newdev->name);
 	/*
 	 * Replace the eventually existing device by the new
 	 * device. If the current device is the broadcast device, do