[PATCH] powerpc/powernv: Support OPAL requested heartbeat

[PATCH] powerpc/powernv: Support OPAL requested heartbeat

[Benjamin Herrenschmidt]

If OPAL requests it, call it back via opal_poll_events() at a
regular interval. Some versions of OPAL on some machines require
this to operate some internal timeouts properly.

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---
 arch/powerpc/platforms/powernv/opal.c | 64 ++++++++++++++++++++++++++++++-----
 1 file changed, 55 insertions(+), 9 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
index f1e0d8c..0153064 100644
--- a/arch/powerpc/platforms/powernv/opal.c
+++ b/arch/powerpc/platforms/powernv/opal.c
@@ -22,6 +22,8 @@
 #include <linux/kobject.h>
 #include <linux/delay.h>
 #include <linux/memblock.h>
+#include <linux/kthread.h>
+#include <linux/freezer.h>
 
 #include <asm/machdep.h>
 #include <asm/opal.h>
@@ -58,6 +60,7 @@ static struct atomic_notifier_head opal_msg_notifier_head[OPAL_MSG_TYPE_MAX];
 static DEFINE_SPINLOCK(opal_notifier_lock);
 static uint64_t last_notified_mask = 0x0ul;
 static atomic_t opal_notifier_hold = ATOMIC_INIT(0);
+static uint32_t opal_heartbeat;
 
 static void opal_reinit_cores(void)
 {
@@ -633,17 +636,9 @@ static void opal_ipmi_init(struct device_node *opal_node)
 			of_platform_device_create(np, NULL, NULL);
 }
 
-static int __init opal_init(void)
+static void opal_console_create_devs(void)
 {
 	struct device_node *np, *consoles;
-	const __be32 *irqs;
-	int rc, i, irqlen;
-
-	opal_node = of_find_node_by_path("/ibm,opal");
-	if (!opal_node) {
-		pr_warn("opal: Node not found\n");
-		return -ENODEV;
-	}
 
 	/* Register OPAL consoles if any ports */
 	if (firmware_has_feature(FW_FEATURE_OPALv2))
@@ -659,6 +654,13 @@ static int __init opal_init(void)
 		of_node_put(consoles);
 	}
 
+}
+
+static void opal_request_interrupts(void)
+{
+	const __be32 *irqs;
+	int rc, i, irqlen;
+
 	/* Find all OPAL interrupts and request them */
 	irqs = of_get_property(opal_node, "opal-interrupts", &irqlen);
 	pr_debug("opal: Found %d interrupts reserved for OPAL\n",
@@ -678,6 +680,49 @@ static int __init opal_init(void)
 				   " (0x%x)\n", rc, irq, hwirq);
 		opal_irqs[i] = irq;
 	}
+}
+
+static int kopald(void *unused)
+{
+	set_freezable();
+	do {
+		try_to_freeze();
+		opal_poll_events(NULL);
+		msleep_interruptible(opal_heartbeat);
+	} while (!kthread_should_stop());
+
+	return 0;
+}
+
+static void opal_init_heartbeat(void)
+{
+	/* Old firwmware, we assume the HVC heartbeat is sufficient */
+	if (of_property_read_u32(opal_node, "ibm,heartbeat-freq",
+				 &opal_heartbeat) != 0)
+		opal_heartbeat = 0;
+
+	if (opal_heartbeat)
+		kthread_run(kopald, NULL, "kopald");
+}
+
+static int __init opal_init(void)
+{
+	int rc;
+
+	opal_node = of_find_node_by_path("/ibm,opal");
+	if (!opal_node) {
+		pr_warn("opal: Node not found\n");
+		return -ENODEV;
+	}
+
+	/* Setup a heatbeat thread if requested by OPAL */
+	opal_init_heartbeat();
+
+	/* Create console platform devices */
+	opal_console_create_devs();
+
+	/* Register OPAL interrupts */
+	opal_request_interrupts();
 
 	/* Create "opal" kobject under /sys/firmware */
 	rc = opal_sysfs_init();
@@ -696,6 +741,7 @@ static int __init opal_init(void)
 		opal_msglog_init();
 	}
 
+	/* Initialize OPAL IPMI backend */
 	opal_ipmi_init(opal_node);
 
 	return 0;

Re: powerpc/powernv: Support OPAL requested heartbeat

[Michael Ellerman]

On Wed, 2014-12-11 at 06:03:14 UTC, Benjamin Herrenschmidt wrote:
> If OPAL requests it, call it back via opal_poll_events() at a
> regular interval. Some versions of OPAL on some machines require
> this to operate some internal timeouts properly.
> 
> diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
> index f1e0d8c..0153064 100644
> --- a/arch/powerpc/platforms/powernv/opal.c
> +++ b/arch/powerpc/platforms/powernv/opal.c
> @@ -678,6 +680,49 @@ static int __init opal_init(void)
>  				   " (0x%x)\n", rc, irq, hwirq);
>  		opal_irqs[i] = irq;
>  	}
> +}
> +
> +static int kopald(void *unused)
> +{
> +	set_freezable();
> +	do {
> +		try_to_freeze();
> +		opal_poll_events(NULL);
> +		msleep_interruptible(opal_heartbeat);
> +	} while (!kthread_should_stop());
> +
> +	return 0;
> +}
> +
> +static void opal_init_heartbeat(void)
> +{
> +	/* Old firwmware, we assume the HVC heartbeat is sufficient */
> +	if (of_property_read_u32(opal_node, "ibm,heartbeat-freq",
> +				 &opal_heartbeat) != 0)

I'd assume "freq" with no units is in HZ, but looks like it's milliseconds. I
guess it's too late to rename it.

cheers

Re: powerpc/powernv: Support OPAL requested heartbeat

[Benjamin Herrenschmidt]

On Thu, 2014-11-13 at 16:29 +1100, Michael Ellerman wrote:
> I'd assume "freq" with no units is in HZ, but looks like it's
> milliseconds. I guess it's too late to rename it.

We still can, we haven't cut an official build with that fw, jk ?

Cheers,
Ben.

Re: powerpc/powernv: Support OPAL requested heartbeat

[Jeremy Kerr]

Hi Ben,

>> I'd assume "freq" with no units is in HZ, but looks like it's
>> milliseconds. I guess it's too late to rename it.
> 
> We still can, we haven't cut an official build with that fw, jk ?

Not yet. You got in by a matter of minutes :)

Can you send me an updated skiboot patch too then?

Cheers,


Jeremy

Re: [PATCH] powerpc/powernv: Support OPAL requested heartbeat

[Paul Clarke]

On 11/12/2014 12:03 AM, Benjamin Herrenschmidt wrote:
> If OPAL requests it, call it back via opal_poll_events() at a
> regular interval. Some versions of OPAL on some machines require
> this to operate some internal timeouts properly.

I don't see any reason why not, but wanted to ask to make sure: this 
daemon can be bound to a core, correct?  At least manually by taskset or 
similar?

It sounds a lot like the RTAS thread, which I was happy to see go away 
when running directly on OPAL, and don't want something else to take its 
place.  It was not bindable, but ran periodically on EVERY core, err 
hardware thread.

(Just looking to avoid latency blips.)

PC

Re: [PATCH] powerpc/powernv: Support OPAL requested heartbeat

[Benjamin Herrenschmidt]

On Thu, 2014-11-13 at 10:59 -0600, Paul Clarke wrote:
> On 11/12/2014 12:03 AM, Benjamin Herrenschmidt wrote:
> > If OPAL requests it, call it back via opal_poll_events() at a
> > regular interval. Some versions of OPAL on some machines require
> > this to operate some internal timeouts properly.
> 
> I don't see any reason why not, but wanted to ask to make sure: this 
> daemon can be bound to a core, correct?  At least manually by taskset or 
> similar?

I think so... it's a kernel thread, but I haven't tried. It should be
akin to the existing khvcd.

> It sounds a lot like the RTAS thread, which I was happy to see go away 
> when running directly on OPAL, and don't want something else to take its 
> place.  It was not bindable, but ran periodically on EVERY core, err 
> hardware thread.
> 
> (Just looking to avoid latency blips.)

Right, I know where you come from. But you'll get those blips whether I
use a thread or an interrupt. Basically, I'm trying to get the HW guys
to give me a modified SLW engine image that I can use to request time by
setting up a delayed interrupt.

I'm trying to reduce the work done in there, and in fact I'm hoping to
only schedule my internal timers that rely on this (or the SLW
interrupt) for cases where we have a pending i2c or IPMI command, and
possibly only on machines without a functional external interrupt (so
lab bringup) or when accessing Centaur i2c (very very rarely).

But I do need that source of time...

Cheers,
Ben.

Re: [PATCH] powerpc/powernv: Support OPAL requested heartbeat

[Alistair Popple]

Hi Ben,

On Wed, 12 Nov 2014 17:03:14 Benjamin Herrenschmidt wrote:

<snip>

> +
> +static int kopald(void *unused)
> +{
> +	set_freezable();
> +	do {
> +		try_to_freeze();
> +		opal_poll_events(NULL);

Do we need to check for outstanding events (and call opal_do_notifier)? The 
Linux OPAL-IPMI interface signals an event which the interrupt handler 
(opal_interrupt) checks for, but if interrupts aren't functional no event will 
be signalled and hence the Linux IPMI layer won't get a response.

It's probably only an issue for lab bring up though as I would expect 
interrupts to generally be working...

> +		msleep_interruptible(opal_heartbeat);
> +	} while (!kthread_should_stop());
> +
> +	return 0;
> +}
> +

Regards,

Alistair

Re: [PATCH] powerpc/powernv: Support OPAL requested heartbeat

[Benjamin Herrenschmidt]

On Fri, 2014-11-14 at 14:01 +1100, Alistair Popple wrote:
> Hi Ben,
> 
> On Wed, 12 Nov 2014 17:03:14 Benjamin Herrenschmidt wrote:
> 
> <snip>
> 
> > +
> > +static int kopald(void *unused)
> > +{
> > +	set_freezable();
> > +	do {
> > +		try_to_freeze();
> > +		opal_poll_events(NULL);
> 
> Do we need to check for outstanding events (and call opal_do_notifier)? The 
> Linux OPAL-IPMI interface signals an event which the interrupt handler 
> (opal_interrupt) checks for, but if interrupts aren't functional no event will 
> be signalled and hence the Linux IPMI layer won't get a response.
> 
> It's probably only an issue for lab bring up though as I would expect 
> interrupts to generally be working...

Well, it would somewhat double up with the OCC interrupt ... OPAL will
sent a local OCC interrupt if we change the event mask as a result of a
call that isn't opal_handle_interrupts() ... but yes, I suppose if that
doesn't work either... though this is not known to be problematic unless
the external one.

> > +		msleep_interruptible(opal_heartbeat);
> > +	} while (!kthread_should_stop());
> > +
> > +	return 0;
> > +}
> > +
> 
> Regards,
> 
> Alistair