powerpc/powernv: Call OPAL sync before kexec'ing

powerpc/powernv: Call OPAL sync before kexec'ing

[Benjamin Herrenschmidt]

From: Vasant Hegde <hegdevasant@linux.vnet.ibm.com>
    
Its possible that OPAL may be writing to host memory during
kexec (like dump retrieve scenario). In this situation we might
end up corrupting host memory.
    
This patch makes OPAL sync call to make sure OPAL stops
writing to host memory before kexec'ing.
    
Signed-off-by: Vasant Hegde <hegdevasant@linux.vnet.ibm.com>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---

diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h
index 0ac6f04..1920f70 100644
--- a/arch/powerpc/include/asm/opal.h
+++ b/arch/powerpc/include/asm/opal.h
@@ -162,6 +162,7 @@ extern int opal_enter_rtas(struct rtas_args *args,
 #define OPAL_DUMP_ACK				84
 #define OPAL_GET_MSG				85
 #define OPAL_CHECK_ASYNC_COMPLETION		86
+#define OPAL_SYNC_HOST_REBOOT			87
 
 #ifndef __ASSEMBLY__
 
@@ -841,6 +842,7 @@ int64_t opal_dump_read(uint32_t dump_id, uint64_t buffer);
 int64_t opal_dump_ack(uint32_t dump_id);
 int64_t opal_get_msg(uint64_t buffer, size_t size);
 int64_t opal_check_completion(uint64_t buffer, size_t size, uint64_t token);
+int64_t opal_sync_host_reboot(void);
 
 /* Internal functions */
 extern int early_init_dt_scan_opal(unsigned long node, const char *uname, int depth, void *data);
diff --git a/arch/powerpc/platforms/powernv/opal-wrappers.S b/arch/powerpc/platforms/powernv/opal-wrappers.S
index d58fcae..9e7ca21 100644
--- a/arch/powerpc/platforms/powernv/opal-wrappers.S
+++ b/arch/powerpc/platforms/powernv/opal-wrappers.S
@@ -127,3 +127,4 @@ OPAL_CALL(opal_dump_read,			OPAL_DUMP_READ);
 OPAL_CALL(opal_dump_ack,			OPAL_DUMP_ACK);
 OPAL_CALL(opal_get_msg,				OPAL_GET_MSG);
 OPAL_CALL(opal_check_completion,		OPAL_CHECK_ASYNC_COMPLETION);
+OPAL_CALL(opal_sync_host_reboot,		OPAL_SYNC_HOST_REBOOT);
diff --git a/arch/powerpc/platforms/powernv/opal.c b/arch/powerpc/platforms/powernv/opal.c
index 0ba1ccb..619b94a 100644
--- a/arch/powerpc/platforms/powernv/opal.c
+++ b/arch/powerpc/platforms/powernv/opal.c
@@ -21,6 +21,7 @@
 #include <linux/kobject.h>
 #include <linux/debugfs.h>
 #include <linux/uaccess.h>
+#include <linux/delay.h>
 #include <asm/opal.h>
 #include <asm/firmware.h>
 #include <asm/mce.h>
@@ -581,10 +582,25 @@ subsys_initcall(opal_init);
 void opal_shutdown(void)
 {
 	unsigned int i;
+	long rc = OPAL_BUSY;
 
+	/* First free interrupts, which will also mask them */
 	for (i = 0; i < opal_irq_count; i++) {
 		if (opal_irqs[i])
 			free_irq(opal_irqs[i], 0);
 		opal_irqs[i] = 0;
 	}
+
+	/*
+	 * Then sync with OPAL which ensure anything that can
+	 * potentially write to our memory has completed such
+	 * as an ongoing dump retrieval
+	 */
+	while (rc == OPAL_BUSY || rc == OPAL_BUSY_EVENT) {
+		rc = opal_sync_host_reboot();
+		if (rc == OPAL_BUSY)
+			opal_poll_events(NULL);
+		else
+			mdelay(10);
+	}
 }
diff --git a/arch/powerpc/platforms/powernv/setup.c b/arch/powerpc/platforms/powernv/setup.c
index ddc9690..86733d6 100644
--- a/arch/powerpc/platforms/powernv/setup.c
+++ b/arch/powerpc/platforms/powernv/setup.c
@@ -153,8 +153,10 @@ static void pnv_shutdown(void)
 	/* Let the PCI code clear up IODA tables */
 	pnv_pci_shutdown();
 
-	/* And unregister all OPAL interrupts so they don't fire
-	 * up while we kexec
+	/*
+	 * Stop OPAL activity: Unregister all OPAL interrupts so they
+	 * don't fire up while we kexec and make sure all potentially
+	 * DMA'ing ops are complete (such as dump retrieval).
 	 */
 	opal_shutdown();
 }

Re: powerpc/powernv: Call OPAL sync before kexec'ing

[Michael Ellerman]

On Wed, 2014-01-15 at 17:02 +1100, Benjamin Herrenschmidt wrote:
> From: Vasant Hegde <hegdevasant@linux.vnet.ibm.com>
>     
> Its possible that OPAL may be writing to host memory during
> kexec (like dump retrieve scenario). In this situation we might
> end up corrupting host memory.

Are we happy with that happening during kdump? (which doesn't call any of the
shutdown paths)

cheers

Re: powerpc/powernv: Call OPAL sync before kexec'ing

[Benjamin Herrenschmidt]

On Thu, 2014-01-16 at 11:58 +1100, Michael Ellerman wrote:
> On Wed, 2014-01-15 at 17:02 +1100, Benjamin Herrenschmidt wrote:
> > From: Vasant Hegde <hegdevasant@linux.vnet.ibm.com>
> >     
> > Its possible that OPAL may be writing to host memory during
> > kexec (like dump retrieve scenario). In this situation we might
> > end up corrupting host memory.
> 
> Are we happy with that happening during kdump? (which doesn't call any of the
> shutdown paths)

Obviously not ... There's a problem there. Not sure what the right fix
is. Might need some ifdef KDUMP to take over in the new driver.

Ben.