* [patch 1/8] x86: UV - Introduce a means to translate from gpa -> socket_paddr.
2009-11-19 17:31 [patch 0/8] x86: UV - XPC fixes with related support functionality V2 Robin Holt
@ 2009-11-19 17:31 ` Robin Holt
2009-11-19 17:31 ` [patch 2/8] x86: UV - XPC needs to provide an abstraction for uv_gpa Robin Holt
` (7 subsequent siblings)
8 siblings, 0 replies; 16+ messages in thread
From: Robin Holt @ 2009-11-19 17:31 UTC (permalink / raw)
To: Ingo Molnar, tglx; +Cc: Robin Holt, Jack Steiner, linux-kernel
[-- Attachment #1: uv_gpa_to_soc_phys_ram --]
[-- Type: text/plain, Size: 1416 bytes --]
For SGI UV systems, translate from a global physical address back to
a socket physical address. This does nothing to ensure the socket
physical address is actually addressable by the kernel. That is the
responsibility of the user of the function.
To: Ingo Molnar <mingo@elte.hu>
To: tglx@linutronix.de
Signed-off-by: Robin Holt <holt@sgi.com>
Cc: Jack Steiner <steiner@sgi.com>
Cc: lkml <linux-kernel@vger.kernel.org>
---
arch/x86/include/asm/uv/uv_hub.h | 13 +++++++++++++
1 file changed, 13 insertions(+)
Index: linux-x86/arch/x86/include/asm/uv/uv_hub.h
===================================================================
--- linux-x86.orig/arch/x86/include/asm/uv/uv_hub.h 2009-11-09 08:46:25.000000000 -0600
+++ linux-x86/arch/x86/include/asm/uv/uv_hub.h 2009-11-09 08:50:25.000000000 -0600
@@ -232,6 +232,19 @@ static inline unsigned long uv_gpa(void
return uv_soc_phys_ram_to_gpa(__pa(v));
}
+/* UV global physical address --> socket phys RAM */
+static inline unsigned long uv_gpa_to_soc_phys_ram(unsigned long gpa)
+{
+ unsigned long paddr = gpa & uv_hub_info->gpa_mask;
+ unsigned long remap_base = uv_hub_info->lowmem_remap_base;
+ unsigned long remap_top = uv_hub_info->lowmem_remap_top;
+
+ if (paddr >= remap_base && paddr < remap_base + remap_top)
+ paddr -= remap_base;
+ return paddr;
+}
+
+
/* gnode -> pnode */
static inline unsigned long uv_gpa_to_gnode(unsigned long gpa)
{
^ permalink raw reply [flat|nested] 16+ messages in thread* [patch 2/8] x86: UV - XPC needs to provide an abstraction for uv_gpa.
2009-11-19 17:31 [patch 0/8] x86: UV - XPC fixes with related support functionality V2 Robin Holt
2009-11-19 17:31 ` [patch 1/8] x86: UV - Introduce a means to translate from gpa -> socket_paddr Robin Holt
@ 2009-11-19 17:31 ` Robin Holt
2009-11-19 17:32 ` [patch 3/8] x86: UV - Introduce uv_gpa_is_mmr Robin Holt
` (6 subsequent siblings)
8 siblings, 0 replies; 16+ messages in thread
From: Robin Holt @ 2009-11-19 17:31 UTC (permalink / raw)
To: Ingo Molnar, tglx; +Cc: Robin Holt, Jack Steiner, linux-kernel
[-- Attachment #1: xpc_introduce_xp_socket_pa --]
[-- Type: text/plain, Size: 3486 bytes --]
Provide an SGI SN2/UV agnositic method for converting a global physical
address into a socket physical address.
To: Ingo Molnar <mingo@elte.hu>
To: tglx@linutronix.de
Signed-off-by: Robin Holt <holt@sgi.com>
Cc: Jack Steiner <steiner@sgi.com>
Cc: lkml <linux-kernel@vger.kernel.org>
---
drivers/misc/sgi-xp/xp.h | 1 +
drivers/misc/sgi-xp/xp_main.c | 3 +++
drivers/misc/sgi-xp/xp_sn2.c | 10 ++++++++++
drivers/misc/sgi-xp/xp_uv.c | 10 ++++++++++
4 files changed, 24 insertions(+)
Index: linux-2.6.27/drivers/misc/sgi-xp/xp.h
===================================================================
--- linux-2.6.27.orig/drivers/misc/sgi-xp/xp.h 2009-10-29 10:15:58.000000000 -0500
+++ linux-2.6.27/drivers/misc/sgi-xp/xp.h 2009-10-29 16:19:25.000000000 -0500
@@ -339,6 +339,7 @@ extern short xp_partition_id;
extern u8 xp_region_size;
extern unsigned long (*xp_pa) (void *);
+extern unsigned long (*xp_socket_pa) (unsigned long);
extern enum xp_retval (*xp_remote_memcpy) (unsigned long, const unsigned long,
size_t);
extern int (*xp_cpu_to_nasid) (int);
Index: linux-2.6.27/drivers/misc/sgi-xp/xp_main.c
===================================================================
--- linux-2.6.27.orig/drivers/misc/sgi-xp/xp_main.c 2009-10-29 09:16:19.000000000 -0500
+++ linux-2.6.27/drivers/misc/sgi-xp/xp_main.c 2009-10-29 16:20:01.000000000 -0500
@@ -44,6 +44,9 @@ EXPORT_SYMBOL_GPL(xp_region_size);
unsigned long (*xp_pa) (void *addr);
EXPORT_SYMBOL_GPL(xp_pa);
+unsigned long (*xp_socket_pa) (unsigned long gpa);
+EXPORT_SYMBOL_GPL(xp_socket_pa);
+
enum xp_retval (*xp_remote_memcpy) (unsigned long dst_gpa,
const unsigned long src_gpa, size_t len);
EXPORT_SYMBOL_GPL(xp_remote_memcpy);
Index: linux-2.6.27/drivers/misc/sgi-xp/xp_sn2.c
===================================================================
--- linux-2.6.27.orig/drivers/misc/sgi-xp/xp_sn2.c 2009-10-29 09:16:11.000000000 -0500
+++ linux-2.6.27/drivers/misc/sgi-xp/xp_sn2.c 2009-10-29 16:21:18.000000000 -0500
@@ -84,6 +84,15 @@ xp_pa_sn2(void *addr)
}
/*
+ * Convert a global physical to a socket physical address.
+ */
+static unsigned long
+xp_socket_pa_sn2(unsigned long gpa)
+{
+ return gpa;
+}
+
+/*
* Wrapper for bte_copy().
*
* dst_pa - physical address of the destination of the transfer.
@@ -162,6 +171,7 @@ xp_init_sn2(void)
xp_region_size = sn_region_size;
xp_pa = xp_pa_sn2;
+ xp_socket_pa = xp_socket_pa_sn2;
xp_remote_memcpy = xp_remote_memcpy_sn2;
xp_cpu_to_nasid = xp_cpu_to_nasid_sn2;
xp_expand_memprotect = xp_expand_memprotect_sn2;
Index: linux-2.6.27/drivers/misc/sgi-xp/xp_uv.c
===================================================================
--- linux-2.6.27.orig/drivers/misc/sgi-xp/xp_uv.c 2009-10-29 16:15:05.000000000 -0500
+++ linux-2.6.27/drivers/misc/sgi-xp/xp_uv.c 2009-10-29 16:24:56.000000000 -0500
@@ -32,6 +32,15 @@ xp_pa_uv(void *addr)
return uv_gpa(addr);
}
+/*
+ * Convert a global physical to socket physical address.
+ */
+static unsigned long
+xp_socket_pa_uv(unsigned long gpa)
+{
+ return uv_gpa_to_soc_phys_ram(gpa);
+}
+
static enum xp_retval
xp_remote_memcpy_uv(unsigned long dst_gpa, const unsigned long src_gpa,
size_t len)
@@ -123,6 +132,7 @@ xp_init_uv(void)
xp_region_size = sn_region_size;
xp_pa = xp_pa_uv;
+ xp_socket_pa = xp_socket_pa_uv;
xp_remote_memcpy = xp_remote_memcpy_uv;
xp_cpu_to_nasid = xp_cpu_to_nasid_uv;
xp_expand_memprotect = xp_expand_memprotect_uv;
^ permalink raw reply [flat|nested] 16+ messages in thread* [patch 3/8] x86: UV - Introduce uv_gpa_is_mmr.
2009-11-19 17:31 [patch 0/8] x86: UV - XPC fixes with related support functionality V2 Robin Holt
2009-11-19 17:31 ` [patch 1/8] x86: UV - Introduce a means to translate from gpa -> socket_paddr Robin Holt
2009-11-19 17:31 ` [patch 2/8] x86: UV - XPC needs to provide an abstraction for uv_gpa Robin Holt
@ 2009-11-19 17:32 ` Robin Holt
2009-11-19 17:32 ` [patch 4/8] X86: UV - Implement a gru_read_gpa kernel function Robin Holt
` (5 subsequent siblings)
8 siblings, 0 replies; 16+ messages in thread
From: Robin Holt @ 2009-11-19 17:32 UTC (permalink / raw)
To: Ingo Molnar, tglx; +Cc: Robin Holt, Jack Steiner, linux-kernel
[-- Attachment #1: uv_gpa_is_mmr_space --]
[-- Type: text/plain, Size: 1049 bytes --]
Provide a mechanism for determining if a global physical address is
pointing to a UV hub MMR.
To: Ingo Molnar <mingo@elte.hu>
To: tglx@linutronix.de
Signed-off-by: Robin Holt <holt@sgi.com>
Cc: Jack Steiner <steiner@sgi.com>
Cc: lkml <linux-kernel@vger.kernel.org>
---
arch/x86/include/asm/uv/uv_hub.h | 7 +++++++
1 file changed, 7 insertions(+)
Index: linux-x86/arch/x86/include/asm/uv/uv_hub.h
===================================================================
--- linux-x86.orig/arch/x86/include/asm/uv/uv_hub.h 2009-11-09 08:50:25.000000000 -0600
+++ linux-x86/arch/x86/include/asm/uv/uv_hub.h 2009-11-09 08:52:14.000000000 -0600
@@ -232,6 +232,13 @@ static inline unsigned long uv_gpa(void
return uv_soc_phys_ram_to_gpa(__pa(v));
}
+/* Top two bits indicate the requested address is in MMR space. */
+static inline int
+uv_gpa_in_mmr_space(unsigned long gpa)
+{
+ return (gpa >> 62) == 0x3UL;
+}
+
/* UV global physical address --> socket phys RAM */
static inline unsigned long uv_gpa_to_soc_phys_ram(unsigned long gpa)
{
^ permalink raw reply [flat|nested] 16+ messages in thread* [patch 4/8] X86: UV - Implement a gru_read_gpa kernel function.
2009-11-19 17:31 [patch 0/8] x86: UV - XPC fixes with related support functionality V2 Robin Holt
` (2 preceding siblings ...)
2009-11-19 17:32 ` [patch 3/8] x86: UV - Introduce uv_gpa_is_mmr Robin Holt
@ 2009-11-19 17:32 ` Robin Holt
2009-11-19 17:32 ` [patch 5/8] x86: UV - Update XPC to handle updated BIOS interface Robin Holt
` (4 subsequent siblings)
8 siblings, 0 replies; 16+ messages in thread
From: Robin Holt @ 2009-11-19 17:32 UTC (permalink / raw)
To: Ingo Molnar, tglx; +Cc: Robin Holt, Jack Steiner, linux-kernel
[-- Attachment #1: gru_read_gpa --]
[-- Type: text/plain, Size: 4734 bytes --]
The BIOS has decided to store a pointer to the partition reserved page
in a scratch MMR. The GRU is only able to read an MMR using a vload
instruction. The gru_read_gpa() function will implemented.
To: Ingo Molnar <mingo@elte.hu>
To: tglx@linutronix.de
Signed-off-by: Robin Holt <holt@sgi.com>
Signed-off-by: Jack Steiner <steiner@sgi.com>
Cc: linux-kernel@vger.kernel.org
---
drivers/misc/sgi-gru/gru_instructions.h | 13 +++++++++++++
drivers/misc/sgi-gru/grukservices.c | 23 +++++++++++++++++++++++
drivers/misc/sgi-gru/grukservices.h | 14 ++++++++++++++
drivers/misc/sgi-gru/gruprocfs.c | 1 +
drivers/misc/sgi-gru/grutables.h | 1 +
5 files changed, 52 insertions(+)
Index: linux-x86/drivers/misc/sgi-gru/gru_instructions.h
===================================================================
--- linux-x86.orig/drivers/misc/sgi-gru/gru_instructions.h 2009-10-17 03:59:22.000000000 -0500
+++ linux-x86/drivers/misc/sgi-gru/gru_instructions.h 2009-11-18 21:21:31.000000000 -0600
@@ -340,6 +340,19 @@ static inline void gru_start_instruction
* - nelem and stride are in elements
* - tri0/tri1 is in bytes for the beginning of the data segment.
*/
+static inline void gru_vload_phys(void *cb, unsigned long gpa,
+ unsigned int tri0, int iaa, unsigned long hints)
+{
+ struct gru_instruction *ins = (struct gru_instruction *)cb;
+
+ ins->baddr0 = (long)gpa | ((unsigned long)iaa << 62);
+ ins->nelem = 1;
+ ins->tri0 = tri0;
+ ins->op1_stride = 1;
+ gru_start_instruction(ins, __opword(OP_VLOAD, 0, XTYPE_DW, iaa, 0,
+ CB_IMA(hints)));
+}
+
static inline void gru_vload(void *cb, unsigned long mem_addr,
unsigned int tri0, unsigned char xtype, unsigned long nelem,
unsigned long stride, unsigned long hints)
Index: linux-x86/drivers/misc/sgi-gru/grukservices.c
===================================================================
--- linux-x86.orig/drivers/misc/sgi-gru/grukservices.c 2009-10-17 03:59:22.000000000 -0500
+++ linux-x86/drivers/misc/sgi-gru/grukservices.c 2009-11-18 21:04:40.000000000 -0600
@@ -858,6 +858,29 @@ EXPORT_SYMBOL_GPL(gru_get_next_message);
/* ---------------------- GRU DATA COPY FUNCTIONS ---------------------------*/
/*
+ * Load a DW from a global GPA. The GPA can be a memory or MMR address.
+ */
+int gru_read_gpa(unsigned long *value, unsigned long gpa)
+{
+ void *cb;
+ void *dsr;
+ int ret, iaa;
+
+ STAT(read_gpa);
+ if (gru_get_cpu_resources(GRU_NUM_KERNEL_DSR_BYTES, &cb, &dsr))
+ return MQE_BUG_NO_RESOURCES;
+ iaa = gpa >> 62;
+ gru_vload_phys(cb, gpa, gru_get_tri(dsr), iaa, IMA);
+ ret = gru_wait(cb);
+ if (ret == CBS_IDLE)
+ *value = *(unsigned long *)dsr;
+ gru_free_cpu_resources(cb, dsr);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(gru_read_gpa);
+
+
+/*
* Copy a block of data using the GRU resources
*/
int gru_copy_gpa(unsigned long dest_gpa, unsigned long src_gpa,
Index: linux-x86/drivers/misc/sgi-gru/grukservices.h
===================================================================
--- linux-x86.orig/drivers/misc/sgi-gru/grukservices.h 2009-10-17 03:59:22.000000000 -0500
+++ linux-x86/drivers/misc/sgi-gru/grukservices.h 2009-11-18 21:03:14.000000000 -0600
@@ -131,6 +131,20 @@ extern void *gru_get_next_message(struct
/*
+ * Read a GRU global GPA. Source can be located in a remote partition.
+ *
+ * Input:
+ * value memory address where MMR value is returned
+ * gpa source numalink physical address of GPA
+ *
+ * Output:
+ * 0 OK
+ * >0 error
+ */
+int gru_read_gpa(unsigned long *value, unsigned long gpa);
+
+
+/*
* Copy data using the GRU. Source or destination can be located in a remote
* partition.
*
Index: linux-x86/drivers/misc/sgi-gru/gruprocfs.c
===================================================================
--- linux-x86.orig/drivers/misc/sgi-gru/gruprocfs.c 2009-11-18 13:45:38.000000000 -0600
+++ linux-x86/drivers/misc/sgi-gru/gruprocfs.c 2009-11-18 21:11:34.000000000 -0600
@@ -98,6 +98,7 @@ static int statistics_show(struct seq_fi
printstat(s, flush_tlb_gru_tgh);
printstat(s, flush_tlb_gru_zero_asid);
printstat(s, copy_gpa);
+ printstat(s, read_gpa);
printstat(s, mesq_receive);
printstat(s, mesq_receive_none);
printstat(s, mesq_send);
Index: linux-x86/drivers/misc/sgi-gru/grutables.h
===================================================================
--- linux-x86.orig/drivers/misc/sgi-gru/grutables.h 2009-10-17 03:59:22.000000000 -0500
+++ linux-x86/drivers/misc/sgi-gru/grutables.h 2009-11-18 21:07:58.000000000 -0600
@@ -224,6 +224,7 @@ struct gru_stats_s {
atomic_long_t flush_tlb_gru_zero_asid;
atomic_long_t copy_gpa;
+ atomic_long_t read_gpa;
atomic_long_t mesq_receive;
atomic_long_t mesq_receive_none;
^ permalink raw reply [flat|nested] 16+ messages in thread* [patch 5/8] x86: UV - Update XPC to handle updated BIOS interface.
2009-11-19 17:31 [patch 0/8] x86: UV - XPC fixes with related support functionality V2 Robin Holt
` (3 preceding siblings ...)
2009-11-19 17:32 ` [patch 4/8] X86: UV - Implement a gru_read_gpa kernel function Robin Holt
@ 2009-11-19 17:32 ` Robin Holt
2009-11-19 17:32 ` [patch 6/8] x86: UV - XPC NULL deref when mesq becomes empty Robin Holt
` (3 subsequent siblings)
8 siblings, 0 replies; 16+ messages in thread
From: Robin Holt @ 2009-11-19 17:32 UTC (permalink / raw)
To: Ingo Molnar, tglx; +Cc: Robin Holt, Jack Steiner, linux-kernel
[-- Attachment #1: xpc_uv_bios_changes --]
[-- Type: text/plain, Size: 7522 bytes --]
The UV BIOS has moved the location of some of their pointers to the
"partition reserved page" from memory into a uv hub MMR. The GRU does
not support bcopy operations from MMR space so we need to special case
the MMR addresses using VLOAD operations.
Additionally, the BIOS call for registering a message queue watchlist
has removed the 'blade' value and eliminated the structure that was
being passed in. This is also reflected in this patch.
To: Ingo Molnar <mingo@elte.hu>
To: tglx@linutronix.de
Signed-off-by: Robin Holt <holt@sgi.com>
Cc: Jack Steiner <steiner@sgi.com>
Cc: lkml <linux-kernel@vger.kernel.org>
---
arch/x86/include/asm/uv/bios.h | 11 +----------
arch/x86/kernel/bios_uv.c | 8 ++------
drivers/misc/sgi-xp/xp_uv.c | 23 +++++++++++++++++++++++
drivers/misc/sgi-xp/xpc_partition.c | 13 +++++++++----
drivers/misc/sgi-xp/xpc_uv.c | 27 +++++++++++++++------------
5 files changed, 50 insertions(+), 32 deletions(-)
Index: linux-x86/arch/x86/include/asm/uv/bios.h
===================================================================
--- linux-x86.orig/arch/x86/include/asm/uv/bios.h 2009-10-17 03:59:21.000000000 -0500
+++ linux-x86/arch/x86/include/asm/uv/bios.h 2009-11-18 21:37:58.000000000 -0600
@@ -76,15 +76,6 @@ union partition_info_u {
};
};
-union uv_watchlist_u {
- u64 val;
- struct {
- u64 blade : 16,
- size : 32,
- filler : 16;
- };
-};
-
enum uv_memprotect {
UV_MEMPROT_RESTRICT_ACCESS,
UV_MEMPROT_ALLOW_AMO,
@@ -100,7 +91,7 @@ extern s64 uv_bios_call_reentrant(enum u
extern s64 uv_bios_get_sn_info(int, int *, long *, long *, long *);
extern s64 uv_bios_freq_base(u64, u64 *);
-extern int uv_bios_mq_watchlist_alloc(int, unsigned long, unsigned int,
+extern int uv_bios_mq_watchlist_alloc(unsigned long, unsigned int,
unsigned long *);
extern int uv_bios_mq_watchlist_free(int, int);
extern s64 uv_bios_change_memprotect(u64, u64, enum uv_memprotect);
Index: linux-x86/arch/x86/kernel/bios_uv.c
===================================================================
--- linux-x86.orig/arch/x86/kernel/bios_uv.c 2009-10-17 03:59:21.000000000 -0500
+++ linux-x86/arch/x86/kernel/bios_uv.c 2009-11-18 21:36:45.000000000 -0600
@@ -101,21 +101,17 @@ s64 uv_bios_get_sn_info(int fc, int *uvt
}
int
-uv_bios_mq_watchlist_alloc(int blade, unsigned long addr, unsigned int mq_size,
+uv_bios_mq_watchlist_alloc(unsigned long addr, unsigned int mq_size,
unsigned long *intr_mmr_offset)
{
- union uv_watchlist_u size_blade;
u64 watchlist;
s64 ret;
- size_blade.size = mq_size;
- size_blade.blade = blade;
-
/*
* bios returns watchlist number or negative error number.
*/
ret = (int)uv_bios_call_irqsave(UV_BIOS_WATCHLIST_ALLOC, addr,
- size_blade.val, (u64)intr_mmr_offset,
+ mq_size, (u64)intr_mmr_offset,
(u64)&watchlist, 0);
if (ret < BIOS_STATUS_SUCCESS)
return ret;
Index: linux-x86/drivers/misc/sgi-xp/xp_uv.c
===================================================================
--- linux-x86.orig/drivers/misc/sgi-xp/xp_uv.c 2009-11-18 21:31:01.000000000 -0600
+++ linux-x86/drivers/misc/sgi-xp/xp_uv.c 2009-11-18 21:31:20.000000000 -0600
@@ -42,11 +42,34 @@ xp_socket_pa_uv(unsigned long gpa)
}
static enum xp_retval
+xp_remote_mmr_read(unsigned long dst_gpa, const unsigned long src_gpa,
+ size_t len)
+{
+ int ret;
+ unsigned long *dst_va = __va(uv_gpa_to_soc_phys_ram(dst_gpa));
+
+ BUG_ON(!uv_gpa_in_mmr_space(src_gpa));
+ BUG_ON(len != 8);
+
+ ret = gru_read_gpa(dst_va, src_gpa);
+ if (ret == 0)
+ return xpSuccess;
+
+ dev_err(xp, "gru_read_gpa() failed, dst_gpa=0x%016lx src_gpa=0x%016lx "
+ "len=%ld\n", dst_gpa, src_gpa, len);
+ return xpGruCopyError;
+}
+
+
+static enum xp_retval
xp_remote_memcpy_uv(unsigned long dst_gpa, const unsigned long src_gpa,
size_t len)
{
int ret;
+ if (uv_gpa_in_mmr_space(src_gpa))
+ return xp_remote_mmr_read(dst_gpa, src_gpa, len);
+
ret = gru_copy_gpa(dst_gpa, src_gpa, len);
if (ret == 0)
return xpSuccess;
Index: linux-x86/drivers/misc/sgi-xp/xpc_partition.c
===================================================================
--- linux-x86.orig/drivers/misc/sgi-xp/xpc_partition.c 2009-11-18 21:31:01.000000000 -0600
+++ linux-x86/drivers/misc/sgi-xp/xpc_partition.c 2009-11-18 21:31:20.000000000 -0600
@@ -18,6 +18,7 @@
#include <linux/device.h>
#include <linux/hardirq.h>
#include "xpc.h"
+#include <asm/uv/uv_hub.h>
/* XPC is exiting flag */
int xpc_exiting;
@@ -92,8 +93,12 @@ xpc_get_rsvd_page_pa(int nasid)
break;
/* !!! L1_CACHE_ALIGN() is only a sn2-bte_copy requirement */
- if (L1_CACHE_ALIGN(len) > buf_len) {
- kfree(buf_base);
+ if (is_shub())
+ len = L1_CACHE_ALIGN(len);
+
+ if (len > buf_len) {
+ if (buf_base != NULL)
+ kfree(buf_base);
buf_len = L1_CACHE_ALIGN(len);
buf = xpc_kmalloc_cacheline_aligned(buf_len, GFP_KERNEL,
&buf_base);
@@ -105,7 +110,7 @@ xpc_get_rsvd_page_pa(int nasid)
}
}
- ret = xp_remote_memcpy(xp_pa(buf), rp_pa, buf_len);
+ ret = xp_remote_memcpy(xp_pa(buf), rp_pa, len);
if (ret != xpSuccess) {
dev_dbg(xpc_part, "xp_remote_memcpy failed %d\n", ret);
break;
@@ -143,7 +148,7 @@ xpc_setup_rsvd_page(void)
dev_err(xpc_part, "SAL failed to locate the reserved page\n");
return -ESRCH;
}
- rp = (struct xpc_rsvd_page *)__va(rp_pa);
+ rp = (struct xpc_rsvd_page *)__va(xp_socket_pa(rp_pa));
if (rp->SAL_version < 3) {
/* SAL_versions < 3 had a SAL_partid defined as a u8 */
Index: linux-x86/drivers/misc/sgi-xp/xpc_uv.c
===================================================================
--- linux-x86.orig/drivers/misc/sgi-xp/xpc_uv.c 2009-11-18 21:31:01.000000000 -0600
+++ linux-x86/drivers/misc/sgi-xp/xpc_uv.c 2009-11-18 21:42:40.000000000 -0600
@@ -157,22 +157,24 @@ xpc_gru_mq_watchlist_alloc_uv(struct xpc
{
int ret;
-#if defined CONFIG_X86_64
- ret = uv_bios_mq_watchlist_alloc(mq->mmr_blade, uv_gpa(mq->address),
- mq->order, &mq->mmr_offset);
- if (ret < 0) {
- dev_err(xpc_part, "uv_bios_mq_watchlist_alloc() failed, "
- "ret=%d\n", ret);
- return ret;
- }
-#elif defined CONFIG_IA64_GENERIC || defined CONFIG_IA64_SGI_UV
- ret = sn_mq_watchlist_alloc(mq->mmr_blade, (void *)uv_gpa(mq->address),
+#if defined CONFIG_IA64_GENERIC || defined CONFIG_IA64_SGI_UV
+ int mmr_pnode = uv_blade_to_pnode(mq->mmr_blade);
+
+ ret = sn_mq_watchlist_alloc(mmr_pnode, (void *)uv_gpa(mq->address),
mq->order, &mq->mmr_offset);
if (ret < 0) {
dev_err(xpc_part, "sn_mq_watchlist_alloc() failed, ret=%d\n",
ret);
return -EBUSY;
}
+#elif defined CONFIG_X86_64
+ ret = uv_bios_mq_watchlist_alloc(uv_gpa(mq->address),
+ mq->order, &mq->mmr_offset);
+ if (ret < 0) {
+ dev_err(xpc_part, "uv_bios_mq_watchlist_alloc() failed, "
+ "ret=%d\n", ret);
+ return ret;
+ }
#else
#error not a supported configuration
#endif
@@ -185,12 +187,13 @@ static void
xpc_gru_mq_watchlist_free_uv(struct xpc_gru_mq_uv *mq)
{
int ret;
+ int mmr_pnode = uv_blade_to_pnode(mq->mmr_blade);
#if defined CONFIG_X86_64
- ret = uv_bios_mq_watchlist_free(mq->mmr_blade, mq->watchlist_num);
+ ret = uv_bios_mq_watchlist_free(mmr_pnode, mq->watchlist_num);
BUG_ON(ret != BIOS_STATUS_SUCCESS);
#elif defined CONFIG_IA64_GENERIC || defined CONFIG_IA64_SGI_UV
- ret = sn_mq_watchlist_free(mq->mmr_blade, mq->watchlist_num);
+ ret = sn_mq_watchlist_free(mmr_pnode, mq->watchlist_num);
BUG_ON(ret != SALRET_OK);
#else
#error not a supported configuration
^ permalink raw reply [flat|nested] 16+ messages in thread* [patch 6/8] x86: UV - XPC NULL deref when mesq becomes empty.
2009-11-19 17:31 [patch 0/8] x86: UV - XPC fixes with related support functionality V2 Robin Holt
` (4 preceding siblings ...)
2009-11-19 17:32 ` [patch 5/8] x86: UV - Update XPC to handle updated BIOS interface Robin Holt
@ 2009-11-19 17:32 ` Robin Holt
2009-11-19 17:32 ` [patch 7/8] X86: UV - xpc_make_first_contact hang due to not accepting ACTIVE state Robin Holt
` (2 subsequent siblings)
8 siblings, 0 replies; 16+ messages in thread
From: Robin Holt @ 2009-11-19 17:32 UTC (permalink / raw)
To: Ingo Molnar, tglx; +Cc: Robin Holt, Jack Steiner, linux-kernel
[-- Attachment #1: xpc_fix_xpc_get_fifo_entry_uv --]
[-- Type: text/plain, Size: 1107 bytes --]
Under heavy load conditions, our set of xpc messages may become exhausted.
The code handles this correctly with the exception of the management
code which hits a NULL pointer dereference.
To: Ingo Molnar <mingo@elte.hu>
To: tglx@linutronix.de
Signed-off-by: Robin Holt <holt@sgi.com>
Cc: Jack Steiner <steiner@sgi.com>
Cc: linux-kernel@vger.kernel.org
---
drivers/misc/sgi-xp/xpc_uv.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
Index: linux-x86/drivers/misc/sgi-xp/xpc_uv.c
===================================================================
--- linux-x86.orig/drivers/misc/sgi-xp/xpc_uv.c 2009-11-18 21:41:23.000000000 -0600
+++ linux-x86/drivers/misc/sgi-xp/xpc_uv.c 2009-11-18 21:41:26.000000000 -0600
@@ -949,11 +949,13 @@ xpc_get_fifo_entry_uv(struct xpc_fifo_he
head->first = first->next;
if (head->first == NULL)
head->last = NULL;
+
+ head->n_entries--;
+ BUG_ON(head->n_entries < 0);
+
+ first->next = NULL;
}
- head->n_entries--;
- BUG_ON(head->n_entries < 0);
spin_unlock_irqrestore(&head->lock, irq_flags);
- first->next = NULL;
return first;
}
^ permalink raw reply [flat|nested] 16+ messages in thread* [patch 7/8] X86: UV - xpc_make_first_contact hang due to not accepting ACTIVE state.
2009-11-19 17:31 [patch 0/8] x86: UV - XPC fixes with related support functionality V2 Robin Holt
` (5 preceding siblings ...)
2009-11-19 17:32 ` [patch 6/8] x86: UV - XPC NULL deref when mesq becomes empty Robin Holt
@ 2009-11-19 17:32 ` Robin Holt
2009-11-19 17:32 ` [patch 8/8] X86: UV - XPC receive message reuse triggers invalid BUG_ON() Robin Holt
2009-11-23 18:03 ` [patch 0/8] x86: UV - XPC fixes with related support functionality V2 Ingo Molnar
8 siblings, 0 replies; 16+ messages in thread
From: Robin Holt @ 2009-11-19 17:32 UTC (permalink / raw)
To: Ingo Molnar, tglx; +Cc: Robin Holt, Jack Steiner, linux-kernel
[-- Attachment #1: xpc_first_contact_when_active --]
[-- Type: text/plain, Size: 1404 bytes --]
Many times while the initial connection is being made, the
contacted partition will send back both the ACTIVATING and the ACTIVE
remote_act_state changes in very close succescion. The 1/4 second delay
in the make first contact loop is large enough to nearly always miss
the ACTIVATING state change.
Since either state indicates the remote partition has acknowledged our
state change, accept either.
To: Ingo Molnar <mingo@elte.hu>
To: tglx@linutronix.de
Signed-off-by: Robin Holt <holt@sgi.com>
Cc: Jack Steiner <steiner@sgi.com>
Cc: linux-kernel@vger.kernel.org
---
drivers/misc/sgi-xp/xpc_uv.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
Index: linux-x86/drivers/misc/sgi-xp/xpc_uv.c
===================================================================
--- linux-x86.orig/drivers/misc/sgi-xp/xpc_uv.c 2009-11-18 21:54:46.000000000 -0600
+++ linux-x86/drivers/misc/sgi-xp/xpc_uv.c 2009-11-18 21:55:18.000000000 -0600
@@ -1024,7 +1024,8 @@ xpc_make_first_contact_uv(struct xpc_par
xpc_send_activate_IRQ_part_uv(part, &msg, sizeof(msg),
XPC_ACTIVATE_MQ_MSG_SYNC_ACT_STATE_UV);
- while (part->sn.uv.remote_act_state != XPC_P_AS_ACTIVATING) {
+ while (!((part->sn.uv.remote_act_state == XPC_P_AS_ACTIVATING) ||
+ (part->sn.uv.remote_act_state == XPC_P_AS_ACTIVE))) {
dev_dbg(xpc_part, "waiting to make first contact with "
"partition %d\n", XPC_PARTID(part));
^ permalink raw reply [flat|nested] 16+ messages in thread* [patch 8/8] X86: UV - XPC receive message reuse triggers invalid BUG_ON().
2009-11-19 17:31 [patch 0/8] x86: UV - XPC fixes with related support functionality V2 Robin Holt
` (6 preceding siblings ...)
2009-11-19 17:32 ` [patch 7/8] X86: UV - xpc_make_first_contact hang due to not accepting ACTIVE state Robin Holt
@ 2009-11-19 17:32 ` Robin Holt
2009-11-23 18:03 ` [patch 0/8] x86: UV - XPC fixes with related support functionality V2 Ingo Molnar
8 siblings, 0 replies; 16+ messages in thread
From: Robin Holt @ 2009-11-19 17:32 UTC (permalink / raw)
To: Ingo Molnar, tglx; +Cc: Robin Holt, Jack Steiner, linux-kernel
[-- Attachment #1: xpc_recv_msg_slots_wrap --]
[-- Type: text/plain, Size: 2203 bytes --]
This was a difficult bug to trip. XPC was in the middle of sending an
acknowledgement for a received message.
In xpc_received_payload_uv():
..
ret = xpc_send_gru_msg(ch->sn.uv.cached_notify_gru_mq_desc, msg,
sizeof(struct xpc_notify_mq_msghdr_uv));
if (ret != xpSuccess)
XPC_DEACTIVATE_PARTITION(&xpc_partitions[ch->partid], ret);
msg->hdr.msg_slot_number += ch->remote_nentries;
at the point in xpc_send_gru_msg() where the hardware has dispatched the
acknowledgement, the remote side is able to reuse the message structure
and send a message with a different slot number. This problem is made
worse by interrupts.
The adjustment of msg_slot_number and the BUG_ON in
xpc_handle_notify_mq_msg_uv() which verifies the msg_slot_number is
consistent are only used for debug purposes. Since a fix for this that
preserves the debug functionality would either have to infringe upon
the payload or allocate another structure just for debug, I decided to
remove it entirely.
To: Ingo Molnar <mingo@elte.hu>
To: tglx@linutronix.de
Signed-off-by: Robin Holt <holt@sgi.com>
Cc: Jack Steiner <steiner@sgi.com>
Cc: linux-kernel@vger.kernel.org
---
drivers/misc/sgi-xp/xpc_uv.c | 3 ---
1 file changed, 3 deletions(-)
Index: linux-x86/drivers/misc/sgi-xp/xpc_uv.c
===================================================================
--- linux-x86.orig/drivers/misc/sgi-xp/xpc_uv.c 2009-11-18 22:16:45.000000000 -0600
+++ linux-x86/drivers/misc/sgi-xp/xpc_uv.c 2009-11-18 22:16:45.000000000 -0600
@@ -1428,7 +1428,6 @@ xpc_handle_notify_mq_msg_uv(struct xpc_p
msg_slot = ch_uv->recv_msg_slots +
(msg->hdr.msg_slot_number % ch->remote_nentries) * ch->entry_size;
- BUG_ON(msg->hdr.msg_slot_number != msg_slot->hdr.msg_slot_number);
BUG_ON(msg_slot->hdr.size != 0);
memcpy(msg_slot, msg, msg->hdr.size);
@@ -1652,8 +1651,6 @@ xpc_received_payload_uv(struct xpc_chann
sizeof(struct xpc_notify_mq_msghdr_uv));
if (ret != xpSuccess)
XPC_DEACTIVATE_PARTITION(&xpc_partitions[ch->partid], ret);
-
- msg->hdr.msg_slot_number += ch->remote_nentries;
}
static struct xpc_arch_operations xpc_arch_ops_uv = {
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [patch 0/8] x86: UV - XPC fixes with related support functionality V2.
2009-11-19 17:31 [patch 0/8] x86: UV - XPC fixes with related support functionality V2 Robin Holt
` (7 preceding siblings ...)
2009-11-19 17:32 ` [patch 8/8] X86: UV - XPC receive message reuse triggers invalid BUG_ON() Robin Holt
@ 2009-11-23 18:03 ` Ingo Molnar
2009-11-23 18:10 ` Ingo Molnar
8 siblings, 1 reply; 16+ messages in thread
From: Ingo Molnar @ 2009-11-23 18:03 UTC (permalink / raw)
To: Robin Holt; +Cc: tglx, Jack Steiner, linux-kernel
* Robin Holt <holt@sgi.com> wrote:
>
> The UV BIOS has been updated to implement some of our interface
> functionality differently than originally expected. These patches
> update us to that working bios and include a few minor bug fixes
> which prevent us from doing significant testing on real hardware.
>
> Changes from V1:
>
> - Actually include the patch introducing the gru_read_gpa. This
> was missed in the V1 submission.
>
> - One additional BIOS change has the OS no longer passing blade
> to the BIOS when registering a message queue watchlist.
Note, the patches are a bit hard to review without diffstats - mind
generating them for future patches? Also, putting a 'summary' diffstat
(the total diffstat effect of all patches) into the 0/8 description mail
is common courtesy that is much welcome by maintainers and reviewers as
well.
Thanks,
Ingo
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [patch 0/8] x86: UV - XPC fixes with related support functionality V2.
2009-11-23 18:03 ` [patch 0/8] x86: UV - XPC fixes with related support functionality V2 Ingo Molnar
@ 2009-11-23 18:10 ` Ingo Molnar
2009-11-23 23:57 ` Robin Holt
0 siblings, 1 reply; 16+ messages in thread
From: Ingo Molnar @ 2009-11-23 18:10 UTC (permalink / raw)
To: Robin Holt, Andrew Morton, H. Peter Anvin
Cc: tglx, Jack Steiner, linux-kernel
* Ingo Molnar <mingo@elte.hu> wrote:
> * Robin Holt <holt@sgi.com> wrote:
>
> >
> > The UV BIOS has been updated to implement some of our interface
> > functionality differently than originally expected. These patches
> > update us to that working bios and include a few minor bug fixes
> > which prevent us from doing significant testing on real hardware.
> >
> > Changes from V1:
> >
> > - Actually include the patch introducing the gru_read_gpa. This
> > was missed in the V1 submission.
> >
> > - One additional BIOS change has the OS no longer passing blade
> > to the BIOS when registering a message queue watchlist.
>
> Note, the patches are a bit hard to review without diffstats - mind
> generating them for future patches? Also, putting a 'summary' diffstat
> (the total diffstat effect of all patches) into the 0/8 description mail
> is common courtesy that is much welcome by maintainers and reviewers as
> well.
Ah, sorry - they were there, just merged together with the patch so i
missed them.
Regarding the patches - i still very much dislike how the code
interfaces to the 'BIOS'. The drivers/misc/sgi-xp/ code looks rather
messy.
Why isnt a pure hardware interface exposed, as both hpa and me suggested
in the past?
Anyway ... these bits got merged via -mm in the past so i guess it could
be done there in the future too - the series does not look horrible -
but the whole concept ought to be done in a cleaner way.
Ingo
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [patch 0/8] x86: UV - XPC fixes with related support functionality V2.
2009-11-23 18:10 ` Ingo Molnar
@ 2009-11-23 23:57 ` Robin Holt
2009-11-24 7:19 ` Ingo Molnar
0 siblings, 1 reply; 16+ messages in thread
From: Robin Holt @ 2009-11-23 23:57 UTC (permalink / raw)
To: Ingo Molnar
Cc: Robin Holt, Andrew Morton, H. Peter Anvin, tglx, Jack Steiner,
linux-kernel
> Regarding the patches - i still very much dislike how the code
> interfaces to the 'BIOS'. The drivers/misc/sgi-xp/ code looks rather
> messy.
>
> Why isnt a pure hardware interface exposed, as both hpa and me suggested
> in the past?
The information is stored on hardware (UV-Hub and RAM) outside this instance of hardware.
The only safe way to access that hardware is via the GRU.
No GRU resources are available to the BIOS.
Ergo, this interface method with multiple passes is needed.
If there is a better way, I am open to suggestions.
> Anyway ... these bits got merged via -mm in the past so i guess it could
> be done there in the future too - the series does not look horrible -
> but the whole concept ought to be done in a cleaner way.
I will ask Andrew to take them.
Thanks,
Robin
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [patch 0/8] x86: UV - XPC fixes with related support functionality V2.
2009-11-23 23:57 ` Robin Holt
@ 2009-11-24 7:19 ` Ingo Molnar
0 siblings, 0 replies; 16+ messages in thread
From: Ingo Molnar @ 2009-11-24 7:19 UTC (permalink / raw)
To: Robin Holt
Cc: Andrew Morton, H. Peter Anvin, tglx, Jack Steiner, linux-kernel
* Robin Holt <holt@sgi.com> wrote:
> > Regarding the patches - i still very much dislike how the code
> > interfaces to the 'BIOS'. The drivers/misc/sgi-xp/ code looks rather
> > messy.
> >
> > Why isnt a pure hardware interface exposed, as both hpa and me
> > suggested in the past?
>
> The information is stored on hardware (UV-Hub and RAM) outside this
> instance of hardware.
>
> The only safe way to access that hardware is via the GRU.
>
> No GRU resources are available to the BIOS.
>
> Ergo, this interface method with multiple passes is needed.
>
> If there is a better way, I am open to suggestions.
Erm, the better way is to not use the BIOS but to access those registers
that the BIOS accesses? (which is currently hidden by the BIOS)
In other words, we want to remove arch/x86/kernel/bios_uv.c and the
uv_bios_call*() APIs and replace them via clear implementations for
clear hardware abstractions. That 'bios message queue' thing is absolute
madness for example. But the type-opaque software API too.
Ingo
^ permalink raw reply [flat|nested] 16+ messages in thread