* AMD Family 10H machine check on vmcore read
@ 2008-07-07 23:08 Bob Montgomery
2008-07-08 13:28 ` Vivek Goyal
0 siblings, 1 reply; 9+ messages in thread
From: Bob Montgomery @ 2008-07-07 23:08 UTC (permalink / raw)
To: Vivek Goyal, kexec@lists.infradead.org
We maintain a 2.6.18 derived kernel.
When testing kdump on a new AMD Family 10h (16) processor, once in the
kdump kernel, a read from either /proc/vmcore or /dev/oldmem that
corresponds to the area of memory identified in the original (crashing)
kernel by these boot messages:
Mapping aperture over 65536 KB of RAM @ 1c000000
...
PCI-DMA: Disabling AGP.
PCI-DMA: aperture base @ 1c000000 size 65536 KB
PCI-DMA: using GART IOMMU.
causes a machine check.
On a Family 15 AMD64 processor running this kernel and kdump kernel, I
can read the areas identified as being in the aperture from the kdump
kernel and get values, but on the new processor, reads from the kdump
kernel that are within that address range result in the machine check:
HARDWARE ERROR
CPU 0: Machine Check Exception: 4 Bank 4: be0000010005001b
TSC 141bd974323de ADDR 1c000000 MISC e00c0ffe01000000
While up on the original kernel, attempts to read those locations
from /dev/mem result in EFAULTs on both the old and new processors.
It's just the reads from the kdump kernel's /proc/vmcore or /dev/oldmem
that work ok on the old processor and machine check on the new one.
The output of /proc/iomem does not specifically identify the area. It's
just within the range of:
00100000-cfe4ffff : System RAM
I found code in arch/x86_64/kernel/mce.c:mce_cpu_quirks() that disables
GART TBL walk errors to prevent machine checks. The code checked for
family 15 and so I added family 16 as well. Here is a patch that fixes
my version of the kernel:
--- linux-source-2.6.18-kdump/arch/x86_64/kernel/mce.c.orig 2008-07-02 09:27:39.000000000 -0600
+++ linux-source-2.6.18-kdump/arch/x86_64/kernel/mce.c 2008-07-02 09:28:45.000000000 -0600
@@ -354,7 +354,8 @@ static void mce_init(void *dummy)
static void __cpuinit mce_cpu_quirks(struct cpuinfo_x86 *c)
{
/* This should be disabled by the BIOS, but isn't always */
- if (c->x86_vendor == X86_VENDOR_AMD && c->x86 == 15) {
+ if (c->x86_vendor == X86_VENDOR_AMD &&
+ ((c->x86 == 15) || (c->x86 == 16))) {
/* disable GART TBL walk error reporting, which trips off
incorrectly with the IOMMU & 3ware & Cerberus. */
clear_bit(10, &bank[4]);
But I don't see this fix upstream in the kernel. So I'm wondering if
some other patch protects other kdump kernels from this problem. In
particular, a recent patch that informed the e820 map about the gart
aperture to prevent a normal kernel and a kexec kernel from putting it
at different addresses. It didn't mention machine checks from kdump
kernels, but I wonder if it would have prevented access to that memory
area by having it be excluded from the /proc/vmcore list of areas??
Thanks for any insights,
Bob Montgomery
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: AMD Family 10H machine check on vmcore read 2008-07-07 23:08 AMD Family 10H machine check on vmcore read Bob Montgomery @ 2008-07-08 13:28 ` Vivek Goyal 0 siblings, 0 replies; 9+ messages in thread From: Vivek Goyal @ 2008-07-08 13:28 UTC (permalink / raw) To: Bob Montgomery Cc: airlied, linux kernel mailing list, kexec@lists.infradead.org, Andi Kleen, Dave Jones On Mon, Jul 07, 2008 at 05:08:06PM -0600, Bob Montgomery wrote: > We maintain a 2.6.18 derived kernel. > When testing kdump on a new AMD Family 10h (16) processor, once in the > kdump kernel, a read from either /proc/vmcore or /dev/oldmem that > corresponds to the area of memory identified in the original (crashing) > kernel by these boot messages: > > Mapping aperture over 65536 KB of RAM @ 1c000000 > ... > PCI-DMA: Disabling AGP. > PCI-DMA: aperture base @ 1c000000 size 65536 KB > PCI-DMA: using GART IOMMU. > > causes a machine check. > > On a Family 15 AMD64 processor running this kernel and kdump kernel, I > can read the areas identified as being in the aperture from the kdump > kernel and get values, but on the new processor, reads from the kdump > kernel that are within that address range result in the machine check: > > HARDWARE ERROR > CPU 0: Machine Check Exception: 4 Bank 4: be0000010005001b > TSC 141bd974323de ADDR 1c000000 MISC e00c0ffe01000000 > > While up on the original kernel, attempts to read those locations > from /dev/mem result in EFAULTs on both the old and new processors. > It's just the reads from the kdump kernel's /proc/vmcore or /dev/oldmem > that work ok on the old processor and machine check on the new one. > > The output of /proc/iomem does not specifically identify the area. It's > just within the range of: > > 00100000-cfe4ffff : System RAM > > I found code in arch/x86_64/kernel/mce.c:mce_cpu_quirks() that disables > GART TBL walk errors to prevent machine checks. The code checked for > family 15 and so I added family 16 as well. Here is a patch that fixes > my version of the kernel: > > --- linux-source-2.6.18-kdump/arch/x86_64/kernel/mce.c.orig 2008-07-02 09:27:39.000000000 -0600 > +++ linux-source-2.6.18-kdump/arch/x86_64/kernel/mce.c 2008-07-02 09:28:45.000000000 -0600 > @@ -354,7 +354,8 @@ static void mce_init(void *dummy) > static void __cpuinit mce_cpu_quirks(struct cpuinfo_x86 *c) > { > /* This should be disabled by the BIOS, but isn't always */ > - if (c->x86_vendor == X86_VENDOR_AMD && c->x86 == 15) { > + if (c->x86_vendor == X86_VENDOR_AMD && > + ((c->x86 == 15) || (c->x86 == 16))) { > /* disable GART TBL walk error reporting, which trips off > incorrectly with the IOMMU & 3ware & Cerberus. */ > clear_bit(10, &bank[4]); > > Hi Bob, I am not sure what's happening here. Because in /proc/iomem, GART reserved area is reported as System RAM, kdump kernel will try to read this area and save it. Now I am not sure, what is so special about this area that mapping it and reading it in second kernel would cause a MCE. CCing it to LKML, hoping people knowing GART will be able to provide some input. > But I don't see this fix upstream in the kernel. So I'm wondering if > some other patch protects other kdump kernels from this problem. In > particular, a recent patch that informed the e820 map about the gart > aperture to prevent a normal kernel and a kexec kernel from putting it > at different addresses. It didn't mention machine checks from kdump > kernels, but I wonder if it would have prevented access to that memory > area by having it be excluded from the /proc/vmcore list of areas?? Can you provide a link to the patch above? If /proc/iomem, does not report GART area as system ram then it will be excluded from the dump. (IIUC, IOMMU tables are in GART area and ideally one should be capturing it to find out how IOMMU tables looked like at the time of crash). Thanks Vivek _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: AMD Family 10H machine check on vmcore read @ 2008-07-08 13:28 ` Vivek Goyal 0 siblings, 0 replies; 9+ messages in thread From: Vivek Goyal @ 2008-07-08 13:28 UTC (permalink / raw) To: Bob Montgomery Cc: kexec@lists.infradead.org, Andi Kleen, Dave Jones, airlied, linux kernel mailing list On Mon, Jul 07, 2008 at 05:08:06PM -0600, Bob Montgomery wrote: > We maintain a 2.6.18 derived kernel. > When testing kdump on a new AMD Family 10h (16) processor, once in the > kdump kernel, a read from either /proc/vmcore or /dev/oldmem that > corresponds to the area of memory identified in the original (crashing) > kernel by these boot messages: > > Mapping aperture over 65536 KB of RAM @ 1c000000 > ... > PCI-DMA: Disabling AGP. > PCI-DMA: aperture base @ 1c000000 size 65536 KB > PCI-DMA: using GART IOMMU. > > causes a machine check. > > On a Family 15 AMD64 processor running this kernel and kdump kernel, I > can read the areas identified as being in the aperture from the kdump > kernel and get values, but on the new processor, reads from the kdump > kernel that are within that address range result in the machine check: > > HARDWARE ERROR > CPU 0: Machine Check Exception: 4 Bank 4: be0000010005001b > TSC 141bd974323de ADDR 1c000000 MISC e00c0ffe01000000 > > While up on the original kernel, attempts to read those locations > from /dev/mem result in EFAULTs on both the old and new processors. > It's just the reads from the kdump kernel's /proc/vmcore or /dev/oldmem > that work ok on the old processor and machine check on the new one. > > The output of /proc/iomem does not specifically identify the area. It's > just within the range of: > > 00100000-cfe4ffff : System RAM > > I found code in arch/x86_64/kernel/mce.c:mce_cpu_quirks() that disables > GART TBL walk errors to prevent machine checks. The code checked for > family 15 and so I added family 16 as well. Here is a patch that fixes > my version of the kernel: > > --- linux-source-2.6.18-kdump/arch/x86_64/kernel/mce.c.orig 2008-07-02 09:27:39.000000000 -0600 > +++ linux-source-2.6.18-kdump/arch/x86_64/kernel/mce.c 2008-07-02 09:28:45.000000000 -0600 > @@ -354,7 +354,8 @@ static void mce_init(void *dummy) > static void __cpuinit mce_cpu_quirks(struct cpuinfo_x86 *c) > { > /* This should be disabled by the BIOS, but isn't always */ > - if (c->x86_vendor == X86_VENDOR_AMD && c->x86 == 15) { > + if (c->x86_vendor == X86_VENDOR_AMD && > + ((c->x86 == 15) || (c->x86 == 16))) { > /* disable GART TBL walk error reporting, which trips off > incorrectly with the IOMMU & 3ware & Cerberus. */ > clear_bit(10, &bank[4]); > > Hi Bob, I am not sure what's happening here. Because in /proc/iomem, GART reserved area is reported as System RAM, kdump kernel will try to read this area and save it. Now I am not sure, what is so special about this area that mapping it and reading it in second kernel would cause a MCE. CCing it to LKML, hoping people knowing GART will be able to provide some input. > But I don't see this fix upstream in the kernel. So I'm wondering if > some other patch protects other kdump kernels from this problem. In > particular, a recent patch that informed the e820 map about the gart > aperture to prevent a normal kernel and a kexec kernel from putting it > at different addresses. It didn't mention machine checks from kdump > kernels, but I wonder if it would have prevented access to that memory > area by having it be excluded from the /proc/vmcore list of areas?? Can you provide a link to the patch above? If /proc/iomem, does not report GART area as system ram then it will be excluded from the dump. (IIUC, IOMMU tables are in GART area and ideally one should be capturing it to find out how IOMMU tables looked like at the time of crash). Thanks Vivek ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: AMD Family 10H machine check on vmcore read 2008-07-08 13:28 ` Vivek Goyal @ 2008-07-09 17:32 ` Bob Montgomery -1 siblings, 0 replies; 9+ messages in thread From: Bob Montgomery @ 2008-07-09 17:32 UTC (permalink / raw) To: Vivek Goyal Cc: airlied@linux.ie, linux kernel mailing list, kexec@lists.infradead.org, Andi Kleen, Dave Jones On Tue, 2008-07-08 at 13:28 +0000, Vivek Goyal wrote: > On Mon, Jul 07, 2008 at 05:08:06PM -0600, Bob Montgomery wrote: > > We maintain a 2.6.18 derived kernel. > > When testing kdump on a new AMD Family 10h (16) processor, once in the > > kdump kernel, a read from either /proc/vmcore or /dev/oldmem that > > corresponds to the area of memory identified in the original (crashing) > > kernel by these boot messages: > > > > On a Family 15 AMD64 processor running this kernel and kdump kernel, I > > can read the areas identified as being in the aperture from the kdump > > kernel and get values, but on the new processor, reads from the kdump > > kernel that are within that address range result in the machine check: > > > > HARDWARE ERROR > > CPU 0: Machine Check Exception: 4 Bank 4: be0000010005001b > > TSC 141bd974323de ADDR 1c000000 MISC e00c0ffe01000000 > > > Hi Bob, > > I am not sure what's happening here. Because in /proc/iomem, GART reserved > area is reported as System RAM, kdump kernel will try to read this area > and save it. Now I am not sure, what is so special about this area that > mapping it and reading it in second kernel would cause a MCE. > > CCing it to LKML, hoping people knowing GART will be able to provide some > input. > > > But I don't see this fix upstream in the kernel. So I'm wondering if > > some other patch protects other kdump kernels from this problem. In > > particular, a recent patch that informed the e820 map about the gart > > aperture to prevent a normal kernel and a kexec kernel from putting it > > at different addresses. It didn't mention machine checks from kdump > > kernels, but I wonder if it would have prevented access to that memory > > area by having it be excluded from the /proc/vmcore list of areas?? > > Can you provide a link to the patch above? If /proc/iomem, does not report > GART area as system ram then it will be excluded from the dump. (IIUC, > IOMMU tables are in GART area and ideally one should be capturing it to > find out how IOMMU tables looked like at the time of crash). The patch that I thought might be related is: x86: disable the GART early, 64-bit author Yinghai Lu <Yinghai.Lu@Sun.COM> Wed, 30 Jan 2008 12:33:09 +0000 (13:33 +0100) committer Ingo Molnar <mingo@elte.hu> Wed, 30 Jan 2008 12:33:09 +0000 (13:33 +0100) commit aaf230424204864e2833dcc1da23e2cb0b9f39cd tree a42042f5135aa63a780964bd053ae174211ab62f I thought it might be relevant because of this included comment: > hm, i'm wondering, instead of modifying the GART, why dont we simply > _detect_ whatever GART settings we have inherited, and propagate that > into our e820 maps? I.e. if there's inconsistency, then punch that out > from the memory maps and just dont use that memory. But this patch doesn't mention machine checks as the symptom that initiated the patch. And my reason for looking was because I didn't think I could be the first person to try reading /proc/vmcore on a Family 10h processor. So I wondered why it hadn't been seen by some other tester, and thought some other patch might have "fixed" it a different way on newer kernels than mine. Thanks, Bob Montgomery _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: AMD Family 10H machine check on vmcore read @ 2008-07-09 17:32 ` Bob Montgomery 0 siblings, 0 replies; 9+ messages in thread From: Bob Montgomery @ 2008-07-09 17:32 UTC (permalink / raw) To: Vivek Goyal Cc: kexec@lists.infradead.org, Andi Kleen, Dave Jones, airlied@linux.ie, linux kernel mailing list On Tue, 2008-07-08 at 13:28 +0000, Vivek Goyal wrote: > On Mon, Jul 07, 2008 at 05:08:06PM -0600, Bob Montgomery wrote: > > We maintain a 2.6.18 derived kernel. > > When testing kdump on a new AMD Family 10h (16) processor, once in the > > kdump kernel, a read from either /proc/vmcore or /dev/oldmem that > > corresponds to the area of memory identified in the original (crashing) > > kernel by these boot messages: > > > > On a Family 15 AMD64 processor running this kernel and kdump kernel, I > > can read the areas identified as being in the aperture from the kdump > > kernel and get values, but on the new processor, reads from the kdump > > kernel that are within that address range result in the machine check: > > > > HARDWARE ERROR > > CPU 0: Machine Check Exception: 4 Bank 4: be0000010005001b > > TSC 141bd974323de ADDR 1c000000 MISC e00c0ffe01000000 > > > Hi Bob, > > I am not sure what's happening here. Because in /proc/iomem, GART reserved > area is reported as System RAM, kdump kernel will try to read this area > and save it. Now I am not sure, what is so special about this area that > mapping it and reading it in second kernel would cause a MCE. > > CCing it to LKML, hoping people knowing GART will be able to provide some > input. > > > But I don't see this fix upstream in the kernel. So I'm wondering if > > some other patch protects other kdump kernels from this problem. In > > particular, a recent patch that informed the e820 map about the gart > > aperture to prevent a normal kernel and a kexec kernel from putting it > > at different addresses. It didn't mention machine checks from kdump > > kernels, but I wonder if it would have prevented access to that memory > > area by having it be excluded from the /proc/vmcore list of areas?? > > Can you provide a link to the patch above? If /proc/iomem, does not report > GART area as system ram then it will be excluded from the dump. (IIUC, > IOMMU tables are in GART area and ideally one should be capturing it to > find out how IOMMU tables looked like at the time of crash). The patch that I thought might be related is: x86: disable the GART early, 64-bit author Yinghai Lu <Yinghai.Lu@Sun.COM> Wed, 30 Jan 2008 12:33:09 +0000 (13:33 +0100) committer Ingo Molnar <mingo@elte.hu> Wed, 30 Jan 2008 12:33:09 +0000 (13:33 +0100) commit aaf230424204864e2833dcc1da23e2cb0b9f39cd tree a42042f5135aa63a780964bd053ae174211ab62f I thought it might be relevant because of this included comment: > hm, i'm wondering, instead of modifying the GART, why dont we simply > _detect_ whatever GART settings we have inherited, and propagate that > into our e820 maps? I.e. if there's inconsistency, then punch that out > from the memory maps and just dont use that memory. But this patch doesn't mention machine checks as the symptom that initiated the patch. And my reason for looking was because I didn't think I could be the first person to try reading /proc/vmcore on a Family 10h processor. So I wondered why it hadn't been seen by some other tester, and thought some other patch might have "fixed" it a different way on newer kernels than mine. Thanks, Bob Montgomery ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: AMD Family 10H machine check on vmcore read 2008-07-09 17:32 ` Bob Montgomery @ 2008-07-11 20:50 ` Vivek Goyal -1 siblings, 0 replies; 9+ messages in thread From: Vivek Goyal @ 2008-07-11 20:50 UTC (permalink / raw) To: Bob Montgomery Cc: Andi Kleen, kexec@lists.infradead.org, linux kernel mailing list, Yinghai Lu On Wed, Jul 09, 2008 at 11:32:40AM -0600, Bob Montgomery wrote: > On Tue, 2008-07-08 at 13:28 +0000, Vivek Goyal wrote: > > On Mon, Jul 07, 2008 at 05:08:06PM -0600, Bob Montgomery wrote: > > > We maintain a 2.6.18 derived kernel. > > > When testing kdump on a new AMD Family 10h (16) processor, once in the > > > kdump kernel, a read from either /proc/vmcore or /dev/oldmem that > > > corresponds to the area of memory identified in the original (crashing) > > > kernel by these boot messages: > > > > > > > On a Family 15 AMD64 processor running this kernel and kdump kernel, I > > > can read the areas identified as being in the aperture from the kdump > > > kernel and get values, but on the new processor, reads from the kdump > > > kernel that are within that address range result in the machine check: > > > > > > HARDWARE ERROR > > > CPU 0: Machine Check Exception: 4 Bank 4: be0000010005001b > > > TSC 141bd974323de ADDR 1c000000 MISC e00c0ffe01000000 > > > > > > Hi Bob, > > > > I am not sure what's happening here. Because in /proc/iomem, GART reserved > > area is reported as System RAM, kdump kernel will try to read this area > > and save it. Now I am not sure, what is so special about this area that > > mapping it and reading it in second kernel would cause a MCE. > > > > CCing it to LKML, hoping people knowing GART will be able to provide some > > input. > > > > > But I don't see this fix upstream in the kernel. So I'm wondering if > > > some other patch protects other kdump kernels from this problem. In > > > particular, a recent patch that informed the e820 map about the gart > > > aperture to prevent a normal kernel and a kexec kernel from putting it > > > at different addresses. It didn't mention machine checks from kdump > > > kernels, but I wonder if it would have prevented access to that memory > > > area by having it be excluded from the /proc/vmcore list of areas?? > > > > Can you provide a link to the patch above? If /proc/iomem, does not report > > GART area as system ram then it will be excluded from the dump. (IIUC, > > IOMMU tables are in GART area and ideally one should be capturing it to > > find out how IOMMU tables looked like at the time of crash). > > The patch that I thought might be related is: > x86: disable the GART early, 64-bit > > author Yinghai Lu <Yinghai.Lu@Sun.COM> > Wed, 30 Jan 2008 12:33:09 +0000 (13:33 +0100) > committer Ingo Molnar <mingo@elte.hu> > Wed, 30 Jan 2008 12:33:09 +0000 (13:33 +0100) > commit aaf230424204864e2833dcc1da23e2cb0b9f39cd > tree a42042f5135aa63a780964bd053ae174211ab62f > > I thought it might be relevant because of this included comment: > > > hm, i'm wondering, instead of modifying the GART, why dont we simply > > _detect_ whatever GART settings we have inherited, and propagate that > > into our e820 maps? I.e. if there's inconsistency, then punch that out > > from the memory maps and just dont use that memory. > > But this patch doesn't mention machine checks as the symptom that > initiated the patch. > > And my reason for looking was because I didn't think I could be the > first person to try reading /proc/vmcore on a Family 10h processor. So > I wondered why it hadn't been seen by some other tester, and thought > some other patch might have "fixed" it a different way on newer kernels > than mine. > Hi Bob, So it looks like this patch will mark aperture region as non RAM and kudmp will not try to dump that memory and will not run into MCE. Have you tried the kernel with this patch? Does it work for you? At this point I don't know, why accessing the aperture region of first kernel causes MCE. May be Andi or Yinghai will know, CCing them. So backporting the Yinghai's patch to your kernel should help here. Thanks Vivek _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: AMD Family 10H machine check on vmcore read @ 2008-07-11 20:50 ` Vivek Goyal 0 siblings, 0 replies; 9+ messages in thread From: Vivek Goyal @ 2008-07-11 20:50 UTC (permalink / raw) To: Bob Montgomery Cc: kexec@lists.infradead.org, linux kernel mailing list, Andi Kleen, Yinghai Lu On Wed, Jul 09, 2008 at 11:32:40AM -0600, Bob Montgomery wrote: > On Tue, 2008-07-08 at 13:28 +0000, Vivek Goyal wrote: > > On Mon, Jul 07, 2008 at 05:08:06PM -0600, Bob Montgomery wrote: > > > We maintain a 2.6.18 derived kernel. > > > When testing kdump on a new AMD Family 10h (16) processor, once in the > > > kdump kernel, a read from either /proc/vmcore or /dev/oldmem that > > > corresponds to the area of memory identified in the original (crashing) > > > kernel by these boot messages: > > > > > > > On a Family 15 AMD64 processor running this kernel and kdump kernel, I > > > can read the areas identified as being in the aperture from the kdump > > > kernel and get values, but on the new processor, reads from the kdump > > > kernel that are within that address range result in the machine check: > > > > > > HARDWARE ERROR > > > CPU 0: Machine Check Exception: 4 Bank 4: be0000010005001b > > > TSC 141bd974323de ADDR 1c000000 MISC e00c0ffe01000000 > > > > > > Hi Bob, > > > > I am not sure what's happening here. Because in /proc/iomem, GART reserved > > area is reported as System RAM, kdump kernel will try to read this area > > and save it. Now I am not sure, what is so special about this area that > > mapping it and reading it in second kernel would cause a MCE. > > > > CCing it to LKML, hoping people knowing GART will be able to provide some > > input. > > > > > But I don't see this fix upstream in the kernel. So I'm wondering if > > > some other patch protects other kdump kernels from this problem. In > > > particular, a recent patch that informed the e820 map about the gart > > > aperture to prevent a normal kernel and a kexec kernel from putting it > > > at different addresses. It didn't mention machine checks from kdump > > > kernels, but I wonder if it would have prevented access to that memory > > > area by having it be excluded from the /proc/vmcore list of areas?? > > > > Can you provide a link to the patch above? If /proc/iomem, does not report > > GART area as system ram then it will be excluded from the dump. (IIUC, > > IOMMU tables are in GART area and ideally one should be capturing it to > > find out how IOMMU tables looked like at the time of crash). > > The patch that I thought might be related is: > x86: disable the GART early, 64-bit > > author Yinghai Lu <Yinghai.Lu@Sun.COM> > Wed, 30 Jan 2008 12:33:09 +0000 (13:33 +0100) > committer Ingo Molnar <mingo@elte.hu> > Wed, 30 Jan 2008 12:33:09 +0000 (13:33 +0100) > commit aaf230424204864e2833dcc1da23e2cb0b9f39cd > tree a42042f5135aa63a780964bd053ae174211ab62f > > I thought it might be relevant because of this included comment: > > > hm, i'm wondering, instead of modifying the GART, why dont we simply > > _detect_ whatever GART settings we have inherited, and propagate that > > into our e820 maps? I.e. if there's inconsistency, then punch that out > > from the memory maps and just dont use that memory. > > But this patch doesn't mention machine checks as the symptom that > initiated the patch. > > And my reason for looking was because I didn't think I could be the > first person to try reading /proc/vmcore on a Family 10h processor. So > I wondered why it hadn't been seen by some other tester, and thought > some other patch might have "fixed" it a different way on newer kernels > than mine. > Hi Bob, So it looks like this patch will mark aperture region as non RAM and kudmp will not try to dump that memory and will not run into MCE. Have you tried the kernel with this patch? Does it work for you? At this point I don't know, why accessing the aperture region of first kernel causes MCE. May be Andi or Yinghai will know, CCing them. So backporting the Yinghai's patch to your kernel should help here. Thanks Vivek ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: AMD Family 10H machine check on vmcore read 2008-07-11 20:50 ` Vivek Goyal @ 2008-07-11 21:13 ` Yinghai Lu -1 siblings, 0 replies; 9+ messages in thread From: Yinghai Lu @ 2008-07-11 21:13 UTC (permalink / raw) To: Vivek Goyal Cc: kexec@lists.infradead.org, Andi Kleen, Bob Montgomery, linux kernel mailing list On Fri, Jul 11, 2008 at 1:50 PM, Vivek Goyal <vgoyal@redhat.com> wrote: > On Wed, Jul 09, 2008 at 11:32:40AM -0600, Bob Montgomery wrote: >> On Tue, 2008-07-08 at 13:28 +0000, Vivek Goyal wrote: >> > On Mon, Jul 07, 2008 at 05:08:06PM -0600, Bob Montgomery wrote: >> > > We maintain a 2.6.18 derived kernel. >> > > When testing kdump on a new AMD Family 10h (16) processor, once in the >> > > kdump kernel, a read from either /proc/vmcore or /dev/oldmem that >> > > corresponds to the area of memory identified in the original (crashing) >> > > kernel by these boot messages: >> >> > > >> > > On a Family 15 AMD64 processor running this kernel and kdump kernel, I >> > > can read the areas identified as being in the aperture from the kdump >> > > kernel and get values, but on the new processor, reads from the kdump >> > > kernel that are within that address range result in the machine check: >> > > >> > > HARDWARE ERROR >> > > CPU 0: Machine Check Exception: 4 Bank 4: be0000010005001b >> > > TSC 141bd974323de ADDR 1c000000 MISC e00c0ffe01000000 >> > > >> >> > Hi Bob, >> > >> > I am not sure what's happening here. Because in /proc/iomem, GART reserved >> > area is reported as System RAM, kdump kernel will try to read this area >> > and save it. Now I am not sure, what is so special about this area that >> > mapping it and reading it in second kernel would cause a MCE. >> > >> > CCing it to LKML, hoping people knowing GART will be able to provide some >> > input. >> > >> > > But I don't see this fix upstream in the kernel. So I'm wondering if >> > > some other patch protects other kdump kernels from this problem. In >> > > particular, a recent patch that informed the e820 map about the gart >> > > aperture to prevent a normal kernel and a kexec kernel from putting it >> > > at different addresses. It didn't mention machine checks from kdump >> > > kernels, but I wonder if it would have prevented access to that memory >> > > area by having it be excluded from the /proc/vmcore list of areas?? >> > >> > Can you provide a link to the patch above? If /proc/iomem, does not report >> > GART area as system ram then it will be excluded from the dump. (IIUC, >> > IOMMU tables are in GART area and ideally one should be capturing it to >> > find out how IOMMU tables looked like at the time of crash). >> >> The patch that I thought might be related is: >> x86: disable the GART early, 64-bit >> >> author Yinghai Lu <Yinghai.Lu@Sun.COM> >> Wed, 30 Jan 2008 12:33:09 +0000 (13:33 +0100) >> committer Ingo Molnar <mingo@elte.hu> >> Wed, 30 Jan 2008 12:33:09 +0000 (13:33 +0100) >> commit aaf230424204864e2833dcc1da23e2cb0b9f39cd >> tree a42042f5135aa63a780964bd053ae174211ab62f >> >> I thought it might be relevant because of this included comment: >> >> > hm, i'm wondering, instead of modifying the GART, why dont we simply >> > _detect_ whatever GART settings we have inherited, and propagate that >> > into our e820 maps? I.e. if there's inconsistency, then punch that out >> > from the memory maps and just dont use that memory. >> >> But this patch doesn't mention machine checks as the symptom that >> initiated the patch. >> >> And my reason for looking was because I didn't think I could be the >> first person to try reading /proc/vmcore on a Family 10h processor. So >> I wondered why it hadn't been seen by some other tester, and thought >> some other patch might have "fixed" it a different way on newer kernels >> than mine. >> > > Hi Bob, > > So it looks like this patch will mark aperture region as non RAM and kudmp > will not try to dump that memory and will not run into MCE. Have you tried > the kernel with this patch? Does it work for you? > > At this point I don't know, why accessing the aperture region of first > kernel causes MCE. May be Andi or Yinghai will know, CCing them. > > So backporting the Yinghai's patch to your kernel should help here. > two kernels could have different position with GART aperture allocated from low ram. so need to stop it in shutdown path of first kernel or startup path of second kernel. otherwise second kernel will try to use address of first kernel gart, and later if the gart is disabled for new gart, the setting will be lost. with this patch could use MCE error and HT sync flood and warm reset YH _______________________________________________ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: AMD Family 10H machine check on vmcore read @ 2008-07-11 21:13 ` Yinghai Lu 0 siblings, 0 replies; 9+ messages in thread From: Yinghai Lu @ 2008-07-11 21:13 UTC (permalink / raw) To: Vivek Goyal Cc: Bob Montgomery, kexec@lists.infradead.org, linux kernel mailing list, Andi Kleen On Fri, Jul 11, 2008 at 1:50 PM, Vivek Goyal <vgoyal@redhat.com> wrote: > On Wed, Jul 09, 2008 at 11:32:40AM -0600, Bob Montgomery wrote: >> On Tue, 2008-07-08 at 13:28 +0000, Vivek Goyal wrote: >> > On Mon, Jul 07, 2008 at 05:08:06PM -0600, Bob Montgomery wrote: >> > > We maintain a 2.6.18 derived kernel. >> > > When testing kdump on a new AMD Family 10h (16) processor, once in the >> > > kdump kernel, a read from either /proc/vmcore or /dev/oldmem that >> > > corresponds to the area of memory identified in the original (crashing) >> > > kernel by these boot messages: >> >> > > >> > > On a Family 15 AMD64 processor running this kernel and kdump kernel, I >> > > can read the areas identified as being in the aperture from the kdump >> > > kernel and get values, but on the new processor, reads from the kdump >> > > kernel that are within that address range result in the machine check: >> > > >> > > HARDWARE ERROR >> > > CPU 0: Machine Check Exception: 4 Bank 4: be0000010005001b >> > > TSC 141bd974323de ADDR 1c000000 MISC e00c0ffe01000000 >> > > >> >> > Hi Bob, >> > >> > I am not sure what's happening here. Because in /proc/iomem, GART reserved >> > area is reported as System RAM, kdump kernel will try to read this area >> > and save it. Now I am not sure, what is so special about this area that >> > mapping it and reading it in second kernel would cause a MCE. >> > >> > CCing it to LKML, hoping people knowing GART will be able to provide some >> > input. >> > >> > > But I don't see this fix upstream in the kernel. So I'm wondering if >> > > some other patch protects other kdump kernels from this problem. In >> > > particular, a recent patch that informed the e820 map about the gart >> > > aperture to prevent a normal kernel and a kexec kernel from putting it >> > > at different addresses. It didn't mention machine checks from kdump >> > > kernels, but I wonder if it would have prevented access to that memory >> > > area by having it be excluded from the /proc/vmcore list of areas?? >> > >> > Can you provide a link to the patch above? If /proc/iomem, does not report >> > GART area as system ram then it will be excluded from the dump. (IIUC, >> > IOMMU tables are in GART area and ideally one should be capturing it to >> > find out how IOMMU tables looked like at the time of crash). >> >> The patch that I thought might be related is: >> x86: disable the GART early, 64-bit >> >> author Yinghai Lu <Yinghai.Lu@Sun.COM> >> Wed, 30 Jan 2008 12:33:09 +0000 (13:33 +0100) >> committer Ingo Molnar <mingo@elte.hu> >> Wed, 30 Jan 2008 12:33:09 +0000 (13:33 +0100) >> commit aaf230424204864e2833dcc1da23e2cb0b9f39cd >> tree a42042f5135aa63a780964bd053ae174211ab62f >> >> I thought it might be relevant because of this included comment: >> >> > hm, i'm wondering, instead of modifying the GART, why dont we simply >> > _detect_ whatever GART settings we have inherited, and propagate that >> > into our e820 maps? I.e. if there's inconsistency, then punch that out >> > from the memory maps and just dont use that memory. >> >> But this patch doesn't mention machine checks as the symptom that >> initiated the patch. >> >> And my reason for looking was because I didn't think I could be the >> first person to try reading /proc/vmcore on a Family 10h processor. So >> I wondered why it hadn't been seen by some other tester, and thought >> some other patch might have "fixed" it a different way on newer kernels >> than mine. >> > > Hi Bob, > > So it looks like this patch will mark aperture region as non RAM and kudmp > will not try to dump that memory and will not run into MCE. Have you tried > the kernel with this patch? Does it work for you? > > At this point I don't know, why accessing the aperture region of first > kernel causes MCE. May be Andi or Yinghai will know, CCing them. > > So backporting the Yinghai's patch to your kernel should help here. > two kernels could have different position with GART aperture allocated from low ram. so need to stop it in shutdown path of first kernel or startup path of second kernel. otherwise second kernel will try to use address of first kernel gart, and later if the gart is disabled for new gart, the setting will be lost. with this patch could use MCE error and HT sync flood and warm reset YH ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2008-07-11 21:13 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2008-07-07 23:08 AMD Family 10H machine check on vmcore read Bob Montgomery 2008-07-08 13:28 ` Vivek Goyal 2008-07-08 13:28 ` Vivek Goyal 2008-07-09 17:32 ` Bob Montgomery 2008-07-09 17:32 ` Bob Montgomery 2008-07-11 20:50 ` Vivek Goyal 2008-07-11 20:50 ` Vivek Goyal 2008-07-11 21:13 ` Yinghai Lu 2008-07-11 21:13 ` Yinghai Lu
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.