* [Qemu-devel] [PATCH uq/master -v2 2/2] KVM, MCE, unpoison memory address across reboot
@ 2011-02-09 3:00 Huang Ying
2011-02-09 8:00 ` [Qemu-devel] " Jan Kiszka
0 siblings, 1 reply; 6+ messages in thread
From: Huang Ying @ 2011-02-09 3:00 UTC (permalink / raw)
To: Avi Kivity, Marcelo Tosatti, Anthony Liguori
Cc: Dean Nelson, Andi Kleen, Jan Kiszka, qemu-devel@nongnu.org,
kvm@vger.kernel.org
In Linux kernel HWPoison processing implementation, the virtual
address in processes mapping the error physical memory page is marked
as HWPoison. So that, the further accessing to the virtual
address will kill corresponding processes with SIGBUS.
If the error physical memory page is used by a KVM guest, the SIGBUS
will be sent to QEMU, and QEMU will simulate a MCE to report that
memory error to the guest OS. If the guest OS can not recover from
the error (for example, the page is accessed by kernel code), guest OS
will reboot the system. But because the underlying host virtual
address backing the guest physical memory is still poisoned, if the
guest system accesses the corresponding guest physical memory even
after rebooting, the SIGBUS will still be sent to QEMU and MCE will be
simulated. That is, guest system can not recover via rebooting.
In fact, across rebooting, the contents of guest physical memory page
need not to be kept. We can allocate a new host physical page to
back the corresponding guest physical address.
This patch fixes this issue in QEMU-KVM via calling qemu_ram_remap()
to clear the corresponding page table entry, so that make it possible
to allocate a new page to recover the issue.
Signed-off-by: Huang Ying <ying.huang@intel.com>
---
target-i386/kvm.c | 39 +++++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)
--- a/target-i386/kvm.c
+++ b/target-i386/kvm.c
@@ -508,6 +508,42 @@ static int kvm_get_supported_msrs(KVMSta
return ret;
}
+struct HWPoisonPage;
+typedef struct HWPoisonPage HWPoisonPage;
+struct HWPoisonPage
+{
+ ram_addr_t ram_addr;
+ QLIST_ENTRY(HWPoisonPage) list;
+};
+
+static QLIST_HEAD(hwpoison_page_list, HWPoisonPage) hwpoison_page_list =
+ QLIST_HEAD_INITIALIZER(hwpoison_page_list);
+
+static void kvm_unpoison_all(void *param)
+{
+ HWPoisonPage *page, *next_page;
+
+ QLIST_FOREACH_SAFE(page, &hwpoison_page_list, list, next_page) {
+ QLIST_REMOVE(page, list);
+ qemu_ram_remap(page->ram_addr, TARGET_PAGE_SIZE);
+ qemu_free(page);
+ }
+}
+
+static void kvm_hwpoison_page_add(ram_addr_t ram_addr)
+{
+ HWPoisonPage *page;
+
+ QLIST_FOREACH(page, &hwpoison_page_list, list) {
+ if (page->ram_addr == ram_addr)
+ return;
+ }
+
+ page = qemu_malloc(sizeof(HWPoisonPage));
+ page->ram_addr = ram_addr;
+ QLIST_INSERT_HEAD(&hwpoison_page_list, page, list);
+}
+
int kvm_arch_init(KVMState *s)
{
uint64_t identity_base = 0xfffbc000;
@@ -556,6 +592,7 @@ int kvm_arch_init(KVMState *s)
fprintf(stderr, "e820_add_entry() table is full\n");
return ret;
}
+ qemu_register_reset(kvm_unpoison_all, NULL);
return 0;
}
@@ -1882,6 +1919,7 @@ int kvm_arch_on_sigbus_vcpu(CPUState *en
hardware_memory_error();
}
}
+ kvm_hwpoison_page_add(ram_addr);
if (code == BUS_MCEERR_AR) {
/* Fake an Intel architectural Data Load SRAR UCR */
@@ -1926,6 +1964,7 @@ int kvm_arch_on_sigbus(int code, void *a
"QEMU itself instead of guest system!: %p\n", addr);
return 0;
}
+ kvm_hwpoison_page_add(ram_addr);
kvm_mce_inj_srao_memscrub2(first_cpu, paddr);
} else
#endif
^ permalink raw reply [flat|nested] 6+ messages in thread* [Qemu-devel] Re: [PATCH uq/master -v2 2/2] KVM, MCE, unpoison memory address across reboot
2011-02-09 3:00 [Qemu-devel] [PATCH uq/master -v2 2/2] KVM, MCE, unpoison memory address across reboot Huang Ying
@ 2011-02-09 8:00 ` Jan Kiszka
2011-02-10 0:27 ` Huang Ying
0 siblings, 1 reply; 6+ messages in thread
From: Jan Kiszka @ 2011-02-09 8:00 UTC (permalink / raw)
To: Huang Ying
Cc: kvm@vger.kernel.org, Dean Nelson, Marcelo Tosatti,
qemu-devel@nongnu.org, Anthony Liguori, Andi Kleen, Avi Kivity
[-- Attachment #1: Type: text/plain, Size: 4046 bytes --]
On 2011-02-09 04:00, Huang Ying wrote:
> In Linux kernel HWPoison processing implementation, the virtual
> address in processes mapping the error physical memory page is marked
> as HWPoison. So that, the further accessing to the virtual
> address will kill corresponding processes with SIGBUS.
>
> If the error physical memory page is used by a KVM guest, the SIGBUS
> will be sent to QEMU, and QEMU will simulate a MCE to report that
> memory error to the guest OS. If the guest OS can not recover from
> the error (for example, the page is accessed by kernel code), guest OS
> will reboot the system. But because the underlying host virtual
> address backing the guest physical memory is still poisoned, if the
> guest system accesses the corresponding guest physical memory even
> after rebooting, the SIGBUS will still be sent to QEMU and MCE will be
> simulated. That is, guest system can not recover via rebooting.
Yeah, saw this already during my test...
>
> In fact, across rebooting, the contents of guest physical memory page
> need not to be kept. We can allocate a new host physical page to
> back the corresponding guest physical address.
I just wondering what would be architecturally suboptimal if we simply
remapped on SIGBUS directly. Would save us at least the bookkeeping.
>
> This patch fixes this issue in QEMU-KVM via calling qemu_ram_remap()
> to clear the corresponding page table entry, so that make it possible
> to allocate a new page to recover the issue.
>
> Signed-off-by: Huang Ying <ying.huang@intel.com>
> ---
> target-i386/kvm.c | 39 +++++++++++++++++++++++++++++++++++++++
> 1 file changed, 39 insertions(+)
>
> --- a/target-i386/kvm.c
> +++ b/target-i386/kvm.c
> @@ -508,6 +508,42 @@ static int kvm_get_supported_msrs(KVMSta
> return ret;
> }
>
> +struct HWPoisonPage;
> +typedef struct HWPoisonPage HWPoisonPage;
> +struct HWPoisonPage
> +{
> + ram_addr_t ram_addr;
> + QLIST_ENTRY(HWPoisonPage) list;
> +};
> +
> +static QLIST_HEAD(hwpoison_page_list, HWPoisonPage) hwpoison_page_list =
> + QLIST_HEAD_INITIALIZER(hwpoison_page_list);
> +
> +static void kvm_unpoison_all(void *param)
> +{
> + HWPoisonPage *page, *next_page;
> +
> + QLIST_FOREACH_SAFE(page, &hwpoison_page_list, list, next_page) {
> + QLIST_REMOVE(page, list);
> + qemu_ram_remap(page->ram_addr, TARGET_PAGE_SIZE);
> + qemu_free(page);
> + }
> +}
> +
> +static void kvm_hwpoison_page_add(ram_addr_t ram_addr)
> +{
> + HWPoisonPage *page;
> +
> + QLIST_FOREACH(page, &hwpoison_page_list, list) {
> + if (page->ram_addr == ram_addr)
> + return;
> + }
> +
> + page = qemu_malloc(sizeof(HWPoisonPage));
> + page->ram_addr = ram_addr;
> + QLIST_INSERT_HEAD(&hwpoison_page_list, page, list);
> +}
> +
> int kvm_arch_init(KVMState *s)
> {
> uint64_t identity_base = 0xfffbc000;
> @@ -556,6 +592,7 @@ int kvm_arch_init(KVMState *s)
> fprintf(stderr, "e820_add_entry() table is full\n");
> return ret;
> }
> + qemu_register_reset(kvm_unpoison_all, NULL);
>
> return 0;
> }
> @@ -1882,6 +1919,7 @@ int kvm_arch_on_sigbus_vcpu(CPUState *en
> hardware_memory_error();
> }
> }
> + kvm_hwpoison_page_add(ram_addr);
>
> if (code == BUS_MCEERR_AR) {
> /* Fake an Intel architectural Data Load SRAR UCR */
> @@ -1926,6 +1964,7 @@ int kvm_arch_on_sigbus(int code, void *a
> "QEMU itself instead of guest system!: %p\n", addr);
> return 0;
> }
> + kvm_hwpoison_page_add(ram_addr);
> kvm_mce_inj_srao_memscrub2(first_cpu, paddr);
> } else
> #endif
>
>
Looks fine otherwise. Unless that simplification makes sense, I could
offer to include this into my MCE rework (there is some minor conflict).
If all goes well, that series should be posted during this week.
Jan
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread* [Qemu-devel] Re: [PATCH uq/master -v2 2/2] KVM, MCE, unpoison memory address across reboot
2011-02-09 8:00 ` [Qemu-devel] " Jan Kiszka
@ 2011-02-10 0:27 ` Huang Ying
2011-02-10 8:22 ` Jan Kiszka
2011-02-10 8:52 ` Jan Kiszka
0 siblings, 2 replies; 6+ messages in thread
From: Huang Ying @ 2011-02-10 0:27 UTC (permalink / raw)
To: Jan Kiszka
Cc: Dean, kvm@vger.kernel.org, Nelson, Marcelo Tosatti,
qemu-devel@nongnu.org, Anthony Liguori, Andi Kleen, Avi Kivity
On Wed, 2011-02-09 at 16:00 +0800, Jan Kiszka wrote:
> On 2011-02-09 04:00, Huang Ying wrote:
> > In Linux kernel HWPoison processing implementation, the virtual
> > address in processes mapping the error physical memory page is marked
> > as HWPoison. So that, the further accessing to the virtual
> > address will kill corresponding processes with SIGBUS.
> >
> > If the error physical memory page is used by a KVM guest, the SIGBUS
> > will be sent to QEMU, and QEMU will simulate a MCE to report that
> > memory error to the guest OS. If the guest OS can not recover from
> > the error (for example, the page is accessed by kernel code), guest OS
> > will reboot the system. But because the underlying host virtual
> > address backing the guest physical memory is still poisoned, if the
> > guest system accesses the corresponding guest physical memory even
> > after rebooting, the SIGBUS will still be sent to QEMU and MCE will be
> > simulated. That is, guest system can not recover via rebooting.
>
> Yeah, saw this already during my test...
>
> >
> > In fact, across rebooting, the contents of guest physical memory page
> > need not to be kept. We can allocate a new host physical page to
> > back the corresponding guest physical address.
>
> I just wondering what would be architecturally suboptimal if we simply
> remapped on SIGBUS directly. Would save us at least the bookkeeping.
Because we can not change the content of memory silently during guest OS
running, this may corrupts guest OS data structure and even ruins disk
contents. But during rebooting, all guest OS state are discarded.
[snip]
> > @@ -1882,6 +1919,7 @@ int kvm_arch_on_sigbus_vcpu(CPUState *en
> > hardware_memory_error();
> > }
> > }
> > + kvm_hwpoison_page_add(ram_addr);
> >
> > if (code == BUS_MCEERR_AR) {
> > /* Fake an Intel architectural Data Load SRAR UCR */
> > @@ -1926,6 +1964,7 @@ int kvm_arch_on_sigbus(int code, void *a
> > "QEMU itself instead of guest system!: %p\n", addr);
> > return 0;
> > }
> > + kvm_hwpoison_page_add(ram_addr);
> > kvm_mce_inj_srao_memscrub2(first_cpu, paddr);
> > } else
> > #endif
> >
> >
>
> Looks fine otherwise. Unless that simplification makes sense, I could
> offer to include this into my MCE rework (there is some minor conflict).
> If all goes well, that series should be posted during this week.
Thanks.
Best Regards,
Huang Ying
^ permalink raw reply [flat|nested] 6+ messages in thread* [Qemu-devel] Re: [PATCH uq/master -v2 2/2] KVM, MCE, unpoison memory address across reboot
2011-02-10 0:27 ` Huang Ying
@ 2011-02-10 8:22 ` Jan Kiszka
2011-02-10 8:52 ` Jan Kiszka
1 sibling, 0 replies; 6+ messages in thread
From: Jan Kiszka @ 2011-02-10 8:22 UTC (permalink / raw)
To: Huang Ying
Cc: kvm@vger.kernel.org, Dean Nelson, Marcelo Tosatti,
qemu-devel@nongnu.org, Anthony Liguori, Andi Kleen, Avi Kivity
[-- Attachment #1: Type: text/plain, Size: 1954 bytes --]
On 2011-02-10 01:27, Huang Ying wrote:
> On Wed, 2011-02-09 at 16:00 +0800, Jan Kiszka wrote:
>> On 2011-02-09 04:00, Huang Ying wrote:
>>> In Linux kernel HWPoison processing implementation, the virtual
>>> address in processes mapping the error physical memory page is marked
>>> as HWPoison. So that, the further accessing to the virtual
>>> address will kill corresponding processes with SIGBUS.
>>>
>>> If the error physical memory page is used by a KVM guest, the SIGBUS
>>> will be sent to QEMU, and QEMU will simulate a MCE to report that
>>> memory error to the guest OS. If the guest OS can not recover from
>>> the error (for example, the page is accessed by kernel code), guest OS
>>> will reboot the system. But because the underlying host virtual
>>> address backing the guest physical memory is still poisoned, if the
>>> guest system accesses the corresponding guest physical memory even
>>> after rebooting, the SIGBUS will still be sent to QEMU and MCE will be
>>> simulated. That is, guest system can not recover via rebooting.
>>
>> Yeah, saw this already during my test...
>>
>>>
>>> In fact, across rebooting, the contents of guest physical memory page
>>> need not to be kept. We can allocate a new host physical page to
>>> back the corresponding guest physical address.
>>
>> I just wondering what would be architecturally suboptimal if we simply
>> remapped on SIGBUS directly. Would save us at least the bookkeeping.
>
> Because we can not change the content of memory silently during guest OS
> running, this may corrupts guest OS data structure and even ruins disk
> contents. But during rebooting, all guest OS state are discarded.
I was not talking about remapping more than just the pages that became
inaccessible, just like you do now. But I guess the problem is rather
that insane guests continuing to access those pages before reboot should
also still receive MCEs.
Jan
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Qemu-devel] Re: [PATCH uq/master -v2 2/2] KVM, MCE, unpoison memory address across reboot
2011-02-10 0:27 ` Huang Ying
2011-02-10 8:22 ` Jan Kiszka
@ 2011-02-10 8:52 ` Jan Kiszka
2011-02-11 1:20 ` Huang Ying
1 sibling, 1 reply; 6+ messages in thread
From: Jan Kiszka @ 2011-02-10 8:52 UTC (permalink / raw)
To: Huang Ying
Cc: kvm@vger.kernel.org, Dean Nelson, Marcelo Tosatti,
qemu-devel@nongnu.org, Anthony Liguori, Andi Kleen, Avi Kivity
[-- Attachment #1: Type: text/plain, Size: 1095 bytes --]
On 2011-02-10 01:27, Huang Ying wrote:
>>> @@ -1882,6 +1919,7 @@ int kvm_arch_on_sigbus_vcpu(CPUState *en
>>> hardware_memory_error();
>>> }
>>> }
>>> + kvm_hwpoison_page_add(ram_addr);
>>>
>>> if (code == BUS_MCEERR_AR) {
>>> /* Fake an Intel architectural Data Load SRAR UCR */
>>> @@ -1926,6 +1964,7 @@ int kvm_arch_on_sigbus(int code, void *a
>>> "QEMU itself instead of guest system!: %p\n", addr);
>>> return 0;
>>> }
>>> + kvm_hwpoison_page_add(ram_addr);
>>> kvm_mce_inj_srao_memscrub2(first_cpu, paddr);
>>> } else
>>> #endif
>>>
>>>
>>
>> Looks fine otherwise. Unless that simplification makes sense, I could
>> offer to include this into my MCE rework (there is some minor conflict).
>> If all goes well, that series should be posted during this week.
Please have a look at
git://git.kiszka.org/qemu-kvm.git queues/kvm-upstream
and tell me if it works for you and your signed-off still applies.
Thanks,
Jan
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 259 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread* [Qemu-devel] Re: [PATCH uq/master -v2 2/2] KVM, MCE, unpoison memory address across reboot
2011-02-10 8:52 ` Jan Kiszka
@ 2011-02-11 1:20 ` Huang Ying
0 siblings, 0 replies; 6+ messages in thread
From: Huang Ying @ 2011-02-11 1:20 UTC (permalink / raw)
To: Jan Kiszka
Cc: Dean, kvm@vger.kernel.org, Nelson, Marcelo Tosatti,
qemu-devel@nongnu.org, Anthony Liguori, Andi Kleen, Avi Kivity
On Thu, 2011-02-10 at 16:52 +0800, Jan Kiszka wrote:
> On 2011-02-10 01:27, Huang Ying wrote:
> >>> @@ -1882,6 +1919,7 @@ int kvm_arch_on_sigbus_vcpu(CPUState *en
> >>> hardware_memory_error();
> >>> }
> >>> }
> >>> + kvm_hwpoison_page_add(ram_addr);
> >>>
> >>> if (code == BUS_MCEERR_AR) {
> >>> /* Fake an Intel architectural Data Load SRAR UCR */
> >>> @@ -1926,6 +1964,7 @@ int kvm_arch_on_sigbus(int code, void *a
> >>> "QEMU itself instead of guest system!: %p\n", addr);
> >>> return 0;
> >>> }
> >>> + kvm_hwpoison_page_add(ram_addr);
> >>> kvm_mce_inj_srao_memscrub2(first_cpu, paddr);
> >>> } else
> >>> #endif
> >>>
> >>>
> >>
> >> Looks fine otherwise. Unless that simplification makes sense, I could
> >> offer to include this into my MCE rework (there is some minor conflict).
> >> If all goes well, that series should be posted during this week.
>
> Please have a look at
>
> git://git.kiszka.org/qemu-kvm.git queues/kvm-upstream
>
> and tell me if it works for you and your signed-off still applies.
Thanks! Works as expected in my testing!
Best Regards,
Huang Ying
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-02-11 1:20 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-02-09 3:00 [Qemu-devel] [PATCH uq/master -v2 2/2] KVM, MCE, unpoison memory address across reboot Huang Ying
2011-02-09 8:00 ` [Qemu-devel] " Jan Kiszka
2011-02-10 0:27 ` Huang Ying
2011-02-10 8:22 ` Jan Kiszka
2011-02-10 8:52 ` Jan Kiszka
2011-02-11 1:20 ` Huang Ying
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).