[Bug] Real mode guests never wake up after an HLT instruction

[Bug] Real mode guests never wake up after an HLT instruction

[Mohammed Gamal]

CPU Model:  Intel(R) Core(TM)2 Duo CPU T7250  @ 2.00GHz
KVM version: kvm-69-1687-gd660add
Host Kernel: 2.6.26-rc5 (x86_64)
Guests: FreeDOS and Minix 3.1.2 32-bit
Command: qemu-system-x86_64 -hda /media/sda6/<dos or minix image>

Real mode guests (namely Minix 3 and FreeDOS with HIMEM XMS driver)
freeze after issuing an hlt instruction and never wake up. The problem
occurs both with and without commit 36742c5470. Problem disappears
using -no-kvm switch.

Re: [Bug] Real mode guests never wake up after an HLT instruction

[Avi Kivity]

Mohammed Gamal wrote:
> CPU Model:  Intel(R) Core(TM)2 Duo CPU T7250  @ 2.00GHz
> KVM version: kvm-69-1687-gd660add
> Host Kernel: 2.6.26-rc5 (x86_64)
> Guests: FreeDOS and Minix 3.1.2 32-bit
> Command: qemu-system-x86_64 -hda /media/sda6/<dos or minix image>
>
> Real mode guests (namely Minix 3 and FreeDOS with HIMEM XMS driver)
> freeze after issuing an hlt instruction and never wake up. The problem
> occurs both with and without commit 36742c5470. Problem disappears
> using -no-kvm switch.
>   

Can you check if halt_request is being set properly?

Looking at the code, all seems fine.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: [Bug] Real mode guests never wake up after an HLT instruction

[Mohammed Gamal]

On Sun, Jun 22, 2008 at 10:15 AM, Avi Kivity <avi@qumranet.com> wrote:
> Mohammed Gamal wrote:
>>
>> CPU Model:  Intel(R) Core(TM)2 Duo CPU T7250  @ 2.00GHz
>> KVM version: kvm-69-1687-gd660add
>> Host Kernel: 2.6.26-rc5 (x86_64)
>> Guests: FreeDOS and Minix 3.1.2 32-bit
>> Command: qemu-system-x86_64 -hda /media/sda6/<dos or minix image>
>>
>> Real mode guests (namely Minix 3 and FreeDOS with HIMEM XMS driver)
>> freeze after issuing an hlt instruction and never wake up. The problem
>> occurs both with and without commit 36742c5470. Problem disappears
>> using -no-kvm switch.
>>
>
> Can you check if halt_request is being set properly?
>
> Looking at the code, all seems fine.
>

halt_request is set properly indeed. I think the problem might be
happening in kvm_vcpu_block() but I'm not entirely sure . I'll look
into it.

Re: [Bug] Real mode guests never wake up after an HLT instruction

[Glauber Costa]

On Sun, Jun 22, 2008 at 3:34 PM, Mohammed Gamal <m.gamal005@gmail.com> wrote:
> On Sun, Jun 22, 2008 at 10:15 AM, Avi Kivity <avi@qumranet.com> wrote:
>> Mohammed Gamal wrote:
>>>
>>> CPU Model:  Intel(R) Core(TM)2 Duo CPU T7250  @ 2.00GHz
>>> KVM version: kvm-69-1687-gd660add
>>> Host Kernel: 2.6.26-rc5 (x86_64)
>>> Guests: FreeDOS and Minix 3.1.2 32-bit
>>> Command: qemu-system-x86_64 -hda /media/sda6/<dos or minix image>
>>>
>>> Real mode guests (namely Minix 3 and FreeDOS with HIMEM XMS driver)
>>> freeze after issuing an hlt instruction and never wake up. The problem
>>> occurs both with and without commit 36742c5470. Problem disappears
>>> using -no-kvm switch.
>>>
>>
>> Can you check if halt_request is being set properly?
>>
>> Looking at the code, all seems fine.
>>
>
> halt_request is set properly indeed. I think the problem might be
> happening in kvm_vcpu_block() but I'm not entirely sure . I'll look
> into it.

It is not, since it also happens with -no-kvm-irqchip, that does not
exercise this path.
I looked into it today, will look further, but my current impression
is that hlt is working fine.

It might be the case that the problem lies in any kind of interrupt
injection that were supposed to wake it up.

-- 
Glauber Costa.
"Free as in Freedom"
http://glommer.net

"The less confident you are, the more serious you have to act."

Re: [Bug] Real mode guests never wake up after an HLT instruction

[Mohammed Gamal]

On Wed, Jul 2, 2008 at 11:18 PM, Glauber Costa <glommer@gmail.com> wrote:
> On Sun, Jun 22, 2008 at 3:34 PM, Mohammed Gamal <m.gamal005@gmail.com> wrote:
>> On Sun, Jun 22, 2008 at 10:15 AM, Avi Kivity <avi@qumranet.com> wrote:
>>> Mohammed Gamal wrote:
>>>>
>>>> CPU Model:  Intel(R) Core(TM)2 Duo CPU T7250  @ 2.00GHz
>>>> KVM version: kvm-69-1687-gd660add
>>>> Host Kernel: 2.6.26-rc5 (x86_64)
>>>> Guests: FreeDOS and Minix 3.1.2 32-bit
>>>> Command: qemu-system-x86_64 -hda /media/sda6/<dos or minix image>
>>>>
>>>> Real mode guests (namely Minix 3 and FreeDOS with HIMEM XMS driver)
>>>> freeze after issuing an hlt instruction and never wake up. The problem
>>>> occurs both with and without commit 36742c5470. Problem disappears
>>>> using -no-kvm switch.
>>>>
>>>
>>> Can you check if halt_request is being set properly?
>>>
>>> Looking at the code, all seems fine.
>>>
>>
>> halt_request is set properly indeed. I think the problem might be
>> happening in kvm_vcpu_block() but I'm not entirely sure . I'll look
>> into it.
>
> It is not, since it also happens with -no-kvm-irqchip, that does not
> exercise this path.
> I looked into it today, will look further, but my current impression
> is that hlt is working fine.
>
> It might be the case that the problem lies in any kind of interrupt
> injection that were supposed to wake it up.
>

I'm also suspecting that, a patch for Xen sent a few weeks ago solved
that issue for FreeDOS. Have a look at it here:
http://xenbits.xensource.com/xen-3.2-testing.hg?rev/6d81a6f6cb21

I have written some code to implement debug exception injection and
single-step status checking, but it didn't seem to work. I'll post my
attempt for review in a later post, maybe there was something I was
missing.

Re: [Bug] Real mode guests never wake up after an HLT instruction

[Mohammed Gamal]

[-- Attachment #1: Type: text/plain, Size: 3108 bytes --]

>> It is not, since it also happens with -no-kvm-irqchip, that does not
>> exercise this path.
>> I looked into it today, will look further, but my current impression
>> is that hlt is working fine.
>>
>> It might be the case that the problem lies in any kind of interrupt
>> injection that were supposed to wake it up.
>>
>
> I'm also suspecting that, a patch for Xen sent a few weeks ago solved
> that issue for FreeDOS. Have a look at it here:
> http://xenbits.xensource.com/xen-3.2-testing.hg?rev/6d81a6f6cb21
>
> I have written some code to implement debug exception injection and
> single-step status checking, but it didn't seem to work. I'll post my
> attempt for review in a later post, maybe there was something I was
> missing.
>

OK, here is the code. Please inform me if there is something I am missing.
-----
>> It is not, since it also happens with -no-kvm-irqchip, that does not
>> exercise this path.
>> I looked into it today, will look further, but my current impression
>> is that hlt is working fine.
>>
>> It might be the case that the problem lies in any kind of interrupt
>> injection that were supposed to wake it up.
>>
>
> I'm also suspecting that, a patch for Xen sent a few weeks ago solved
> that issue for FreeDOS. Have a look at it here:
> http://xenbits.xensource.com/xen-3.2-testing.hg?rev/6d81a6f6cb21
>
> I have written some code to implement debug exception injection and
> single-step status checking, but it didn't seem to work. I'll post my
> attempt for review in a later post, maybe there was something I was
> missing.
>

OK, here is the code. Please inform me if I am missing something.

----
diff --git a/arch/x86/kvm/x86_emulate.c b/arch/x86/kvm/x86_emulate.c
index dd4efe1..8643ee5 100644
--- a/arch/x86/kvm/x86_emulate.c
+++ b/arch/x86/kvm/x86_emulate.c
@@ -294,6 +294,7 @@ static u16 group2_table[] = {
 /* EFLAGS bit definitions. */
 #define EFLG_OF (1<<11)
 #define EFLG_DF (1<<10)
+#define EFLG_TF (1<<8)
 #define EFLG_SF (1<<7)
 #define EFLG_ZF (1<<6)
 #define EFLG_AF (1<<4)
@@ -1767,6 +1768,9 @@ writeback:
 	if (rc != 0)
 		goto done;

+	if (ctxt->eflags & EFLG_TF)
+		kvm_inject_db(ctxt->vcpu,0);
+
 	/* Commit shadow register state. */
 	memcpy(ctxt->vcpu->arch.regs, c->regs, sizeof c->regs);
 	kvm_rip_write(ctxt->vcpu, c->eip);
diff --git a/include/asm-x86/kvm_host.h b/include/asm-x86/kvm_host.h
index 02b88ec..2af18f7 100644
--- a/include/asm-x86/kvm_host.h
+++ b/include/asm-x86/kvm_host.h
@@ -56,6 +56,7 @@
 #define KVM_PAGES_PER_HPAGE (KVM_HPAGE_SIZE / PAGE_SIZE)

 #define DE_VECTOR 0
+#define DB_VECTOR 1
 #define UD_VECTOR 6
 #define NM_VECTOR 7
 #define DF_VECTOR 8
@@ -663,6 +664,11 @@ static inline void kvm_inject_gp(struct kvm_vcpu
*vcpu, u32 error_code)
 	kvm_queue_exception_e(vcpu, GP_VECTOR, error_code);
 }

+static inline void kvm_inject_db(struct kvm_vcpu *vcpu, u32 error_code)
+{
+	kvm_queue_exception_e(vcpu, DB_VECTOR, error_code);
+}
+
 #define ASM_VMX_VMCLEAR_RAX       ".byte 0x66, 0x0f, 0xc7, 0x30"
 #define ASM_VMX_VMLAUNCH          ".byte 0x0f, 0x01, 0xc2"
 #define ASM_VMX_VMRESUME          ".byte 0x0f, 0x01, 0xc3"

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: debug.patch --]
[-- Type: text/x-diff; name=debug.patch, Size: 1474 bytes --]

diff --git a/arch/x86/kvm/x86_emulate.c b/arch/x86/kvm/x86_emulate.c
index dd4efe1..8643ee5 100644
--- a/arch/x86/kvm/x86_emulate.c
+++ b/arch/x86/kvm/x86_emulate.c
@@ -294,6 +294,7 @@ static u16 group2_table[] = {
 /* EFLAGS bit definitions. */
 #define EFLG_OF (1<<11)
 #define EFLG_DF (1<<10)
+#define EFLG_TF (1<<8)
 #define EFLG_SF (1<<7)
 #define EFLG_ZF (1<<6)
 #define EFLG_AF (1<<4)
@@ -1767,6 +1768,9 @@ writeback:
 	if (rc != 0)
 		goto done;
 
+	if (ctxt->eflags & EFLG_TF)
+		kvm_inject_db(ctxt->vcpu,0);
+
 	/* Commit shadow register state. */
 	memcpy(ctxt->vcpu->arch.regs, c->regs, sizeof c->regs);
 	kvm_rip_write(ctxt->vcpu, c->eip);
diff --git a/include/asm-x86/kvm_host.h b/include/asm-x86/kvm_host.h
index 02b88ec..2af18f7 100644
--- a/include/asm-x86/kvm_host.h
+++ b/include/asm-x86/kvm_host.h
@@ -56,6 +56,7 @@
 #define KVM_PAGES_PER_HPAGE (KVM_HPAGE_SIZE / PAGE_SIZE)
 
 #define DE_VECTOR 0
+#define DB_VECTOR 1
 #define UD_VECTOR 6
 #define NM_VECTOR 7
 #define DF_VECTOR 8
@@ -663,6 +664,11 @@ static inline void kvm_inject_gp(struct kvm_vcpu *vcpu, u32 error_code)
 	kvm_queue_exception_e(vcpu, GP_VECTOR, error_code);
 }
 
+static inline void kvm_inject_db(struct kvm_vcpu *vcpu, u32 error_code)
+{
+	kvm_queue_exception_e(vcpu, DB_VECTOR, error_code);
+}
+
 #define ASM_VMX_VMCLEAR_RAX       ".byte 0x66, 0x0f, 0xc7, 0x30"
 #define ASM_VMX_VMLAUNCH          ".byte 0x0f, 0x01, 0xc2"
 #define ASM_VMX_VMRESUME          ".byte 0x0f, 0x01, 0xc3"

Re: [Bug] Real mode guests never wake up after an HLT instruction

[Mohammed Gamal]

[-- Attachment #1: Type: text/plain, Size: 51 bytes --]

Sorry for the scrambled message. Here is the patch

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: debug.patch --]
[-- Type: text/x-diff; name=debug.patch, Size: 1474 bytes --]

diff --git a/arch/x86/kvm/x86_emulate.c b/arch/x86/kvm/x86_emulate.c
index dd4efe1..8643ee5 100644
--- a/arch/x86/kvm/x86_emulate.c
+++ b/arch/x86/kvm/x86_emulate.c
@@ -294,6 +294,7 @@ static u16 group2_table[] = {
 /* EFLAGS bit definitions. */
 #define EFLG_OF (1<<11)
 #define EFLG_DF (1<<10)
+#define EFLG_TF (1<<8)
 #define EFLG_SF (1<<7)
 #define EFLG_ZF (1<<6)
 #define EFLG_AF (1<<4)
@@ -1767,6 +1768,9 @@ writeback:
 	if (rc != 0)
 		goto done;
 
+	if (ctxt->eflags & EFLG_TF)
+		kvm_inject_db(ctxt->vcpu,0);
+
 	/* Commit shadow register state. */
 	memcpy(ctxt->vcpu->arch.regs, c->regs, sizeof c->regs);
 	kvm_rip_write(ctxt->vcpu, c->eip);
diff --git a/include/asm-x86/kvm_host.h b/include/asm-x86/kvm_host.h
index 02b88ec..2af18f7 100644
--- a/include/asm-x86/kvm_host.h
+++ b/include/asm-x86/kvm_host.h
@@ -56,6 +56,7 @@
 #define KVM_PAGES_PER_HPAGE (KVM_HPAGE_SIZE / PAGE_SIZE)
 
 #define DE_VECTOR 0
+#define DB_VECTOR 1
 #define UD_VECTOR 6
 #define NM_VECTOR 7
 #define DF_VECTOR 8
@@ -663,6 +664,11 @@ static inline void kvm_inject_gp(struct kvm_vcpu *vcpu, u32 error_code)
 	kvm_queue_exception_e(vcpu, GP_VECTOR, error_code);
 }
 
+static inline void kvm_inject_db(struct kvm_vcpu *vcpu, u32 error_code)
+{
+	kvm_queue_exception_e(vcpu, DB_VECTOR, error_code);
+}
+
 #define ASM_VMX_VMCLEAR_RAX       ".byte 0x66, 0x0f, 0xc7, 0x30"
 #define ASM_VMX_VMLAUNCH          ".byte 0x0f, 0x01, 0xc2"
 #define ASM_VMX_VMRESUME          ".byte 0x0f, 0x01, 0xc3"

Re: [Bug] Real mode guests never wake up after an HLT instruction

[Avi Kivity]

Mohammed Gamal wrote:
> Sorry for the scrambled message. Here is the patch
>   
>  
> +static inline void kvm_inject_db(struct kvm_vcpu *vcpu, u32 error_code)
> +{
> +	kvm_queue_exception_e(vcpu, DB_VECTOR, error_code);
> +}

IIRC debug exceptions don't push an error code.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.