writable page tables appear to be causing xen to lock up

writable page tables appear to be causing xen to lock up

[Kip Macy]

I've just tried enabling writable page tables in FreeBSD.

I'm seeing the following code in traps.c hit over again. Disabling the
DPRINTK on line 341 doesn't help, it just causes the machine to become
unresponsive quietly.


xen_fault:      

if ( likely((fixup = search_exception_table(regs->eip)) != 0) )     {
        perfc_incrc(copy_user_faults);         
        if ( !shadow_mode_enabled(d) )
            DPRINTK("Page fault: %p -> %p\n", regs->eip, fixup);
        regs->eip = fixup;         
        return 0;
    }

RE: writable page tables appear to be causing xen to lock up

[Ian Pratt]

 
> I've just tried enabling writable page tables in FreeBSD.
> 
> I'm seeing the following code in traps.c hit over again. 
> Disabling the DPRINTK on line 341 doesn't help, it just 
> causes the machine to become unresponsive quietly.

You'll have to give us a bit more to go on. I presume this is in
unstable?

It's probably worth adding a show_stack and a show_guest_stack to see
how you're getting there.

Ian
 
> xen_fault:      
> 
> if ( likely((fixup = search_exception_table(regs->eip)) != 0) )     {
>         perfc_incrc(copy_user_faults);         
>         if ( !shadow_mode_enabled(d) )
>             DPRINTK("Page fault: %p -> %p\n", regs->eip, fixup);
>         regs->eip = fixup;         
>         return 0;
>     }
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 

Re: writable page tables appear to be causing xen to lock up

[Kip Macy]

> You'll have to give us a bit more to go on. I presume this is in
> unstable?

Yes this is as of your last pushed checkin from 72hrs ago. LOL. If I
had more I'd give it to you. I've just disabled writable pagetables in
FreeBSD for the moment.


> It's probably worth adding a show_stack and a show_guest_stack to see
> how you're getting there.

I'll stick that in and let you know.

 -Kip

Re: writable page tables appear to be causing xen to lock up

[Kip Macy]

My guess from this is FreeBSD is DOSing xen by endlessly triggering
failsafe_callback. Although this is *extremely* likely to be my fault,
it raises an interesting point about the need for heavily reducing the
scheduling priority of misbehaved guests.


0xfc550862 <__copy_from_user_ll+53>:    repz movsb %ds:(%esi),%es:(%edi) 
0xfc552df4 <failsafe_callback+233>:     push   %ecx 
0xfc54df1e <error_code+94>:     add    $0x4,%esp 
0xfc550862 <__copy_from_user_ll+53>:    repz movsb %ds:(%esi),%es:(%edi)
0xfc5508f4 <copy_from_user+60>: mov    %eax,%ebx
0xfc54da4f <x86_emulate_read_std+40>:   test   %eax,%eax
0xfc54abfc <x86_emulate_memop+176>:     mov    %eax,0xffffffd4(%ebp)
0xfc54dae5 <memcpy+31>: leave
0xfc54cdca <x86_emulate_memop+8830>:    cmpl $0x1,0xffffffd4(%ebp)
0xfc52ba0f <alloc_l1_table+211>:        movl   $0x1,0xffffffec(%ebp)
0xfc52c3a0 <alloc_page_type+72>:        mov    %eax,0xfffffffc(%ebp)
0xfc52cab6 <get_page_type+860>: movl   $0x1,0xffffffd8(%ebp)
0xfc52e6a6 <do_mmu_update+2826>:        mov    0xffffffd8(%ebp),%eax
0xfc5304c1 <ptwr_do_page_fault+1070>:   test   %eax,%eax
0xfc5426b3 <debugger_trap_entry+11>:    mov    %eax,0xfffffffc(%ebp)
0xfc540100 <do_page_fault+300>: test   %eax,%eax
0xfc53ff60 <propagate_page_fault+18>:   mov    %eax,0xfffffff4(%ebp)
0xfc540330 <do_page_fault+860>: movl   $0x0,0xffffffe0(%ebp)
0xfc54df1e <error_code+94>:     add    $0x4,%esp
0xfc54dda3 <FLT14>:     mov    %eax,%gs:(%esi)
0xfc54de75 <process_guest_exception_and_events+21>:     jmp   
0xfc54dcd7 <test_all_events>

On 4/18/05, Kip Macy <kip.macy@gmail.com> wrote:
> > You'll have to give us a bit more to go on. I presume this is in
> > unstable?
> 
> Yes this is as of your last pushed checkin from 72hrs ago. LOL. If I
> had more I'd give it to you. I've just disabled writable pagetables in
> FreeBSD for the moment.
> 
> 
> > It's probably worth adding a show_stack and a show_guest_stack to see
> > how you're getting there.
> 
> I'll stick that in and let you know.
> 
>  -Kip
>

Re: writable page tables appear to be causing xen to lock up

[Kip Macy]

Sticking a printk in the writable page table macros tracked down the
problem immediately. I had gone through pmap.c carefully
differentiating between page table and page directory updates, but I
neglected to do that in machdep.c.

WP PT_SET ../../../i386-xen/i386-xen/machdep.c:1461
(XEN) (file=traps.c, line=343) Page fault: fc550862 -> fc552df4
(XEN) Stack trace from ESP=fc503c04:
...

Easy enough to solve on my end. As of this point I did not have my
trap table setup so I would have thought that xen should've simply
killed the guest rather than retrying indefinitely.

                              -Kip

On 4/19/05, Kip Macy <kip.macy@gmail.com> wrote:
> My guess from this is FreeBSD is DOSing xen by endlessly triggering
> failsafe_callback. Although this is *extremely* likely to be my fault,
> it raises an interesting point about the need for heavily reducing the
> scheduling priority of misbehaved guests.
> 
> 0xfc550862 <__copy_from_user_ll+53>:    repz movsb %ds:(%esi),%es:(%edi)
> 0xfc552df4 <failsafe_callback+233>:     push   %ecx
> 0xfc54df1e <error_code+94>:     add    $0x4,%esp
> 0xfc550862 <__copy_from_user_ll+53>:    repz movsb %ds:(%esi),%es:(%edi)
> 0xfc5508f4 <copy_from_user+60>: mov    %eax,%ebx
> 0xfc54da4f <x86_emulate_read_std+40>:   test   %eax,%eax
> 0xfc54abfc <x86_emulate_memop+176>:     mov    %eax,0xffffffd4(%ebp)
> 0xfc54dae5 <memcpy+31>: leave
> 0xfc54cdca <x86_emulate_memop+8830>:    cmpl $0x1,0xffffffd4(%ebp)
> 0xfc52ba0f <alloc_l1_table+211>:        movl   $0x1,0xffffffec(%ebp)
> 0xfc52c3a0 <alloc_page_type+72>:        mov    %eax,0xfffffffc(%ebp)
> 0xfc52cab6 <get_page_type+860>: movl   $0x1,0xffffffd8(%ebp)
> 0xfc52e6a6 <do_mmu_update+2826>:        mov    0xffffffd8(%ebp),%eax
> 0xfc5304c1 <ptwr_do_page_fault+1070>:   test   %eax,%eax
> 0xfc5426b3 <debugger_trap_entry+11>:    mov    %eax,0xfffffffc(%ebp)
> 0xfc540100 <do_page_fault+300>: test   %eax,%eax
> 0xfc53ff60 <propagate_page_fault+18>:   mov    %eax,0xfffffff4(%ebp)
> 0xfc540330 <do_page_fault+860>: movl   $0x0,0xffffffe0(%ebp)
> 0xfc54df1e <error_code+94>:     add    $0x4,%esp
> 0xfc54dda3 <FLT14>:     mov    %eax,%gs:(%esi)
> 0xfc54de75 <process_guest_exception_and_events+21>:     jmp
> 0xfc54dcd7 <test_all_events>
> 
> On 4/18/05, Kip Macy <kip.macy@gmail.com> wrote:
> > > You'll have to give us a bit more to go on. I presume this is in
> > > unstable?
> >
> > Yes this is as of your last pushed checkin from 72hrs ago. LOL. If I
> > had more I'd give it to you. I've just disabled writable pagetables in
> > FreeBSD for the moment.
> >
> >
> > > It's probably worth adding a show_stack and a show_guest_stack to see
> > > how you're getting there.
> >
> > I'll stick that in and let you know.
> >
> >  -Kip
> >
>

RE: writable page tables appear to be causing xen to lock up

[Ian Pratt]

 
> My guess from this is FreeBSD is DOSing xen by endlessly 
> triggering failsafe_callback. Although this is *extremely* 
> likely to be my fault, it raises an interesting point about 
> the need for heavily reducing the scheduling priority of 
> misbehaved guests.

Hmm, was the failsafe callback failing? That should result in the domain
being crashed.

If the guest was actually leaving Xen, then it should be preempted and
scheduled as per normal.

Ian