Dom0 pvops kernel booting: release dom0-2.6.24-B0

Dom0 pvops kernel booting: release dom0-2.6.24-B0

[Stephen C. Tweedie]

Hi all,

I've substantially cleaned up the Xen contiguous region handling and the
pfn/mfn translations in the dma mapping layer, and rebased everything
cleanly on top of v2.6.24.  The resulting patch queue can be viewed as
tag "dom0-2.6.24-B0" on the dom0-pvops git tree at

	git://et.redhat.com/linux-2.6-dom0-pvops.git

or equivalently on gitweb at

	http://git.et.redhat.com/?p=linux-2.6-dom0-pvops.git;a=shortlog;h=dom0-2.6.24-B0

As usual, a mergeable, fast-forwarding development branch is being
maintained on the git master branch, with tag "dom0-2.6.24-B0-dev",
which has an identical git tree but with the full development history
behind it.

New in this release:

      * Added full mtrr support from Mark McLoughlin
      * Added xen_(create|destroy)_contiguous region handling
      * Added full Xen dma support to enable booting a dom0 kernel

Support for the dom0 xend userland is still missing, and there is no
swiotlb yet; but the dom0 kernel boots successfully on a HV and I've had
a best uptime of nearly an hour so far.  I'm getting soft lockups under
load, but it's not clear if that's a dom0 problem or a latent pv_ops
bug.

Enjoy!

Cheers,
 Stephen

Re: Dom0 pvops kernel booting: release dom0-2.6.24-B0

[Stephen C. Tweedie]

Hi,

On Wed, 2008-02-06 at 15:47 +0000, Stephen C. Tweedie wrote:

> I've substantially cleaned up the Xen contiguous region handling and the
> pfn/mfn translations in the dma mapping layer, and rebased everything
> cleanly on top of v2.6.24.  The resulting patch queue can be viewed as
> tag "dom0-2.6.24-B0" on the dom0-pvops git tree
...
> Support for the dom0 xend userland is still missing, and there is no
> swiotlb yet; but the dom0 kernel boots successfully on a HV and I've had
> a best uptime of nearly an hour so far.

Just to add to that status: all the normal kernel device drivers I've
tried seem to be working with this release --- usb, network and disk.  X
is still known NOT to work, but that's not top priority to deal with.

Next steps include:

      * Making sure domU runs correctly with the same kernel
      * Cleaning up the apic handling to be pvops-friendly, so the same
        kernel also boots on baremetal (all the code except for apic
        should be OK already)
      * Adding the necessary /dev and /proc support for xend
      * Tracking 2.6.25-pre

At some point we'll also want to start tracking the xen-64 bit work,
too.

--Stephen

Re: Dom0 pvops kernel booting: release dom0-2.6.24-B0

[Michael Marineau]

On Wed, Feb 6, 2008 at 7:47 AM, Stephen C. Tweedie <sct@redhat.com> wrote:
> Hi all,
>
>  I've substantially cleaned up the Xen contiguous region handling and the
>  pfn/mfn translations in the dma mapping layer, and rebased everything
>  cleanly on top of v2.6.24.  The resulting patch queue can be viewed as
>  tag "dom0-2.6.24-B0" on the dom0-pvops git tree at
>
>         git://et.redhat.com/linux-2.6-dom0-pvops.git
>
>  or equivalently on gitweb at
>
>         http://git.et.redhat.com/?p=linux-2.6-dom0-pvops.git;a=shortlog;h=dom0-2.6.24-B0
>
>  As usual, a mergeable, fast-forwarding development branch is being
>  maintained on the git master branch, with tag "dom0-2.6.24-B0-dev",
>  which has an identical git tree but with the full development history
>  behind it.
>
>  New in this release:
>
>       * Added full mtrr support from Mark McLoughlin
>       * Added xen_(create|destroy)_contiguous region handling
>       * Added full Xen dma support to enable booting a dom0 kernel
>
>  Support for the dom0 xend userland is still missing, and there is no
>  swiotlb yet; but the dom0 kernel boots successfully on a HV and I've had
>  a best uptime of nearly an hour so far.  I'm getting soft lockups under
>  load, but it's not clear if that's a dom0 problem or a latent pv_ops
>  bug.
>
>  Enjoy!
>
>  Cheers,
>   Stephen

I took a stab at getting this booting but it dies when trying to make
a call to bios32_indirect in arch/x86/pci/pcbios.c, the address
apparently is triggering a page fault that of course cannot be
handled. Here is the resulting oops:

PCI: BIOS32 Service Directory structure at 0xc00f62d0
PCI: BIOS32 Service Directory entry at 0xfd5d7
BUG: unable to handle kernel paging request at virtual address c00fd5d7
printing eip: c00fd5d7 *pdpt = 000000003e006001
Oops: 0011 [#1] SMP
Modules linked in:

Pid: 1, comm: swapper Not tainted (2.6.24-osl1-gd076f0f0-dirty #9)
EIP: 0061:[<c00fd5d7>] EFLAGS: 00010046 CPU: 0
EIP is at 0xc00fd5d7
EAX: 49435024 EBX: 00000000 ECX: 00000000 EDX: c0600b36
ESI: 00000200 EDI: c0675a84 EBP: c06d0ff4 ESP: ed023ed4
 DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: e021
Process swapper (pid: 1, ti=ed023000 task=ed022a90 task.ti=ed023000)
Stack: c0459af4 00000061 00000001 c0166fd7 c15a0900 ed0007c0 49435024 000fd5d7
       c00f62d0 00000000 c06d0ff4 c0459b68 c05a851d ed045e00 00000200 00000000
       c0675a9c c06d0ff4 c0450135 c06bf56b ed047480 c062f7e0 00000cff 000fd5d7
Call Trace:
 [<c0459af4>] bios32_service+0x30/0x93
 [<c0166fd7>] kfree+0x95/0x9b
 [<c0459b68>] check_pcibios+0x11/0x14d
 [<c0450135>] dmi_get_year+0x28/0x52
 [<c06bf56b>] pci_sanity_check+0x2c/0xe8
 [<c0459f20>] pci_find_bios+0xc8/0xec
 [<c06bf521>] pci_pcbios_init+0xe/0x2c
 [<c06bf4f2>] pci_access_init+0x17/0x38
 [<c06a3cb8>] kernel_init+0x0/0xa3
 [<c06a3b37>] do_initcalls+0x73/0x1b2
 [<c019688e>] get_inode_number+0x30/0x54
 [<c0196c11>] proc_register+0x85/0x9f
 [<c0196e4a>] create_proc_entry+0x76/0x89
 [<c014b051>] register_irq_proc+0x77/0x93
 [<c0196db5>] proc_mkdir_mode+0x30/0x43
 [<c06a3cb8>] kernel_init+0x0/0xa3
 [<c06a3d19>] kernel_init+0x61/0xa3
 [<c01080a3>] kernel_thread_helper+0x7/0x10
 =======================
Code: 8b 0e 23 3b 66 2b d1 66 b8 01 00 eb 04 66 b8 00 00 66 59 66 81 e1 00 f0 6
EIP: [<c00fd5d7>] 0xc00fd5d7 SS:ESP e021:ed023ed4
---[ end trace ca143223eefdc828 ]---
-- 
Michael Marineau
Oregon State University
mike@marineau.org

Re: Dom0 pvops kernel booting: release dom0-2.6.24-B0

[Pasi Kärkkäinen]

On Fri, Feb 15, 2008 at 03:10:22PM -0800, Michael Marineau wrote:
> On Wed, Feb 6, 2008 at 7:47 AM, Stephen C. Tweedie <sct@redhat.com> wrote:
> > Hi all,
> >
> >  I've substantially cleaned up the Xen contiguous region handling and the
> >  pfn/mfn translations in the dma mapping layer, and rebased everything
> >  cleanly on top of v2.6.24.  The resulting patch queue can be viewed as
> >  tag "dom0-2.6.24-B0" on the dom0-pvops git tree at
> >
> >         git://et.redhat.com/linux-2.6-dom0-pvops.git
> >
> >  or equivalently on gitweb at
> >
> >         http://git.et.redhat.com/?p=linux-2.6-dom0-pvops.git;a=shortlog;h=dom0-2.6.24-B0
> >
> >  As usual, a mergeable, fast-forwarding development branch is being
> >  maintained on the git master branch, with tag "dom0-2.6.24-B0-dev",
> >  which has an identical git tree but with the full development history
> >  behind it.
> >
> >  New in this release:
> >
> >       * Added full mtrr support from Mark McLoughlin
> >       * Added xen_(create|destroy)_contiguous region handling
> >       * Added full Xen dma support to enable booting a dom0 kernel
> >
> >  Support for the dom0 xend userland is still missing, and there is no
> >  swiotlb yet; but the dom0 kernel boots successfully on a HV and I've had
> >  a best uptime of nearly an hour so far.  I'm getting soft lockups under
> >  load, but it's not clear if that's a dom0 problem or a latent pv_ops
> >  bug.
> >
> >  Enjoy!
> >


Hello!

Are there any plans to update dom0 pv_ops tree for 2.6.26 or other newer
kernels? 

It seems Jeremy has gotten x86_64 xen domU patches submitted for 2.6.27
inclusion.. 

-- Pasi


> >  Cheers,
> >   Stephen
> 
> I took a stab at getting this booting but it dies when trying to make
> a call to bios32_indirect in arch/x86/pci/pcbios.c, the address
> apparently is triggering a page fault that of course cannot be
> handled. Here is the resulting oops:
> 
> PCI: BIOS32 Service Directory structure at 0xc00f62d0
> PCI: BIOS32 Service Directory entry at 0xfd5d7
> BUG: unable to handle kernel paging request at virtual address c00fd5d7
> printing eip: c00fd5d7 *pdpt = 000000003e006001
> Oops: 0011 [#1] SMP
> Modules linked in:
> 
> Pid: 1, comm: swapper Not tainted (2.6.24-osl1-gd076f0f0-dirty #9)
> EIP: 0061:[<c00fd5d7>] EFLAGS: 00010046 CPU: 0
> EIP is at 0xc00fd5d7
> EAX: 49435024 EBX: 00000000 ECX: 00000000 EDX: c0600b36
> ESI: 00000200 EDI: c0675a84 EBP: c06d0ff4 ESP: ed023ed4
>  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: e021
> Process swapper (pid: 1, ti=ed023000 task=ed022a90 task.ti=ed023000)
> Stack: c0459af4 00000061 00000001 c0166fd7 c15a0900 ed0007c0 49435024 000fd5d7
>        c00f62d0 00000000 c06d0ff4 c0459b68 c05a851d ed045e00 00000200 00000000
>        c0675a9c c06d0ff4 c0450135 c06bf56b ed047480 c062f7e0 00000cff 000fd5d7
> Call Trace:
>  [<c0459af4>] bios32_service+0x30/0x93
>  [<c0166fd7>] kfree+0x95/0x9b
>  [<c0459b68>] check_pcibios+0x11/0x14d
>  [<c0450135>] dmi_get_year+0x28/0x52
>  [<c06bf56b>] pci_sanity_check+0x2c/0xe8
>  [<c0459f20>] pci_find_bios+0xc8/0xec
>  [<c06bf521>] pci_pcbios_init+0xe/0x2c
>  [<c06bf4f2>] pci_access_init+0x17/0x38
>  [<c06a3cb8>] kernel_init+0x0/0xa3
>  [<c06a3b37>] do_initcalls+0x73/0x1b2
>  [<c019688e>] get_inode_number+0x30/0x54
>  [<c0196c11>] proc_register+0x85/0x9f
>  [<c0196e4a>] create_proc_entry+0x76/0x89
>  [<c014b051>] register_irq_proc+0x77/0x93
>  [<c0196db5>] proc_mkdir_mode+0x30/0x43
>  [<c06a3cb8>] kernel_init+0x0/0xa3
>  [<c06a3d19>] kernel_init+0x61/0xa3
>  [<c01080a3>] kernel_thread_helper+0x7/0x10
>  =======================
> Code: 8b 0e 23 3b 66 2b d1 66 b8 01 00 eb 04 66 b8 00 00 66 59 66 81 e1 00 f0 6
> EIP: [<c00fd5d7>] 0xc00fd5d7 SS:ESP e021:ed023ed4
> ---[ end trace ca143223eefdc828 ]---
> -- 
> Michael Marineau
> Oregon State University
> mike@marineau.org
> 

Re: Dom0 pvops kernel booting: release dom0-2.6.24-B0

[Pasi Kärkkäinen]

On Wed, Jul 16, 2008 at 03:43:13PM +0300, Pasi Kärkkäinen wrote:
> On Fri, Feb 15, 2008 at 03:10:22PM -0800, Michael Marineau wrote:
> > On Wed, Feb 6, 2008 at 7:47 AM, Stephen C. Tweedie <sct@redhat.com> wrote:
> > > Hi all,
> > >
> > >  I've substantially cleaned up the Xen contiguous region handling and the
> > >  pfn/mfn translations in the dma mapping layer, and rebased everything
> > >  cleanly on top of v2.6.24.  The resulting patch queue can be viewed as
> > >  tag "dom0-2.6.24-B0" on the dom0-pvops git tree at
> > >
> > >         git://et.redhat.com/linux-2.6-dom0-pvops.git
> > >
> > >  or equivalently on gitweb at
> > >
> > >         http://git.et.redhat.com/?p=linux-2.6-dom0-pvops.git;a=shortlog;h=dom0-2.6.24-B0
> > >
> > >  As usual, a mergeable, fast-forwarding development branch is being
> > >  maintained on the git master branch, with tag "dom0-2.6.24-B0-dev",
> > >  which has an identical git tree but with the full development history
> > >  behind it.
> > >
> > >  New in this release:
> > >
> > >       * Added full mtrr support from Mark McLoughlin
> > >       * Added xen_(create|destroy)_contiguous region handling
> > >       * Added full Xen dma support to enable booting a dom0 kernel
> > >
> > >  Support for the dom0 xend userland is still missing, and there is no
> > >  swiotlb yet; but the dom0 kernel boots successfully on a HV and I've had
> > >  a best uptime of nearly an hour so far.  I'm getting soft lockups under
> > >  load, but it's not clear if that's a dom0 problem or a latent pv_ops
> > >  bug.
> > >
> > >  Enjoy!
> > >
> 
> 
> Hello!
> 
> Are there any plans to update dom0 pv_ops tree for 2.6.26 or other newer
> kernels? 
> 
> It seems Jeremy has gotten x86_64 xen domU patches submitted for 2.6.27
> inclusion.. 
> 

And also Save/restore/migration is queued for 2.6.27.. 

-- Pasi

> 
> 
> > >  Cheers,
> > >   Stephen
> > 
> > I took a stab at getting this booting but it dies when trying to make
> > a call to bios32_indirect in arch/x86/pci/pcbios.c, the address
> > apparently is triggering a page fault that of course cannot be
> > handled. Here is the resulting oops:
> > 
> > PCI: BIOS32 Service Directory structure at 0xc00f62d0
> > PCI: BIOS32 Service Directory entry at 0xfd5d7
> > BUG: unable to handle kernel paging request at virtual address c00fd5d7
> > printing eip: c00fd5d7 *pdpt = 000000003e006001
> > Oops: 0011 [#1] SMP
> > Modules linked in:
> > 
> > Pid: 1, comm: swapper Not tainted (2.6.24-osl1-gd076f0f0-dirty #9)
> > EIP: 0061:[<c00fd5d7>] EFLAGS: 00010046 CPU: 0
> > EIP is at 0xc00fd5d7
> > EAX: 49435024 EBX: 00000000 ECX: 00000000 EDX: c0600b36
> > ESI: 00000200 EDI: c0675a84 EBP: c06d0ff4 ESP: ed023ed4
> >  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: e021
> > Process swapper (pid: 1, ti=ed023000 task=ed022a90 task.ti=ed023000)
> > Stack: c0459af4 00000061 00000001 c0166fd7 c15a0900 ed0007c0 49435024 000fd5d7
> >        c00f62d0 00000000 c06d0ff4 c0459b68 c05a851d ed045e00 00000200 00000000
> >        c0675a9c c06d0ff4 c0450135 c06bf56b ed047480 c062f7e0 00000cff 000fd5d7
> > Call Trace:
> >  [<c0459af4>] bios32_service+0x30/0x93
> >  [<c0166fd7>] kfree+0x95/0x9b
> >  [<c0459b68>] check_pcibios+0x11/0x14d
> >  [<c0450135>] dmi_get_year+0x28/0x52
> >  [<c06bf56b>] pci_sanity_check+0x2c/0xe8
> >  [<c0459f20>] pci_find_bios+0xc8/0xec
> >  [<c06bf521>] pci_pcbios_init+0xe/0x2c
> >  [<c06bf4f2>] pci_access_init+0x17/0x38
> >  [<c06a3cb8>] kernel_init+0x0/0xa3
> >  [<c06a3b37>] do_initcalls+0x73/0x1b2
> >  [<c019688e>] get_inode_number+0x30/0x54
> >  [<c0196c11>] proc_register+0x85/0x9f
> >  [<c0196e4a>] create_proc_entry+0x76/0x89
> >  [<c014b051>] register_irq_proc+0x77/0x93
> >  [<c0196db5>] proc_mkdir_mode+0x30/0x43
> >  [<c06a3cb8>] kernel_init+0x0/0xa3
> >  [<c06a3d19>] kernel_init+0x61/0xa3
> >  [<c01080a3>] kernel_thread_helper+0x7/0x10
> >  =======================
> > Code: 8b 0e 23 3b 66 2b d1 66 b8 01 00 eb 04 66 b8 00 00 66 59 66 81 e1 00 f0 6
> > EIP: [<c00fd5d7>] 0xc00fd5d7 SS:ESP e021:ed023ed4
> > ---[ end trace ca143223eefdc828 ]---
> > -- 
> > Michael Marineau
> > Oregon State University
> > mike@marineau.org
> > 
> 

Re: Dom0 pvops kernel booting: release dom0-2.6.24-B0

[Jeremy Fitzhardinge]

Pasi Kärkkäinen wrote:
>> Are there any plans to update dom0 pv_ops tree for 2.6.26 or other newer
>> kernels? 
>>
>> It seems Jeremy has gotten x86_64 xen domU patches submitted for 2.6.27
>> inclusion.. 
>>
>>     
>
> And also Save/restore/migration is queued for 2.6.27.. 
>   

I jumped the gun a bit.  The patches are queued, but are not yet in 
linux-2.6.git.  Later this week, I hope...

    J