[ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Stefano Stabellini]

Hi all,
a few weeks ago I (and a few others) started hacking on a
proof-of-concept hypervisor port to Cortex-A15 which uses and requires
ARMv7 virtualization extensions. The intention of this work was to find
out how to best support ARM v7+ on Xen. See
http://old-list-archives.xen.org/archives/html/xen-arm/2011-09/msg00013.html
for more details. 

I am pleased to announce that significant progress has been made, and
that we now have a nascent Xen port for Cortex-A15. The port is based on
xen-unstable (HG CS 8d6edc3d26d2) and written from scratch exploiting
the latest virtualization, LPAE, GIC and generic timer support in
hardware.

We started the work less than three months ago, but the port is already
capable of booting a Linux 3.0 based virtual machine (dom0) up to a
shell prompt on an ARM Architecture Envelope Model, configured to emulate
an A15-based Versatile Express. In this context, we wanted to thank ARM
for making the model available to us.
Now we are looking forward to porting the tools and running multiple
guests.

The code requires virtualization, LPAE and GIC support and therefore it
won't be able to run on anything older than a Cortex-A15.
On the other hand, thanks to this, it is very small and easy to read,
write and understand.
The implementation does not distinguish between PV and HVM guests: there
is just one type of guests that would be comparable to Linux PV on HVM
in the Xen X86 world, but with no need for Qemu emulation.
The code only requires minimal changes to the Linux kernel: just enough
to support PV drivers.

Even though we are currently targeting Versatile Express and Cortex-A15
we do intend to support other machines and other ARMv7 with
virtualization extensions CPUs.
We are also looking forward to ARMv8 and 64 bits support.

Given that porting Xen to Cortex-A15 could be done with so little code,
we believe that the best course of action is to merge it into
xen-unstable as quickly as possible. There are still few rough edges to
sort out but we should be able to produce a clean and digestible patch
series for submission to xen-devel within the next couple of months. I
hope to see the first patches going to the list as soon as possible.
We would very welcome any contributions, in the form of testing, code
reviews and, of course, patches! 


A git tree is available here:

git://xenbits.xen.org/people/sstabellini/xen-unstable.git arm

the gitweb url is the following:

http://xenbits.xen.org/gitweb/?p=people/sstabellini/xen-unstable.git/.git;a=shortlog;h=refs/heads/arm

And here is the full diff:

http://xenbits.xen.org/people/sstabellini/diff


We want to point out that this effort is in addition to Samsung's
ongoing efforts to upstream Xen ARM to xen-unstable. Samsung's XenARM
port allows virtualization of Xen on ARM CPUs prior to virtualization
extensions and supports traditional PV guests.

I would like to thank Tim Deegan and Ian Campbell: if you spend some
time reading the history, you'll see that this project wouldn't have
been possible in such a short time without great contributions from
them.

Cheers,
Stefano

[ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Arnd Bergmann]

On Tuesday 29 November 2011, Stefano Stabellini wrote:
> Hi all,
> a few weeks ago I (and a few others) started hacking on a
> proof-of-concept hypervisor port to Cortex-A15 which uses and requires
> ARMv7 virtualization extensions. The intention of this work was to find
> out how to best support ARM v7+ on Xen. See
> http://old-list-archives.xen.org/archives/html/xen-arm/2011-09/msg00013.html
> for more details. 
> 
> I am pleased to announce that significant progress has been made, and
> that we now have a nascent Xen port for Cortex-A15. The port is based on
> xen-unstable (HG CS 8d6edc3d26d2) and written from scratch exploiting
> the latest virtualization, LPAE, GIC and generic timer support in
> hardware.

Very nice!

Do you have a pointer to the kernel sources for the Linux guest?
Since Xen and KVM are both in an early working state right now,
it would be very nice if we could agree on the guest model to make
sure that it's always possible to run the same kernel in both
(and potentially other future) hypervisors without modifications.

	Arnd

[ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Stefano Stabellini]

On Tue, 29 Nov 2011, Arnd Bergmann wrote:
> On Tuesday 29 November 2011, Stefano Stabellini wrote:
> > Hi all,
> > a few weeks ago I (and a few others) started hacking on a
> > proof-of-concept hypervisor port to Cortex-A15 which uses and requires
> > ARMv7 virtualization extensions. The intention of this work was to find
> > out how to best support ARM v7+ on Xen. See
> > http://old-list-archives.xen.org/archives/html/xen-arm/2011-09/msg00013.html
> > for more details. 
> > 
> > I am pleased to announce that significant progress has been made, and
> > that we now have a nascent Xen port for Cortex-A15. The port is based on
> > xen-unstable (HG CS 8d6edc3d26d2) and written from scratch exploiting
> > the latest virtualization, LPAE, GIC and generic timer support in
> > hardware.
> 
> Very nice!
> 
> Do you have a pointer to the kernel sources for the Linux guest?

We have very few changes to the Linux kernel at the moment (only 3
commits!), just enough to be able to issue hypercalls and start a PV
console.


A git branch is available here (not ready for submission):

git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git arm

the branch above is based on git://linux-arm.org/linux-2.6.git arm-lpae,
even though guests don't really need lpae support to run on Xen.


> Since Xen and KVM are both in an early working state right now,
> it would be very nice if we could agree on the guest model to make
> sure that it's always possible to run the same kernel in both
> (and potentially other future) hypervisors without modifications.

Yes, that would be ideal.
We don't plan on making many changes other than enabling PV frontends
and backends. 

[ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Arnd Bergmann]

On Wednesday 30 November 2011, Stefano Stabellini wrote:
> On Tue, 29 Nov 2011, Arnd Bergmann wrote:
> > On Tuesday 29 November 2011, Stefano Stabellini wrote:
> > 
> > Do you have a pointer to the kernel sources for the Linux guest?
> 
> We have very few changes to the Linux kernel at the moment (only 3
> commits!), just enough to be able to issue hypercalls and start a PV
> console.
> 
> A git branch is available here (not ready for submission):
> 
> git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git arm

Ok, interesting. There really isn't much of the platform support
that I was expecting there. I finally found the information
I was looking for in the xen construct_dom0() function:

 167     regs->r0 = 0; /* SBZ */
 168     regs->r1 = 2272; /* Machine NR: Versatile Express */
 169     regs->r2 = 0xc0000100; /* ATAGS */

What this means is that you are emulating the current ARM/Keil reference
board, at least to the degree that is necessary to get the guest started.

This is the same choice people have made for KVM, but it's not
necessarily the best option in the long run. In particular, this
board has a lot of hardware that you claim to have by putting the
machine number there, when you don't really want to emulate it.

Pawell Moll is working on a variant of the vexpress code that uses
the flattened device tree to describe the present hardware [1], and
I think that would be a much better target for an official release.
Ideally, the hypervisor should provide the device tree binary (dtb)
to the guest OS describing the hardware that is actually there.

This would also be the place where you tell the guest that it should
look for PV devices. I'm not familiar with how Xen announces PV
devices to the guest on other architectures, but you have the
choice between providing a full "binding", i.e. a formal specification
in device tree format for the guest to detect PV devices in the
same way as physical or emulated devices, or just providing a single
place in the device tree in which the guest detects the presence
of a xen device bus and then uses hcalls to find the devices on that
bus.

Another topic is the question whether there are any hcalls that
we should try to standardize before we get another architecture
with multiple conflicting hcall APIs as we have on x86 and powerpc.

	Arnd

[1] http://www.spinics.net/lists/arm-kernel/msg149604.html

[Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Ian Campbell]

On Wed, 2011-11-30 at 13:03 +0000, Arnd Bergmann wrote:
> On Wednesday 30 November 2011, Stefano Stabellini wrote:
> > On Tue, 29 Nov 2011, Arnd Bergmann wrote:
> > > On Tuesday 29 November 2011, Stefano Stabellini wrote:
> > > 
> > > Do you have a pointer to the kernel sources for the Linux guest?
> > 
> > We have very few changes to the Linux kernel at the moment (only 3
> > commits!), just enough to be able to issue hypercalls and start a PV
> > console.
> > 
> > A git branch is available here (not ready for submission):
> > 
> > git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git arm
> 
> Ok, interesting. There really isn't much of the platform support
> that I was expecting there. I finally found the information
> I was looking for in the xen construct_dom0() function:
> 
>  167     regs->r0 = 0; /* SBZ */
>  168     regs->r1 = 2272; /* Machine NR: Versatile Express */
>  169     regs->r2 = 0xc0000100; /* ATAGS */
> 
> What this means is that you are emulating the current ARM/Keil reference
> board, at least to the degree that is necessary to get the guest started.
> 
> This is the same choice people have made for KVM, but it's not
> necessarily the best option in the long run. In particular, this
> board has a lot of hardware that you claim to have by putting the
> machine number there, when you don't really want to emulate it.

This code is actually setting up dom0 which (for the most part) sees the
real hardware.

The hardcoding of the platform is just a short term hack.

> Pawell Moll is working on a variant of the vexpress code that uses
> the flattened device tree to describe the present hardware [1], and
> I think that would be a much better target for an official release.
> Ideally, the hypervisor should provide the device tree binary (dtb)
> to the guest OS describing the hardware that is actually there.

Agreed. Our intention was to use DT so this fits perfectly with our
plans.

For dom0 we would expose a (possibly filtered) version of the DT given
to us by the firmware (e.g. we might hide a serial port to reserve it
for Xen's use, we'd likely fiddle with the memory map etc).

For domU the DT would presumably be constructed by the toolstack (in
dom0 userspace) as appropriate for the guest configuration. I guess this
needn't correspond to any particular "real" hardware platform.

> This would also be the place where you tell the guest that it should
> look for PV devices. I'm not familiar with how Xen announces PV
> devices to the guest on other architectures, but you have the
> choice between providing a full "binding", i.e. a formal specification
> in device tree format for the guest to detect PV devices in the
> same way as physical or emulated devices, or just providing a single
> place in the device tree in which the guest detects the presence
> of a xen device bus and then uses hcalls to find the devices on that
> bus.

On x86 there is an emulated PCI device which serves as the hooking point
for the PV drivers. For ARM I don't think it would be unreasonable to
have a DT entry instead. I think it would be fine just represent the
root of the "xenbus" and further discovery would occur using the normal
xenbus mechanisms (so not a full binding). AIUI for buses which are
enumerable this is the preferred DT scheme to use.

> Another topic is the question whether there are any hcalls that
> we should try to standardize before we get another architecture
> with multiple conflicting hcall APIs as we have on x86 and powerpc.

The hcall API we are currently targeting is the existing Xen API (at
least the generic parts of it). These generally deal with fairly Xen
specific concepts like grant tables etc.

Ian.

> 
> 	Arnd
> 
> [1] http://www.spinics.net/lists/arm-kernel/msg149604.html
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel at lists.xensource.com
> http://lists.xensource.com/xen-devel

[Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Arnd Bergmann]

On Wednesday 30 November 2011, Ian Campbell wrote:
> On Wed, 2011-11-30 at 13:03 +0000, Arnd Bergmann wrote:
> > On Wednesday 30 November 2011, Stefano Stabellini wrote:
> > This is the same choice people have made for KVM, but it's not
> > necessarily the best option in the long run. In particular, this
> > board has a lot of hardware that you claim to have by putting the
> > machine number there, when you don't really want to emulate it.
> 
> This code is actually setting up dom0 which (for the most part) sees the
> real hardware.

Ok, I see.

> > Pawell Moll is working on a variant of the vexpress code that uses
> > the flattened device tree to describe the present hardware [1], and
> > I think that would be a much better target for an official release.
> > Ideally, the hypervisor should provide the device tree binary (dtb)
> > to the guest OS describing the hardware that is actually there.
> 
> Agreed. Our intention was to use DT so this fits perfectly with our
> plans.
> 
> For dom0 we would expose a (possibly filtered) version of the DT given
> to us by the firmware (e.g. we might hide a serial port to reserve it
> for Xen's use, we'd likely fiddle with the memory map etc).

Ah, very good.

> For domU the DT would presumably be constructed by the toolstack (in
> dom0 userspace) as appropriate for the guest configuration. I guess this
> needn't correspond to any particular "real" hardware platform.

Correct, but it needs to correspond to some platform that is supported
by the guest OS, which leaves the choice between emulating a real
hardware platform, adding a completely new platform specifically for
virtual machines, or something in between the two.

What I suggested to the KVM developers is to start out with the
vexpress platform, but then generalize it to the point where it fits
your needs. All hardware that one expects a guest to have (GIC, timer,
...) will still show up in the same location as on a real vexpress,
while anything that makes no sense or is better paravirtualized (LCD,
storage, ...) just becomes optional and has to be described in the
device tree if it's actually there.

> > This would also be the place where you tell the guest that it should
> > look for PV devices. I'm not familiar with how Xen announces PV
> > devices to the guest on other architectures, but you have the
> > choice between providing a full "binding", i.e. a formal specification
> > in device tree format for the guest to detect PV devices in the
> > same way as physical or emulated devices, or just providing a single
> > place in the device tree in which the guest detects the presence
> > of a xen device bus and then uses hcalls to find the devices on that
> > bus.
> 
> On x86 there is an emulated PCI device which serves as the hooking point
> for the PV drivers. For ARM I don't think it would be unreasonable to
> have a DT entry instead. I think it would be fine just represent the
> root of the "xenbus" and further discovery would occur using the normal
> xenbus mechanisms (so not a full binding). AIUI for buses which are
> enumerable this is the preferred DT scheme to use.

In general that is the case, yes. One could argue that any software
protocol between Xen and the guest is as good as any other, so it
makes sense to use the device tree to describe all devices here.
The counterargument to that is that Linux and other OSs already
support Xenbus, so there is no need to come up with a new binding.

I don't care much either way, but I think it would be good to
use similar solutions across all hypervisors. The two options
that I've seen discussed for KVM were to use either a virtual PCI
bus with individual virtio-pci devices as on the PC, or to
use the new virtio-mmio driver and individually put virtio devices
into the device tree.

> > Another topic is the question whether there are any hcalls that
> > we should try to standardize before we get another architecture
> > with multiple conflicting hcall APIs as we have on x86 and powerpc.
> 
> The hcall API we are currently targeting is the existing Xen API (at
> least the generic parts of it). These generally deal with fairly Xen
> specific concepts like grant tables etc.

Ok. It would of course still be possible to agree on an argument passing
convention so that we can share the macros used to issue the hcalls,
even if the individual commands are all different. I think I also
remember talk about the need for a set of hypervisor independent calls
that everyone should implement, but I can't remember what those were.
Maybe we can split the number space into a range of some generic and
some vendor specific hcalls?

	Arnd

[Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Pawel Moll]

On Wed, 2011-11-30 at 14:32 +0000, Arnd Bergmann wrote:
> I don't care much either way, but I think it would be good to
> use similar solutions across all hypervisors. The two options
> that I've seen discussed for KVM were to use either a virtual PCI
> bus with individual virtio-pci devices as on the PC, or to
> use the new virtio-mmio driver and individually put virtio devices
> into the device tree.

Let me just add that the virtio-mmio devices can already be instantiated
from DT (see Documentation/devicetree/bindings/virtio/mmio.txt).

For A9-based VE I'd suggest placing them around 0x1001e000, eg.:

        virtio_block at 1001e000 {
                compatible = "virtio,mmio";
                reg = <0x1001e000 0x100>;
                interrupts = <41>;
        }

Cheers!

Pawe?

[Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Ian Campbell]

On Wed, 2011-11-30 at 14:32 +0000, Arnd Bergmann wrote:
> On Wednesday 30 November 2011, Ian Campbell wrote:
> > On Wed, 2011-11-30 at 13:03 +0000, Arnd Bergmann wrote:
> > For domU the DT would presumably be constructed by the toolstack (in
> > dom0 userspace) as appropriate for the guest configuration. I guess this
> > needn't correspond to any particular "real" hardware platform.
> 
> Correct, but it needs to correspond to some platform that is supported
> by the guest OS, which leaves the choice between emulating a real
> hardware platform, adding a completely new platform specifically for
> virtual machines, or something in between the two.
> 
> What I suggested to the KVM developers is to start out with the
> vexpress platform, but then generalize it to the point where it fits
> your needs. All hardware that one expects a guest to have (GIC, timer,
> ...) will still show up in the same location as on a real vexpress,
> while anything that makes no sense or is better paravirtualized (LCD,
> storage, ...) just becomes optional and has to be described in the
> device tree if it's actually there.

That's along the lines of what I was thinking as well.

The DT contains the address of GIC, timer etc as well right? So at least
in principal we needn't provide e.g. the GIC at the same address as any
real platform but in practice I expect we will.

In principal we could also offer the user options as to which particular
platform a guest looks like.

> > > This would also be the place where you tell the guest that it should
> > > look for PV devices. I'm not familiar with how Xen announces PV
> > > devices to the guest on other architectures, but you have the
> > > choice between providing a full "binding", i.e. a formal specification
> > > in device tree format for the guest to detect PV devices in the
> > > same way as physical or emulated devices, or just providing a single
> > > place in the device tree in which the guest detects the presence
> > > of a xen device bus and then uses hcalls to find the devices on that
> > > bus.
> > 
> > On x86 there is an emulated PCI device which serves as the hooking point
> > for the PV drivers. For ARM I don't think it would be unreasonable to
> > have a DT entry instead. I think it would be fine just represent the
> > root of the "xenbus" and further discovery would occur using the normal
> > xenbus mechanisms (so not a full binding). AIUI for buses which are
> > enumerable this is the preferred DT scheme to use.
> 
> In general that is the case, yes. One could argue that any software
> protocol between Xen and the guest is as good as any other, so it
> makes sense to use the device tree to describe all devices here.
> The counterargument to that is that Linux and other OSs already
> support Xenbus, so there is no need to come up with a new binding.

Right.

> I don't care much either way, but I think it would be good to
> use similar solutions across all hypervisors. The two options
> that I've seen discussed for KVM were to use either a virtual PCI
> bus with individual virtio-pci devices as on the PC, or to
> use the new virtio-mmio driver and individually put virtio devices
> into the device tree.
> 
> > > Another topic is the question whether there are any hcalls that
> > > we should try to standardize before we get another architecture
> > > with multiple conflicting hcall APIs as we have on x86 and powerpc.
> > 
> > The hcall API we are currently targeting is the existing Xen API (at
> > least the generic parts of it). These generally deal with fairly Xen
> > specific concepts like grant tables etc.
> 
> Ok. It would of course still be possible to agree on an argument passing
> convention so that we can share the macros used to issue the hcalls,
> even if the individual commands are all different.

I think it likely that we can all agree on a common calling convention
for N-argument hypercalls. It doubt there are that many useful choices
with conflicting requirements yet strongly compelling advantages.

>  I think I also
> remember talk about the need for a set of hypervisor independent calls
> that everyone should implement, but I can't remember what those were.

I'd not heard of this, maybe I just wasn't looking the right way though.

> Maybe we can split the number space into a range of some generic and
> some vendor specific hcalls?

Ian.

[Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Arnd Bergmann]

On Wednesday 30 November 2011, Ian Campbell wrote:
> On Wed, 2011-11-30 at 14:32 +0000, Arnd Bergmann wrote:
> > On Wednesday 30 November 2011, Ian Campbell wrote:
> > What I suggested to the KVM developers is to start out with the
> > vexpress platform, but then generalize it to the point where it fits
> > your needs. All hardware that one expects a guest to have (GIC, timer,
> > ...) will still show up in the same location as on a real vexpress,
> > while anything that makes no sense or is better paravirtualized (LCD,
> > storage, ...) just becomes optional and has to be described in the
> > device tree if it's actually there.
> 
> That's along the lines of what I was thinking as well.
> 
> The DT contains the address of GIC, timer etc as well right? So at least
> in principal we needn't provide e.g. the GIC at the same address as any
> real platform but in practice I expect we will.

Yes.

> In principal we could also offer the user options as to which particular
> platform a guest looks like.

At least when using a qemu based simulation. Most platforms have some
characteristics that are not meaningful in a classic virtualization
scenario, but it would certainly be helpful to use the virtualization
extensions to run a kernel that was built for a particular platform
faster than with pure qemu, when you want to test that kernel image.

It has been suggested in the past that it would be nice to run the
guest kernel built for the same platform as the host kernel by
default, but I think it would be much better to have just one
platform that we end up using for guests on any host platform,
unless there is a strong reason to do otherwise.

There is also ongoing restructuring in the ARM Linux kernel to
allow running the same kernel binary on multiple platforms. While
there is still a lot of work to be done, you should assume that
we will finish it before you see lots of users in production, there
is no need to plan for the current one-kernel-per-board case.

> > Ok. It would of course still be possible to agree on an argument passing
> > convention so that we can share the macros used to issue the hcalls,
> > even if the individual commands are all different.
> 
> I think it likely that we can all agree on a common calling convention
> for N-argument hypercalls. It doubt there are that many useful choices
> with conflicting requirements yet strongly compelling advantages.

Exactly. I think it's only lack of communication that has resulted in
different interfaces for each hypervisor on the other architectures.

KVM and Xen at least both fall into the single-return-value category,
so we should be able to agree on a calling conventions. KVM does not
have an hcall API on ARM yet, and I see no reason not to use the
same implementation that you have in the Xen guest.

Stefano, can you split out the generic parts of your asm/xen/hypercall.h
file into a common asm/hypercall.h and submit it for review to the
arm kernel list?

	Arnd

[Embeddedxen-devel] [Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Stefano Stabellini]

On Wed, 30 Nov 2011, Arnd Bergmann wrote:
> > In principal we could also offer the user options as to which particular
> > platform a guest looks like.
> 
> At least when using a qemu based simulation. Most platforms have some
> characteristics that are not meaningful in a classic virtualization
> scenario, but it would certainly be helpful to use the virtualization
> extensions to run a kernel that was built for a particular platform
> faster than with pure qemu, when you want to test that kernel image.
> 
> It has been suggested in the past that it would be nice to run the
> guest kernel built for the same platform as the host kernel by
> default, but I think it would be much better to have just one
> platform that we end up using for guests on any host platform,
> unless there is a strong reason to do otherwise.
> 
> There is also ongoing restructuring in the ARM Linux kernel to
> allow running the same kernel binary on multiple platforms. While
> there is still a lot of work to be done, you should assume that
> we will finish it before you see lots of users in production, there
> is no need to plan for the current one-kernel-per-board case.

It is very good to hear, I am counting on it.


> > > Ok. It would of course still be possible to agree on an argument passing
> > > convention so that we can share the macros used to issue the hcalls,
> > > even if the individual commands are all different.
> > 
> > I think it likely that we can all agree on a common calling convention
> > for N-argument hypercalls. It doubt there are that many useful choices
> > with conflicting requirements yet strongly compelling advantages.
> 
> Exactly. I think it's only lack of communication that has resulted in
> different interfaces for each hypervisor on the other architectures.

It is also due to history: on X86 it was possible to issue hypercalls to
Xen before VMCALL (the X86 version of HVC) was available.


> KVM and Xen at least both fall into the single-return-value category,
> so we should be able to agree on a calling conventions. KVM does not
> have an hcall API on ARM yet, and I see no reason not to use the
> same implementation that you have in the Xen guest.
> 
> Stefano, can you split out the generic parts of your asm/xen/hypercall.h
> file into a common asm/hypercall.h and submit it for review to the
> arm kernel list?

Sure, I can do that.
Usually the hypercall calling convention is very hypervisor specific,
but if it turns out that we have the same requirements I happy to design
a common interface.

[Embeddedxen-devel] [Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Ian Campbell]

On Wed, 2011-11-30 at 18:32 +0000, Stefano Stabellini wrote:
> On Wed, 30 Nov 2011, Arnd Bergmann wrote:

> 
> > KVM and Xen at least both fall into the single-return-value category,
> > so we should be able to agree on a calling conventions. KVM does not
> > have an hcall API on ARM yet, and I see no reason not to use the
> > same implementation that you have in the Xen guest.
> > 
> > Stefano, can you split out the generic parts of your asm/xen/hypercall.h
> > file into a common asm/hypercall.h and submit it for review to the
> > arm kernel list?
> 
> Sure, I can do that.
> Usually the hypercall calling convention is very hypervisor specific,
> but if it turns out that we have the same requirements I happy to design
> a common interface.

I expect the only real decision to be made is hypercall page vs. raw hvc
instruction.

The page was useful on x86 where there is a variety of instructions
which could be used (at least for PV there was systenter/syscall/int, I
think vmcall instruction differs between AMD and Intel also) and gives
some additional flexibility. It's hard to predict but I don't think I'd
expect that to be necessary on ARM.

Another reason for having a hypercall page instead of a raw instruction
might be wanting to support 32 bit guests (from ~today) on a 64 bit
hypervisor in the future and perhaps needing to do some shimming/arg
translation. It would be better to aim for having the interface just be
32/64 agnostic but mistakes do happen.

Ian.

[Android-virt] [Embeddedxen-devel] [Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Catalin Marinas]

On Thu, Dec 01, 2011 at 10:26:37AM +0000, Ian Campbell wrote:
> On Wed, 2011-11-30 at 18:32 +0000, Stefano Stabellini wrote:
> > On Wed, 30 Nov 2011, Arnd Bergmann wrote:
> > > KVM and Xen at least both fall into the single-return-value category,
> > > so we should be able to agree on a calling conventions. KVM does not
> > > have an hcall API on ARM yet, and I see no reason not to use the
> > > same implementation that you have in the Xen guest.
> > > 
> > > Stefano, can you split out the generic parts of your asm/xen/hypercall.h
> > > file into a common asm/hypercall.h and submit it for review to the
> > > arm kernel list?
> > 
> > Sure, I can do that.
> > Usually the hypercall calling convention is very hypervisor specific,
> > but if it turns out that we have the same requirements I happy to design
> > a common interface.
> 
> I expect the only real decision to be made is hypercall page vs. raw hvc
> instruction.
> 
> The page was useful on x86 where there is a variety of instructions
> which could be used (at least for PV there was systenter/syscall/int, I
> think vmcall instruction differs between AMD and Intel also) and gives
> some additional flexibility. It's hard to predict but I don't think I'd
> expect that to be necessary on ARM.
> 
> Another reason for having a hypercall page instead of a raw instruction
> might be wanting to support 32 bit guests (from ~today) on a 64 bit
> hypervisor in the future and perhaps needing to do some shimming/arg
> translation. It would be better to aim for having the interface just be
> 32/64 agnostic but mistakes do happen.

Given the way register banking is done on AArch64, issuing an HVC on a
32-bit guest OS doesn't require translation on a 64-bit hypervisor. We
have a similar implementation at the SVC level (for 32-bit user apps on
a 64-bit kernel), the only modification was where a 32-bit SVC takes a
64-bit parameter in two separate 32-bit registers, so packing needs to
be done in a syscall wrapper.

I'm not closely involved with any of the Xen or KVM work but I would
vote for using HVC than a hypercall page.

-- 
Catalin

[Android-virt] [Embeddedxen-devel] [Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Arnd Bergmann]

On Thursday 01 December 2011, Catalin Marinas wrote:
> Given the way register banking is done on AArch64, issuing an HVC on a
> 32-bit guest OS doesn't require translation on a 64-bit hypervisor. We
> have a similar implementation at the SVC level (for 32-bit user apps on
> a 64-bit kernel), the only modification was where a 32-bit SVC takes a
> 64-bit parameter in two separate 32-bit registers, so packing needs to
> be done in a syscall wrapper.

How do you deal with signed integer arguments passed into SVC or HVC from
a caller? If I understand the architecture correctly, the upper
halves of the argument register end up zero-padded, while the callee
expects sign-extension.

	Arnd

[Android-virt] [Embeddedxen-devel] [Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Catalin Marinas]

On Thu, Dec 01, 2011 at 03:42:19PM +0000, Arnd Bergmann wrote:
> On Thursday 01 December 2011, Catalin Marinas wrote:
> > Given the way register banking is done on AArch64, issuing an HVC on a
> > 32-bit guest OS doesn't require translation on a 64-bit hypervisor. We
> > have a similar implementation at the SVC level (for 32-bit user apps on
> > a 64-bit kernel), the only modification was where a 32-bit SVC takes a
> > 64-bit parameter in two separate 32-bit registers, so packing needs to
> > be done in a syscall wrapper.
> 
> How do you deal with signed integer arguments passed into SVC or HVC from
> a caller? If I understand the architecture correctly, the upper
> halves of the argument register end up zero-padded, while the callee
> expects sign-extension.

If you treat it as an "int" (32-bit) and function prototype defined
accordingly, then the generated code only accesses it as a W (rather
than X) register and the top 32-bit part is ignored (no need for
sign-extension). If it is defined as a "long" in the 32-bit world, then
it indeed needs explicit conversion given the different sizes for long
(for example sys_lseek, the second argument is a 'long' and we do
explicit sign extension in the wrapper).

-- 
Catalin

[Android-virt] [Embeddedxen-devel] [Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Arnd Bergmann]

On Thursday 01 December 2011, Catalin Marinas wrote:
> On Thu, Dec 01, 2011 at 03:42:19PM +0000, Arnd Bergmann wrote:
> > On Thursday 01 December 2011, Catalin Marinas wrote:
> > How do you deal with signed integer arguments passed into SVC or HVC from
> > a caller? If I understand the architecture correctly, the upper
> > halves of the argument register end up zero-padded, while the callee
> > expects sign-extension.
> 
> If you treat it as an "int" (32-bit) and function prototype defined
> accordingly, then the generated code only accesses it as a W (rather
> than X) register and the top 32-bit part is ignored (no need for
> sign-extension). If it is defined as a "long" in the 32-bit world, then
> it indeed needs explicit conversion given the different sizes for long
> (for example sys_lseek, the second argument is a 'long' and we do
> explicit sign extension in the wrapper).

Ok, so it's actually different from most other 64 bit architectures, which
normally operate on 64-bit registers and expect the caller to do the
correct sign-extension.

Doing the sign-extension for long arguments then falls into the same
category as long long and unsigned long long arguments, which also need
a wrapper, as you mentioned. Strictly speaking, we only need to do it
for those where the long argument has a meaning outside of the 0..2^31
range, e.g. io_submit can only take small positive numbers although
the type is 'long'.

What about unsigned long and pointer? Can we always rely on the upper
half of the register to be zero-filled when we get an exception from 32
bit into 64 bit state, or do we also have to zero-extend those?

	Arnd

[Android-virt] [Embeddedxen-devel] [Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Catalin Marinas]

On Thu, Dec 01, 2011 at 04:44:40PM +0000, Arnd Bergmann wrote:
> On Thursday 01 December 2011, Catalin Marinas wrote:
> > On Thu, Dec 01, 2011 at 03:42:19PM +0000, Arnd Bergmann wrote:
> > > On Thursday 01 December 2011, Catalin Marinas wrote:
> > > How do you deal with signed integer arguments passed into SVC or HVC from
> > > a caller? If I understand the architecture correctly, the upper
> > > halves of the argument register end up zero-padded, while the callee
> > > expects sign-extension.
> > 
> > If you treat it as an "int" (32-bit) and function prototype defined
> > accordingly, then the generated code only accesses it as a W (rather
> > than X) register and the top 32-bit part is ignored (no need for
> > sign-extension). If it is defined as a "long" in the 32-bit world, then
> > it indeed needs explicit conversion given the different sizes for long
> > (for example sys_lseek, the second argument is a 'long' and we do
> > explicit sign extension in the wrapper).
...
> What about unsigned long and pointer? Can we always rely on the upper
> half of the register to be zero-filled when we get an exception from 32
> bit into 64 bit state, or do we also have to zero-extend those?

They are also fine, no need for zero-extension.

-- 
Catalin

[Android-virt] [Embeddedxen-devel] [Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Ian Campbell]

On Thu, 2011-12-01 at 15:10 +0000, Catalin Marinas wrote:
> On Thu, Dec 01, 2011 at 10:26:37AM +0000, Ian Campbell wrote:
> > On Wed, 2011-11-30 at 18:32 +0000, Stefano Stabellini wrote:
> > > On Wed, 30 Nov 2011, Arnd Bergmann wrote:
> > > > KVM and Xen at least both fall into the single-return-value category,
> > > > so we should be able to agree on a calling conventions. KVM does not
> > > > have an hcall API on ARM yet, and I see no reason not to use the
> > > > same implementation that you have in the Xen guest.
> > > > 
> > > > Stefano, can you split out the generic parts of your asm/xen/hypercall.h
> > > > file into a common asm/hypercall.h and submit it for review to the
> > > > arm kernel list?
> > > 
> > > Sure, I can do that.
> > > Usually the hypercall calling convention is very hypervisor specific,
> > > but if it turns out that we have the same requirements I happy to design
> > > a common interface.
> > 
> > I expect the only real decision to be made is hypercall page vs. raw hvc
> > instruction.
> > 
> > The page was useful on x86 where there is a variety of instructions
> > which could be used (at least for PV there was systenter/syscall/int, I
> > think vmcall instruction differs between AMD and Intel also) and gives
> > some additional flexibility. It's hard to predict but I don't think I'd
> > expect that to be necessary on ARM.
> > 
> > Another reason for having a hypercall page instead of a raw instruction
> > might be wanting to support 32 bit guests (from ~today) on a 64 bit
> > hypervisor in the future and perhaps needing to do some shimming/arg
> > translation. It would be better to aim for having the interface just be
> > 32/64 agnostic but mistakes do happen.
> 
> Given the way register banking is done on AArch64, issuing an HVC on a
> 32-bit guest OS doesn't require translation on a 64-bit hypervisor.

The issue I was thinking about was struct packing for arguments passed
as pointers etc rather than the argument registers themselves. Since the
preference appears to be for raw hvc we should just be careful that they
are agnostic in these.

Ian.

>  We
> have a similar implementation at the SVC level (for 32-bit user apps on
> a 64-bit kernel), the only modification was where a 32-bit SVC takes a
> 64-bit parameter in two separate 32-bit registers, so packing needs to
> be done in a syscall wrapper.
> 
> I'm not closely involved with any of the Xen or KVM work but I would
> vote for using HVC than a hypercall page.
> 

[Embeddedxen-devel] [Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Stefano Stabellini]

On Thu, 1 Dec 2011, Ian Campbell wrote:
> On Wed, 2011-11-30 at 18:32 +0000, Stefano Stabellini wrote:
> > On Wed, 30 Nov 2011, Arnd Bergmann wrote:
> > > KVM and Xen at least both fall into the single-return-value category,
> > > so we should be able to agree on a calling conventions. KVM does not
> > > have an hcall API on ARM yet, and I see no reason not to use the
> > > same implementation that you have in the Xen guest.
> > > 
> > > Stefano, can you split out the generic parts of your asm/xen/hypercall.h
> > > file into a common asm/hypercall.h and submit it for review to the
> > > arm kernel list?
> > 
> > Sure, I can do that.
> > Usually the hypercall calling convention is very hypervisor specific,
> > but if it turns out that we have the same requirements I happy to design
> > a common interface.
> 
> I expect the only real decision to be made is hypercall page vs. raw hvc
> instruction.
> 
> The page was useful on x86 where there is a variety of instructions
> which could be used (at least for PV there was systenter/syscall/int, I
> think vmcall instruction differs between AMD and Intel also) and gives
> some additional flexibility. It's hard to predict but I don't think I'd
> expect that to be necessary on ARM.
>
> Another reason for having a hypercall page instead of a raw instruction
> might be wanting to support 32 bit guests (from ~today) on a 64 bit
> hypervisor in the future and perhaps needing to do some shimming/arg
> translation. It would be better to aim for having the interface just be
> 32/64 agnostic but mistakes do happen.

I always like to keep things as simple as possible, so I am in favor of
using the raw hvc instruction.
Besides with the bulk of mmu hypercalls gone, it should not be difficult
to design a 32/64 bit agnostic interface.

[Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Ian Campbell]

On Wed, 2011-11-30 at 18:15 +0000, Arnd Bergmann wrote:
> On Wednesday 30 November 2011, Ian Campbell wrote:
> > On Wed, 2011-11-30 at 14:32 +0000, Arnd Bergmann wrote:
> > > On Wednesday 30 November 2011, Ian Campbell wrote:
> > > What I suggested to the KVM developers is to start out with the
> > > vexpress platform, but then generalize it to the point where it fits
> > > your needs. All hardware that one expects a guest to have (GIC, timer,
> > > ...) will still show up in the same location as on a real vexpress,
> > > while anything that makes no sense or is better paravirtualized (LCD,
> > > storage, ...) just becomes optional and has to be described in the
> > > device tree if it's actually there.
> > 
> > That's along the lines of what I was thinking as well.
> > 
> > The DT contains the address of GIC, timer etc as well right? So at least
> > in principal we needn't provide e.g. the GIC at the same address as any
> > real platform but in practice I expect we will.
> 
> Yes.
> 
> > In principal we could also offer the user options as to which particular
> > platform a guest looks like.
> 
> At least when using a qemu based simulation. Most platforms have some
> characteristics that are not meaningful in a classic virtualization
> scenario, but it would certainly be helpful to use the virtualization
> extensions to run a kernel that was built for a particular platform
> faster than with pure qemu, when you want to test that kernel image.
> 
> It has been suggested in the past that it would be nice to run the
> guest kernel built for the same platform as the host kernel by
> default, but I think it would be much better to have just one
> platform that we end up using for guests on any host platform,
> unless there is a strong reason to do otherwise.

Yes, I agree, certainly that is what we were planning to target in the
first instance. Doing this means that we can get away with minimal
emulation of actual hardware, relying instead on PV drivers or hardware
virtualisation features.

Supporting specific board platforms as guests would be nice to have
eventually. We would need to do more emulation (e.g. running qemu as a
device model) for that case.

> There is also ongoing restructuring in the ARM Linux kernel to
> allow running the same kernel binary on multiple platforms. While
> there is still a lot of work to be done, you should assume that
> we will finish it before you see lots of users in production, there
> is no need to plan for the current one-kernel-per-board case.

We were absolutely banking on targeting the results of this work, so
that's good ;-)

Ian.

[ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Catalin Marinas]

On 30 November 2011 11:39, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> A git branch is available here (not ready for submission):
>
> git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git arm
>
> the branch above is based on git://linux-arm.org/linux-2.6.git arm-lpae,
> even though guests don't really need lpae support to run on Xen.

Indeed, you don't really need LPAE. What you may need though is
generic timers support for A15, it would allow less Hypervisor traps.
For up-to-date architecture patches (well, development tree, not
guaranteed to be stable), I would recommend this (they get into
mainline at some point):

http://git.kernel.org/?p=linux/kernel/git/cmarinas/linux-arm-arch.git;a=summary

Either use master or just cherry-pick the branches that you are interested in.

-- 
Catalin

[ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Stefano Stabellini]

On Wed, 30 Nov 2011, Catalin Marinas wrote:
> On 30 November 2011 11:39, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > A git branch is available here (not ready for submission):
> >
> > git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git arm
> >
> > the branch above is based on git://linux-arm.org/linux-2.6.git arm-lpae,
> > even though guests don't really need lpae support to run on Xen.
> 
> Indeed, you don't really need LPAE. What you may need though is
> generic timers support for A15, it would allow less Hypervisor traps.
> For up-to-date architecture patches (well, development tree, not
> guaranteed to be stable), I would recommend this (they get into
> mainline at some point):
> 
> http://git.kernel.org/?p=linux/kernel/git/cmarinas/linux-arm-arch.git;a=summary
> 
> Either use master or just cherry-pick the branches that you are interested in.

Thanks, I'll rebase on that.

[Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[David Vrabel]

On 30/11/11 14:11, Catalin Marinas wrote:
> On 30 November 2011 11:39, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
>> A git branch is available here (not ready for submission):
>>
>> git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git arm
>>
>> the branch above is based on git://linux-arm.org/linux-2.6.git arm-lpae,
>> even though guests don't really need lpae support to run on Xen.
> 
> Indeed, you don't really need LPAE. What you may need though is
> generic timers support for A15, it would allow less Hypervisor traps.
> For up-to-date architecture patches (well, development tree, not
> guaranteed to be stable), I would recommend this (they get into
> mainline at some point):
> 
> http://git.kernel.org/?p=linux/kernel/git/cmarinas/linux-arm-arch.git;a=summary
> 
> Either use master or just cherry-pick the branches that you are interested in.

Hi Catalin,

Which branches are required for the Versatile Express with the
Cortex-A15? I merged linux-arm-arch/arm-arch/vexpress in but I get a
instruction fault immediately after branching to __mmap_switched.

Is it not setting up the MMU correctly?

David

[Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Catalin Marinas]

On Fri, Dec 16, 2011 at 03:43:57PM +0000, David Vrabel wrote:
> On 30/11/11 14:11, Catalin Marinas wrote:
> > On 30 November 2011 11:39, Stefano Stabellini
> > <stefano.stabellini@eu.citrix.com> wrote:
> >> A git branch is available here (not ready for submission):
> >>
> >> git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git arm
> >>
> >> the branch above is based on git://linux-arm.org/linux-2.6.git arm-lpae,
> >> even though guests don't really need lpae support to run on Xen.
> > 
> > Indeed, you don't really need LPAE. What you may need though is
> > generic timers support for A15, it would allow less Hypervisor traps.
> > For up-to-date architecture patches (well, development tree, not
> > guaranteed to be stable), I would recommend this (they get into
> > mainline at some point):
> > 
> > http://git.kernel.org/?p=linux/kernel/git/cmarinas/linux-arm-arch.git;a=summary
> > 
> > Either use master or just cherry-pick the branches that you are interested in.
> 
> Which branches are required for the Versatile Express with the
> Cortex-A15? I merged linux-arm-arch/arm-arch/vexpress in

That's the branch if you only need VE and A15 support

> but I get a
> instruction fault immediately after branching to __mmap_switched.
> 
> Is it not setting up the MMU correctly?

Do you run this on a software model? What config options do you use? You
would need to enable VEXPRESS_EXTENDED_MEMORY_MAP and
ARCH_VEXPRESS_CA15X4.

-- 
Catalin

[Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[David Vrabel]

On 16/12/11 16:54, Catalin Marinas wrote:
> On Fri, Dec 16, 2011 at 03:43:57PM +0000, David Vrabel wrote:
>> On 30/11/11 14:11, Catalin Marinas wrote:
>>> On 30 November 2011 11:39, Stefano Stabellini
>>> <stefano.stabellini@eu.citrix.com> wrote:
>>>> A git branch is available here (not ready for submission):
>>>>
>>>> git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git arm
>>>>
>>>> the branch above is based on git://linux-arm.org/linux-2.6.git arm-lpae,
>>>> even though guests don't really need lpae support to run on Xen.
>>>
>>> Indeed, you don't really need LPAE. What you may need though is
>>> generic timers support for A15, it would allow less Hypervisor traps.
>>> For up-to-date architecture patches (well, development tree, not
>>> guaranteed to be stable), I would recommend this (they get into
>>> mainline at some point):
>>>
>>> http://git.kernel.org/?p=linux/kernel/git/cmarinas/linux-arm-arch.git;a=summary
>>>
>>> Either use master or just cherry-pick the branches that you are interested in.
>>
>> Which branches are required for the Versatile Express with the
>> Cortex-A15? I merged linux-arm-arch/arm-arch/vexpress in
> 
> That's the branch if you only need VE and A15 support
> 
>> but I get a
>> instruction fault immediately after branching to __mmap_switched.
>>
>> Is it not setting up the MMU correctly?
> 
> Do you run this on a software model? What config options do you use? You
> would need to enable VEXPRESS_EXTENDED_MEMORY_MAP and
> ARCH_VEXPRESS_CA15X4.

The envelope model, yes.  Both those options are enabled.  I've also
attached the complete config and the model configuration.

I took a closer look at the diffs between what Stefano had in his tree
(which included a bunch of LPAE support which I don't have enabled) and
the kernel boots the addition of some isb's when the MMU is switched on.

These were added by: "ARM: LPAE: add ISBs around MMU enabling code"
(commit 1c553c2 in your tree) which is think is only present in the LPAE
branch.  Is this patch not actually specific to LPAE?  Are there other
similar patches?

diff --git a/arch/arm/kernel/head.S b/arch/arm/kernel/head.S
index 566c54c..b4e84e2 100644
--- a/arch/arm/kernel/head.S
+++ b/arch/arm/kernel/head.S
@@ -400,8 +400,10 @@ ENDPROC(__enable_mmu)
        .align  5
 __turn_mmu_on:
        mov     r0, r0
+        isb
        mcr     p15, 0, r0, c1, c0, 0           @ write control reg
        mrc     p15, 0, r3, c0, c0, 0           @ read id reg
+        isb
        mov     r3, r3
        mov     r3, r13
        mov     pc, r3

David
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: .config
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20111216/b7b92ed0/attachment-0002.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: fm.cfg
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20111216/b7b92ed0/attachment-0003.ksh>

[Xen-devel] [ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Catalin Marinas]

On Fri, Dec 16, 2011 at 05:38:52PM +0000, David Vrabel wrote:
> On 16/12/11 16:54, Catalin Marinas wrote:
> > On Fri, Dec 16, 2011 at 03:43:57PM +0000, David Vrabel wrote:
> >> On 30/11/11 14:11, Catalin Marinas wrote:
> >>> On 30 November 2011 11:39, Stefano Stabellini
> >>> <stefano.stabellini@eu.citrix.com> wrote:
> >>>> A git branch is available here (not ready for submission):
> >>>>
> >>>> git://xenbits.xen.org/people/sstabellini/linux-pvhvm.git arm
> >>>>
> >>>> the branch above is based on git://linux-arm.org/linux-2.6.git arm-lpae,
> >>>> even though guests don't really need lpae support to run on Xen.
> >>>
> >>> Indeed, you don't really need LPAE. What you may need though is
> >>> generic timers support for A15, it would allow less Hypervisor traps.
> >>> For up-to-date architecture patches (well, development tree, not
> >>> guaranteed to be stable), I would recommend this (they get into
> >>> mainline at some point):
> >>>
> >>> http://git.kernel.org/?p=linux/kernel/git/cmarinas/linux-arm-arch.git;a=summary
> >>>
> >>> Either use master or just cherry-pick the branches that you are interested in.
> >>
> >> Which branches are required for the Versatile Express with the
> >> Cortex-A15? I merged linux-arm-arch/arm-arch/vexpress in
> > 
> > That's the branch if you only need VE and A15 support
> > 
> >> but I get a
> >> instruction fault immediately after branching to __mmap_switched.
> >>
> >> Is it not setting up the MMU correctly?
> > 
> > Do you run this on a software model? What config options do you use? You
> > would need to enable VEXPRESS_EXTENDED_MEMORY_MAP and
> > ARCH_VEXPRESS_CA15X4.
> 
> The envelope model, yes.  Both those options are enabled.  I've also
> attached the complete config and the model configuration.
> 
> I took a closer look at the diffs between what Stefano had in his tree
> (which included a bunch of LPAE support which I don't have enabled) and
> the kernel boots the addition of some isb's when the MMU is switched on.
> 
> These were added by: "ARM: LPAE: add ISBs around MMU enabling code"
> (commit 1c553c2 in your tree) which is think is only present in the LPAE
> branch.  Is this patch not actually specific to LPAE?  Are there other
> similar patches?

This patch is not specific to LPAE, it's an architecture requirement and
it became visible with A15. The complete patch is here:

http://git.kernel.org/?p=linux/kernel/git/cmarinas/linux-arm-arch.git;a=commitdiff;h=1c553c2afdd7a57cf874a38925bc58381b28150b

though I only kept it in my LPAE branch (which is on it's way to
mainline for 3.3-rc1).

-- 
Catalin

[ANNOUNCE] Xen port to Cortex-A15 / ARMv7 with virt extensions

[Stefano Stabellini]

On Wed, 30 Nov 2011, Anup Patel wrote:
> Hi all,
> 
> I wanted to know how Xen-ARM for A15 will address following concerns:
> 
> - How will Xen-ARM for A15 support legacy guest environment like ARMv5 or ARMv6 ?

It is not our focus at the moment; we are targeting operating systems
that support a modern ARMv7 machine with GIC support. 
That said, it might be possible to run legacy guests in the future,
introducing more emulation to the hypervisor.


> - What if my Cortex-A15 board does not have a GIC with virtualization support ?

We expect most hardware vendors to provide a GIC with virtualization
support. However if they do not, in order to support their boards we'll
have to do more emulation in the hypervisor.