Enable virtual memory for Mini-os on ARM

Enable virtual memory for Mini-os on ARM

[karim.allah.ahmed]

[-- Attachment #1.1: Type: text/plain, Size: 1221 bytes --]

Hi,

I'm currently working on porting mini-os to ARM.

The work I've done so far was related:
* Changes to build system to build mini-os as zImage
* adding a linker script for arm target
* Splitting some source files between common and arch specific bits
* adding some hypercalls for arm ( mainly used at the moment for console_io
and getting shared_info page ).
* some assembly code for early startup ( stack setup, mmu stuff etc )
before jumping to c code.
* , etc

Now, it compiles and boots ( as zImage ). Obviously at the moment most of
the ARM specific bits are only stubs and I've a while(1){} in the
initialization code because it's not complete yet. So, it's still useless
for now :)

My question is the following:
Why do I need to create pagetables and fill them for mini-os ? as far as I
can see, mini-os is always statically built with another app ( or the dummy
app_main ) .. Is there an actual reason for needing virtual memory in that
setup ? Can't we just run on physical memory ?

At the moment I've some crude assembly code that was creating initial
pagetables, but I honestly don't know why I even need that!


-- 
Karim Allah Ahmed.
LinkedIn <http://eg.linkedin.com/pub/karim-allah-ahmed/13/829/550/>

[-- Attachment #1.2: Type: text/html, Size: 1620 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Enable virtual memory for Mini-os on ARM

[Samuel Thibault]

karim.allah.ahmed@gmail.com, le Mon 23 Dec 2013 10:23:01 +0000, a écrit :
> Why do I need to create pagetables and fill them for mini-os ?

I don't know the virtual memory of ARM, but on x86 we definitely need a
pagetable because very often we need to map/unmap things, like granted
mfns, i/o areas, foreign pages, etc.

Samuel

Re: Enable virtual memory for Mini-os on ARM

[Andrew Cooper]

On 23/12/2013 10:38, Samuel Thibault wrote:
> karim.allah.ahmed@gmail.com, le Mon 23 Dec 2013 10:23:01 +0000, a écrit :
>> Why do I need to create pagetables and fill them for mini-os ?
> I don't know the virtual memory of ARM, but on x86 we definitely need a
> pagetable because very often we need to map/unmap things, like granted
> mfns, i/o areas, foreign pages, etc.
>
> Samuel

More importantly, an x86 PV guest *cannot* run in non-paging mode.

This, I believe, is different to ARM, where an ARM guest is rather more
like x86 HVM guests, which have a full processor to play with.

~Andrew

>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Enable virtual memory for Mini-os on ARM

[Samuel Thibault]

Andrew Cooper, le Mon 23 Dec 2013 12:07:38 +0000, a écrit :
> On 23/12/2013 10:38, Samuel Thibault wrote:
> > karim.allah.ahmed@gmail.com, le Mon 23 Dec 2013 10:23:01 +0000, a écrit :
> >> Why do I need to create pagetables and fill them for mini-os ?
> > I don't know the virtual memory of ARM, but on x86 we definitely need a
> > pagetable because very often we need to map/unmap things, like granted
> > mfns, i/o areas, foreign pages, etc.
> 
> More importantly, an x86 PV guest *cannot* run in non-paging mode.

Sure, but that's not a ground reason like having to map some extra pages
etc.

Samuel

Re: Enable virtual memory for Mini-os on ARM

[karim.allah.ahmed]

[-- Attachment #1.1: Type: text/plain, Size: 1232 bytes --]

On Mon, Dec 23, 2013 at 10:38 AM, Samuel Thibault <
samuel.thibault@ens-lyon.org> wrote:

> karim.allah.ahmed@gmail.com, le Mon 23 Dec 2013 10:23:01 +0000, a écrit :
> > Why do I need to create pagetables and fill them for mini-os ?
>
> I don't know the virtual memory of ARM, but on x86 we definitely need a
> pagetable because very often we need to map/unmap things, like granted
> mfns, i/o areas, foreign pages, etc.
>

Can you please refer me to the part of the code related to any of these
cases ?

In ARM MMU is virtualized in HW. So you've an extra level of translation
that's controlled by the hypervisor.
So, Is there a reason for doing map/unmap on virtual address space not on
physical one ?

For me I would use virtual memory for:
1- Isolation between different entities ( processes ) - ( Is that
applicable to mini-os ? I can see that things like fork isn't supported by
mini-os [unsupported_function_crash(fork);], which makes me think that this
isn't applicable )
2- creating a contiguous mapping of scattered pages.

Are any of these a concern at the moment ?

Thanks.


> Samuel
>



-- 
Karim Allah Ahmed.
LinkedIn <http://eg.linkedin.com/pub/karim-allah-ahmed/13/829/550/>

[-- Attachment #1.2: Type: text/html, Size: 2082 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Enable virtual memory for Mini-os on ARM

[Samuel Thibault]

karim.allah.ahmed@gmail.com, le Mon 23 Dec 2013 12:52:44 +0000, a écrit :
> On Mon, Dec 23, 2013 at 10:38 AM, Samuel Thibault <samuel.thibault@ens-lyon.org
> > wrote:
> 
>     karim.allah.ahmed@gmail.com, le Mon 23 Dec 2013 10:23:01 +0000, a écrit :
>     > Why do I need to create pagetables and fill them for mini-os ?
> 
>     I don't know the virtual memory of ARM, but on x86 we definitely need a
>     pagetable because very often we need to map/unmap things, like granted
>     mfns, i/o areas, foreign pages, etc.
> 
> Can you please refer me to the part of the code related to any of these cases ?

Essentially everything that calls map_frames and friends.

- mini-os/gnttab.c calls gnttab_table = map_frames(frames, NR_GRANT_FRAMES);.
- I/O can be seen in mini-os/lib/sys.c: map_frames_ex(&first_mfn, n, 0, 1, 1,
DOMID_IO, NULL, _PAGE_PRESENT|_PAGE_RW);
- foreign pages can be seen in tools/libxc/xc_minios.c in
minios_privcmd_map_foreign_*

> In ARM MMU is virtualized in HW. So you've an extra level of translation that's
> controlled by the hypervisor.
> So, Is there a reason for doing map/unmap on virtual address space not on
> physical one ?

The physical one might be enough, I don't know the details of ARM
memory to be able to tell for sure.

> 1- Isolation between different entities ( processes ) - ( Is that applicable to
> mini-os ?

It is not.

> 2- creating a contiguous mapping of scattered pages.

Which can perhaps be done at the physical layer?

Samuel

Re: Enable virtual memory for Mini-os on ARM

[karim.allah.ahmed]

[-- Attachment #1.1: Type: text/plain, Size: 2435 bytes --]

On Mon, Dec 23, 2013 at 1:21 PM, Samuel Thibault <
samuel.thibault@ens-lyon.org> wrote:

> karim.allah.ahmed@gmail.com, le Mon 23 Dec 2013 12:52:44 +0000, a écrit :
> > On Mon, Dec 23, 2013 at 10:38 AM, Samuel Thibault <
> samuel.thibault@ens-lyon.org
> > > wrote:
> >
> >     karim.allah.ahmed@gmail.com, le Mon 23 Dec 2013 10:23:01 +0000, a
> écrit :
> >     > Why do I need to create pagetables and fill them for mini-os ?
> >
> >     I don't know the virtual memory of ARM, but on x86 we definitely
> need a
> >     pagetable because very often we need to map/unmap things, like
> granted
> >     mfns, i/o areas, foreign pages, etc.
> >
> > Can you please refer me to the part of the code related to any of these
> cases ?
>
> Essentially everything that calls map_frames and friends.
>
> - mini-os/gnttab.c calls gnttab_table = map_frames(frames,
> NR_GRANT_FRAMES);.
> - I/O can be seen in mini-os/lib/sys.c: map_frames_ex(&first_mfn, n, 0, 1,
> 1,
> DOMID_IO, NULL, _PAGE_PRESENT|_PAGE_RW);
> - foreign pages can be seen in tools/libxc/xc_minios.c in
> minios_privcmd_map_foreign_*
>
> > In ARM MMU is virtualized in HW. So you've an extra level of translation
> that's
> > controlled by the hypervisor.
> > So, Is there a reason for doing map/unmap on virtual address space not on
> > physical one ?
>
> The physical one might be enough, I don't know the details of ARM
> memory to be able to tell for sure.
>
> > 1- Isolation between different entities ( processes ) - ( Is that
> applicable to
> > mini-os ?
>
> It is not.
>
> > 2- creating a contiguous mapping of scattered pages.
>
> Which can perhaps be done at the physical layer?
>

I think even if we can change the "guest_to_machine translation" ( maybe
through XENMEM_populate_physmap ? ) that's controlled by the hypervisor,
this is going to be more expensive than simply updating a memory location
(page table) since all updates in the former case will have to go through
the hypervisor!

For me using physical addresses directly was useful for:
1- No code for setting up pagetables.
2- No memory wasted/used as pagetables. ( it's a tiny amount of memory
anyway )

So, if this [using physical addresses directly] ends up causing any
performance loss, I think it's not worth it then.


> Samuel
>



-- 
Karim Allah Ahmed.
LinkedIn <http://eg.linkedin.com/pub/karim-allah-ahmed/13/829/550/>

[-- Attachment #1.2: Type: text/html, Size: 3546 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Enable virtual memory for Mini-os on ARM

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1276 bytes --]

On lun, 2013-12-23 at 10:23 +0000, karim.allah.ahmed@gmail.com wrote:
> Hi,
> 
Hello!

> I'm currently working on porting mini-os to ARM.
> 
Cool... What's your usecase for that (I'm nosy, I know :-P)

> The work I've done so far was related:
> 
> * Changes to build system to build mini-os as zImage
> 
> * adding a linker script for arm target
> 
> * Splitting some source files between common and arch specific bits
> 
> * adding some hypercalls for arm ( mainly used at the moment for
> console_io and getting shared_info page ).
> 
> * some assembly code for early startup ( stack setup, mmu stuff etc )
> before jumping to c code.
> 
> * , etc
> 
> Now, it compiles and boots ( as zImage ). 
>
Cool... If I can ask, do you have the code somewhere publicly
accessible?

I appreciate what you said about this being still deeply experimental,
but it's something really interesting, and there may be people
interested in seeing/collaborating! :-)

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Enable virtual memory for Mini-os on ARM

[Chen Baozi]

On Dec 24, 2013, at 0:01, Dario Faggioli <dario.faggioli@citrix.com> wrote:

> On lun, 2013-12-23 at 10:23 +0000, karim.allah.ahmed@gmail.com wrote:
>> Hi,
>> 
> Hello!
> 
>> I'm currently working on porting mini-os to ARM.
>> 
> Cool... What's your usecase for that (I'm nosy, I know :-P)
> 
>> The work I've done so far was related:
>> 
>> * Changes to build system to build mini-os as zImage
>> 
>> * adding a linker script for arm target
>> 
>> * Splitting some source files between common and arch specific bits
>> 
>> * adding some hypercalls for arm ( mainly used at the moment for
>> console_io and getting shared_info page ).
>> 
>> * some assembly code for early startup ( stack setup, mmu stuff etc )
>> before jumping to c code.
>> 
>> * , etc
>> 
>> Now, it compiles and boots ( as zImage ). 
>> 
> Cool... If I can ask, do you have the code somewhere publicly
> accessible?
> 
> I appreciate what you said about this being still deeply experimental,
> but it's something really interesting, and there may be people
> interested in seeing/collaborating! :-)

+1

I’ve tried to build a basic framework to port mini-os on ARM this summer,
but stopped to fix my dev-board problem then. Looking forward to it.

Regards,

Baozi

> 
> Thanks and Regards,
> Dario
> 
> -- 
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Enable virtual memory for Mini-os on ARM

[karim.allah.ahmed]

[-- Attachment #1.1: Type: text/plain, Size: 1727 bytes --]

On Mon, Dec 23, 2013 at 4:01 PM, Dario Faggioli
<dario.faggioli@citrix.com>wrote:

> On lun, 2013-12-23 at 10:23 +0000, karim.allah.ahmed@gmail.com wrote:
> > Hi,
> >
> Hello!
>
> > I'm currently working on porting mini-os to ARM.
> >
> Cool... What's your usecase for that (I'm nosy, I know :-P)
>

I'm planning to clean up this work during Xmas vacation and will make it
public then.


> > The work I've done so far was related:
> >
> > * Changes to build system to build mini-os as zImage
> >
> > * adding a linker script for arm target
> >
> > * Splitting some source files between common and arch specific bits
> >
> > * adding some hypercalls for arm ( mainly used at the moment for
> > console_io and getting shared_info page ).
> >
> > * some assembly code for early startup ( stack setup, mmu stuff etc )
> > before jumping to c code.
> >
> > * , etc
> >
> > Now, it compiles and boots ( as zImage ).
> >
> Cool... If I can ask, do you have the code somewhere publicly
> accessible?
>
> No, not at the moment.

I appreciate what you said about this being still deeply experimental,
> but it's something really interesting, and there may be people
> interested in seeing/collaborating! :-)
>

I'm planning to clean up and push it a little bit further during Xmas time
and then I'll put the code in a public place early Jan.


> Thanks and Regards,
> Dario
>
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>
>


-- 
Karim Allah Ahmed.
LinkedIn <http://eg.linkedin.com/pub/karim-allah-ahmed/13/829/550/>

[-- Attachment #1.2: Type: text/html, Size: 3076 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Enable virtual memory for Mini-os on ARM

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1140 bytes --]

Re-adding the list, hope you don't mind]

On mar, 2013-12-24 at 01:35 +0000, karim.allah.ahmed@gmail.com wrote:
> On Mon, Dec 23, 2013 at 4:01 PM, Dario Faggioli
> <dario.faggioli@citrix.com> wrote:
>         
>         Cool... What's your usecase for that (I'm nosy, I know :-P) 
> 
> I'm just playing with my cubieboard, so this can be hardly considered
> as a use case :)
> 
Well, more than enough! :-)

>         I appreciate what you said about this being still deeply
>         experimental,
>         but it's something really interesting, and there may be people
>         interested in seeing/collaborating! :-)
>
> I'm planning to clean up and push it a little bit further during Xmas
> time and then I'll put the code in a public place early Jan.
> 
Ok... Looking forward to it. Let us know when it's online! :-)

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Enable virtual memory for Mini-os on ARM

[Ian Campbell]

On Mon, 2013-12-23 at 12:52 +0000, karim.allah.ahmed@gmail.com wrote:
> So, Is there a reason for doing map/unmap on virtual address space not
> on physical one ? 

The only thing which springs to mind is if you needed to control the
cacheability attributes of the mappings (see [0] for the requirements).
However I think these are met by just enabling the SCTLR.C bit, SCTLR.M
isn't needed.

Things like foreign page mappings (e.g. via grant tables) and ballooning
happen via the stage 2 paging, so no need for stage 1 MMU to be enabled
there, I don't think.

I think it would be reasonable to omit page table setup from arm mini-os
for now and reconsider it if/when we discover a need for it.

Ian.

[0]
http://xenbits.xen.org/docs/unstable/hypercall/arm/include,public,arch-arm.h.html#incontents_arm_abi