Re: "Barry Silverman": Setting GDT entries for Thread Local Storage

Re: "Barry Silverman": Setting GDT entries for Thread Local Storage

[Keir Fraser]

> I am trying to port the Fedora version of Linux to xeno-linux. It involves
> porting the NPTL Library (New Posix Thread Library), and TLS (Thread Local
> Storage) kernel changes which Redhat backported from 2.6 into 2.4.
> 
> It seems that with TLS the guest Fedora must:
> 1) Have a per-cpu GDT
> 2) Be able to change some descriptors within the GDT when the
> "set_thread_area" syscall is executed, and on thread context switches. TLS
> maintain a small array of free descriptors which can be freely used by for
> thread local storage by the guest libc - (mediated by the guest syscall)
> 
> In looking at the code, I have seen how a Xen can set up its own per CPU GDT
> when Domain 0 is building it, but how would a non-Domain 0 guest change GDT
> descriptors on the fly for "set_thread_area"?

You would use HYPERVISOR_update_descriptor(), which takes the physical
address of the descriptor-table entry to be updated, and the two
32-bit values to write into that descriptor entry.

This shoudl all work straightforwardly, unless TLS tries to install
segments that are "full size" (i.e. 4GB). I know that one of the
recent glibc threading solutions tries to do this --- it won't work
with Xen because we clip segments so that they do not overlap with
Xen's reserved area in the top 64MB of the virtual address space.

 -- Keir


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn

Re: Re: "Barry Silverman": Setting GDT entries for Thread Local Storage

[Zachary Amsden]

Keir Fraser wrote:

>>I am trying to port the Fedora version of Linux to xeno-linux. It involves
>>porting the NPTL Library (New Posix Thread Library), and TLS (Thread Local
>>Storage) kernel changes which Redhat backported from 2.6 into 2.4.
>>
>>    
>>
>...
>This shoudl all work straightforwardly, unless TLS tries to install
>segments that are "full size" (i.e. 4GB). I know that one of the
>recent glibc threading solutions tries to do this --- it won't work
>with Xen because we clip segments so that they do not overlap with
>Xen's reserved area in the top 64MB of the virtual address space.
>  
>
I don't want to throw a monkeywrench in your plans, but another 
potential trouble spot is the x86 vsyscall interface in 2.6.  The 
vsyscall region sits at the top of the virtual address space, and could 
conflict with the Xen mapping.  You may have to consider remapping Xen 
to another memory region, but I'm not sure where there may be other 
trouble areas with Windows domains (map?).  At some point in the 
forseeable future, it might no longer be possible to locate Xen at an OS 
neutral location, so perhaps it is worth considering the remapping 
problem now.

Zachary Amsden
zamsden@cisco.com


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn

Re: Re: "Barry Silverman": Setting GDT entries for Thread Local Storage

[Keir Fraser]

> I don't want to throw a monkeywrench in your plans, but another 
> potential trouble spot is the x86 vsyscall interface in 2.6.  The 
> vsyscall region sits at the top of the virtual address space, and could 
> conflict with the Xen mapping.  You may have to consider remapping Xen 
> to another memory region, but I'm not sure where there may be other 
> trouble areas with Windows domains (map?).  At some point in the 
> forseeable future, it might no longer be possible to locate Xen at an OS 
> neutral location, so perhaps it is worth considering the remapping 
> problem now.

The vsyscall interface happens to sit at the top of virtual memory in
native Linux, I think mainly because the vsyscall page is allocated
using the Linux 'fixmap' mechanism.

However, the address of the vsyscall interface is passed to
applications as part of the ELF-loading protocol. It should therefore
be possible to map the vsyscall stubs to a different virtual address
in Xenolinux (Fingers crossed!).

I hope we don't have to go down a 'map Xen to guest-specific location'
kind of route. I think that position-independent code in x86 is likely
to perform below-par (no PC-relative addressing; allocating a base
register is painful on a register-starved architecture).

 -- Keir


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn

Re: Re: "Barry Silverman": Setting GDT entries for Thread Local Storage

[Keir Fraser]

> However, the address of the vsyscall interface is passed to
> applications as part of the ELF-loading protocol. It should therefore
> be possible to map the vsyscall stubs to a different virtual address
> in Xenolinux (Fingers crossed!).

Just to clarify: the address of the vsyscall code appears to be
specified on by *kernel* side of the ELF-loading protocol. It
shouldn't be hardcoded in any user-space application or library (so,
for example, you shouldn't need to recompile ld.so).

 -- Keir


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn

Re: Re: "Barry Silverman": Setting GDT entries for Thread Local Storage

[Zachary Amsden]

Keir Fraser wrote:

>The vsyscall interface happens to sit at the top of virtual memory in
>native Linux, I think mainly because the vsyscall page is allocated
>using the Linux 'fixmap' mechanism.
>  
>
>However, the address of the vsyscall interface is passed to
>applications as part of the ELF-loading protocol. It should therefore
>be possible to map the vsyscall stubs to a different virtual address
>in Xenolinux (Fingers crossed!).
>  
>
I think that might work, but it seems somewhat unnecessary.  Does Xen 
use the very top part of VM space for any special purpose - e.g. stack?  
Why not just make Xen itself a kernel fixmap entry to avoid collisions?

>I hope we don't have to go down a 'map Xen to guest-specific location'
>kind of route. I think that position-independent code in x86 is likely
>to perform below-par (no PC-relative addressing; allocating a base
>register is painful on a register-starved architecture).
>  
>
Not to mention having to deal with relocatable pointers across different 
domains!  Ugh.

Zachary Amsden
zamsden@cisco.com


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn

RE: Re: "Barry Silverman": Setting GDT entries for Thread Local Storage

[Barry Silverman]

Re: vsyscall mapping issues

It seems that QEMU has the same issue as Xen with vsyscall (their fast
emulator can't map kernel virtual addresses > 0xa8000000) so they created a
kernel patch to dynamically generate the "fixed" symbol FIXADDR_TOP from a
config parameter, and to have the Makefile generate the .lds file used to
build the vsyscall DSO images. It seems to me that a similar solution can be
used for Xen.

http://mail.gnu.org/archive/html/qemu-devel/2003-12/msg00020.html


>>This should all work straightforwardly, unless TLS tries to install
>>segments that are "full size" (i.e. 4GB). I know that one of the
>>recent glibc threading solutions tries to do this --- it won't work
>>with Xen because we clip segments so that they do not overlap with
>>Xen's reserved area in the top 64MB of the virtual address space.

 -- Keir


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel



-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn

Re: Re: "Barry Silverman": Setting GDT entries for Thread Local Storage

[Keir Fraser]

> Re: vsyscall mapping issues
> 
> It seems that QEMU has the same issue as Xen with vsyscall (their fast
> emulator can't map kernel virtual addresses > 0xa8000000) so they created a
> kernel patch to dynamically generate the "fixed" symbol FIXADDR_TOP from a
> config parameter, and to have the Makefile generate the .lds file used to
> build the vsyscall DSO images. It seems to me that a similar solution can be
> used for Xen.
> 
> http://mail.gnu.org/archive/html/qemu-devel/2003-12/msg00020.html

The Xen port of Linux already chnages values like FIXADDR_TOP,
compared with native x86 Linux. The vsyscall stuff will therefore 'just
work'. 

 -- Keir


-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn