[Qemu-devel] Using native libraries

[Qemu-devel] Using native libraries

[Luke -Jr]

I'm sure someone's probably had a similar idea before, and it's probably not 
practical for some reason I'm overlooking-- but is there a reason Qemu can't 
dynamically translate library calls to use the native libraries instead of 
requiring emulated libraries as well?

Re: [Qemu-devel] Using native libraries

[Carlos A. M. dos Santos]

On 5/21/07, Luke -Jr <luke@dashjr.org> wrote:
> I'm sure someone's probably had a similar idea before, and it's probably not
> practical for some reason I'm overlooking-- but is there a reason Qemu can't
> dynamically translate library calls to use the native libraries instead of
> requiring emulated libraries as well?

That would be a totally different class of emulation, much more
similar to the FreeBSD "linuxulator" than to what QEMU is intended to
do.

-- 
Carlos A. M. dos Santos

Re: [Qemu-devel] Using native libraries

[H. Peter Anvin]

Luke -Jr wrote:
> I'm sure someone's probably had a similar idea before, and it's probably not 
> practical for some reason I'm overlooking-- but is there a reason Qemu can't 
> dynamically translate library calls to use the native libraries instead of 
> requiring emulated libraries as well?

The easiest way to do that is by having an "RPC stub library" in the
emulated environment which contains nothing but trap instructions --
like system calls -- that can be intercepted on the other side.  At that
point, one has to do translation of data types and pointers (called
"thunking") before invoking the native library.  Generating such
"thunks" is usually done automatically, but it is still a fairly major
effort.

This kind of stuff is what underlies things like running PowerMac
applications on IntelMacs (or 68kmac apps on PowerMac).

	-hpa

Re: [Qemu-devel] Using native libraries

[Luke -Jr]

On Monday 21 May 2007 16:13, H. Peter Anvin wrote:
> The easiest way to do that is by having an "RPC stub library" in the
> emulated environment which contains nothing but trap instructions --
> like system calls -- that can be intercepted on the other side.  At that
> point, one has to do translation of data types and pointers (called
> "thunking") before invoking the native library.  Generating such
> "thunks" is usually done automatically, but it is still a fairly major
> effort.

But can it be done at the JIT stage?

Re: [Qemu-devel] Using native libraries

[Gwenole Beauchesne]

Hi,

> I'm sure someone's probably had a similar idea before, and it's  
> probably not
> practical for some reason I'm overlooking-- but is there a reason  
> Qemu can't
> dynamically translate library calls to use the native libraries  
> instead of
> requiring emulated libraries as well?

It should be possible, and I have just released an old PoC. I might  
work again on that once I get more free time in 3 weeks. ;-)
<http://gwenole.beauchesne.info/blog#qemuabout_native_libraries_bridge>

I am copying the article hereunder for reference.

[...]

So, what’s the status on the QEMU project side? Well, here is a  
tarball and a patch provided as is for the QEMU PoC. It passes the  
QEMU testthread test and bridges enough of libpthread and librt for  
that purpose. That implementation is weak insofar as nativelib lookup  
is really sub-optimal. Other implementations are meant to run-time  
patch ld.so, this provides more flexibily and permits to selectively  
nativelib a few functions (e.g. memcpy et al.), not the whole library.

How to build?

Please bear in mind this is just a PoC and is very incomplete in  
function coverage. If you still want to have a look, grab qemu 0.9.0  
sources, preferably a 2007/02/14 snapshot, and apply both tarball and  
patch. Configure for a i386-linux-user target and make it. Then, in  
the i386-linux-user directory, delete the nativelib-*.so files and re- 
make them with CC=”gcc -m32”. This assumes your host was x86_64.  
Otherwise, simply go to an i386 system and manually build the  
nativelibs.

How does it work?

* Target libraries (here, linux/i386) are statically compiled with  
the functions reducing to a few assembly code: the nativelib cookie.  
The target libraries (e.g. nativelib-libpthread.so) are placed in a  
special directory so that the open() syscall can actually resolve to  
this directory when target system libraries are looked up for.

* The nativelib cookie is actually an illegal instruction (0x0f 0x3c)  
that is caught at code translation time. This illegal instruction is  
followed by two other instructions describing the native function to  
call (library code, function code). This could have been raw IDs but  
this makes debugging/disassembling trickier.

* The do_nativelib() hook is then called and the library and function  
IDs are decoded and dispatched.

Regards,
Gwenolé.