[uml-devel] system call accessing the host os

[uml-devel] system call accessing the host os

[Olivier Crameri]

Hi,

I'm currently working on a project in which we are using UML to  
access the host os and perform some operations.
More precisely, we would like to have a system call in UML that  
parses a file in the host os.

Since the UML kernel is a host process, I naively thought that I  
could use regular C directly. Indeed, I was able to create my system  
call, and then have some other C files compiled using host os headers  
such as <stdio.h> and <stdlib.h> by putting them in the USER_OBJS  
list of the Makefile.

Unfortunately, I'm having some weird issues that I can't really  
understand. I can read the file using fread, but only in a buffer  
that I allocated using um_kmalloc. If I use a buffer allocated by  
malloc, the fread fails. Then, even if I replace all  my mallocs by  
um_kmallocs, some libc functions (such as sscanf) don't seem to work  
properly. I guess I'm missing something, but I can't figure out what.

If anyone of you can help me, I'd really appreciate,

With best regards,

Olivier

P.S.: I'm using 2.6.15.6 with the latest 2.6.15 patch and the guest  
kernel runs in skas3 mode.


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] system call accessing the host os

[D. Bahi]

[-- Attachment #1: Type: text/plain, Size: 1857 bytes --]

what about this?

http://user-mode-linux.sourceforge.net/iomem.html

Olivier Crameri wrote:
> Hi,
>
> I'm currently working on a project in which we are using UML to access
> the host os and perform some operations.
> More precisely, we would like to have a system call in UML that parses
> a file in the host os.
>
> Since the UML kernel is a host process, I naively thought that I could
> use regular C directly. Indeed, I was able to create my system call,
> and then have some other C files compiled using host os headers such
> as <stdio.h> and <stdlib.h> by putting them in the USER_OBJS list of
> the Makefile.
>
> Unfortunately, I'm having some weird issues that I can't really
> understand. I can read the file using fread, but only in a buffer that
> I allocated using um_kmalloc. If I use a buffer allocated by malloc,
> the fread fails. Then, even if I replace all  my mallocs by
> um_kmallocs, some libc functions (such as sscanf) don't seem to work
> properly. I guess I'm missing something, but I can't figure out what.
>
> If anyone of you can help me, I'd really appreciate,
>
> With best regards,
>
> Olivier
>
> P.S.: I'm using 2.6.15.6 with the latest 2.6.15 patch and the guest
> kernel runs in skas3 mode.
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting
> language
> that extends applications into web and mobile media. Attend the live
> webcast
> and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> User-mode-linux-devel mailing list
> User-mode-linux-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 187 bytes --]

Re: [uml-devel] system call accessing the host os

[Olivier Crameri]

Well, yes, I could use this.

But what I'm actually trying to understand is how to properly execute  
regular C within the UML kernel to access the host OS.

Thks,

Olivier

On 5 avr. 06, at 19:18, D. Bahi wrote:

> what about this?
>
> http://user-mode-linux.sourceforge.net/iomem.html
>
> Olivier Crameri wrote:
>> Hi,
>>
>> I'm currently working on a project in which we are using UML to  
>> access
>> the host os and perform some operations.
>> More precisely, we would like to have a system call in UML that  
>> parses
>> a file in the host os.
>>
>> Since the UML kernel is a host process, I naively thought that I  
>> could
>> use regular C directly. Indeed, I was able to create my system call,
>> and then have some other C files compiled using host os headers such
>> as <stdio.h> and <stdlib.h> by putting them in the USER_OBJS list of
>> the Makefile.
>>
>> Unfortunately, I'm having some weird issues that I can't really
>> understand. I can read the file using fread, but only in a buffer  
>> that
>> I allocated using um_kmalloc. If I use a buffer allocated by malloc,
>> the fread fails. Then, even if I replace all  my mallocs by
>> um_kmallocs, some libc functions (such as sscanf) don't seem to work
>> properly. I guess I'm missing something, but I can't figure out what.
>>
>> If anyone of you can help me, I'd really appreciate,
>>
>> With best regards,
>>
>> Olivier
>>
>> P.S.: I'm using 2.6.15.6 with the latest 2.6.15 patch and the guest
>> kernel runs in skas3 mode.
>>
>>
>> -------------------------------------------------------
>> This SF.Net email is sponsored by xPML, a groundbreaking scripting
>> language
>> that extends applications into web and mobile media. Attend the live
>> webcast
>> and join the prime developer group breaking into this new coding
>> territory!
>> http://sel.as-us.falkag.net/sel? 
>> cmd=lnk&kid=110944&bid=241720&dat=121642
>> _______________________________________________
>> User-mode-linux-devel mailing list
>> User-mode-linux-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel
>




-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] system call accessing the host os

[D. Bahi]

[-- Attachment #1: Type: text/plain, Size: 3007 bytes --]

well there are several examples of this in the arch/um/drivers area.
<blah>_user.c and <blah>_kern.c break out the userspace and
kernel space parts of UML's dual personality.

Olivier Crameri wrote:
> Well, yes, I could use this.
>
> But what I'm actually trying to understand is how to properly execute
> regular C within the UML kernel to access the host OS.
>
> Thks,
>
> Olivier
>
> On 5 avr. 06, at 19:18, D. Bahi wrote:
>
>> what about this?
>>
>> http://user-mode-linux.sourceforge.net/iomem.html
>>
>> Olivier Crameri wrote:
>>> Hi,
>>>
>>> I'm currently working on a project in which we are using UML to access
>>> the host os and perform some operations.
>>> More precisely, we would like to have a system call in UML that parses
>>> a file in the host os.
>>>
>>> Since the UML kernel is a host process, I naively thought that I could
>>> use regular C directly. Indeed, I was able to create my system call,
>>> and then have some other C files compiled using host os headers such
>>> as <stdio.h> and <stdlib.h> by putting them in the USER_OBJS list of
>>> the Makefile.
>>>
>>> Unfortunately, I'm having some weird issues that I can't really
>>> understand. I can read the file using fread, but only in a buffer that
>>> I allocated using um_kmalloc. If I use a buffer allocated by malloc,
>>> the fread fails. Then, even if I replace all  my mallocs by
>>> um_kmallocs, some libc functions (such as sscanf) don't seem to work
>>> properly. I guess I'm missing something, but I can't figure out what.
>>>
>>> If anyone of you can help me, I'd really appreciate,
>>>
>>> With best regards,
>>>
>>> Olivier
>>>
>>> P.S.: I'm using 2.6.15.6 with the latest 2.6.15 patch and the guest
>>> kernel runs in skas3 mode.
>>>
>>>
>>> -------------------------------------------------------
>>> This SF.Net email is sponsored by xPML, a groundbreaking scripting
>>> language
>>> that extends applications into web and mobile media. Attend the live
>>> webcast
>>> and join the prime developer group breaking into this new coding
>>> territory!
>>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
>>>
>>> _______________________________________________
>>> User-mode-linux-devel mailing list
>>> User-mode-linux-devel@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel
>>
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting
> language
> that extends applications into web and mobile media. Attend the live
> webcast
> and join the prime developer group breaking into this new coding
> territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> User-mode-linux-devel mailing list
> User-mode-linux-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 187 bytes --]

Re: [uml-devel] system call accessing the host os

[Jeff Dike]

On Wed, Apr 05, 2006 at 07:16:22PM +0200, Olivier Crameri wrote:
> Unfortunately, I'm having some weird issues that I can't really  
> understand. I can read the file using fread, but only in a buffer  
> that I allocated using um_kmalloc. If I use a buffer allocated by  
> malloc, the fread fails. Then, even if I replace all  my mallocs by  
> um_kmallocs, some libc functions (such as sscanf) don't seem to work  
> properly. I guess I'm missing something, but I can't figure out what.

Define "fails" and "don't seem to work properly".

If your buffers are larger than 128K, then libc malloc gets turned into
UML kernel vmalloc.  In this case, the buffer isn't mapped, and
passing it into a system call will make it return -EFAULT.  The
easiest workaround for this is to memset the thing immediately after
allocating it.

Also, if you're using the libc things you're talking about, watch out
for your stack consumption.  By default, you get two pages (8K).
printf will completely use it up, so it is unusable in kernel code.

UML kernel stack size is configurable - CONFIG_KERNEL_STACK_ORDER -
bumping that to 3 will double the kernel stack size.  If problems then
go away, then you know that libc is overflowing your stack.

				Jeff


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] system call accessing the host os

[Olivier Crameri]

Thanks for the answer. I bumped CONFIG_KERNEL_STACK_ORDER to 3, but it
doesn't help.

I have a system call that does nothing but execute my code to parse my
file in the host.
When I said that using malloc didn't work, I meant the following: I'm
using malloc to allocate a buffer in the UML kernel. I'm not passing a
user level buffer to the system call. Then, when I use fread to read
my file into this buffer, if I read a small number of bytes, it works.
However when I try to fread the entire file (38k), fread returns 0.
The same thing using a buffer returned by um_kmalloc works perfectly
(i.e. returns 1 and the buffer is filled with the file data).

Later on, I use sscanf to parse my buffer. No matter what happens,
sscanf returns 0, which is wrong even if there is no error. The exact
same code compiled in the host works perfectly.

Again, thank you for your help, I really appreciate.
Olivier

On 4/5/06, Jeff Dike <jdike@addtoit.com> wrote:
> On Wed, Apr 05, 2006 at 07:16:22PM +0200, Olivier Crameri wrote:
> > Unfortunately, I'm having some weird issues that I can't really
> > understand. I can read the file using fread, but only in a buffer
> > that I allocated using um_kmalloc. If I use a buffer allocated by
> > malloc, the fread fails. Then, even if I replace all  my mallocs by
> > um_kmallocs, some libc functions (such as sscanf) don't seem to work
> > properly. I guess I'm missing something, but I can't figure out what.
>
> Define "fails" and "don't seem to work properly".
>
> If your buffers are larger than 128K, then libc malloc gets turned into
> UML kernel vmalloc.  In this case, the buffer isn't mapped, and
> passing it into a system call will make it return -EFAULT.  The
> easiest workaround for this is to memset the thing immediately after
> allocating it.
>
> Also, if you're using the libc things you're talking about, watch out
> for your stack consumption.  By default, you get two pages (8K).
> printf will completely use it up, so it is unusable in kernel code.
>
> UML kernel stack size is configurable - CONFIG_KERNEL_STACK_ORDER -
> bumping that to 3 will double the kernel stack size.  If problems then
> go away, then you know that libc is overflowing your stack.
>
>                                 Jeff
>


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] system call accessing the host os

[Jeff Dike]

On Wed, Apr 05, 2006 at 10:20:44PM +0200, Olivier Crameri wrote:
> Thanks for the answer. I bumped CONFIG_KERNEL_STACK_ORDER to 3, but it
> doesn't help.

I didn't think so, since the symptoms were wrong.  However, keep that
in the back of your mind since you're dealing with some piggy parts of
libc.

> When I said that using malloc didn't work, I meant the following: I'm
> using malloc to allocate a buffer in the UML kernel. I'm not passing a
> user level buffer to the system call. 

38K isn't big enough for it to fall back to vmalloc, so the pages not
being present probably isn't an issue.

> Then, when I use fread to read
> my file into this buffer, if I read a small number of bytes, it works.
> However when I try to fread the entire file (38k), fread returns 0.

What's the break point between working and non-working?

> Later on, I use sscanf to parse my buffer. No matter what happens,
> sscanf returns 0, which is wrong even if there is no error. The exact
> same code compiled in the host works perfectly.

This, I don't understand.

What version of UML are you using?

				Jeff


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] system call accessing the host os

[Olivier Crameri]

>
>> Then, when I use fread to read
>> my file into this buffer, if I read a small number of bytes, it  
>> works.
>> However when I try to fread the entire file (38k), fread returns 0.
>
> What's the break point between working and non-working?

It looks like it's on a page boundary. 4095 bytes is ok, 4096 is not
I'm using 2.6.15.6 with the latest 2.6.15 patch from Blaisorblade's  
website.

Regards,

Olivier


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] system call accessing the host os

[Jeff Dike]

On Fri, Apr 07, 2006 at 02:20:00AM +0200, Blaisorblade wrote:
> No, I made it "greater than 4k" in 2.4.24 time... in fact it's what's 
> happening to him (see his last mail):

Oh yeah.  In that case, the memset thing I mentioned earlier might help.

				Jeff


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] system call accessing the host os

[Blaisorblade]

On Wednesday 05 April 2006 20:18, Jeff Dike wrote:
> On Wed, Apr 05, 2006 at 07:16:22PM +0200, Olivier Crameri wrote:
> > Unfortunately, I'm having some weird issues that I can't really
> > understand. I can read the file using fread, but only in a buffer
> > that I allocated using um_kmalloc. If I use a buffer allocated by
> > malloc, the fread fails. Then, even if I replace all  my mallocs by
> > um_kmallocs, some libc functions (such as sscanf) don't seem to work
> > properly. I guess I'm missing something, but I can't figure out what.
>
> Define "fails" and "don't seem to work properly".
>
> If your buffers are larger than 128K, then libc malloc gets turned into
> UML kernel vmalloc.

No, I made it "greater than 4k" in 2.4.24 time... in fact it's what's 
happening to him (see his last mail):

void *__wrap_malloc(int size)
{
        void *ret;

        if(!CAN_KMALLOC())
                return(__real_malloc(size));
        else if(size <= PAGE_SIZE) /* finding contiguos pages can be hard*/
                ret = um_kmalloc(size);
        else ret = um_vmalloc(size);
}

128K is the maximum conceivable kmallocable size, but it hardly works due to 
fragmentation.

> In this case, the buffer isn't mapped, and 
> passing it into a system call will make it return -EFAULT.  The
> easiest workaround for this is to memset the thing immediately after
> allocating it.

> Also, if you're using the libc things you're talking about, watch out
> for your stack consumption.  By default, you get two pages (8K).

By default on UML you get 4 pages, 16K, i.e. PAGE_SIZE *  (1 << 
CONFIG_KERNEL_STACK_ORDER) == 4k * (1<<2) == 4k * 4.

> printf will completely use it up, so it is unusable in kernel code.

Indeed. sprintf() + write(1) works, instead - grep for my_printf in the source 
code, I've coded this hack up.

> UML kernel stack size is configurable - CONFIG_KERNEL_STACK_ORDER -
> bumping that to 3 will double the kernel stack size.  If problems then
> go away, then you know that libc is overflowing your stack.

> 				Jeff

-- 
Inform me of my mistakes, so I can keep imitating Homer Simpson's "Doh!".
Paolo Giarrusso, aka Blaisorblade (Skype ID "PaoloGiarrusso", ICQ 215621894)
http://www.user-mode-linux.org/~blaisorblade

	

	
		
___________________________________ 
Yahoo! Mail: gratis 1GB per i messaggi e allegati da 10MB 
http://mail.yahoo.it



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] system call accessing the host os

[Olivier Crameri]

Great, thanks a lot. Now I understand what's happening. It was  
actually calling um_vmalloc....

Olivier


On 7 avr. 06, at 01:42, Jeff Dike wrote:

> On Fri, Apr 07, 2006 at 02:20:00AM +0200, Blaisorblade wrote:
>> No, I made it "greater than 4k" in 2.4.24 time... in fact it's what's
>> happening to him (see his last mail):
>
> Oh yeah.  In that case, the memset thing I mentioned earlier might  
> help.
>
> 				Jeff



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel