how to deal with copy_to_user returning non zero

how to deal with copy_to_user returning non zero

[Anh Dinh]

[-- Attachment #1.1: Type: text/plain, Size: 1271 bytes --]

I'm testing out a hypercall that takes in large data (many MB) from the
user space and simply copy the data back.

For copying in, I call xmalloc_array() for about 4MB at a time and call
copy_from_user() successfully for the entire input.

The problem is with copy_to_user() which returns non-zero values. I tried
to vary the size of the data being copy, but still unable to copy the whole
data back.

What are the cause of copy_to_user() failure? I checked the source I got
quite lost at the code.

How could I make sure all data is copy back? Surely it is not a memory
limitation problem, because I copy_from_user() succeeds for over 100MB.


******
long do_test_copy(long n, unsigned char *in, unsigned char *out){
        unsigned char **tmp;
        long ret;
        int m = n/(MAX_ALLOC);
        tmp = xmalloc_array(unsigned char*,m);

        for (int i=0; i<m; i++){
                tmp[i] = xmalloc_array(unsigned char, MAX_ALLOC);
                copy_from_user(tmp[i],in+i*MAX_ALLOC,MAX_ALLOC);
        }

        for (int i=0; i<m; i++){
                if (tmp[i]){
                        ret =
copy_to_user(out+i*MAX_ALLOC,tmp[i],MAX_ALLOC);
                        xfree(tmp[i]);
                }
        }
        xfree(tmp);
        return 0;
}
****

[-- Attachment #1.2: Type: text/html, Size: 1845 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: how to deal with copy_to_user returning non zero

[Andrew Cooper]

[-- Attachment #1.1: Type: text/plain, Size: 1814 bytes --]

On 07/11/14 10:38, Anh Dinh wrote:
> I'm testing out a hypercall that takes in large data (many MB) from
> the user space and simply copy the data back. 
>
> For copying in, I call xmalloc_array() for about 4MB at a time and
> call copy_from_user() successfully for the entire input. 
>
> The problem is with copy_to_user() which returns non-zero values. I
> tried to vary the size of the data being copy, but still unable to
> copy the whole data back. 
>
> What are the cause of copy_to_user() failure? I checked the source I
> got quite lost at the code. 
>
> How could I make sure all data is copy back? Surely it is not a memory
> limitation problem, because I copy_from_user() succeeds for over 100MB. 
>

copy_to_user() returns non-zero if it encounters a fault while moving
data, generally a pagefault.

>
> ******
> long do_test_copy(long n, unsigned char *in, unsigned char *out){

This looks quite bogus.  You should have XEN_GUEST_HANDLE() for a
pointer into guest memory.  A bare pointer like *in is completely wrong.

~Andrew

>         unsigned char **tmp; 
>         long ret;
>         int m = n/(MAX_ALLOC); 
>         tmp = xmalloc_array(unsigned char*,m); 
>
>         for (int i=0; i<m; i++){
>                 tmp[i] = xmalloc_array(unsigned char, MAX_ALLOC); 
>                 copy_from_user(tmp[i],in+i*MAX_ALLOC,MAX_ALLOC); 
>         }
>         
>         for (int i=0; i<m; i++){
>                 if (tmp[i]){
>                         ret =
> copy_to_user(out+i*MAX_ALLOC,tmp[i],MAX_ALLOC);                 
>                         xfree(tmp[i]); 
>                 }
>         }
>         xfree(tmp);
>         return 0;       
> }
> ****
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


[-- Attachment #1.2: Type: text/html, Size: 4489 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: how to deal with copy_to_user returning non zero

[And Dinh]

[-- Attachment #1.1: Type: text/plain, Size: 2237 bytes --]

how does it get page fault? I made sure the output buffer at the user space is properly allocated with the correct  size.

When page fault, do I have no choice but abort? It seems calling the hypercall again does not solve it.




> On 7 Nov 2014, at 6:44 pm, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> 
>> On 07/11/14 10:38, Anh Dinh wrote:
>> I'm testing out a hypercall that takes in large data (many MB) from the user space and simply copy the data back. 
>> 
>> For copying in, I call xmalloc_array() for about 4MB at a time and call copy_from_user() successfully for the entire input. 
>> 
>> The problem is with copy_to_user() which returns non-zero values. I tried to vary the size of the data being copy, but still unable to copy the whole data back. 
>> 
>> What are the cause of copy_to_user() failure? I checked the source I got quite lost at the code. 
>> 
>> How could I make sure all data is copy back? Surely it is not a memory limitation problem, because I copy_from_user() succeeds for over 100MB. 
> 
> copy_to_user() returns non-zero if it encounters a fault while moving data, generally a pagefault.
> 
>> 
>> ******
>> long do_test_copy(long n, unsigned char *in, unsigned char *out){
> 
> This looks quite bogus.  You should have XEN_GUEST_HANDLE() for a pointer into guest memory.  A bare pointer like *in is completely wrong.
> 
> ~Andrew
> 
>>         unsigned char **tmp; 
>>         long ret;
>>         int m = n/(MAX_ALLOC); 
>>         tmp = xmalloc_array(unsigned char*,m); 
>> 
>>         for (int i=0; i<m; i++){
>>                 tmp[i] = xmalloc_array(unsigned char, MAX_ALLOC); 
>>                 copy_from_user(tmp[i],in+i*MAX_ALLOC,MAX_ALLOC); 
>>         }
>>         
>>         for (int i=0; i<m; i++){
>>                 if (tmp[i]){
>>                         ret = copy_to_user(out+i*MAX_ALLOC,tmp[i],MAX_ALLOC);                 
>>                         xfree(tmp[i]); 
>>                 }
>>         }
>>         xfree(tmp);
>>         return 0;       
>> }
>> ****
>> 
>> 
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel
> 

[-- Attachment #1.2: Type: text/html, Size: 5009 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: how to deal with copy_to_user returning non zero

[Andrew Cooper]

[-- Attachment #1.1: Type: text/plain, Size: 758 bytes --]

On 07/11/14 10:57, And Dinh wrote:
> how does it get page fault? I made sure the output buffer at the user
> space is properly allocated with the correct  size.
>
> When page fault, do I have no choice but abort? It seems calling the
> hypercall again does not solve it.
>

And nothing guarentees that your userspace process is in context when
Xen is running, or that the kernel hasn't played with the pagetables
behind your back.

You must use the hypercall buffer mechanism to avoid issues like this. 
See the hypercall implementations in libxc.  In Xen, you must have a
XEN_GUEST_HANDLE() which is an opaque reference to your buffer, and use
copy_{to,from}_guest() rather than {to/from}_user(), which is generally
only safe for kernel addresses.

~Andrew

[-- Attachment #1.2: Type: text/html, Size: 1360 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: how to deal with copy_to_user returning non zero

[Anh Dinh]

[-- Attachment #1.1: Type: text/plain, Size: 1186 bytes --]

Thanks,

I've found out the reason it page-faulting is because I used malloc() to
allocate the output buffer, which turns out to allocate lazily. Therefore
the hypervisor page-fault because the memory is still waiting to be mapped
by the kernel.

I simply touched all the allocated memory, and it works fine now.

Cheers.


On 7 November 2014 19:01, Andrew Cooper <andrew.cooper3@citrix.com> wrote:

>  On 07/11/14 10:57, And Dinh wrote:
>
> how does it get page fault? I made sure the output buffer at the user
> space is properly allocated with the correct  size.
>
>  When page fault, do I have no choice but abort? It seems calling the
> hypercall again does not solve it.
>
>
> And nothing guarentees that your userspace process is in context when Xen
> is running, or that the kernel hasn't played with the pagetables behind
> your back.
>
> You must use the hypercall buffer mechanism to avoid issues like this.
> See the hypercall implementations in libxc.  In Xen, you must have a
> XEN_GUEST_HANDLE() which is an opaque reference to your buffer, and use
> copy_{to,from}_guest() rather than {to/from}_user(), which is generally
> only safe for kernel addresses.
>
> ~Andrew
>

[-- Attachment #1.2: Type: text/html, Size: 1979 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: how to deal with copy_to_user returning non zero

[Ian Campbell]

On Sun, 2014-11-09 at 09:22 +0800, Anh Dinh wrote:
> Thanks,
> 
> 
> I've found out the reason it page-faulting is because I used malloc()
> to allocate the output buffer, which turns out to allocate lazily.
> Therefore the hypervisor page-fault because the memory is still
> waiting to be mapped by the kernel. 
> 
> 
> I simply touched all the allocated memory, and it works fine now. 

It might work, but unless it is using hypercall buffers and
XEN_GUEST_HANDLE then it is still subtly broken.

Ian.