How to transfer memory from PCI memory directly to user space safely and portable?

How to transfer memory from PCI memory directly to user space safely and portable?

[Anders Torger]

I'm writing a sound card driver where I need to transfer memory from the card 
to user space using the CPU. Ideally I'd like to do that without needing to 
have an intermediate buffer in kernel memory. I have implemented the copy 
functions like this:

>From user space to the sound card:

	if (verify_area(VERIFY_READ, user_space_src, count) != 0) {
	    return -EFAULT;
	}
	memcpy_toio(iobase, user_space_src, count);
	return 0;

>From the sound card to user space:

	if (verify_area(VERIFY_WRITE, user_space_dst, count) != 0) {
	    return -EFAULT;
	}
	memcpy_fromio(user_space_dst, iobase, count);
	return 0;


These functions are called on the behalf of the user, so the current process 
should be the correct one. The iobase is ioremapped: 

	iobase = ioremap(sound_card_port, sound_card_io_size);

Now, this code works, I have a working driver. However, some questions have 
been raised about the code, namely the following:

1. What happens if the user space memory is swapped to disk? Will 
verify_area() make sure that the memory is in physical RAM when it returns, 
or will it return -EFAULT, or will something even worse happen?

2. Is this code really portable? I currently have an I386 architecture, and I 
could use copy_to/from_user on that instead, but that is not portable. Now, 
by using memcpy_to/fromio instead, is this code fully portable?

3. Will the current process always be the correct one? The copy functions is 
directly initiated by the user, and not through an interrupt, so I think the 
user space mapping will always be to the correct process. Is that correct?

Since I'm not on the list, I'd like to have answers CC'd to my address. Thank 
you.

/Anders Torger
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: How to transfer memory from PCI memory directly to user space safely and portable?

[Philipp Rumpf]

On Sun, Nov 26, 2000 at 02:21:31PM +0100, Anders Torger wrote:
> 	memcpy_toio(iobase, user_space_src, count);

I hope count isn't provided by userspace here ?

> 1. What happens if the user space memory is swapped to disk? Will 
> verify_area() make sure that the memory is in physical RAM when it returns, 
> or will it return -EFAULT, or will something even worse happen?

On i386, you'll sleep implicitly waiting for the page fault to be handled;  in
the generic case, anything could happen.

> 2. Is this code really portable? I currently have an I386 architecture, and I 
> could use copy_to/from_user on that instead, but that is not portable. Now, 
> by using memcpy_to/fromio instead, is this code fully portable?

No.  It would be portable if you were using memcpy_fromuser_toio and it
existed.

> 3. Will the current process always be the correct one? The copy functions is 
> directly initiated by the user, and not through an interrupt, so I think the 
> user space mapping will always be to the correct process. Is that correct?

current should be fine if you're not in a bh/interrupt/kernel thread.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: How to transfer memory from PCI memory directly to user space safely and portable?

[Anders Torger]

On Sun, 26 Nov 2000, you wrote:
> On Sun, Nov 26, 2000 at 02:21:31PM +0100, Anders Torger wrote:
> > 	memcpy_toio(iobase, user_space_src, count);
>
> I hope count isn't provided by userspace here ?

Fortunately, 'count' is controlled by the driver architecture (ALSA), and not 
the user.

> > 1. What happens if the user space memory is swapped to disk? Will
> > verify_area() make sure that the memory is in physical RAM when it
> > returns, or will it return -EFAULT, or will something even worse happen?
>
> On i386, you'll sleep implicitly waiting for the page fault to be handled; 
> in the generic case, anything could happen.

Do you know of an architecture that will not do like i386 in this case?

> > 2. Is this code really portable? I currently have an I386 architecture,
> > and I could use copy_to/from_user on that instead, but that is not
> > portable. Now, by using memcpy_to/fromio instead, is this code fully
> > portable?
>
> No.  It would be portable if you were using memcpy_fromuser_toio and it
> existed.

Oh, I see. Again, I wonder, do you know of any architecture, currently 
supported by Linux, where my code would fail? It would be helpful to know.

/Anders Torger
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: How to transfer memory from PCI memory directly to user space safely and portable?

[H. Peter Anvin]

Followup to:  <20001126151120.V2272@parcelfarce.linux.theplanet.co.uk>
By author:    Philipp Rumpf <prumpf@parcelfarce.linux.theplanet.co.uk>
In newsgroup: linux.dev.kernel
> 
> I hope count isn't provided by userspace here ?
> 
> > 1. What happens if the user space memory is swapped to disk? Will 
> > verify_area() make sure that the memory is in physical RAM when it returns, 
> > or will it return -EFAULT, or will something even worse happen?
> 
> On i386, you'll sleep implicitly waiting for the page fault to be handled;  in
> the generic case, anything could happen.
> 

That doesn't sound right.  I would expect it to wait for the page to
be brought in on any and all architectures, otherwise it seems rather
impossible to write portable Linux kernel code.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: How to transfer memory from PCI memory directly to user space safely and portable?

[Philipp Rumpf]

On Mon, Nov 27, 2000 at 10:36:34AM -0800, H. Peter Anvin wrote:
> Followup to:  <20001126151120.V2272@parcelfarce.linux.theplanet.co.uk>
> By author:    Philipp Rumpf <prumpf@parcelfarce.linux.theplanet.co.uk>
> In newsgroup: linux.dev.kernel
> > 
> > I hope count isn't provided by userspace here ?
> > 
> > > 1. What happens if the user space memory is swapped to disk? Will 
> > > verify_area() make sure that the memory is in physical RAM when it returns, 
> > > or will it return -EFAULT, or will something even worse happen?
> > 
> > On i386, you'll sleep implicitly waiting for the page fault to be handled;  in
> > the generic case, anything could happen.
> > 
> 
> That doesn't sound right.  I would expect it to wait for the page to
> be brought in on any and all architectures, otherwise it seems rather
> impossible to write portable Linux kernel code.

The code in question was
	memcpy_fromio(user_space_dst, iobase, count);

Assuming user_space_dst is a userspace pointer, what I said is true;  on some
architectures we will be dereferencing random pointers in kernel space, and
we won't get -EFAULT right on any architecture.

Or did I miss something ?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: How to transfer memory from PCI memory directly to user space safely and portable?

[H. Peter Anvin]

Philipp Rumpf wrote:
> 
> On Mon, Nov 27, 2000 at 10:36:34AM -0800, H. Peter Anvin wrote:
> > Followup to:  <20001126151120.V2272@parcelfarce.linux.theplanet.co.uk>
> > By author:    Philipp Rumpf <prumpf@parcelfarce.linux.theplanet.co.uk>
> > In newsgroup: linux.dev.kernel
> > >
> > > I hope count isn't provided by userspace here ?
> > >
> > > > 1. What happens if the user space memory is swapped to disk? Will
> > > > verify_area() make sure that the memory is in physical RAM when it returns,
> > > > or will it return -EFAULT, or will something even worse happen?
> > >
> > > On i386, you'll sleep implicitly waiting for the page fault to be handled;  in
> > > the generic case, anything could happen.
> > >
> >
> > That doesn't sound right.  I would expect it to wait for the page to
> > be brought in on any and all architectures, otherwise it seems rather
> > impossible to write portable Linux kernel code.
> 
> The code in question was
>         memcpy_fromio(user_space_dst, iobase, count);
> 
> Assuming user_space_dst is a userspace pointer, what I said is true;  on some
> architectures we will be dereferencing random pointers in kernel space, and
> we won't get -EFAULT right on any architecture.
> 
> Or did I miss something ?
> 

Yes, the post you responded to was talking about verify_area() [which is,
admittedly, obsolete.]

	-hpa

-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: How to transfer memory from PCI memory directly to user space safely and portable?

[Richard B. Johnson]

On Sun, 26 Nov 2000, Anders Torger wrote:

> 
> I'm writing a sound card driver where I need to transfer memory from the card 
> to user space using the CPU. Ideally I'd like to do that without needing to 
> have an intermediate buffer in kernel memory. I have implemented the copy 
> functions like this:
> 
> >From user space to the sound card:
> 
> 	if (verify_area(VERIFY_READ, user_space_src, count) != 0) {
> 	    return -EFAULT;
> 	}
> 	memcpy_toio(iobase, user_space_src, count);
> 	return 0;
> 
> >From the sound card to user space:
> 
> 	if (verify_area(VERIFY_WRITE, user_space_dst, count) != 0) {
> 	    return -EFAULT;
> 	}
> 	memcpy_fromio(user_space_dst, iobase, count);
> 	return 0;
> 

This contains a known bug. The user's buffer may be paged out even
though it is called by the current process. 

> 
> These functions are called on the behalf of the user, so the current process 
> should be the correct one. The iobase is ioremapped: 
> 
> 	iobase = ioremap(sound_card_port, sound_card_io_size);

ioremap() does not return a pointer. Although this may seem to work
for some addresses (below 1 megabyte), this is also a bug.

> 
> Now, this code works, I have a working driver. However, some questions have 
> been raised about the code, namely the following:
> 
> 1. What happens if the user space memory is swapped to disk? Will 
> verify_area() make sure that the memory is in physical RAM when it returns, 
> or will it return -EFAULT, or will something even worse happen?
> 

verify_area() is not used for this and, in fact, it is obsolete in new
kernels. You need to use copy_to/from_user(). This has the capability
of generating the proper page-fault machanism.

> 2. Is this code really portable? I currently have an I386 architecture, and I 
> could use copy_to/from_user on that instead, but that is not portable. Now, 
> by using memcpy_to/fromio instead, is this code fully portable?
> 

Portable? If you use the necessary kernel functions, then your
driver can run on various Linux architectures. However I don't
think you can call this "portable".


> 3. Will the current process always be the correct one? The copy functions is 
> directly initiated by the user, and not through an interrupt, so I think the 
> user space mapping will always be to the correct process. Is that correct?
> 

If your user calls open(), close(), read(), write(), ioctl(), etc., in
your driver, 'current' will be correct. However, this does not mean that
any of your data space is present in memory.


A guaranteed procedure to get data from your device to a user involves
a temporary buffer.


 	iobase = ioremap(sound_card_port, sound_card_io_size);
 	memcpy_fromio(kmalloced_buffer, iobase, count);
        copy_to_user(user_space_dst, kmalloced_buffer, count);


This involves an extra copy. You can get rid of the extra copy by
doing the following:

(1)	Using an ioctl() from user-space, return the physical address of
	'iobase' there is a macro for this.

(2)	From user-space, mmap(), MAP_FIXED this address space.

Here is a test program that does that (shows how mmap works):


#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <signal.h>
#include <fcntl.h>
#include <termios.h>
#include <sys/types.h>
#include <sys/mman.h>

#define NoDEBUG

#if !defined(PAGE_SIZE)
#define PAGE_SIZE 0x1000
#endif
#if !defined(PAGE_MASK)
#define PAGE_MASK ~(PAGE_SIZE - 1)
#endif
#if !defined(MAP_FAILED)
#define MAP_FAILED (void *) -1
#endif
#define UL unsigned long

#define ERRORS \
    { fprintf(stderr, errmsg, __LINE__,\
      __FILE__, errno, strerror(errno));\
      exit(EXIT_FAILURE); }

#ifdef DEBUG
#define DEB(f) (f)
#else
#define DEB(f)
#endif

static const char errmsg[]="Error at line %d, file %s (%d) [%s]\n";
static const char mapfile[]="/dev/mem";
#define PROT (PROT_READ|PROT_WRITE)
#define FLAGS (MAP_FIXED|MAP_SHARED)
/*-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-*/
/*
 *  This writes a 'debug-like' dump out the screen. It always writes
 *  16 lines of 16 characters. It returns the last address displayed.
 */
static off_t dump(off_t addr);

static off_t dump(off_t addr)
{
    size_t i, j;
    int fd;
    caddr_t mem;
    off_t next;
    unsigned char c, *buf;

    next = addr; 
    addr &= PAGE_MASK;
    if((fd = open(mapfile, O_RDWR)) < 0)
        ERRORS;

    if((mem = mmap((caddr_t)addr, PAGE_SIZE * 2, PROT, FLAGS, fd, addr)) == MAP_FAILED)
    {
        (void)close(fd);
        fprintf(stderr, "\tCan't map address %08lX\n", (UL) addr);
        return (off_t)mem;
    } 
    buf = (unsigned char *) next;
    DEB(fprintf(stderr, "Mapped address = 0x%lx\n", addr));
    DEB(fprintf(stderr, "Buffer address = %p\n", buf));

    for(i=0; i< 0x10; i++)
    {
        fprintf(stdout, "\n%08lX ", (UL) next);
        next += 0x10;
        for(j=0; j< 0x10; j++)
            fprintf(stdout, "%02X ", (unsigned int) *buf++);
        buf -= 0x10;
        for(j=0; j<0x10; j++)
        {
            c = *buf++;
            if((c < (unsigned char) ' ') || (c > (unsigned char) 'z'))
                c = (unsigned char) '.';
            fprintf(stdout, "%c", c);
        }
    }
    fprintf(stdout, "\n");
    (void)fflush(stdout);
    (void)close(fd);
    if(munmap(mem, PAGE_SIZE)< 0)
        ERRORS;
    return next;
}

void usage(char *cp)
{
        fprintf(stderr, "Usage\n%s <start address> [end address]\n", cp);
        exit(EXIT_FAILURE);
}

int main(int args, char *argv[])
{
    off_t addr;
    off_t end;

    if(args < 2)
        usage(argv[0]);
    end = 0;
    if(sscanf(argv[1], "%lx", &addr) != 1)
        usage(argv[0]);
    if(argv[2] != NULL)
        (void)sscanf(argv[2], "%lx", &end);
    do {    
        if((addr = dump(addr)) == 0xffffffff)
            break;
        } while (end > addr);
    return 0;
}

-----------





Cheers,
Dick Johnson

Penguin : Linux version 2.4.0 on an i686 machine (799.54 BogoMips).

"Memory is like gasoline. You use it up when you are running. Of
course you get it all back when you reboot..."; Actual explanation
obtained from the Micro$oft help desk.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: How to transfer memory from PCI memory directly to user space safely and portable?

[aprasad]

>I'm writing a sound card driver where I need to transfer memory from the
card
>to user space using the CPU. Ideally I'd like to do that without needing
to
>have an intermediate buffer in kernel memory. I have implemented the copy
>functions like this:

>From user space to the sound card:
>
>    if (verify_area(VERIFY_READ, user_space_src, count) != 0) {
>        return -EFAULT;
>    }
>    memcpy_toio(iobase, user_space_src, count);
>    return 0;

>From the sound card to user space:
>
>    if (verify_area(VERIFY_WRITE, user_space_dst, count) != 0) {
>        return -EFAULT;
>    }
>    memcpy_fromio(user_space_dst, iobase, count);
>    return 0;


>These functions are called on the behalf of the user, so the current
process
>should be the correct one. The iobase is ioremapped:
>
>    iobase = ioremap(sound_card_port, sound_card_io_size);


The best solution will be to let the user mmap the device memory to his
address space.The driver need to provide the interface through ioctl cmd or
mmap file operations.


http://www2.linuxjournal.com/lj-issues/issue28/1287.html
The above link might be usefull though this is for pre2.4 kernel, it needs
some modification for 2.4 kernels.

Regards
Anil


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: How to transfer memory from PCI memory directly to user space safely and portable?

[Anders Torger]

On Mon, 27 Nov 2000, you wrote:
> >have an intermediate buffer in kernel memory. I have implemented the copy
> >functions like this:
> >
> >From user space to the sound card:
> >
> >    if (verify_area(VERIFY_READ, user_space_src, count) != 0) {
> >        return -EFAULT;
> >    }
> >    memcpy_toio(iobase, user_space_src, count);
> >    return 0;
> >
> >From the sound card to user space:
> >
> >    if (verify_area(VERIFY_WRITE, user_space_dst, count) != 0) {
> >        return -EFAULT;
> >    }
> >    memcpy_fromio(user_space_dst, iobase, count);
> >    return 0;
> >
> >
> >These functions are called on the behalf of the user, so the current
>
> process
>
> >should be the correct one. The iobase is ioremapped:
> >
> >    iobase = ioremap(sound_card_port, sound_card_io_size);

> The best solution will be to let the user mmap the device memory to his
> address space.The driver need to provide the interface through ioctl cmd or
> mmap file operations.

I have learnt that it is not portable to mmap the IO memory, since some 
architectures (for example Alpha) cannot access IO memory the same was as 
physical RAM. That is why I am using memcpy_to/fromio instead of 
copy_to/from_user. If I am correctly informed, when mmaping is used, the card 
use to have some sort of DMA mechanism, so the mmaping is done on ordinary 
RAM, not IO memory. Unfortunately, on this sound card (RME Audio Digi96), I 
have to access the on-board buffers directly, using the CPU. Thus, to be 
portable, I cannot use mmap directly on the ioremap'd memory, right?

/Anders Torger
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/