Probs with PCI bus master DMA to user space

Probs with PCI bus master DMA to user space

[Norbert Roos]

Hello!

I think the following is general problem, but i haven't found any
information about that yet..

I'm currently writing a driver which wants to transfer data between main
memory and a PCI device. The data buffers are allocated by the program
which uses the driver and therefore lie in the user space. Pointers to
the data buffers are as usual passed to the driver with read() and
write() calls. The driver then initializes a DMA transfer where the PCI
device is bus master.

The driver of course has to split the transfer into several smaller
ones, because the memory pages which contain the data buffers could be
spread across the complete memory.

Before i can actually start the transfer, i have to make sure that the
pages are not swapped and remain locked during the transfer.

The problem I have is: Is there an efficient way to lock the pages which
are accessed by the DMA?

The program using the driver can't use mlock(), because the program does
not know about DMA transfers somewhere below the driver.

The driver should not use mlock() either, because i think that the
execution of mlock() takes about as long as copying the data directly
with copy_from/to_user(). Additionally, I could not unlock() the memory
after the transfer, because the main program might have locked the same
memory area and wants the memory to remain locked.

So what can i do then?

bye

Re: Probs with PCI bus master DMA to user space

[Alan Cox]

> I'm currently writing a driver which wants to transfer data between main
> memory and a PCI device. The data buffers are allocated by the program
> which uses the driver and therefore lie in the user space. Pointers to

Allocate the buffers in the kernel and mmap() them into user space

Re: Probs with PCI bus master DMA to user space

[David Woodhouse]

norbert.roos@cluster-labs.de said:
>  The problem I have is: Is there an efficient way to lock the pages
> which are accessed by the DMA?

map_user_kiobuf, lock it, DMA into it, unlock it and unmap it again?

--
dwmw2

Re: Probs with PCI bus master DMA to user space

[Norbert Roos]

> Allocate the buffers in the kernel and mmap() them into user space

But the buffers are usually allocated with malloc() by any application
which wants to use my driver.. otherwise my driver would have to offer a
malloc-like function, but I can hardly force the application to use my
own malloc function.

Re: Probs with PCI bus master DMA to user space

[Jeff Garzik]

On Tue, 20 Feb 2001, Norbert Roos wrote:
> > Allocate the buffers in the kernel and mmap() them into user space
> 
> But the buffers are usually allocated with malloc() by any application
> which wants to use my driver.. otherwise my driver would have to offer a
> malloc-like function, but I can hardly force the application to use my
> own malloc function.

If you are writing the driver, sure you can.

	Jeff

Re: Probs with PCI bus master DMA to user space

[Norbert Roos]

Jeff Garzik wrote:

> > But the buffers are usually allocated with malloc() by any application
> > which wants to use my driver.. otherwise my driver would have to offer a
> > malloc-like function, but I can hardly force the application to use my
> > own malloc function.
> 
> If you are writing the driver, sure you can.

??

The application is doing something like

  fd = open("/dev/mydriver");
  buf = malloc();
  fill_buffer_with_data(buf);
\x18 write(fd,buf);

And now i should tell the programmer not to use malloc() but my special
driver-malloc?
Or do you mean something different?

Norbert

Re: Probs with PCI bus master DMA to user space

[Jeff Garzik]

On Tue, 20 Feb 2001, Norbert Roos wrote:

> Jeff Garzik wrote:
> 
> > > But the buffers are usually allocated with malloc() by any application
> > > which wants to use my driver.. otherwise my driver would have to offer a
> > > malloc-like function, but I can hardly force the application to use my
> > > own malloc function.
> > 
> > If you are writing the driver, sure you can.
> 
> ??
> 
> The application is doing something like
> 
>   fd = open("/dev/mydriver");
>   buf = malloc();
>   fill_buffer_with_data(buf);
> \x18 write(fd,buf);
> 
> And now i should tell the programmer not to use malloc() but my special
> driver-malloc?
> Or do you mean something different?

fd = open(...);
buf = mmap(fd, ...);
fill_buffer_with_data(buf);
ioctl(fd, ...); /* tell kernel data is there */

There are variations depending on the application, but you get the
picture.  A buffer copy is eliminated when mmap is used, too, making
your application faster.

	Jeff

Re: Probs with PCI bus master DMA to user space

[Norbert Roos]

Jeff Garzik wrote:
> 
> On Tue, 20 Feb 2001, Norbert Roos wrote:
> 
> > Jeff Garzik wrote:
> >
> > > > But the buffers are usually allocated with malloc() by any application
> > > > which wants to use my driver.. otherwise my driver would have to offer a

> 
> fd = open(...);
> buf = mmap(fd, ...);
> fill_buffer_with_data(buf);
> ioctl(fd, ...); /* tell kernel data is there */
> 

Hm hm - this is exactly what i wanted to avoid: The application should
not
be modified only to be able to use my driver - and if it is using
malloc(), it would have to be modified..

Thanks anyway!

Norbert