sparc/staging compile error

sparc/staging compile error

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 282 bytes --]

Hi Greg,

Today's tree from Linus gets the following error from a sparc
allmodconfig build:

ERROR: "___f_flush_cache_range" [drivers/staging/poch/poch.ko] undefined!

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

sparc/staging compile error

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 282 bytes --]

Hi Greg,

Today's tree from Linus gets the following error from a sparc
allmodconfig build:

ERROR: "___f_flush_cache_range" [drivers/staging/poch/poch.ko] undefined!

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: sparc/staging compile error

[Greg KH]

On Thu, Nov 06, 2008 at 04:36:26PM +1100, Stephen Rothwell wrote:
> Hi Greg,
> 
> Today's tree from Linus gets the following error from a sparc
> allmodconfig build:
> 
> ERROR: "___f_flush_cache_range" [drivers/staging/poch/poch.ko] undefined!

Odd, is flush_cache_range() not allowed on the sparc platform?

thanks,

greg k-h

Re: sparc/staging compile error

[Greg KH]

On Thu, Nov 06, 2008 at 04:36:26PM +1100, Stephen Rothwell wrote:
> Hi Greg,
> 
> Today's tree from Linus gets the following error from a sparc
> allmodconfig build:
> 
> ERROR: "___f_flush_cache_range" [drivers/staging/poch/poch.ko] undefined!

Odd, is flush_cache_range() not allowed on the sparc platform?

thanks,

greg k-h

Re: sparc/staging compile error

[David Miller]

From: Greg KH <greg@kroah.com>
Date: Wed, 5 Nov 2008 22:37:09 -0800

> On Thu, Nov 06, 2008 at 04:36:26PM +1100, Stephen Rothwell wrote:
> > Hi Greg,
> > 
> > Today's tree from Linus gets the following error from a sparc
> > allmodconfig build:
> > 
> > ERROR: "___f_flush_cache_range" [drivers/staging/poch/poch.ko] undefined!
> 
> Odd, is flush_cache_range() not allowed on the sparc platform?

No, just nobody used it from a module before.

Re: sparc/staging compile error

[David Miller]

From: Greg KH <greg@kroah.com>
Date: Wed, 5 Nov 2008 22:37:09 -0800

> On Thu, Nov 06, 2008 at 04:36:26PM +1100, Stephen Rothwell wrote:
> > Hi Greg,
> > 
> > Today's tree from Linus gets the following error from a sparc
> > allmodconfig build:
> > 
> > ERROR: "___f_flush_cache_range" [drivers/staging/poch/poch.ko] undefined!
> 
> Odd, is flush_cache_range() not allowed on the sparc platform?

No, just nobody used it from a module before.

Re: sparc/staging compile error

[Paul Mackerras]

Greg KH writes:

> On Thu, Nov 06, 2008 at 04:36:26PM +1100, Stephen Rothwell wrote:
> > Hi Greg,
> > 
> > Today's tree from Linus gets the following error from a sparc
> > allmodconfig build:
> > 
> > ERROR: "___f_flush_cache_range" [drivers/staging/poch/poch.ko] undefined!
> 
> Odd, is flush_cache_range() not allowed on the sparc platform?

I'm curious, what is a driver doing calling flush_cache_range()?
What does it expect it to do precisely?

Paul.

Re: sparc/staging compile error

[Paul Mackerras]

Greg KH writes:

> On Thu, Nov 06, 2008 at 04:36:26PM +1100, Stephen Rothwell wrote:
> > Hi Greg,
> > 
> > Today's tree from Linus gets the following error from a sparc
> > allmodconfig build:
> > 
> > ERROR: "___f_flush_cache_range" [drivers/staging/poch/poch.ko] undefined!
> 
> Odd, is flush_cache_range() not allowed on the sparc platform?

I'm curious, what is a driver doing calling flush_cache_range()?
What does it expect it to do precisely?

Paul.

Re: sparc/staging compile error

[J.R. Mauro]

On Thu, Nov 6, 2008 at 5:32 AM, Paul Mackerras <paulus@samba.org> wrote:
> Greg KH writes:
>
>> On Thu, Nov 06, 2008 at 04:36:26PM +1100, Stephen Rothwell wrote:
>> > Hi Greg,
>> >
>> > Today's tree from Linus gets the following error from a sparc
>> > allmodconfig build:
>> >
>> > ERROR: "___f_flush_cache_range" [drivers/staging/poch/poch.ko] undefined!
>>
>> Odd, is flush_cache_range() not allowed on the sparc platform?
>
> I'm curious, what is a driver doing calling flush_cache_range()?
> What does it expect it to do precisely?

It's part of the driver's ioctl. Relevant lines:

static int poch_ioctl(struct inode *inode, struct file *filp,
                      unsigned int cmd, unsigned long arg)
{
/* ---snip---*/
        case POCH_IOC_SYNC_GROUP_FOR_USER:
        case POCH_IOC_SYNC_GROUP_FOR_DEVICE:
                vms = find_vma(current->mm, arg);
                if (!vms)
                        /* Address not mapped. */
                        return -EINVAL;
                if (vms->vm_file != filp)
                        /* Address mapped from different device/file. */
                        return -EINVAL;

                flush_cache_range(vms, arg, arg + channel->group_size);


>
> Paul.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: sparc/staging compile error

[J.R. Mauro]

On Thu, Nov 6, 2008 at 5:32 AM, Paul Mackerras <paulus@samba.org> wrote:
> Greg KH writes:
>
>> On Thu, Nov 06, 2008 at 04:36:26PM +1100, Stephen Rothwell wrote:
>> > Hi Greg,
>> >
>> > Today's tree from Linus gets the following error from a sparc
>> > allmodconfig build:
>> >
>> > ERROR: "___f_flush_cache_range" [drivers/staging/poch/poch.ko] undefined!
>>
>> Odd, is flush_cache_range() not allowed on the sparc platform?
>
> I'm curious, what is a driver doing calling flush_cache_range()?
> What does it expect it to do precisely?

It's part of the driver's ioctl. Relevant lines:

static int poch_ioctl(struct inode *inode, struct file *filp,
                      unsigned int cmd, unsigned long arg)
{
/* ---snip---*/
        case POCH_IOC_SYNC_GROUP_FOR_USER:
        case POCH_IOC_SYNC_GROUP_FOR_DEVICE:
                vms = find_vma(current->mm, arg);
                if (!vms)
                        /* Address not mapped. */
                        return -EINVAL;
                if (vms->vm_file != filp)
                        /* Address mapped from different device/file. */
                        return -EINVAL;

                flush_cache_range(vms, arg, arg + channel->group_size);


>
> Paul.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: sparc/staging compile error

[Christoph Hellwig]

On Thu, Nov 06, 2008 at 09:06:36AM -0500, J.R. Mauro wrote:
> static int poch_ioctl(struct inode *inode, struct file *filp,
>                       unsigned int cmd, unsigned long arg)
> {
> /* ---snip---*/
>         case POCH_IOC_SYNC_GROUP_FOR_USER:
>         case POCH_IOC_SYNC_GROUP_FOR_DEVICE:
>                 vms = find_vma(current->mm, arg);
>                 if (!vms)
>                         /* Address not mapped. */
>                         return -EINVAL;
>                 if (vms->vm_file != filp)
>                         /* Address mapped from different device/file. */
>                         return -EINVAL;
> 
>                 flush_cache_range(vms, arg, arg + channel->group_size);

This doesn't look like something a driver should ever do.  Could someone
explain what it's trying to do from a high level point of view?

Re: sparc/staging compile error

[Christoph Hellwig]

On Thu, Nov 06, 2008 at 09:06:36AM -0500, J.R. Mauro wrote:
> static int poch_ioctl(struct inode *inode, struct file *filp,
>                       unsigned int cmd, unsigned long arg)
> {
> /* ---snip---*/
>         case POCH_IOC_SYNC_GROUP_FOR_USER:
>         case POCH_IOC_SYNC_GROUP_FOR_DEVICE:
>                 vms = find_vma(current->mm, arg);
>                 if (!vms)
>                         /* Address not mapped. */
>                         return -EINVAL;
>                 if (vms->vm_file != filp)
>                         /* Address mapped from different device/file. */
>                         return -EINVAL;
> 
>                 flush_cache_range(vms, arg, arg + channel->group_size);

This doesn't look like something a driver should ever do.  Could someone
explain what it's trying to do from a high level point of view?

Re: sparc/staging compile error

[J.R. Mauro]

On Thu, Nov 6, 2008 at 12:32 PM, Christoph Hellwig <hch@infradead.org> wrote:
> On Thu, Nov 06, 2008 at 09:06:36AM -0500, J.R. Mauro wrote:
>> static int poch_ioctl(struct inode *inode, struct file *filp,
>>                       unsigned int cmd, unsigned long arg)
>> {
>> /* ---snip---*/
>>         case POCH_IOC_SYNC_GROUP_FOR_USER:
>>         case POCH_IOC_SYNC_GROUP_FOR_DEVICE:
>>                 vms = find_vma(current->mm, arg);
>>                 if (!vms)
>>                         /* Address not mapped. */
>>                         return -EINVAL;
>>                 if (vms->vm_file != filp)
>>                         /* Address mapped from different device/file. */
>>                         return -EINVAL;
>>
>>                 flush_cache_range(vms, arg, arg + channel->group_size);
>
> This doesn't look like something a driver should ever do.  Could someone
> explain what it's trying to do from a high level point of view?
>
>

CC'd driver maintainers mentioned in the README

Re: sparc/staging compile error

[J.R. Mauro]

On Thu, Nov 6, 2008 at 12:32 PM, Christoph Hellwig <hch@infradead.org> wrote:
> On Thu, Nov 06, 2008 at 09:06:36AM -0500, J.R. Mauro wrote:
>> static int poch_ioctl(struct inode *inode, struct file *filp,
>>                       unsigned int cmd, unsigned long arg)
>> {
>> /* ---snip---*/
>>         case POCH_IOC_SYNC_GROUP_FOR_USER:
>>         case POCH_IOC_SYNC_GROUP_FOR_DEVICE:
>>                 vms = find_vma(current->mm, arg);
>>                 if (!vms)
>>                         /* Address not mapped. */
>>                         return -EINVAL;
>>                 if (vms->vm_file != filp)
>>                         /* Address mapped from different device/file. */
>>                         return -EINVAL;
>>
>>                 flush_cache_range(vms, arg, arg + channel->group_size);
>
> This doesn't look like something a driver should ever do.  Could someone
> explain what it's trying to do from a high level point of view?
>
>

CC'd driver maintainers mentioned in the README

Re: sparc/staging compile error

[Vijay Kumar]

J.R. Mauro wrote:
> On Thu, Nov 6, 2008 at 12:32 PM, Christoph Hellwig <hch@infradead.org> wrote:
>> On Thu, Nov 06, 2008 at 09:06:36AM -0500, J.R. Mauro wrote:
>>> static int poch_ioctl(struct inode *inode, struct file *filp,
>>>                       unsigned int cmd, unsigned long arg)
>>> {
>>> /* ---snip---*/
>>>         case POCH_IOC_SYNC_GROUP_FOR_USER:
>>>         case POCH_IOC_SYNC_GROUP_FOR_DEVICE:
>>>                 vms = find_vma(current->mm, arg);
>>>                 if (!vms)
>>>                         /* Address not mapped. */
>>>                         return -EINVAL;
>>>                 if (vms->vm_file != filp)
>>>                         /* Address mapped from different device/file. */
>>>                         return -EINVAL;
>>>
>>>                 flush_cache_range(vms, arg, arg + channel->group_size);
>> This doesn't look like something a driver should ever do.  Could someone
>> explain what it's trying to do from a high level point of view?
>
> CC'd driver maintainers mentioned in the README

May be the code is not doing what is supposed to do, but here is what the 
driver is trying to achieve:

The driver allocates a set of buffers for DMA. These buffers are mapped 
into user space, when the user does an mmap. In transmit, when the user 
space writes to these buffers, the data has to reach physical memory so 
that the device can access them. For receive, the cache has to be 
invalidated before the user space reads the buffer.

Do let me know if further clarification is required. Any inputs and 
suggestions are welcome.

BTW, please do send in the compiler error message.

Regards,
Vijay

Re: sparc/staging compile error

[Vijay Kumar]

J.R. Mauro wrote:
> On Thu, Nov 6, 2008 at 12:32 PM, Christoph Hellwig <hch@infradead.org> wrote:
>> On Thu, Nov 06, 2008 at 09:06:36AM -0500, J.R. Mauro wrote:
>>> static int poch_ioctl(struct inode *inode, struct file *filp,
>>>                       unsigned int cmd, unsigned long arg)
>>> {
>>> /* ---snip---*/
>>>         case POCH_IOC_SYNC_GROUP_FOR_USER:
>>>         case POCH_IOC_SYNC_GROUP_FOR_DEVICE:
>>>                 vms = find_vma(current->mm, arg);
>>>                 if (!vms)
>>>                         /* Address not mapped. */
>>>                         return -EINVAL;
>>>                 if (vms->vm_file != filp)
>>>                         /* Address mapped from different device/file. */
>>>                         return -EINVAL;
>>>
>>>                 flush_cache_range(vms, arg, arg + channel->group_size);
>> This doesn't look like something a driver should ever do.  Could someone
>> explain what it's trying to do from a high level point of view?
>
> CC'd driver maintainers mentioned in the README

May be the code is not doing what is supposed to do, but here is what the 
driver is trying to achieve:

The driver allocates a set of buffers for DMA. These buffers are mapped 
into user space, when the user does an mmap. In transmit, when the user 
space writes to these buffers, the data has to reach physical memory so 
that the device can access them. For receive, the cache has to be 
invalidated before the user space reads the buffer.

Do let me know if further clarification is required. Any inputs and 
suggestions are welcome.

BTW, please do send in the compiler error message.

Regards,
Vijay

Re: sparc/staging compile error

[Benjamin Herrenschmidt]

> May be the code is not doing what is supposed to do, but here is what the 
> driver is trying to achieve:
> 
> The driver allocates a set of buffers for DMA. These buffers are mapped 
> into user space, when the user does an mmap. In transmit, when the user 
> space writes to these buffers, the data has to reach physical memory so 
> that the device can access them. For receive, the cache has to be 
> invalidated before the user space reads the buffer.
> 
> Do let me know if further clarification is required. Any inputs and 
> suggestions are welcome.

Well, this is the wrong API to do that. First, some platforms are cache
coherent and thus don't need anything there other than maybe a barrier.

Then, the dma_* API is what you want to ensure cache coherency with DMA
transfers within the kernel.

If userspace can directly trigger DMA from pre-allocated buffers, then
doing an ioctl to sync is a major wastage, since most platforms are
cache coherent (and thus don't need it) and those who are not generally
provide user-space accessible ways of doing the flush to some extent
(not all, for example, BookE PPC cannot invalidate the data cache from
userspace, only flush it).

In any case, we would need to understand better your driver userspace
API and transfer model to find the right solution.

Ben.

Re: sparc/staging compile error

[Benjamin Herrenschmidt]

> May be the code is not doing what is supposed to do, but here is what the 
> driver is trying to achieve:
> 
> The driver allocates a set of buffers for DMA. These buffers are mapped 
> into user space, when the user does an mmap. In transmit, when the user 
> space writes to these buffers, the data has to reach physical memory so 
> that the device can access them. For receive, the cache has to be 
> invalidated before the user space reads the buffer.
> 
> Do let me know if further clarification is required. Any inputs and 
> suggestions are welcome.

Well, this is the wrong API to do that. First, some platforms are cache
coherent and thus don't need anything there other than maybe a barrier.

Then, the dma_* API is what you want to ensure cache coherency with DMA
transfers within the kernel.

If userspace can directly trigger DMA from pre-allocated buffers, then
doing an ioctl to sync is a major wastage, since most platforms are
cache coherent (and thus don't need it) and those who are not generally
provide user-space accessible ways of doing the flush to some extent
(not all, for example, BookE PPC cannot invalidate the data cache from
userspace, only flush it).

In any case, we would need to understand better your driver userspace
API and transfer model to find the right solution.

Ben.

Re: sparc/staging compile error

[Vijay Kumar]

Benjamin Herrenschmidt wrote:
> In any case, we would need to understand better your driver userspace
> API and transfer model to find the right solution.

Please find below the user space skeleton code, to access the driver. It 
was written by Robert Fitzsimons robfitz@273k.net The driver is based on 
this user space interface.

In short, there is a ring buffer that is shared between the _device_ and 
_user-space_. The ring buffer has a header, that is shared between the 
_kernel_ and _user-space_.

The header specifies the offsets of various buffers in the ring buffer. In 
Rx, after user space has read a buffer, the user space writes -1 to the 
offset in the header. When the device has filled the buffer, the kernel 
space writes the offset back to the header.

#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <poll.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>

#define G_USED -1

struct group_circular_buffer {
	int32_t group_size_bytes;
	int32_t group_count;
	int32_t group_offset[0];
};

int sysfs_write_long(const char *name, long value);
int sysfs_read_long(const char *name, long *value);
int sysfs_read_buffer(const char *name, void *buffer, int len);
int process_group(void *group, int32_t group_size_bytes);

int main(int argc, char **argv)
{
	int uio_fd;
	int rx_fd;
	int group_index;
	long mmap_buffer_size;
	struct pollfd poll_fds;
	struct group_circular_buffer *ring;
	char buffer[128];

	/* Query supported device and channels */
	sysfs_read_buffer("/sys/.../uioX/name", buffer, sizeof(buffer));
	sysfs_read_buffer("/sys/.../uioX/version", buffer, sizeof(buffer));
	sysfs_read_buffer("/sys/.../uioX/channels", buffer, sizeof(buffer));

	/* Open and mmap UIO device */
	uio_fd = open("/dev/.../uioX", O_RDWR);
	// uio_mmap = mmap(..., uio_fd, ...); /* map BAR1 */

	/* Configure 1 megabyte receive dma buffer */
	sysfs_write_long("/sys/.../uioX/rx0/block_size", 512);
	sysfs_write_long("/sys/.../uioX/rx0/group_size", 1);
	sysfs_write_long("/sys/.../uioX/rx0/group_count", 256);

	// configure user firmware, uio_mmap

	/* Open rx char device, which allocates suitable dma/mmap buffers */
	rx_fd = open("/dev/.../uioX/rx0", O_RDWR);

	/* Query size of mmap buffer, maybe use mmap for this value? */
	sysfs_read_long("/sys/.../uioX/rx0/mmap_buffer_size", &mmap_buffer_size);

	ring = mmap(NULL, mmap_buffer_size, PROT_READ | PROT_WRITE, MAP_SHARED, 
rx_fd, 0);

	// start/reset capture, uio_mmap

	group_index = 0;

	while (1) {
		if (ring->group_offset[group_index] > 0) {
			process_group(ring + ring->group_offset[group_index], 
ring->group_size_bytes);

			ring->group_offset[group_index] = G_USED;

			group_index = (group_index + 1) % ring->group_count;
		} else {
			poll_fds.fd = rx_fd;
			poll_fds.events = POLLIN | POLLERR;
			poll_fds.revents = 0;

			poll(&poll_fds, 1, -1);
		}
	}

	// stop capture, uio_mmap

	// munmap(ring)
	// munmap(uio_mmap)
	close(rx_fd);
	close(uio_fd);

	return 0;
}

int sysfs_write_long(const char *name, long value)
{
	return 0;
}

int sysfs_read_long(const char *name, long *value)
{
	return 0;
}

int sysfs_read_buffer(const char *name, void *buffer, int len)
{
	return 0;
}

int process_group(void *group, int32_t group_size_bytes)
{
	return 0;
}


Regards,
Vijay

Re: sparc/staging compile error

[Vijay Kumar]

Benjamin Herrenschmidt wrote:
> In any case, we would need to understand better your driver userspace
> API and transfer model to find the right solution.

Please find below the user space skeleton code, to access the driver. It 
was written by Robert Fitzsimons robfitz@273k.net The driver is based on 
this user space interface.

In short, there is a ring buffer that is shared between the _device_ and 
_user-space_. The ring buffer has a header, that is shared between the 
_kernel_ and _user-space_.

The header specifies the offsets of various buffers in the ring buffer. In 
Rx, after user space has read a buffer, the user space writes -1 to the 
offset in the header. When the device has filled the buffer, the kernel 
space writes the offset back to the header.

#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <poll.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>

#define G_USED -1

struct group_circular_buffer {
	int32_t group_size_bytes;
	int32_t group_count;
	int32_t group_offset[0];
};

int sysfs_write_long(const char *name, long value);
int sysfs_read_long(const char *name, long *value);
int sysfs_read_buffer(const char *name, void *buffer, int len);
int process_group(void *group, int32_t group_size_bytes);

int main(int argc, char **argv)
{
	int uio_fd;
	int rx_fd;
	int group_index;
	long mmap_buffer_size;
	struct pollfd poll_fds;
	struct group_circular_buffer *ring;
	char buffer[128];

	/* Query supported device and channels */
	sysfs_read_buffer("/sys/.../uioX/name", buffer, sizeof(buffer));
	sysfs_read_buffer("/sys/.../uioX/version", buffer, sizeof(buffer));
	sysfs_read_buffer("/sys/.../uioX/channels", buffer, sizeof(buffer));

	/* Open and mmap UIO device */
	uio_fd = open("/dev/.../uioX", O_RDWR);
	// uio_mmap = mmap(..., uio_fd, ...); /* map BAR1 */

	/* Configure 1 megabyte receive dma buffer */
	sysfs_write_long("/sys/.../uioX/rx0/block_size", 512);
	sysfs_write_long("/sys/.../uioX/rx0/group_size", 1);
	sysfs_write_long("/sys/.../uioX/rx0/group_count", 256);

	// configure user firmware, uio_mmap

	/* Open rx char device, which allocates suitable dma/mmap buffers */
	rx_fd = open("/dev/.../uioX/rx0", O_RDWR);

	/* Query size of mmap buffer, maybe use mmap for this value? */
	sysfs_read_long("/sys/.../uioX/rx0/mmap_buffer_size", &mmap_buffer_size);

	ring = mmap(NULL, mmap_buffer_size, PROT_READ | PROT_WRITE, MAP_SHARED, 
rx_fd, 0);

	// start/reset capture, uio_mmap

	group_index = 0;

	while (1) {
		if (ring->group_offset[group_index] > 0) {
			process_group(ring + ring->group_offset[group_index], 
ring->group_size_bytes);

			ring->group_offset[group_index] = G_USED;

			group_index = (group_index + 1) % ring->group_count;
		} else {
			poll_fds.fd = rx_fd;
			poll_fds.events = POLLIN | POLLERR;
			poll_fds.revents = 0;

			poll(&poll_fds, 1, -1);
		}
	}

	// stop capture, uio_mmap

	// munmap(ring)
	// munmap(uio_mmap)
	close(rx_fd);
	close(uio_fd);

	return 0;
}

int sysfs_write_long(const char *name, long value)
{
	return 0;
}

int sysfs_read_long(const char *name, long *value)
{
	return 0;
}

int sysfs_read_buffer(const char *name, void *buffer, int len)
{
	return 0;
}

int process_group(void *group, int32_t group_size_bytes)
{
	return 0;
}


Regards,
Vijay