Can't mmap the top 1MB of /dev/mem?

Can't mmap the top 1MB of /dev/mem?

[Mark S. Mathews]

Hi Dan (and all),

I've been playing w/ mmap for flash and nvram access on the rpxlite(MV
derived kernel). One thing I've noticed is that it won't let me mmap the
top 1MB of the address space.  I'm guessing that the region isn't
represented in the MMU stuff or this is a deliberate block in the /dev/mem
driver.

I've done a little digging on my own and haven't found the cause.  Can
anyone point me to the right file/function(s)?

Many Thanks,
-Mark

Mark S. Mathews

AbsoluteValue Systems      Web:    http://www.linux-wlan.com
P.O. Box 941149            e-mail: mark@linux-wlan.com
Maitland, FL 32794-1149    Phone:  407.644.8582
USA                        Fax:    407.539.1294


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Can't mmap the top 1MB of /dev/mem?

[Dan Malek]

"Mark S. Mathews" wrote:


> ...... One thing I've noticed is that it won't let me mmap the
> top 1MB of the address space.

That should work, I do this quite regularly.  What actually fails,
the mmap() or your access to the mapped region?

There is what I do:

	mem_addr = (u_char *)mmap(NULL, FLASH_MEM_SIZE,
			(PROT_READ | PROT_WRITE), MAP_SHARED,
			mem_fd, FLASH_MEM_ADDR);

#define FLASH_MEM_SIZE and FLASH_MEM_ADDR accordingly.  I hope there
isn't some weird arithmetic rounding problem when we hit the top.  I
just usually tell this to map the upper 8M bytes.

The MMU doesn't care, if the generic Linux VM subsystem creates the PTEs
it just loads them.  I don't remember any code in the mem driver that
would prevent this either.  None of this is unique to the 8xx.


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Can't mmap the top 1MB of /dev/mem?

[Pavel Roskin]

Hello, Dan!

> That should work, I do this quite regularly.  What actually fails,
> the mmap() or your access to the mapped region?

mmap fails with EINVAL.

> There is what I do:
>
> 	mem_addr = (u_char *)mmap(NULL, FLASH_MEM_SIZE,
> 			(PROT_READ | PROT_WRITE), MAP_SHARED,
> 			mem_fd, FLASH_MEM_ADDR);

I know that program. FLASH_MEM_SIZE is a small number used only in the
first mmap() to determine the size of the flash. If you make it so big
that (int)(FLASH_MEM_ADDR+FLASH_MEM_SIZE)==0 then mmap() fails.

The test program below works on LinuxPPC (2.2.14 with tweaks for G3) but
fails on RPX/Lite (MontaVista kernel):

mmap at 0xffc00000, size 0x400000 failed: Invalid argument

Regards,
Pavel Roskin

=================================
#include <sys/types.h>
#include <sys/mman.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#define FLASH_MEM_ADDR	((uint)0xffc00000)
#define FLASH_MEM_SIZE	((uint)0x00400000)
int
main(void)
{
	int	mem_fd, i;
	uint	mfg, dev;
	void* mem_addr = NULL;

	if ((mem_fd = open("/dev/mem", O_RDWR)) < 0) {
		perror("cannot open /dev/mem");
		exit(1);
	}

	mem_addr = (u_int *)mmap(NULL, FLASH_MEM_SIZE,
				(PROT_READ | PROT_WRITE), MAP_SHARED,
				mem_fd, FLASH_MEM_ADDR);

	if ((int)mem_addr < 0) {
		fprintf(stderr, "mmap at 0x%x, size 0x%x failed: %s\n",
			FLASH_MEM_ADDR, FLASH_MEM_SIZE, strerror(errno));
		exit(1);
	}
}
=================================


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Can't mmap the top 1MB of /dev/mem?

[Dan Malek]

Pavel Roskin wrote:

> I know that program. FLASH_MEM_SIZE is a small number used only in the
> first mmap() to determine the size of the flash.

Yes, but then it re-maps based upon the flash devices found....but I'll
bet I don't map the top boot sector.......

> ..... If you make it so big
> that (int)(FLASH_MEM_ADDR+FLASH_MEM_SIZE)==0 then mmap() fails.

So, this should be a pretty obvious boundary condition bug in
drivers/char/mem.c or one of the Linux MM functions.  You want to
learn about MMUs?  Go check some of those files for inspiration......


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Can't mmap the top 1MB of /dev/mem?

[Pavel Roskin]

> So, this should be a pretty obvious boundary condition bug in
> drivers/char/mem.c or one of the Linux MM functions.  You want to
> learn about MMUs?  Go check some of those files for inspiration......

Already. This should be The Right Thing. Anybody wants to send it to
Linus?

len is not 0, it is clear from the code above. (off + len) points to the
byte that is after the allocated area, (off + len - 1) points to the
last byte. The last byte should not be on the "other end of memory"

=======================
diff -u -r1.1.1.1 mmap.c
--- mm/mmap.c	2000/01/03 20:11:04	1.1.1.1
+++ mm/mmap.c	2000/06/22 19:52:51
@@ -186,7 +186,7 @@
 		return -EINVAL;

 	/* offset overflow? */
-	if (off + len < off)
+	if (off + len - 1 < off)
 		return -EINVAL;

 	/* Too many mappings? */
=======================

Regards,
Pavel Roskin


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Can't mmap the top 1MB of /dev/mem?

[Dan Malek]

Pavel Roskin wrote:

> Already. This should be The Right Thing. Anybody wants to send it to
> Linus?

I'll try to sneak it in.  We are already "discussing" some of my
other changes :-).

There you go Mark.  Patch it and try it.

Thanks.


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Can't mmap the top 1MB of /dev/mem?

[Mark S. Mathews]

Sorry, I should have been more specific....aren't you all mind-readers?
;-)

The mmap is failing w/ errno=EINVAL.  Here's the call:

mem_fd = open("/dev/mem", ORDWR);

p = mmap( NULL, (16*1024*1024),
          (PROT_READ|PROT_WRITE), MAP_SHARED, mem_fd, 0xff000000UL);

15MB works just fine.  I just tried a test w/ offset 0xff800000 and size
of 8MB.  That failed too.  Here's a couple of other tests I just tried:

offset      len        result
-----------------------------------
0xfff00000, 1MB        failed
0xffe00000, 1MB        success

So it looks like it's specifically related to the top 1MB, not the size.

BTW: Dan, I think you may have run into this before.  The sample code you
once sent me only mmaps the lower 7MB of an 8MB flash setup.

This whole thing isn't really a critical issue for me.  More of a
curiosity.  I'm not using those sectors anyway.  ;-)

If it _were_ extremely  important I could always trace the mmap system
call.
_That's_ why Linux is so cool.

Thanks to everyone for the responses,
-Mark

On Thu, 22 Jun 2000, Dan Malek wrote:

> "Mark S. Mathews" wrote:
>
>
> > ...... One thing I've noticed is that it won't let me mmap the
> > top 1MB of the address space.
>
> That should work, I do this quite regularly.  What actually fails,
> the mmap() or your access to the mapped region?
>
> There is what I do:
>
> 	mem_addr = (u_char *)mmap(NULL, FLASH_MEM_SIZE,
> 			(PROT_READ | PROT_WRITE), MAP_SHARED,
> 			mem_fd, FLASH_MEM_ADDR);
>
> #define FLASH_MEM_SIZE and FLASH_MEM_ADDR accordingly.  I hope there
> isn't some weird arithmetic rounding problem when we hit the top.  I
> just usually tell this to map the upper 8M bytes.
>
> The MMU doesn't care, if the generic Linux VM subsystem creates the PTEs
> it just loads them.  I don't remember any code in the mem driver that
> would prevent this either.  None of this is unique to the 8xx.
>
>
> 	-- Dan
>


Mark S. Mathews

AbsoluteValue Systems      Web:    http://www.linux-wlan.com
P.O. Box 941149            e-mail: mark@linux-wlan.com
Maitland, FL 32794-1149    Phone:  407.644.8582
USA                        Fax:    407.539.1294


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Can't mmap the top 1MB of /dev/mem?

[Pavel Roskin]

Hello, Mark!

> If it _were_ extremely  important I could always trace the mmap system
> call.

I already know that it happens in do_mmap(), at this point:

/* offset overflow? */
if (off + len < off)
        return -EINVAL;

I'm puzzled why it doesn't happen on G3.

> _That's_ why Linux is so cool.

Yup.

Pavel


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Can't mmap the top 1MB of /dev/mem?

[Mark S. Mathews]

I figured it was something like that.  Shouldn't that expression be
testing the topmost address of the mmap'd region instead of one beyond it?

i.e.:

   if ( (off+len-1) < off )
	return EINVAL;


It works on the G3?  Perhaps the compiler/processor difference is getting
you into some of the 64-bit register goofiness or a slightly different
code sequence that does something different w/ the overflow flag.  Just
guessing here.

-Mark


On Thu, 22 Jun 2000, Pavel Roskin wrote:

>
> Hello, Mark!
>
> > If it _were_ extremely  important I could always trace the mmap system
> > call.
>
> I already know that it happens in do_mmap(), at this point:
>
> /* offset overflow? */
> if (off + len < off)
>         return -EINVAL;
>
> I'm puzzled why it doesn't happen on G3.
>
> > _That's_ why Linux is so cool.
>
> Yup.
>
> Pavel
>
>
>


Mark S. Mathews

AbsoluteValue Systems      Web:    http://www.linux-wlan.com
P.O. Box 941149            e-mail: mark@linux-wlan.com
Maitland, FL 32794-1149    Phone:  407.644.8582
USA                        Fax:    407.539.1294


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Can't mmap the top 1MB of /dev/mem?

[Pavel Roskin]

Hello, Mark!

>    if ( (off+len-1) < off )
> 	return EINVAL;

It's done this way in Linux-2.2.14 :-)

> It works on the G3?  Perhaps the compiler/processor difference is getting
> you into some of the 64-bit register goofiness or a slightly different
> code sequence that does something different w/ the overflow flag.  Just
> guessing here.

Absolutely no "64-bit register goofiness" :-)))

Pavel


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Can't mmap the top 1MB of /dev/mem?

[Pavel Roskin]

Hello, Mark!

Hint: if you are going to program the flash you don't want to allocate the
amount of memory larger than the file you are going to program. And you
probably don't want to erase the initial loader ;-)

But I agree that the kernel should allow you do that and it doesn't.

Regards,
Pavel Roskin


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Can't mmap the top 1MB of /dev/mem?

[Frank Smith]

On Thu, 22 Jun 2000, Mark S. Mathews wrote:

>
> Hi Dan (and all),
>
> I've been playing w/ mmap for flash and nvram access on the rpxlite(MV
> derived kernel). One thing I've noticed is that it won't let me mmap the
> top 1MB of the address space.  I'm guessing that the region isn't
> represented in the MMU stuff or this is a deliberate block in the /dev/mem
> driver.
>
> I've done a little digging on my own and haven't found the cause.  Can
> anyone point me to the right file/function(s)?

include/asm/processor.h: -------------------------------------------------

...
#define TASK_SIZE    (0x80000000UL)
...

mm/mmap.c: ---------------------------------------------------------------

...
unsigned long do_mmap(struct file * file, unsigned long addr, unsigned
long len,
        unsigned long prot, unsigned long flags, unsigned long off)
{
...
        if (len > TASK_SIZE || addr > TASK_SIZE-len)
                return -EINVAL;
...


-------------------------------------------------------------------------

Wouldn't this account for it?  I'm not sure where your top 1M of
address space is on your board, but the DY4 board I'm working on
has flash up around 0xfe000000. And we ran into the same problem trying
to map our flash.

We solved the problem in a roundabout way by doing what we need to do
from inside the kernel, calling remap_page_range() directly like
mmap_mem() [drivers/char/mmap.c] does.


Frank.

-----
Frank Smith, MCompSci
Principal Software Designer      frank.smith@amirix.com
AMIRIX Systems Inc.              http://www.amirix.com/
Embedded Debian Project          http://www.emdebian.org/
77 Chain Lake Drive              902-450-1700 x289 (Phone)
Halifax, N.S. B3S 1E1            902-450-1704 (FAX)


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Can't mmap the top 1MB of /dev/mem?

[Dan Malek]

Frank Smith wrote:

> Wouldn't this account for it?


No.  Those are virtual address calculations in that file........


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Can't mmap the top 1MB of /dev/mem?

[Frank Smith]

On Thu, 22 Jun 2000, Dan Malek wrote:

>
> Frank Smith wrote:
>
> > Wouldn't this account for it?
>
>
> No.  Those are virtual address calculations in that file........

Oops. Upon thinking about this a bit more, the problem we were having
was that we were trying to get physical 0xfe000000 mapped to virtual
0xfe000000.

The board we're using has firmware that we want to be able to call
after Linux is up and running. The FW runs from 0x3000-0x60000, but
refers to data up in 0xfe space. Our first naive attempts were to
use mmap from userland, but we ended up putting stuff in the kernel
and mapping flash directly with remap_page_range().


Frank.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/