Uncached mmap

Uncached mmap

[Mile Davidovic]

Hello to all,
I am working on fb device and I have general questions regarding cache/uncache
access to MIPS memory. 

User space application use mmap access for r/w access to fb device. If fb use
cached memory, user space application has to call cacheflush to flush contents
instruction and/or data cache. In this case there are no limitations regarding
byte access to this memory. Is this correct statement?

If fb use uncached memory, there is no need for cacheflush call in user space
application but limitation is access to mmap uncached memory. There is no byte
access to uncached mmaped memory. Is this correct statement?


Many thanks in advance.
Best regards Mile

Uncached mmap

[Mile Davidovic]

Hello to all,
I am working on fb device and I have general questions regarding cache/uncache
access to MIPS memory. 

User space application use mmap access for r/w access to fb device. If fb use
cached memory, user space application has to call cacheflush to flush contents
instruction and/or data cache. In this case there are no limitations regarding
byte access to this memory. Is this correct statement?

If fb use uncached memory, there is no need for cacheflush call in user space
application but limitation is access to mmap uncached memory. There is no byte
access to uncached mmaped memory. Is this correct statement?


Many thanks in advance.
Best regards Mile

Re: Uncached mmap

[Ralf Baechle]

On Mon, Nov 13, 2006 at 10:18:52AM +0100, Mile Davidovic wrote:

> Hello to all,
> I am working on fb device and I have general questions regarding cache/uncache
> access to MIPS memory. 
> 
> User space application use mmap access for r/w access to fb device. If fb use
> cached memory, user space application has to call cacheflush to flush contents
> instruction and/or data cache. In this case there are no limitations regarding
> byte access to this memory. Is this correct statement?

Yes.

With caching bus transfers from and to the device happen at the full width
of the bus, so a device won't see byte accesses ever.

> If fb use uncached memory, there is no need for cacheflush call in user space
> application but limitation is access to mmap uncached memory.

Correct.

> There is no byte access to uncached mmaped memory. Is this correct statement?

Definately wrong.  For example alot of mmapped I/O devices use uncached
byte accesses.

This stament if of course limited to the CPU's part of the system.  Devices
may have their specific restrictions on access size and its not uncommon to
have such restrictions though that would seem unlikely for framebuffer
memory.

If your particular CPU support it you may want to use cache mode "uncached
accellerated" for a framebuffer.  It should deliver significtn performance
gains yet avoid the need for cache flushes.

  Ralf

RE: Uncached mmap

[Mile Davidovic]

Hello again,
first I want to thank You for Your fast answer. I work on MIPS 4Kec with linux
kernel in version 2.6.15 from linux-mips (gcc 4.0.3 and gcc 3.4.3).

>> There is no byte access to uncached mmaped memory. Is this correct statement?
>Definately wrong.  For example alot of mmapped I/O devices use uncached
>byte accesses.

Ok, in that case I have problem with byte access on mmaped uncached memory. 
Reason for previous post is next:
If I write bytes to mmaped uncached memory like:
...
ptr = (unsigned char*)mmap(0,lineSize,PROT_READ|PROT_WRITE,MAP_SHARED,fd0,0);
...
for (i = 0; i < 12; i++) 
   *ptr++ = 0xaa;

this loop will not write all bytes correctly (every 4 bytes will have 0xaa as
value), here is dump from Lauterbach debugger:
___address__|_0________4________8________C________0123456789ABCDEF
  D:83660000|>FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF ................
  D:83660010| 000000AA 000000AA 000000AA 0000AA02 ................

and if I use bigger loop
for (i = 0; i < 20; i++) 
   *ptr++ = 0xaa;
My linux will be crashed on 13 write. So, this is reason why I thought that
byte access is not allowed on mmaped uncached memory. 

Is it possible that problem with byte access is related with device mmap
function?

>This stament if of course limited to the CPU's part of the system.  Devices
>may have their specific restrictions on access size and its not uncommon to
>have such restrictions though that would seem unlikely for framebuffer
>memory.

Ok, I understood this.

>If your particular CPU support it you may want to use cache mode "uncached
>accellerated" for a framebuffer.  It should deliver significtn performance
>gains yet avoid the need for cache flushes.



Thanks in advance
Mile

RE: Uncached mmap

[Mile Davidovic]

Hello again,
first I want to thank You for Your fast answer. I work on MIPS 4Kec with linux
kernel in version 2.6.15 from linux-mips (gcc 4.0.3 and gcc 3.4.3).

>> There is no byte access to uncached mmaped memory. Is this correct statement?
>Definately wrong.  For example alot of mmapped I/O devices use uncached
>byte accesses.

Ok, in that case I have problem with byte access on mmaped uncached memory. 
Reason for previous post is next:
If I write bytes to mmaped uncached memory like:
...
ptr = (unsigned char*)mmap(0,lineSize,PROT_READ|PROT_WRITE,MAP_SHARED,fd0,0);
...
for (i = 0; i < 12; i++) 
   *ptr++ = 0xaa;

this loop will not write all bytes correctly (every 4 bytes will have 0xaa as
value), here is dump from Lauterbach debugger:
___address__|_0________4________8________C________0123456789ABCDEF
  D:83660000|>FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF ................
  D:83660010| 000000AA 000000AA 000000AA 0000AA02 ................

and if I use bigger loop
for (i = 0; i < 20; i++) 
   *ptr++ = 0xaa;
My linux will be crashed on 13 write. So, this is reason why I thought that
byte access is not allowed on mmaped uncached memory. 

Is it possible that problem with byte access is related with device mmap
function?

>This stament if of course limited to the CPU's part of the system.  Devices
>may have their specific restrictions on access size and its not uncommon to
>have such restrictions though that would seem unlikely for framebuffer
>memory.

Ok, I understood this.

>If your particular CPU support it you may want to use cache mode "uncached
>accellerated" for a framebuffer.  It should deliver significtn performance
>gains yet avoid the need for cache flushes.



Thanks in advance
Mile

Re: Uncached mmap

[Ralf Baechle]

On Mon, Nov 13, 2006 at 04:30:26PM +0100, Mile Davidovic wrote:

> ptr = (unsigned char*)mmap(0,lineSize,PROT_READ|PROT_WRITE,MAP_SHARED,fd0,0);
> ...
> for (i = 0; i < 12; i++) 
>    *ptr++ = 0xaa;
> 
> this loop will not write all bytes correctly (every 4 bytes will have 0xaa as
> value), here is dump from Lauterbach debugger:
> ___address__|_0________4________8________C________0123456789ABCDEF
>   D:83660000|>FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF ................
>   D:83660010| 000000AA 000000AA 000000AA 0000AA02 ................

Is 83660000 a proper physical address or a virtual address?  A common
mistake is mapping a KSEG _virtual_ address to a userspace _virtual_
address.  Obviously mapping anything virtual to something else virtual
doesn't work ...

> and if I use bigger loop
> for (i = 0; i < 20; i++) 
>    *ptr++ = 0xaa;
> My linux will be crashed on 13 write. So, this is reason why I thought that
> byte access is not allowed on mmaped uncached memory. 

Let me guess, you filled up some write queue which now is waiting for
an acknowledge which never arrives.

> Is it possible that problem with byte access is related with device mmap
> function?

That is fairly simple code.

  Ralf

RE: Uncached mmap

[Mile Davidovic]

Hello all again,

> ptr = (unsigned char*)mmap(0,lineSize,PROT_READ|PROT_WRITE,MAP_SHARED,fd0,0);
> ...
> for (i = 0; i < 12; i++) 
>    *ptr++ = 0xaa;
> 
> this loop will not write all bytes correctly (every 4 bytes will have 0xaa as
> value), here is dump from Lauterbach debugger:
> ___address__|_0________4________8________C________0123456789ABCDEF
>   D:83660000|>FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF ................
>   D:83660010| 000000AA 000000AA 000000AA 0000AA02 ................

Is 83660000 a proper physical address or a virtual address?  A common
mistake is mapping a KSEG _virtual_ address to a userspace _virtual_
address.  Obviously mapping anything virtual to something else virtual
doesn't work ...

Ok, physical address for fb mmap is 0x03660000, mmap return 0x2aaa8000 to user
space application and ptr is shifted for 16 bytes (only for test purpose), so
loop will start writing from 0x2aaa8010, with Lauterbach I can check only
0x83660000 or 0xA3660000 address and both are different then expected. Also
small test loop also show that values on 0x2aaa8010 are not ok.
Also if I write/read to this memory as u32 everything work as expected.

I also have here IDT board with 79RC32K438 (4Kc core) and I will same test.

> My linux will be crashed on 13 write. So, this is reason why I thought that
> byte access is not allowed on mmaped uncached memory. 

Let me guess, you filled up some write queue which now is waiting for
an acknowledge which never arrives.

Unfortunately no, linux stopped and ejtag debugger is dead. 


> Is it possible that problem with byte access is related with device mmap
> function?

That is fairly simple code.

  Ralf

RE: Uncached mmap

[Mile Davidovic]

Hello all again,

> ptr = (unsigned char*)mmap(0,lineSize,PROT_READ|PROT_WRITE,MAP_SHARED,fd0,0);
> ...
> for (i = 0; i < 12; i++) 
>    *ptr++ = 0xaa;
> 
> this loop will not write all bytes correctly (every 4 bytes will have 0xaa as
> value), here is dump from Lauterbach debugger:
> ___address__|_0________4________8________C________0123456789ABCDEF
>   D:83660000|>FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF ................
>   D:83660010| 000000AA 000000AA 000000AA 0000AA02 ................

Is 83660000 a proper physical address or a virtual address?  A common
mistake is mapping a KSEG _virtual_ address to a userspace _virtual_
address.  Obviously mapping anything virtual to something else virtual
doesn't work ...

Ok, physical address for fb mmap is 0x03660000, mmap return 0x2aaa8000 to user
space application and ptr is shifted for 16 bytes (only for test purpose), so
loop will start writing from 0x2aaa8010, with Lauterbach I can check only
0x83660000 or 0xA3660000 address and both are different then expected. Also
small test loop also show that values on 0x2aaa8010 are not ok.
Also if I write/read to this memory as u32 everything work as expected.

I also have here IDT board with 79RC32K438 (4Kc core) and I will same test.

> My linux will be crashed on 13 write. So, this is reason why I thought that
> byte access is not allowed on mmaped uncached memory. 

Let me guess, you filled up some write queue which now is waiting for
an acknowledge which never arrives.

Unfortunately no, linux stopped and ejtag debugger is dead. 


> Is it possible that problem with byte access is related with device mmap
> function?

That is fairly simple code.

  Ralf

RE: Uncached mmap

[Mile Davidovic]

Hello again,
I just want to make apologies for bothering, I contacted our vendor and it seems
that uncached DRAM byte accesses on their core (based on MIPS 4Kec) are
impossible. 
Thanks a lot for Your effort.

Best regards Mile

Hello all again,

> ptr = (unsigned char*)mmap(0,lineSize,PROT_READ|PROT_WRITE,MAP_SHARED,fd0,0);
> ...
> for (i = 0; i < 12; i++) 
>    *ptr++ = 0xaa;
> 
> this loop will not write all bytes correctly (every 4 bytes will have 0xaa as
> value), here is dump from Lauterbach debugger:
> ___address__|_0________4________8________C________0123456789ABCDEF
>   D:83660000|>FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF ................
>   D:83660010| 000000AA 000000AA 000000AA 0000AA02 ................

Is 83660000 a proper physical address or a virtual address?  A common
mistake is mapping a KSEG _virtual_ address to a userspace _virtual_
address.  Obviously mapping anything virtual to something else virtual
doesn't work ...

Ok, physical address for fb mmap is 0x03660000, mmap return 0x2aaa8000 to user
space application and ptr is shifted for 16 bytes (only for test purpose), so
loop will start writing from 0x2aaa8010, with Lauterbach I can check only
0x83660000 or 0xA3660000 address and both are different then expected. Also
small test loop also show that values on 0x2aaa8010 are not ok.
Also if I write/read to this memory as u32 everything work as expected.

I also have here IDT board with 79RC32K438 (4Kc core) and I will same test.

> My linux will be crashed on 13 write. So, this is reason why I thought that
> byte access is not allowed on mmaped uncached memory. 

Let me guess, you filled up some write queue which now is waiting for
an acknowledge which never arrives.

Unfortunately no, linux stopped and ejtag debugger is dead. 


> Is it possible that problem with byte access is related with device mmap
> function?

That is fairly simple code.

  Ralf

RE: Uncached mmap

[Mile Davidovic]

Hello again,
I just want to make apologies for bothering, I contacted our vendor and it seems
that uncached DRAM byte accesses on their core (based on MIPS 4Kec) are
impossible. 
Thanks a lot for Your effort.

Best regards Mile

Hello all again,

> ptr = (unsigned char*)mmap(0,lineSize,PROT_READ|PROT_WRITE,MAP_SHARED,fd0,0);
> ...
> for (i = 0; i < 12; i++) 
>    *ptr++ = 0xaa;
> 
> this loop will not write all bytes correctly (every 4 bytes will have 0xaa as
> value), here is dump from Lauterbach debugger:
> ___address__|_0________4________8________C________0123456789ABCDEF
>   D:83660000|>FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF ................
>   D:83660010| 000000AA 000000AA 000000AA 0000AA02 ................

Is 83660000 a proper physical address or a virtual address?  A common
mistake is mapping a KSEG _virtual_ address to a userspace _virtual_
address.  Obviously mapping anything virtual to something else virtual
doesn't work ...

Ok, physical address for fb mmap is 0x03660000, mmap return 0x2aaa8000 to user
space application and ptr is shifted for 16 bytes (only for test purpose), so
loop will start writing from 0x2aaa8010, with Lauterbach I can check only
0x83660000 or 0xA3660000 address and both are different then expected. Also
small test loop also show that values on 0x2aaa8010 are not ok.
Also if I write/read to this memory as u32 everything work as expected.

I also have here IDT board with 79RC32K438 (4Kc core) and I will same test.

> My linux will be crashed on 13 write. So, this is reason why I thought that
> byte access is not allowed on mmaped uncached memory. 

Let me guess, you filled up some write queue which now is waiting for
an acknowledge which never arrives.

Unfortunately no, linux stopped and ejtag debugger is dead. 


> Is it possible that problem with byte access is related with device mmap
> function?

That is fairly simple code.

  Ralf