mmap on 440gx

mmap on 440gx

[Ed Goforth]

I've been struggling with implementing mmap on a 440gx-based custom
board.  I have been able to use ioremap(), but we really need a mmap()
for our software.  The kernel is 2.4.18 (TimeSys 4.0).

I'm trying to access one of our FPGA's located at 0x50000000.  Offsets
0x00000000 thru
0x000FFFFF are control and status registers.  Offsets 0x01000000 thru
0x017FFFFF are the QDR SRAM on the FPGA.

Using ioremap_nocache(0x50000000, bytes_to_map) works (so long as
bytes_to_map isn't too large).  I can then use in_be32() and
out_be32() to read/write the registers and the SRAM.  However, trying
to implement mmap() causes the board to lock up hard.  I have tried
iterations both where the driver first does an ioremap(), and where it
doesn't do the ioremap().  Some bits:

In driver module:
static struct file_operations test_mmap_fops =3D {
  read : test_mmap_read,
  mmap : test_mmap_mmap,
  open : test_mmap_open,
  release : test_mmap_release,
};

static int test_mmap_mmap (struct file *filp, struct vm_area_struct *vma) {
  vma->vm_flags |=3D (VM_SHM | VM_LOCKED | VM_IO | VM_RESERVED);

  if (remap_page_range(vma->vm_start, 0x50000000, 4096, vma->vm_page_prot))
    return(-EAGAIN);

  return(0);

} /* test_mmap_mmap() */


When the test code calls mmap():
  unsigned long *mmap_region;
  unsigned long data;
  mmap_region =3D mmap(0, 4096, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
it seems to work.  However, dereferencing mmap_region, either thru
  data =3D *mmap_region;
or
  data =3D mmap_region[0];
causes the board to lock hard.

I would appreciate any suggestions or hints.

Thanks,
Ed

Re: mmap on 440gx

[Matt Porter]

On Thu, Jun 16, 2005 at 05:33:44PM -0400, Ed Goforth wrote:
> I've been struggling with implementing mmap on a 440gx-based custom
> board.  I have been able to use ioremap(), but we really need a mmap()
> for our software.  The kernel is 2.4.18 (TimeSys 4.0).

There are some special things done for handling [io]_remap_page/pfn_range
on other vendor kernels and the current mainline kernel.  I'm not sure
if your vendor kernel addressed them since it would be a vendor-specific
patch in that timeframe.  The special things are due to 36-bit
addressing.

> I'm trying to access one of our FPGA's located at 0x50000000.  Offsets

Let's start at the beginning.  How do you have FPGA's at 0x50000000?
that address falls with the fix DDR SDRAM area on the 440GX memory
map. All peripheral and EBC space is mapped by 4GB. You lost me
right here. Oh wait, are you referring to the least significant 32-bits
of the physical mapping. It's not really at 0x50000000.

You need something like the bigphys_remap patch for 2.4 that can be
found at ftp://source.mvista.com/pub/linuxppc/

It does the same fixup for remap_page_range() that happens for
ioremap().  In 2.6 this stuff is all in the mainline kernel but
you use io_rename_[page|pfn]_range() there.

-Matt

Re: mmap on 440gx

[Ed Goforth]

On 6/16/05, Matt Porter <mporter@kernel.crashing.org> wrote:
> On Thu, Jun 16, 2005 at 05:33:44PM -0400, Ed Goforth wrote:

Thanks for taking the time to reply.

> > I've been struggling with implementing mmap on a 440gx-based custom
> > board.  I have been able to use ioremap(), but we really need a mmap()
> > for our software.  The kernel is 2.4.18 (TimeSys 4.0).
>=20
> There are some special things done for handling [io]_remap_page/pfn_range
> on other vendor kernels and the current mainline kernel.  I'm not sure
> if your vendor kernel addressed them since it would be a vendor-specific
> patch in that timeframe.  The special things are due to 36-bit
> addressing.
>=20
> > I'm trying to access one of our FPGA's located at 0x50000000.  Offsets
>=20
> Let's start at the beginning.  How do you have FPGA's at 0x50000000?
> that address falls with the fix DDR SDRAM area on the 440GX memory
> map. All peripheral and EBC space is mapped by 4GB. You lost me
> right here. Oh wait, are you referring to the least significant 32-bits
> of the physical mapping. It's not really at 0x50000000.

Of course, you're correct.  The 36-bit base address is 0x1_0000_0000.=20
It is mapped to the 32-bit address 0x5000_0000.

Should I pass the 36-bit address to remap_page_range()?

For what it's worth, __pa(0x5000_0000) returns 0x9000_0000.

>=20
> You need something like the bigphys_remap patch for 2.4 that can be
> found at ftp://source.mvista.com/pub/linuxppc/

I just checked the source, and that patch has indeed been applied.

>=20
> It does the same fixup for remap_page_range() that happens for
> ioremap().  In 2.6 this stuff is all in the mainline kernel but
> you use io_rename_[page|pfn]_range() there.
>=20
> -Matt
>=20
Thanks,
Ed

Re: mmap on 440gx

[Matt Porter]

On Thu, Jun 16, 2005 at 11:47:22PM -0400, Ed Goforth wrote:
> On 6/16/05, Matt Porter <mporter@kernel.crashing.org> wrote:
> > On Thu, Jun 16, 2005 at 05:33:44PM -0400, Ed Goforth wrote:
> 
> Thanks for taking the time to reply.
> 
> > > I've been struggling with implementing mmap on a 440gx-based custom
> > > board.  I have been able to use ioremap(), but we really need a mmap()
> > > for our software.  The kernel is 2.4.18 (TimeSys 4.0).
> > 
> > There are some special things done for handling [io]_remap_page/pfn_range
> > on other vendor kernels and the current mainline kernel.  I'm not sure
> > if your vendor kernel addressed them since it would be a vendor-specific
> > patch in that timeframe.  The special things are due to 36-bit
> > addressing.
> > 
> > > I'm trying to access one of our FPGA's located at 0x50000000.  Offsets
> > 
> > Let's start at the beginning.  How do you have FPGA's at 0x50000000?
> > that address falls with the fix DDR SDRAM area on the 440GX memory
> > map. All peripheral and EBC space is mapped by 4GB. You lost me
> > right here. Oh wait, are you referring to the least significant 32-bits
> > of the physical mapping. It's not really at 0x50000000.
> 
> Of course, you're correct.  The 36-bit base address is 0x1_0000_0000. 
> It is mapped to the 32-bit address 0x5000_0000.

That's an interesting way to look at it but there isn't really a
"mapping".  You have configured a EBC chip selects to decode at
1_5000_0000.

> Should I pass the 36-bit address to remap_page_range()?

No, it takes a 32-bit physical address and "fixes it up" using
the fixup_bigphys_addr() method.  Double check that the trap ranges
are correct for you board port.  They need to be modified to add
the appropriate MS 4-bit.  Since ioremap() also uses
fixup_bigphys_addr() to "fixup" 32-bit addresses to 36-bit,
it should just work. 

> For what it's worth, __pa(0x5000_0000) returns 0x9000_0000.

Sure, you just asked it to subtract KERNELBASE from a physical address.
Don't use __pa() in drivers. That's expected behavior. Why are
you doing that?
 
> > You need something like the bigphys_remap patch for 2.4 that can be
> > found at ftp://source.mvista.com/pub/linuxppc/
> 
> I just checked the source, and that patch has indeed been applied.

Ok, assuming it's all correct then I don't know why it does work
for you in your vendor tree. Have you asked Timesys for help?

Put some debug statements throughout fixup_bigphys_addr() to see
what's going on.

-Matt

Re: mmap on 440gx

[Ed Goforth]

On 6/17/05, Matt Porter <mporter@kernel.crashing.org> wrote:
> On Thu, Jun 16, 2005 at 11:47:22PM -0400, Ed Goforth wrote:
> > On 6/16/05, Matt Porter <mporter@kernel.crashing.org> wrote:
> > > On Thu, Jun 16, 2005 at 05:33:44PM -0400, Ed Goforth wrote:
> >
> > Thanks for taking the time to reply.
> >
> > > > I've been struggling with implementing mmap on a 440gx-based custom
> > > > board.  I have been able to use ioremap(), but we really need a mma=
p()
> > > > for our software.  The kernel is 2.4.18 (TimeSys 4.0).
> > >
>=20
>=20
> > For what it's worth, __pa(0x5000_0000) returns 0x9000_0000.
>=20
> Sure, you just asked it to subtract KERNELBASE from a physical address.
> Don't use __pa() in drivers. That's expected behavior. Why are
> you doing that?

Desperation.

>=20
> > > You need something like the bigphys_remap patch for 2.4 that can be
> > > found at ftp://source.mvista.com/pub/linuxppc/
> >
> > I just checked the source, and that patch has indeed been applied.
>=20
> Ok, assuming it's all correct then I don't know why it does work
> for you in your vendor tree. Have you asked Timesys for help?

Not yet.  I wanted to get some confidence that I was using the correct
APIs appropriately.

>=20
> Put some debug statements throughout fixup_bigphys_addr() to see
> what's going on.

My include/asm/mmu.h has it defined as:
#ifndef CONFIG_440
#include <asm-generic/mmu.h>
#else
typedef unsigned long long phys_addr_t;
#define fixup_bigphys_addr(addr, size)   (addr)
#endif

I see that in later 2.4 kernels and in mainline 2.6 it is implemented
as an actual working routine.  Perhaps that explains it all.  The
ioremap() in my tree implements the fixup code inline, which would
explain why it works, right?

>=20
> -Matt
>=20

Thanks again.
Ed

Re: mmap on 440gx

[Matt Porter]

On Fri, Jun 17, 2005 at 01:42:07AM -0400, Ed Goforth wrote:
> On 6/17/05, Matt Porter <mporter@kernel.crashing.org> wrote:
> > On Thu, Jun 16, 2005 at 11:47:22PM -0400, Ed Goforth wrote:
> > > On 6/16/05, Matt Porter <mporter@kernel.crashing.org> wrote:
> > > > On Thu, Jun 16, 2005 at 05:33:44PM -0400, Ed Goforth wrote:
> > >
> > > Thanks for taking the time to reply.
> > >
> > > > > I've been struggling with implementing mmap on a 440gx-based custom
> > > > > board.  I have been able to use ioremap(), but we really need a mmap()
> > > > > for our software.  The kernel is 2.4.18 (TimeSys 4.0).
> > > >
> > 
> > 
> > > For what it's worth, __pa(0x5000_0000) returns 0x9000_0000.
> > 
> > Sure, you just asked it to subtract KERNELBASE from a physical address.
> > Don't use __pa() in drivers. That's expected behavior. Why are
> > you doing that?
> 
> Desperation.

Heh, I like that. :)

<snip>

> > Put some debug statements throughout fixup_bigphys_addr() to see
> > what's going on.
> 
> My include/asm/mmu.h has it defined as:
> #ifndef CONFIG_440
> #include <asm-generic/mmu.h>
> #else
> typedef unsigned long long phys_addr_t;
> #define fixup_bigphys_addr(addr, size)   (addr)
> #endif
> 
> I see that in later 2.4 kernels and in mainline 2.6 it is implemented
> as an actual working routine.  Perhaps that explains it all.  The
> ioremap() in my tree implements the fixup code inline, which would
> explain why it works, right?

It sure does.  That code is 100% wrong.  Just merge in mainline 2.4
stuff and you should be golden. 

-Matt