[Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writing.

[Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writing.

[Mike Fox]

[-- Attachment #1: Type: text/plain, Size: 1339 bytes --]

Hi,

 

I am running kernel version 2.4.19 and I am getting a system crash when I
mmap my PCI device into user space and access it.

 

It seems to be the same issue that was posted earlier in the following post,
which had no response:

 

Has anyone tried using mmap to remap PCI device memory into user space?  I
have seen the following problem:  If I do the mmap, it returns successfully,
and I can indeed access the device memory.  Unfortunately, the system will
hang within about a minute after doing so.  I do not get an oops, or crash
of any kind, simply no further response form the machine.  If I just do the
mmap and then munmap the system will not hang.  But if I do one access to
the mmapped address, the system will hang.  I am using the remap_page_range
call in my mmap routine, and this same code works on x86 and Alpha
platforms.  I found it odd that it takes a while for the system to hang
after the access, is their something special I have to do for this to work
on IA64?  I am running 2.4.1 with the 1/31 patch, although this problem
occurs on all previous kernels I have tried.  Thanks in advance for any
insight anyone might have.

 

Thanks for any help you can give. I'm new to Linux and IA64 and I am in a
rush to port a PCI driver and API.

 

Thanks,

Mike Fox.

 

 


[-- Attachment #2: Type: text/html, Size: 4902 bytes --]

Re: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin

[Grant Grundler]

On Tue, Apr 08, 2003 at 05:59:56PM -0400, Mike Fox wrote:
...
> Has anyone tried using mmap to remap PCI device memory into user space?  I
> have seen the following problem:  If I do the mmap, it returns successfully,
> and I can indeed access the device memory.  Unfortunately, the system will
> hang within about a minute after doing so.
...

David Mosberger passed on a very short program who's essence is:
  mem = mmap (NULL, page_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, offset);
  if (mem = MAP_FAILED)
  {
    perror ("memmap");
    exit (-1);
  }

This works on the HP ZX1 platforms with 2.4.20+patches kernel.

Be warned, it's *very* easy to crash a machine when using /dev/mem
and scribbling in misc parts of IO space. The symptom will typically
be an MCA (see "errdump MCA" output from EFI to get the xip).

grant

RE: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin

[Mike Fox]

Ok I have some more info. The crash only happens after I do a write to my
PCI controller. I map the PCI controller into user space and I can read and
write a scratch register ok. I can also read other initialized registers to
confirm I am reading my PCI controller. I can do many reads, but after some
number of writes the system goes down. I went ahead and updated the kernel
to 2.4.20 with the latest patches. Any thoughts or a push in the right
direction are very much appreciated.

My mmap call in user space looks something like this:

    pRegRegionMappedAddr = (DWORD*)mmap( 0, 
                                         WSII_REG_SPACE_SIZE_BYTES,
                                         PROT_WRITE | PROT_READ,
                                         MAP_SHARED,
                                         DriverHandle,
                                         0 );

In the driver I am basically doing this in my mmap entry point:

  vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
  vma->vm_flags |= VM_IO;

  /*
   * this maps the physical memory to the user
   */
  if (remap_page_range(vma->vm_start, offset, vma->vm_end-vma->vm_start,
                            vma->vm_page_prot))
  {
    return -EAGAIN;
  }

Thanks,
Mike.



-----Original Message-----
From: linux-ia64-admin@linuxia64.org [mailto:linux-ia64-admin@linuxia64.org]
On Behalf Of Grant Grundler
Sent: Tuesday, April 08, 2003 8:43 PM
To: Mike Fox
Cc: linux-ia64@linuxia64.org
Subject: Re: [Linux-ia64] mmap crashes system after mapping PCI device into
user space and reading and writing.

On Tue, Apr 08, 2003 at 05:59:56PM -0400, Mike Fox wrote:
...
> Has anyone tried using mmap to remap PCI device memory into user space?  I
> have seen the following problem:  If I do the mmap, it returns
successfully,
> and I can indeed access the device memory.  Unfortunately, the system will
> hang within about a minute after doing so.
...

David Mosberger passed on a very short program who's essence is:
  mem = mmap (NULL, page_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd,
offset);
  if (mem == MAP_FAILED)
  {
    perror ("memmap");
    exit (-1);
  }

This works on the HP ZX1 platforms with 2.4.20+patches kernel.

Be warned, it's *very* easy to crash a machine when using /dev/mem
and scribbling in misc parts of IO space. The symptom will typically
be an MCA (see "errdump MCA" output from EFI to get the xip).

grant

_______________________________________________
Linux-IA64 mailing list
Linux-IA64@linuxia64.org
http://lists.linuxia64.org/lists/listinfo/linux-ia64

Re: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin

[Grant Grundler]

On Wed, Apr 09, 2003 at 01:36:33AM -0400, Mike Fox wrote:
>   /*
>    * this maps the physical memory to the user
>    */
>   if (remap_page_range(vma->vm_start, offset, vma->vm_end-vma->vm_start,
>                             vma->vm_page_prot))
>   {
>     return -EAGAIN;
>   }

The example code I posted (a) used physical addresses and (b)
assume the "user" was root (to access /dev/mem/).

XFree86 pokes around in PCI MMIO and IO space and does work.
Have you tried to dig through the source of XF86 4.2?

Have you checked the return address (to user space) is uncached?
If I understood details of VM it might be obvious from the code
you posted.

hth,
grant

> 
> Thanks,
> Mike.
> 
> 
> 
> -----Original Message-----
> From: linux-ia64-admin@linuxia64.org [mailto:linux-ia64-admin@linuxia64.org]
> On Behalf Of Grant Grundler
> Sent: Tuesday, April 08, 2003 8:43 PM
> To: Mike Fox
> Cc: linux-ia64@linuxia64.org
> Subject: Re: [Linux-ia64] mmap crashes system after mapping PCI device into
> user space and reading and writing.
> 
> On Tue, Apr 08, 2003 at 05:59:56PM -0400, Mike Fox wrote:
> ...
> > Has anyone tried using mmap to remap PCI device memory into user space?  I
> > have seen the following problem:  If I do the mmap, it returns
> successfully,
> > and I can indeed access the device memory.  Unfortunately, the system will
> > hang within about a minute after doing so.
> ...
> 
> David Mosberger passed on a very short program who's essence is:
>   mem = mmap (NULL, page_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd,
> offset);
>   if (mem = MAP_FAILED)
>   {
>     perror ("memmap");
>     exit (-1);
>   }
> 
> This works on the HP ZX1 platforms with 2.4.20+patches kernel.
> 
> Be warned, it's *very* easy to crash a machine when using /dev/mem
> and scribbling in misc parts of IO space. The symptom will typically
> be an MCA (see "errdump MCA" output from EFI to get the xip).
> 
> grant
> 
> _______________________________________________
> Linux-IA64 mailing list
> Linux-IA64@linuxia64.org
> http://lists.linuxia64.org/lists/listinfo/linux-ia64
> 
> 
> _______________________________________________
> Linux-IA64 mailing list
> Linux-IA64@linuxia64.org
> http://lists.linuxia64.org/lists/listinfo/linux-ia64

-- 
Grant Grundler                           Linux Open Systems Lab/BCST
grundler@cup.hp.com                      +1.408.447.7253

Re: [Linux-ia64] mmap crashes system after mapping PCI device into user

[Christian Hinkelbein]

Hi,

Mike Fox schrieb:
> 
> 
> In the driver I am basically doing this in my mmap entry point:
> 
>   vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
>   vma->vm_flags |= VM_IO;

i know this is ugly stuff, but ...
maybe you want to try this instead of the above:

#if defined(__ia64__)
	vmap->vm_flags |= (VM_SHM | VM_LOCKED | VM_IO);
	vmap->vm_flags &= ~(VM_SHARED | VM_MAYSHARE);
	vmap->vm_page_prot = __pgprot(pgprot_val(vmap->vm_page_prot)& ~_PAGE_AR_RW);
#endif

> 
> Thanks,
> Mike.
> 

 christian

Re: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin

[David Mosberger]

>>>>> On Tue, 8 Apr 2003 17:59:56 -0400, "Mike Fox" <mfox@annapmicro.com> said:

  Mike> Hi, I am running kernel version 2.4.19 and I am getting a
  Mike> system crash when I mmap my PCI device into user space and
  Mike> access it.

One thing you need to be careful about is memory attribute aliasing.
The CPU (or platform chips) can MCA if you try to access the same
physical location with different memory attributes (e.g., cached vs.
uncached).  Also, keep in mind that the kernel uses a 64MB page size
for the identity-mapped regions (regions 6 and 7).  This means that
you cannot map stuff requiring different attributes within the same
64MB page.  I kind of doubt this is what's causing the problem for
you, but since you didn't give any details about the hw you're trying
to map, it's hard to be more specific.

As others have pointed out, it's of course also important to verify
that you access I/O space uncached.

	--david

RE: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin

[David Mosberger]

>>>>> On Wed, 9 Apr 2003 01:36:33 -0400, "Mike Fox" <mfox@annapmicro.com> said:

  Mike> In the driver I am basically doing this in my mmap entry point:

  Mike>   vma-> vm_page_prot = pgprot_noncached(vma->vm_page_prot);
  Mike>   vma-> vm_flags |= VM_IO;

  Mike> /*
  Mike> * this maps the physical memory to the user
  Mike> */
  Mike> if (remap_page_range(vma->vm_start, offset, vma->vm_end-vma->vm_start, vma->vm_page_prot))
  Mike> {
  Mike>     return -EAGAIN;
  Mike> }

This is basically what drivers/char/mem.c is doing, so there is
nothing obviously wrong here.

	--david

RE: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin

[Mike Fox]

Hey all, I got it working! I was porting this code from a 32 bit version,
and it was as follows:

  /*
   * Accessing memory above the top the kernel knows about or
   * through a file pointer that was marked O_SYNC will be
   * done non-cached.
   *
   * Set VM_IO, as this is likely a non-cached access to an
   * I/O area, and we don't want to include that in a core
   * file.
   */
  if (offset >= __pa(high_memory) || (file->f_flags & O_SYNC))
  {
     vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
     vma->vm_flags |= VM_IO;
  }

Turns out my pgprot_noncached was not getting called, so I took it out of
the conditional. Thanks for all of your help.

I just started working at my current company 3 weeks ago, but it looks like
we will be doing a lot of work in the Linux environment on 32 and 64 bit
architectures.

I went ahead and bought the book, "ia-64 linux kernel", by David Mosberger
and Stephane Eranian. My background is in EE, and I never got a chance to
take an OS class, but I figure this book is my chance.

Thanks again,
Mike Fox.


-----Original Message-----
From: David Mosberger [mailto:davidm@napali.hpl.hp.com] 
Sent: Wednesday, April 09, 2003 1:46 PM
To: mfox@annapmicro.com
Cc: linux-ia64@linuxia64.org
Subject: RE: [Linux-ia64] mmap crashes system after mapping PCI device into
user space and reading and writing.

>>>>> On Wed, 9 Apr 2003 01:36:33 -0400, "Mike Fox" <mfox@annapmicro.com>
said:

  Mike> In the driver I am basically doing this in my mmap entry point:

  Mike>   vma-> vm_page_prot = pgprot_noncached(vma->vm_page_prot);
  Mike>   vma-> vm_flags |= VM_IO;

  Mike> /*
  Mike> * this maps the physical memory to the user
  Mike> */
  Mike> if (remap_page_range(vma->vm_start, offset,
vma->vm_end-vma->vm_start, vma->vm_page_prot))
  Mike> {
  Mike>     return -EAGAIN;
  Mike> }

This is basically what drivers/char/mem.c is doing, so there is
nothing obviously wrong here.

	--david

RE: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin

[David Mosberger]

>>>>> On Wed, 9 Apr 2003 19:53:38 -0400, "Mike Fox" <mfox@annapmicro.com> said:

  Mike> Turns out my pgprot_noncached was not getting called, so I
  Mike> took it out of the conditional. Thanks for all of your help.

Ah, that explains it.

  Mike> I went ahead and bought the book, "ia-64 linux kernel", by
  Mike> David Mosberger and Stephane Eranian.

Great choice! ;-))

Seriously though: an errata, kernel change log, and a searchable index
are available at:

  http://www.lia64.org/book/

There were some stupid typos that snuck by, so the errata is
especially important.

	--david