From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Mike Fox" Date: Tue, 08 Apr 2003 21:59:56 +0000 Subject: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writing. MIME-Version: 1 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0020_01C2FDF8.AA4069A0" Message-Id: List-Id: To: linux-ia64@vger.kernel.org This is a multi-part message in MIME format. ------=_NextPart_000_0020_01C2FDF8.AA4069A0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, =20 I am running kernel version 2.4.19 and I am getting a system crash when = I mmap my PCI device into user space and access it. =20 It seems to be the same issue that was posted earlier in the following = post, which had no response: =20 Has anyone tried using mmap to remap PCI device memory into user space? = I have seen the following problem: If I do the mmap, it returns = successfully, and I can indeed access the device memory. Unfortunately, the system = will hang within about a minute after doing so. I do not get an oops, or = crash of any kind, simply no further response form the machine. If I just do = the mmap and then munmap the system will not hang. But if I do one access = to the mmapped address, the system will hang. I am using the = remap_page_range call in my mmap routine, and this same code works on x86 and Alpha platforms. I found it odd that it takes a while for the system to hang after the access, is their something special I have to do for this to = work on IA64? I am running 2.4.1 with the 1/31 patch, although this problem occurs on all previous kernels I have tried. Thanks in advance for any insight anyone might have. =20 Thanks for any help you can give. I'm new to Linux and IA64 and I am in = a rush to port a PCI driver and API. =20 Thanks, Mike Fox. =20 =20 ------=_NextPart_000_0020_01C2FDF8.AA4069A0 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,

I am running kernel version 2.4.19 and I am getting a = system crash when I mmap my PCI device into user space and access = it.

It seems to be the same issue that was posted earlier = in the following post, which had no response:

Has anyone tried using mmap to remap PCI =
device memory into user space?  I

have seen =
the following problem:  If I do the mmap, it returns =
successfully,

and I can =
indeed access the device memory.  Unfortunately, the system =
will

hang =
within about a minute after doing so.  I do not get an oops, or =
crash

of any =
kind, simply no further response form the machine.  If I just do =
the

mmap and =
then munmap the system will not hang.  But if I do one access =
to

the =
mmapped address, the system will hang.  I am using the =
remap_page_range

call in =
my mmap routine, and this same code works on x86 and =
Alpha

platforms.  I found it odd that it takes =
a while for the system to hang

after the =
access, is their something special I have to do for this to =
work

on =
IA64?  I am running 2.4.1 with the 1/31 patch, although this =
problem

occurs on =
all previous kernels I have tried.  Thanks in advance for =
any

insight =
anyone might have.

Thanks for any help you can give. I’m new to = Linux and IA64 and I am in a rush to port a PCI driver and API.

Thanks,

Mike Fox.

------=_NextPart_000_0020_01C2FDF8.AA4069A0-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: grundler@cup.hp.com (Grant Grundler) Date: Wed, 09 Apr 2003 00:42:59 +0000 Subject: Re: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin Message-Id: List-Id: References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: linux-ia64@vger.kernel.org On Tue, Apr 08, 2003 at 05:59:56PM -0400, Mike Fox wrote: ... > Has anyone tried using mmap to remap PCI device memory into user space? I > have seen the following problem: If I do the mmap, it returns successfully, > and I can indeed access the device memory. Unfortunately, the system will > hang within about a minute after doing so. ... David Mosberger passed on a very short program who's essence is: mem = mmap (NULL, page_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, offset); if (mem = MAP_FAILED) { perror ("memmap"); exit (-1); } This works on the HP ZX1 platforms with 2.4.20+patches kernel. Be warned, it's *very* easy to crash a machine when using /dev/mem and scribbling in misc parts of IO space. The symptom will typically be an MCA (see "errdump MCA" output from EFI to get the xip). grant From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Mike Fox" Date: Wed, 09 Apr 2003 05:36:33 +0000 Subject: RE: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin Message-Id: List-Id: References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: linux-ia64@vger.kernel.org Ok I have some more info. The crash only happens after I do a write to my PCI controller. I map the PCI controller into user space and I can read and write a scratch register ok. I can also read other initialized registers to confirm I am reading my PCI controller. I can do many reads, but after some number of writes the system goes down. I went ahead and updated the kernel to 2.4.20 with the latest patches. Any thoughts or a push in the right direction are very much appreciated. My mmap call in user space looks something like this: pRegRegionMappedAddr = (DWORD*)mmap( 0, WSII_REG_SPACE_SIZE_BYTES, PROT_WRITE | PROT_READ, MAP_SHARED, DriverHandle, 0 ); In the driver I am basically doing this in my mmap entry point: vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); vma->vm_flags |= VM_IO; /* * this maps the physical memory to the user */ if (remap_page_range(vma->vm_start, offset, vma->vm_end-vma->vm_start, vma->vm_page_prot)) { return -EAGAIN; } Thanks, Mike. -----Original Message----- From: linux-ia64-admin@linuxia64.org [mailto:linux-ia64-admin@linuxia64.org] On Behalf Of Grant Grundler Sent: Tuesday, April 08, 2003 8:43 PM To: Mike Fox Cc: linux-ia64@linuxia64.org Subject: Re: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writing. On Tue, Apr 08, 2003 at 05:59:56PM -0400, Mike Fox wrote: ... > Has anyone tried using mmap to remap PCI device memory into user space? I > have seen the following problem: If I do the mmap, it returns successfully, > and I can indeed access the device memory. Unfortunately, the system will > hang within about a minute after doing so. ... David Mosberger passed on a very short program who's essence is: mem = mmap (NULL, page_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, offset); if (mem == MAP_FAILED) { perror ("memmap"); exit (-1); } This works on the HP ZX1 platforms with 2.4.20+patches kernel. Be warned, it's *very* easy to crash a machine when using /dev/mem and scribbling in misc parts of IO space. The symptom will typically be an MCA (see "errdump MCA" output from EFI to get the xip). grant _______________________________________________ Linux-IA64 mailing list Linux-IA64@linuxia64.org http://lists.linuxia64.org/lists/listinfo/linux-ia64 From mboxrd@z Thu Jan 1 00:00:00 1970 From: grundler@cup.hp.com (Grant Grundler) Date: Wed, 09 Apr 2003 06:20:46 +0000 Subject: Re: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin Message-Id: List-Id: References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: linux-ia64@vger.kernel.org On Wed, Apr 09, 2003 at 01:36:33AM -0400, Mike Fox wrote: > /* > * this maps the physical memory to the user > */ > if (remap_page_range(vma->vm_start, offset, vma->vm_end-vma->vm_start, > vma->vm_page_prot)) > { > return -EAGAIN; > } The example code I posted (a) used physical addresses and (b) assume the "user" was root (to access /dev/mem/). XFree86 pokes around in PCI MMIO and IO space and does work. Have you tried to dig through the source of XF86 4.2? Have you checked the return address (to user space) is uncached? If I understood details of VM it might be obvious from the code you posted. hth, grant > > Thanks, > Mike. > > > > -----Original Message----- > From: linux-ia64-admin@linuxia64.org [mailto:linux-ia64-admin@linuxia64.org] > On Behalf Of Grant Grundler > Sent: Tuesday, April 08, 2003 8:43 PM > To: Mike Fox > Cc: linux-ia64@linuxia64.org > Subject: Re: [Linux-ia64] mmap crashes system after mapping PCI device into > user space and reading and writing. > > On Tue, Apr 08, 2003 at 05:59:56PM -0400, Mike Fox wrote: > ... > > Has anyone tried using mmap to remap PCI device memory into user space? I > > have seen the following problem: If I do the mmap, it returns > successfully, > > and I can indeed access the device memory. Unfortunately, the system will > > hang within about a minute after doing so. > ... > > David Mosberger passed on a very short program who's essence is: > mem = mmap (NULL, page_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, > offset); > if (mem = MAP_FAILED) > { > perror ("memmap"); > exit (-1); > } > > This works on the HP ZX1 platforms with 2.4.20+patches kernel. > > Be warned, it's *very* easy to crash a machine when using /dev/mem > and scribbling in misc parts of IO space. The symptom will typically > be an MCA (see "errdump MCA" output from EFI to get the xip). > > grant > > _______________________________________________ > Linux-IA64 mailing list > Linux-IA64@linuxia64.org > http://lists.linuxia64.org/lists/listinfo/linux-ia64 > > > _______________________________________________ > Linux-IA64 mailing list > Linux-IA64@linuxia64.org > http://lists.linuxia64.org/lists/listinfo/linux-ia64 -- Grant Grundler Linux Open Systems Lab/BCST grundler@cup.hp.com +1.408.447.7253 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christian Hinkelbein Date: Wed, 09 Apr 2003 08:57:12 +0000 Subject: Re: [Linux-ia64] mmap crashes system after mapping PCI device into user Message-Id: List-Id: References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: linux-ia64@vger.kernel.org Hi, Mike Fox schrieb: > > > In the driver I am basically doing this in my mmap entry point: > > vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); > vma->vm_flags |= VM_IO; i know this is ugly stuff, but ... maybe you want to try this instead of the above: #if defined(__ia64__) vmap->vm_flags |= (VM_SHM | VM_LOCKED | VM_IO); vmap->vm_flags &= ~(VM_SHARED | VM_MAYSHARE); vmap->vm_page_prot = __pgprot(pgprot_val(vmap->vm_page_prot)& ~_PAGE_AR_RW); #endif > > Thanks, > Mike. > christian From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Mosberger Date: Wed, 09 Apr 2003 17:33:59 +0000 Subject: Re: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin Message-Id: List-Id: References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: linux-ia64@vger.kernel.org >>>>> On Tue, 8 Apr 2003 17:59:56 -0400, "Mike Fox" said: Mike> Hi, I am running kernel version 2.4.19 and I am getting a Mike> system crash when I mmap my PCI device into user space and Mike> access it. One thing you need to be careful about is memory attribute aliasing. The CPU (or platform chips) can MCA if you try to access the same physical location with different memory attributes (e.g., cached vs. uncached). Also, keep in mind that the kernel uses a 64MB page size for the identity-mapped regions (regions 6 and 7). This means that you cannot map stuff requiring different attributes within the same 64MB page. I kind of doubt this is what's causing the problem for you, but since you didn't give any details about the hw you're trying to map, it's hard to be more specific. As others have pointed out, it's of course also important to verify that you access I/O space uncached. --david From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Mosberger Date: Wed, 09 Apr 2003 17:45:45 +0000 Subject: RE: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin Message-Id: List-Id: References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: linux-ia64@vger.kernel.org >>>>> On Wed, 9 Apr 2003 01:36:33 -0400, "Mike Fox" said: Mike> In the driver I am basically doing this in my mmap entry point: Mike> vma-> vm_page_prot = pgprot_noncached(vma->vm_page_prot); Mike> vma-> vm_flags |= VM_IO; Mike> /* Mike> * this maps the physical memory to the user Mike> */ Mike> if (remap_page_range(vma->vm_start, offset, vma->vm_end-vma->vm_start, vma->vm_page_prot)) Mike> { Mike> return -EAGAIN; Mike> } This is basically what drivers/char/mem.c is doing, so there is nothing obviously wrong here. --david From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Mike Fox" Date: Wed, 09 Apr 2003 23:53:38 +0000 Subject: RE: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin Message-Id: List-Id: References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: linux-ia64@vger.kernel.org Hey all, I got it working! I was porting this code from a 32 bit version, and it was as follows: /* * Accessing memory above the top the kernel knows about or * through a file pointer that was marked O_SYNC will be * done non-cached. * * Set VM_IO, as this is likely a non-cached access to an * I/O area, and we don't want to include that in a core * file. */ if (offset >= __pa(high_memory) || (file->f_flags & O_SYNC)) { vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); vma->vm_flags |= VM_IO; } Turns out my pgprot_noncached was not getting called, so I took it out of the conditional. Thanks for all of your help. I just started working at my current company 3 weeks ago, but it looks like we will be doing a lot of work in the Linux environment on 32 and 64 bit architectures. I went ahead and bought the book, "ia-64 linux kernel", by David Mosberger and Stephane Eranian. My background is in EE, and I never got a chance to take an OS class, but I figure this book is my chance. Thanks again, Mike Fox. -----Original Message----- From: David Mosberger [mailto:davidm@napali.hpl.hp.com] Sent: Wednesday, April 09, 2003 1:46 PM To: mfox@annapmicro.com Cc: linux-ia64@linuxia64.org Subject: RE: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writing. >>>>> On Wed, 9 Apr 2003 01:36:33 -0400, "Mike Fox" said: Mike> In the driver I am basically doing this in my mmap entry point: Mike> vma-> vm_page_prot = pgprot_noncached(vma->vm_page_prot); Mike> vma-> vm_flags |= VM_IO; Mike> /* Mike> * this maps the physical memory to the user Mike> */ Mike> if (remap_page_range(vma->vm_start, offset, vma->vm_end-vma->vm_start, vma->vm_page_prot)) Mike> { Mike> return -EAGAIN; Mike> } This is basically what drivers/char/mem.c is doing, so there is nothing obviously wrong here. --david From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Mosberger Date: Thu, 10 Apr 2003 00:44:19 +0000 Subject: RE: [Linux-ia64] mmap crashes system after mapping PCI device into user space and reading and writin Message-Id: List-Id: References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: linux-ia64@vger.kernel.org >>>>> On Wed, 9 Apr 2003 19:53:38 -0400, "Mike Fox" said: Mike> Turns out my pgprot_noncached was not getting called, so I Mike> took it out of the conditional. Thanks for all of your help. Ah, that explains it. Mike> I went ahead and bought the book, "ia-64 linux kernel", by Mike> David Mosberger and Stephane Eranian. Great choice! ;-)) Seriously though: an errata, kernel change log, and a searchable index are available at: http://www.lia64.org/book/ There were some stupid typos that snuck by, so the errata is especially important. --david