mmap64/open64/pread etc on 440GX

mmap64/open64/pread etc on 440GX

[Neil Wilson]

Hi,

I am trying to write a command line utility for our dev board
(440GX,2.6.9 kernel) in order to provide support for our hardware
engineers.  As a test I am trying to dump the first few bytes of the
U-boot header.

I though that this would work but using /dev/mem gives me a bad address
error on the pread64, using /dev/kmem only reads back 0 bytes.  Is there
something blindingly obvious that I am missing for using 36bit addresses
? I believe the address is correct as I can see the boot code from
within the kernel. Thanks.

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>

unsigned char buf[10240];

int main(int argc,char **argv)
{
	unsigned long long memAddr=3D0x00000001fff80000ULL;
	int x;
	int fd;
	int readSize;
	int pageSize;
=09
	pageSize=3Dsysconf(_SC_PAGESIZE);
=09
	fd=3Dopen64("/dev/mem", O_RDWR);
	if(fd=3D=3D-1)
	{
		perror("open64");
		return 0;
	}
=09
	readSize=3Dpread64(fd,(void*)buf,pageSize,memAddr);
=09
	if(readSize !=3D pageSize)
	{
		perror("pread64 ");
		printf("only read %d of %d requested bytes\n", readSize,
pageSize);
	}
=09
	close(fd);
=09
	// show some of it
	if(readSize>0)
	{
		for(x=3D0;x<25;x++)
		{
			printf("%02x(%c)
",buf[x],isprint(buf[x])?buf[x]:'.');
		}
		printf("\n");
	}
}

Neil

--
Neil Wilson
Airspan Communications Ltd.
Cambridge House, Oxford Road,
Uxbridge, Middx, UB8 1UN, UK.
Tel: +44(0)1895-467265

neilwilson@airspan.com
www.airspan.com

Re: mmap64/open64/pread etc on 440GX

[Matt Porter]

On Wed, Nov 10, 2004 at 01:58:29PM -0000, Neil Wilson wrote:
> Hi,
> 
> I am trying to write a command line utility for our dev board
> (440GX,2.6.9 kernel) in order to provide support for our hardware
> engineers.  As a test I am trying to dump the first few bytes of the
> U-boot header.
> 
> I though that this would work but using /dev/mem gives me a bad address
> error on the pread64, using /dev/kmem only reads back 0 bytes.  Is there
> something blindingly obvious that I am missing for using 36bit addresses
> ? I believe the address is correct as I can see the boot code from
> within the kernel. Thanks.

No, it's not blindingly obvious at all.  First, mmaping /dev/mem is
a completely non-portable way to get at devices in userspace. It
has no guarantees of working across architectures and is broken
in many ways.

One way is on PPC440 where we have a 36-bit physical address space.
The /dev/mem implementation simply allows you to mmap an offset
within a 32-bit physical address space. On PPC440, this means you can't
get to any of the useful devices since they are all above 4GB.
In post 2.6.9, remap_pfn_range() helps a bit but there is still a
local (arch/ppc/) problem...

I'm working on some patches to fix 36-bit support for
io_remap_page_range() and to fix some problems with set_pte()
and friends on 36-bit platforms. This is the first bit necessary
to make this work at all in 2.6. I should be posting something
RSN. ;)

-Matt

RE: mmap64/open64/pread etc on 440GX

[Neil Wilson]

=20
> I'm working on some patches to fix 36-bit support for
> io_remap_page_range() and to fix some problems with set_pte()=20
> and friends on 36-bit platforms. This is the first bit=20
> necessary to make this work at all in 2.6. I should be=20
> posting something RSN. ;)
>=20
> -Matt

Okay, thanks for the info.

Guess I will have to put together a simple kernel driver so the hardware
boys can get 36bit access for now.

Neil

--
Neil Wilson
Airspan Communications Ltd.
Cambridge House, Oxford Road,
Uxbridge, Middx, UB8 1UN, UK.
Tel: +44(0)1895-467265

neilwilson@airspan.com
www.airspan.com

Re: mmap64/open64/pread etc on 440GX

[Mark Powell]

[-- Attachment #1: Type: text/plain, Size: 988 bytes --]

FWIW, the attached might help.

This is something I added to our 2.4.25 (10-May-04) tree to allow me do 
the same thing on our '440GP platform.
The patch adds a file to arch/ppc/mm that provides an 
io_remap_page_range64() function, analogous to ioremap64().
drv_mmap.c is the body of the mmap() entry point in my driver for an 
example.

Hope it is useful

-- 
Mark Powell, Senior Software Engineer
Primagraphics, Curtiss-Wright Controls Embedded Computing
Tel: +44 (0) 1763 852222
Email: medp@primagraphics.com


Neil Wilson wrote:

> 
>  
>
>>I'm working on some patches to fix 36-bit support for
>>io_remap_page_range() and to fix some problems with set_pte() 
>>and friends on 36-bit platforms. This is the first bit 
>>necessary to make this work at all in 2.6. I should be 
>>posting something RSN. ;)
>>
>>-Matt
>>    
>>
>
>Okay, thanks for the info.
>
>Guess I will have to put together a simple kernel driver so the hardware
>boys can get 36bit access for now.
>
>Neil
>
>  
>

[-- Attachment #2: mm.patch --]
[-- Type: text/plain, Size: 3369 bytes --]

diff -urN --exclude=CVS ../10-May-04/arch/ppc/mm/44x_remap.c ./arch/ppc/mm/44x_remap.c
--- ../10-May-04/arch/ppc/mm/44x_remap.c	Thu Jan  1 01:00:00 1970
+++ ./arch/ppc/mm/44x_remap.c	Thu Jun  3 16:54:44 2004
@@ -0,0 +1,102 @@
+/* 
+ * Implementation of remap_page_range() for IBM440GP where physical
+ * addresses are 36 bits.
+ * 
+ * This differs form ioremap64() in that it manipulates the
+ * current process's page tables.
+ * 
+ * $Id: 44x_remap.c,v 1.1 2004/06/03 15:54:44 medp Exp $
+ * $Log: 44x_remap.c,v $
+ * Revision 1.1  2004/06/03 15:54:44  medp
+ * Added io_remap_page_range64
+ *
+ */
+#include <linux/mm.h>
+
+
+static inline void forget_pte(pte_t page)
+{
+	if (!pte_none(page)) {
+		printk("forget_pte: old mapping existed!\n");
+		BUG();
+	}
+}
+
+/*
+ * maps a range of physical memory into the requested pages. the old
+ * mappings are removed. any references to nonexistent pages results
+ * in null mappings (currently treated as "copy-on-access")
+ */
+static inline void io_remap_pte_range64(pte_t * pte, unsigned long address, unsigned long size,
+	phys_addr_t phys_addr, pgprot_t prot)
+{
+	unsigned long end;
+
+	address &= ~PMD_MASK;
+	end = address + size;
+	if (end > PMD_SIZE)
+		end = PMD_SIZE;
+	do {
+		pte_t oldpage;
+		oldpage = ptep_get_and_clear(pte);
+
+ 		set_pte(pte, mk_pte_phys(phys_addr, prot));
+		forget_pte(oldpage);
+		address += PAGE_SIZE;
+		phys_addr += PAGE_SIZE;
+		pte++;
+	} while (address && (address < end));
+}
+
+static inline int io_remap_pmd_range64(struct mm_struct *mm, pmd_t * pmd, unsigned long address, unsigned long size,
+	phys_addr_t phys_addr, pgprot_t prot)
+{
+	unsigned long end;
+
+	address &= ~PGDIR_MASK;
+	end = address + size;
+	if (end > PGDIR_SIZE)
+		end = PGDIR_SIZE;
+	phys_addr -= address;
+	do {
+		pte_t * pte = pte_alloc(mm, pmd, address);
+		if (!pte)
+			return -ENOMEM;
+		io_remap_pte_range64(pte, address, end - address, address + phys_addr, prot);
+		address = (address + PMD_SIZE) & PMD_MASK;
+		pmd++;
+	} while (address && (address < end));
+	return 0;
+}
+
+/*  Note: this is only safe if the mm semaphore is held when called. */
+int io_remap_page_range64(unsigned long from, phys_addr_t phys_addr, unsigned long size, pgprot_t prot)
+{
+	int error = 0;
+	pgd_t * dir;
+	unsigned long beg = from;
+	unsigned long end = from + size;
+	struct mm_struct *mm = current->mm;
+
+	phys_addr -= from;
+	dir = pgd_offset(mm, from);
+	flush_cache_range(mm, beg, end);
+	if (from >= end)
+		BUG();
+
+	spin_lock(&mm->page_table_lock);
+	do {
+		pmd_t *pmd = pmd_alloc(mm, dir, from);
+		error = -ENOMEM;
+		if (!pmd)
+			break;
+		error = io_remap_pmd_range64(mm, pmd, from, end - from, phys_addr + from, prot);
+		if (error)
+			break;
+		from = (from + PGDIR_SIZE) & PGDIR_MASK;
+		dir++;
+	} while (from && (from < end));
+	spin_unlock(&mm->page_table_lock);
+	flush_tlb_range(mm, beg, end);
+	return error;
+}
diff -urN --exclude=CVS ../10-May-04/arch/ppc/mm/Makefile ./arch/ppc/mm/Makefile
--- ../10-May-04/arch/ppc/mm/Makefile	Tue Dec 16 20:12:55 2003
+++ ./arch/ppc/mm/Makefile	Thu Jun  3 16:54:44 2004
@@ -19,7 +19,7 @@
 
 obj-$(CONFIG_PPC_STD_MMU)	+= hashtable.o ppc_mmu.o tlb.o
 obj-$(CONFIG_40x)		+= 4xx_mmu.o
-obj-$(CONFIG_44x)		+= 44x_mmu.o
+obj-$(CONFIG_44x)		+= 44x_mmu.o 44x_remap.o
 obj-$(CONFIG_NOT_COHERENT_CACHE)	+= cachemap.o
 
 include $(TOPDIR)/Rules.make

[-- Attachment #3: drv_mmap.c --]
[-- Type: text/plain, Size: 1637 bytes --]

/*
 * - Function ----------------------------------------------------------------
 *
 *  sentric_fpga_mmap
 *
 *      This function maps the FPGA register block to user space.
 *
 *  Parameters:
 *      filep           Pointer to file struct for this device.
 *      vma             Pointer to the virtual memory area struct to map to.
 *
 *  Returns:
 *      0 success, -ve number is an error code.
 *
 *  Notes:
 *      None.
 * ---------------------------------------------------------------------------
 */
int
sentric_fpga_mmap(struct file *filep, struct vm_area_struct *vma)
{
    unsigned long off;
    phys_addr_t phys;
    unsigned long vsize, psize;

    /* Convert requested offset in units of pages to bytes */
    /* This is the offset from base of FPGA regs region, should always be 0 */
    off = vma->vm_pgoff << PAGE_SHIFT;

    /* physical address */
    phys = SENTRIC_FPGA_ADDR;

    /* Check that the user hasn't asked for more than is available */
    vsize = vma->vm_end - vma->vm_start;
    psize = PAGE_SIZE - off;
    if (vsize > psize) {
        printk("sentric_fpga:mmap: requested too much i/o space: %ld\n", vsize);
        return -EINVAL;
    }

    /* Use non-cached accesses as this is shared-memory */
    vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);

    /* Don't try to swap out physical pages.. */
    vma->vm_flags |= VM_RESERVED;

    /* Don't dump addresses that are not real memory to a core file. */
    vma->vm_flags |= VM_IO;

    if (io_remap_page_range64(vma->vm_start, phys, vsize, vma->vm_page_prot)) {
        return -EAGAIN;
    }

    return 0;
} /* sentric_fpga_mmap() */