RFC: Reclaim address space on omaps, Tony on vacation in July

RFC: Reclaim address space on omaps, Tony on vacation in July

[Tony Lindgren]

Hi all,

Last week I posted some more omap io.h clean-up patches to the linux-omap
list [1], and started looking at what it would take to reclaim lost address
space on top of the io.h clean-up patches.

Looks like we should be able to reclaim about 454MB of the 640MB
of the lost IO address space by doing the following:

1. Start converting drivers using omap_read/write to use ioremap +
   __raw_read/write

2. Remove OMAP2_IO_ADDRESS macro, and replace it with with finer grained
   OMAP_L3_ADDRESS, OMAP_L4_ADDRESS, OMAP_SDRC_ADDRESS and so on macros

3. Remap the the IO addresses in smaller sections in io.h so the virtual
   address for the blocks stays the same, and is counted down from the
   ARM reserved DMA address.

After these steps, omap io.h would look like this:

#define ARM_RESERVED_FUTURE_DMA	0xff000000 /* Documentation/arm/memory.txt */
#define OMAP23_L4_PHYS		0x48000000
#define OMAP24_L3_PHYS		0x68000000
#define OMAP4_L3_PHYS		0x44000000
#define OMAP4_L4_PHYS		0x4a000000

/* These virtual addresses are the same for all omaps */
#define OMAP_L3_SIZE		SZ_1M
#define OMAP_L3_VIRT		(ARM_RESERVED_FUTURE_DMA - OMAP_L3_SIZE)
#define OMAP_L4_SIZE		(SZ_16M + SZ_1M)
#define OMAP_L4_VIRT		(OMAP_L3_VIRT - OMAP_L4_SIZE)
#define OMAP_SDRC_SIZE		SZ_1M
#define OMAP_SDRC_BASE		(OMAP_L4_VIRT - OMAP_SDRC_SIZE)
...


#define OMAP_L3_ADDRESS(pa)	(((pa) & ~0xff000000) + OMAP_L3_VIRT)
#define OMAP_L4_ADDRESS(pa)	(((pa) & ~0xff000000) + OMAP_L4_VIRT)
#define OMAP_SDRC_ADDRESS(pa)	(((pa) & ~0xff000000) + OMAP_SDRC_VIRT)
...

Anybody got better ideas? In general, while thinking about this, we
should all start working on step 1 above where possible no matter what.

Also, I will be mostly offline in July. Meanwhile, please try to make all
the patches against the mainline kernel where possible to avoid cross
dependencies. Some code of course depends on some topic branches like pm
or dss2. Or the stuff in omap-headers branch for the omap headers.

Cheers,

Tony

[1] http://www.mail-archive.com/linux-omap@vger.kernel.org/msg13874.html

RE: Reclaim address space on omaps, Tony on vacation in July

[Shilimkar, Santosh]

> -----Original Message-----
> From: linux-omap-owner@vger.kernel.org 
> [mailto:linux-omap-owner@vger.kernel.org] On Behalf Of Tony Lindgren
> Sent: Friday, June 26, 2009 11:56 AM
> To: linux-omap@vger.kernel.org
> Cc: linux-arm-kernel@lists.arm.linux.org.uk
> Subject: RFC: Reclaim address space on omaps, Tony on vacation in July
> 
> Hi all,
> 
> Last week I posted some more omap io.h clean-up patches to 
> the linux-omap
> list [1], and started looking at what it would take to 
> reclaim lost address
> space on top of the io.h clean-up patches.
> 
> Looks like we should be able to reclaim about 454MB of the 640MB
> of the lost IO address space by doing the following:
> 
> 1. Start converting drivers using omap_read/write to use ioremap +
>    __raw_read/write
> 
> 2. Remove OMAP2_IO_ADDRESS macro, and replace it with with 
> finer grained
>    OMAP_L3_ADDRESS, OMAP_L4_ADDRESS, OMAP_SDRC_ADDRESS and so 
> on macros

> 3. Remap the the IO addresses in smaller sections in io.h so 
> the virtual
>    address for the blocks stays the same, and is counted down from the
>    ARM reserved DMA address.
> 
> After these steps, omap io.h would look like this:
> 
> #define ARM_RESERVED_FUTURE_DMA	0xff000000 /* 
> Documentation/arm/memory.txt */
> #define OMAP23_L4_PHYS		0x48000000
> #define OMAP24_L3_PHYS		0x68000000
> #define OMAP4_L3_PHYS		0x44000000
> #define OMAP4_L4_PHYS		0x4a000000
> 
> /* These virtual addresses are the same for all omaps */
> #define OMAP_L3_SIZE		SZ_1M
> #define OMAP_L3_VIRT		(ARM_RESERVED_FUTURE_DMA - OMAP_L3_SIZE)
> #define OMAP_L4_SIZE		(SZ_16M + SZ_1M)
> #define OMAP_L4_VIRT		(OMAP_L3_VIRT - OMAP_L4_SIZE)
> #define OMAP_SDRC_SIZE		SZ_1M
> #define OMAP_SDRC_BASE		(OMAP_L4_VIRT - OMAP_SDRC_SIZE)
> ...
> 
> 
> #define OMAP_L3_ADDRESS(pa)	(((pa) & ~0xff000000) + OMAP_L3_VIRT)
> #define OMAP_L4_ADDRESS(pa)	(((pa) & ~0xff000000) + OMAP_L4_VIRT)
> #define OMAP_SDRC_ADDRESS(pa)	(((pa) & ~0xff000000) + OMAP_SDRC_VIRT)
We surely need this going forward since there is also demand to fit higher 
RAMS(256MB,512MB,1GB)in the system. 

Just to confirm, any further patches on this topic should be on top of this:
Tony Lindgren (3):
      OMAP: Remove ifdefs for io.h
      OMAP: Rename OMAP_MPUIO_BASE to OMAP1_MPUIO_BASE
      OMAP: Remove OMAP_IO_ADDRESS, use OMAP1_IO_ADDRESS and OMAP2_IO_ADDRESS instead

Is that correct Tony ?

Regards,
Santosh
 

Re: Reclaim address space on omaps, Tony on vacation in July

[Tony Lindgren]

* Shilimkar, Santosh <santosh.shilimkar@ti.com> [090626 10:45]:
> > -----Original Message-----
> > From: linux-omap-owner@vger.kernel.org 
> > [mailto:linux-omap-owner@vger.kernel.org] On Behalf Of Tony Lindgren
> > Sent: Friday, June 26, 2009 11:56 AM
> > To: linux-omap@vger.kernel.org
> > Cc: linux-arm-kernel@lists.arm.linux.org.uk
> > Subject: RFC: Reclaim address space on omaps, Tony on vacation in July
> > 
> > Hi all,
> > 
> > Last week I posted some more omap io.h clean-up patches to 
> > the linux-omap
> > list [1], and started looking at what it would take to 
> > reclaim lost address
> > space on top of the io.h clean-up patches.
> > 
> > Looks like we should be able to reclaim about 454MB of the 640MB
> > of the lost IO address space by doing the following:
> > 
> > 1. Start converting drivers using omap_read/write to use ioremap +
> >    __raw_read/write
> > 
> > 2. Remove OMAP2_IO_ADDRESS macro, and replace it with with 
> > finer grained
> >    OMAP_L3_ADDRESS, OMAP_L4_ADDRESS, OMAP_SDRC_ADDRESS and so 
> > on macros
> 
> > 3. Remap the the IO addresses in smaller sections in io.h so 
> > the virtual
> >    address for the blocks stays the same, and is counted down from the
> >    ARM reserved DMA address.
> > 
> > After these steps, omap io.h would look like this:
> > 
> > #define ARM_RESERVED_FUTURE_DMA	0xff000000 /* 
> > Documentation/arm/memory.txt */
> > #define OMAP23_L4_PHYS		0x48000000
> > #define OMAP24_L3_PHYS		0x68000000
> > #define OMAP4_L3_PHYS		0x44000000
> > #define OMAP4_L4_PHYS		0x4a000000
> > 
> > /* These virtual addresses are the same for all omaps */
> > #define OMAP_L3_SIZE		SZ_1M
> > #define OMAP_L3_VIRT		(ARM_RESERVED_FUTURE_DMA - OMAP_L3_SIZE)
> > #define OMAP_L4_SIZE		(SZ_16M + SZ_1M)
> > #define OMAP_L4_VIRT		(OMAP_L3_VIRT - OMAP_L4_SIZE)
> > #define OMAP_SDRC_SIZE		SZ_1M
> > #define OMAP_SDRC_BASE		(OMAP_L4_VIRT - OMAP_SDRC_SIZE)
> > ...
> > 
> > 
> > #define OMAP_L3_ADDRESS(pa)	(((pa) & ~0xff000000) + OMAP_L3_VIRT)
> > #define OMAP_L4_ADDRESS(pa)	(((pa) & ~0xff000000) + OMAP_L4_VIRT)
> > #define OMAP_SDRC_ADDRESS(pa)	(((pa) & ~0xff000000) + OMAP_SDRC_VIRT)
> We surely need this going forward since there is also demand to fit higher 
> RAMS(256MB,512MB,1GB)in the system. 
> 
> Just to confirm, any further patches on this topic should be on top of this:
> Tony Lindgren (3):
>       OMAP: Remove ifdefs for io.h
>       OMAP: Rename OMAP_MPUIO_BASE to OMAP1_MPUIO_BASE
>       OMAP: Remove OMAP_IO_ADDRESS, use OMAP1_IO_ADDRESS and OMAP2_IO_ADDRESS instead
> 
> Is that correct Tony ?

Yes it makes it easier as the patches above remove all the ifdefs for io.h.

Tony

RE: Reclaim address space on omaps, Tony on vacation in July

[Woodruff, Richard]

> From: linux-omap-owner@vger.kernel.org [mailto:linux-omap-
> owner@vger.kernel.org] On Behalf Of Tony Lindgren

> > > Last week I posted some more omap io.h clean-up patches to
> > > the linux-omap
> > > list [1], and started looking at what it would take to
> > > reclaim lost address
> > > space on top of the io.h clean-up patches.
> > >
> > > Looks like we should be able to reclaim about 454MB of the 640MB
> > > of the lost IO address space by doing the following:

Using section entries to cover ranges provides superior performance.  1 TLB will cover a whole series of accesses when doing things like servicing an interrupt.  Using fewer TLBs to cover system accesses leaves more in tact for application usage.  I've not see recent data but long back TLB optimization provided sizeable performance boost for many applications.

Additionally debug is simplified by having the simple static map around.

Russell recently pointed out and David showed an example of how ioremap() can take an arch specific function which can return virtual address from already defined static ranges.  I do think this is a nice clean up and optimization (to stop some double mapping which now happens with ioremap).

http://www.mail-archive.com/linux-omap@vger.kernel.org/msg03813.html

So, I think yes moving to ioremap in all drivers is good, but it would use underlying static maps where they exist.

It might be not _all_ ranges need to be fully static mapped, but the common device ones should be.

On this thread there has been a mounting request for more physical memory.  Starting to make a 2x2gig split option and perhaps understanding more how to use extended addressing like xscale is doing would be good road map stuff.

If io.h is getting messy maybe it should be broken in to a couple files... Anyway, driver cleanup using common ioremap would be the big winner.

Regards,
Richard W.

Re: Reclaim address space on omaps, Tony on vacation in July

[Kevin Hilman]

"Woodruff, Richard" <r-woodruff2@ti.com> writes:

>> From: linux-omap-owner@vger.kernel.org [mailto:linux-omap-
>> owner@vger.kernel.org] On Behalf Of Tony Lindgren
>
>> > > Last week I posted some more omap io.h clean-up patches to
>> > > the linux-omap
>> > > list [1], and started looking at what it would take to
>> > > reclaim lost address
>> > > space on top of the io.h clean-up patches.
>> > >
>> > > Looks like we should be able to reclaim about 454MB of the 640MB
>> > > of the lost IO address space by doing the following:
>
> Using section entries to cover ranges provides superior performance.  1 TLB will cover a whole series of accesses when doing things like servicing an interrupt.  Using fewer TLBs to cover system accesses leaves more in tact for application usage.  I've not see recent data but long back TLB optimization provided sizeable performance boost for many applications.
>
> Additionally debug is simplified by having the simple static map around.
>
> Russell recently pointed out and David showed an example of how ioremap() can take an arch specific function which can return virtual address from already defined static ranges.  I do think this is a nice clean up and optimization (to stop some double mapping which now happens with ioremap).
>
> http://www.mail-archive.com/linux-omap@vger.kernel.org/msg03813.html
>
> So, I think yes moving to ioremap in all drivers is good, but it would use underlying static maps where they exist.

Current linux-omap has worked like this for a while now.  IOW,
ioremap() will first check any static mappings before doing a real
remap.

So, without any changes to the memory map, everything could be converted
to ioremap + read/write already.

Kevin

Re: RFC: Reclaim address space on omaps, Tony on vacation in July

[Ben Dooks]

On Fri, Jun 26, 2009 at 09:25:40AM +0300, Tony Lindgren wrote:
> Hi all,
> 
> Last week I posted some more omap io.h clean-up patches to the linux-omap
> list [1], and started looking at what it would take to reclaim lost address
> space on top of the io.h clean-up patches.
> 
> Looks like we should be able to reclaim about 454MB of the 640MB
> of the lost IO address space by doing the following:
> 
> 1. Start converting drivers using omap_read/write to use ioremap +
>    __raw_read/write

ioremap implies readl/writel, the __raw stuff is for items that have been
mapped by iodesc (iirc).

-- 
Ben

Q:      What's a light-year?
A:      One-third less calories than a regular year.

Re: RFC: Reclaim address space on omaps, Tony on vacation in July

[Tony Lindgren]

* Ben Dooks <ben-linux@fluff.org> [090626 11:37]:
> On Fri, Jun 26, 2009 at 09:25:40AM +0300, Tony Lindgren wrote:
> > Hi all,
> > 
> > Last week I posted some more omap io.h clean-up patches to the linux-omap
> > list [1], and started looking at what it would take to reclaim lost address
> > space on top of the io.h clean-up patches.
> > 
> > Looks like we should be able to reclaim about 454MB of the 640MB
> > of the lost IO address space by doing the following:
> > 
> > 1. Start converting drivers using omap_read/write to use ioremap +
> >    __raw_read/write
> 
> ioremap implies readl/writel, the __raw stuff is for items that have been
> mapped by iodesc (iirc).

Thanks, will take a look at the iodesc.

Tony

Re: RFC: Reclaim address space on omaps, Tony on vacation in July

[Tony Lindgren]

* Tony Lindgren <tony@atomide.com> [090626 11:58]:
> * Ben Dooks <ben-linux@fluff.org> [090626 11:37]:
> > On Fri, Jun 26, 2009 at 09:25:40AM +0300, Tony Lindgren wrote:
> > > Hi all,
> > > 
> > > Last week I posted some more omap io.h clean-up patches to the linux-omap
> > > list [1], and started looking at what it would take to reclaim lost address
> > > space on top of the io.h clean-up patches.
> > > 
> > > Looks like we should be able to reclaim about 454MB of the 640MB
> > > of the lost IO address space by doing the following:
> > > 
> > > 1. Start converting drivers using omap_read/write to use ioremap +
> > >    __raw_read/write
> > 
> > ioremap implies readl/writel, the __raw stuff is for items that have been
> > mapped by iodesc (iirc).
> 
> Thanks, will take a look at the iodesc.

Hmm, OK so we should just use readl/writel. No need to map everything
statically with struct map_desc.

Tony