SCU registers mapping for CA9/CA5 cores

SCU registers mapping for CA9/CA5 cores

[Chao Xie]

hi, Russell
When i sbumited the patch for pxa988 which is CA9, Arnd gave me some
suggestions of SCU mapping.
I used static mapping in the platform file for SCU registers mapping,
but Arnd think there should be a more general way to do it, and others
can benefit from it. I agree with him.

The following is his question for you when reply to my patch.
"Russell, do you think using a fixmap page for the SCU makes sense?
I'm looking for a method to consolidate the various methods of
doing early mappings of the SCU for device tree based platforms.
Where do you think that should be implemented?"

So can you give your idea and suggestion? Thanks.

SCU registers mapping for CA9/CA5 cores

[Russell King - ARM Linux]

On Fri, Jun 21, 2013 at 02:11:31PM +0800, Chao Xie wrote:
> hi, Russell
> When i sbumited the patch for pxa988 which is CA9, Arnd gave me some
> suggestions of SCU mapping.
> I used static mapping in the platform file for SCU registers mapping,
> but Arnd think there should be a more general way to do it, and others
> can benefit from it. I agree with him.
> 
> The following is his question for you when reply to my patch.
> "Russell, do you think using a fixmap page for the SCU makes sense?
> I'm looking for a method to consolidate the various methods of
> doing early mappings of the SCU for device tree based platforms.
> Where do you think that should be implemented?"
> 
> So can you give your idea and suggestion? Thanks.

With our current fixmap region, we have 224 pages available, currently
all dedicated to kmap.  kmap uses 16 pages per CPU at the moment, which
we hope is enough (it's only checked when CONFIG_DEBUG_HIGHMEM is set.)

Assuming four CPUs, that makes it 64 pages, but then we have big.LITTLE
with potentially 8 (I guess they map as separate smp_processor_id()'s)
which makes that 128.  128 is over half of the available fixmap.

If we ever saw 16 CPUs, we'd need to find a new home for fixmap.  If we
needed to increase KM_TYPE_NR beyond 28, we'd be in the same situation.

We do have the 0xffff1000 - 0xffff4000 range which is currently unused.
We could put the SCU at 0xffff1000 for a single page.  IOW, not a fixmap
page.

SCU registers mapping for CA9/CA5 cores

[Arnd Bergmann]

On Friday 21 June 2013, Russell King - ARM Linux wrote:
> On Fri, Jun 21, 2013 at 02:11:31PM +0800, Chao Xie wrote:
> > hi, Russell
> > When i sbumited the patch for pxa988 which is CA9, Arnd gave me some
> > suggestions of SCU mapping.
> > I used static mapping in the platform file for SCU registers mapping,
> > but Arnd think there should be a more general way to do it, and others
> > can benefit from it. I agree with him.
> > 
> > The following is his question for you when reply to my patch.
> > "Russell, do you think using a fixmap page for the SCU makes sense?
> > I'm looking for a method to consolidate the various methods of
> > doing early mappings of the SCU for device tree based platforms.
> > Where do you think that should be implemented?"
> > 
> > So can you give your idea and suggestion? Thanks.
> 
> With our current fixmap region, we have 224 pages available, currently
> all dedicated to kmap.  kmap uses 16 pages per CPU at the moment, which
> we hope is enough (it's only checked when CONFIG_DEBUG_HIGHMEM is set.)
> 
> Assuming four CPUs, that makes it 64 pages, but then we have big.LITTLE
> with potentially 8 (I guess they map as separate smp_processor_id()'s)
> which makes that 128.  128 is over half of the available fixmap.
> 
> If we ever saw 16 CPUs, we'd need to find a new home for fixmap.  If we
> needed to increase KM_TYPE_NR beyond 28, we'd be in the same situation.
> 
> We do have the 0xffff1000 - 0xffff4000 range which is currently unused.
> We could put the SCU at 0xffff1000 for a single page.  IOW, not a fixmap
> page.

I think we need to separate the specific discussion on the SCU page
from the question about how to use fixmap. Putting the SCU at 0xffff1000
sounds reasonable to me, so let's do that.

On the matter of fixmap, I'm missing what the requirement is to have
fixmap be a single section. Right now as you say, we have a bit under
half of the fixmap unused in the worst-case (8 CPU) scenario, so we
could put other things in the remaining space. If we get to 16 CPUs
(which I think is specified already but not implemented anywhere so
far) or beyond, we need to grow fixmap, but then there would be
free space again.

I think we could actually move everything between VMALLOC_END and
ffff0000 /into/ a larger fixmap. This would be a substantial change,
but it would give us more vmalloc space for most configurations
and more flexibility with what else we could put into fixmap.

Something like:

enum fixed_addresses {
#ifdef CONFIG_HIGHMEM
	FIX_KMAP_BEGIN,
	FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_TYPE_NR*NR_CPUS)-1,
#endif
#ifdef CONFIG_CPU_XSCALE
	XSCALE_CLEAN_BEGIN,
	XSCALE_CLEAN_END = XSCALE_CLEAN_START + 15,
#endif
#ifdef CONFIG_HAVE_TCM
	ITCM_BEGIN,
	ITCM_END = ITCM_SIZE / PAGE_SIZE - 1,
	DTCM_BEGIN,
	DTCM_END = DTCM_SIZE / PAGE_SIZE - 1,
#endif
#ifdef CONFIG_PCI
	PCI_IO_BEGIN,
	PCI_IO_END = PCI_IO_BEGIN + SZ_1M / PAGE_SIZE -1,
#endif
	__end_of_fixed_addresses,
};

Obviously we'd have to be careful to change all references to the
current well-known addresses to use the fixmap infrastructure
intead.

Any opinion on this? I haven't found any downsides yet, other than
having to do the work. I suppose we could even move the DEBUG_LL
mapping in there, but that would require some more rework.

	Arnd

SCU registers mapping for CA9/CA5 cores

[Chao Xie]

On Fri, Jun 21, 2013 at 5:29 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Friday 21 June 2013, Russell King - ARM Linux wrote:
>> On Fri, Jun 21, 2013 at 02:11:31PM +0800, Chao Xie wrote:
>> > hi, Russell
>> > When i sbumited the patch for pxa988 which is CA9, Arnd gave me some
>> > suggestions of SCU mapping.
>> > I used static mapping in the platform file for SCU registers mapping,
>> > but Arnd think there should be a more general way to do it, and others
>> > can benefit from it. I agree with him.
>> >
>> > The following is his question for you when reply to my patch.
>> > "Russell, do you think using a fixmap page for the SCU makes sense?
>> > I'm looking for a method to consolidate the various methods of
>> > doing early mappings of the SCU for device tree based platforms.
>> > Where do you think that should be implemented?"
>> >
>> > So can you give your idea and suggestion? Thanks.
>>
>> With our current fixmap region, we have 224 pages available, currently
>> all dedicated to kmap.  kmap uses 16 pages per CPU at the moment, which
>> we hope is enough (it's only checked when CONFIG_DEBUG_HIGHMEM is set.)
>>
>> Assuming four CPUs, that makes it 64 pages, but then we have big.LITTLE
>> with potentially 8 (I guess they map as separate smp_processor_id()'s)
>> which makes that 128.  128 is over half of the available fixmap.
>>
>> If we ever saw 16 CPUs, we'd need to find a new home for fixmap.  If we
>> needed to increase KM_TYPE_NR beyond 28, we'd be in the same situation.
>>
>> We do have the 0xffff1000 - 0xffff4000 range which is currently unused.
>> We could put the SCU at 0xffff1000 for a single page.  IOW, not a fixmap
>> page.
>
> I think we need to separate the specific discussion on the SCU page
> from the question about how to use fixmap. Putting the SCU at 0xffff1000
> sounds reasonable to me, so let's do that.
>

Where can i put the mapping code for SCU registers?
The mapping code will make use of some static APIs in
arch/arm/mm/mm.h, for example
set_top_pte(). These APIs can only be used by files under arch/arm/mm/
It seems that all the mm related operations will be located at
arch/arm/mm, and it is strange
if i copied the code of the functions in mm.h to SCU driver located at
arch/arm/kernel/ for the mapping.

> On the matter of fixmap, I'm missing what the requirement is to have
> fixmap be a single section. Right now as you say, we have a bit under
> half of the fixmap unused in the worst-case (8 CPU) scenario, so we
> could put other things in the remaining space. If we get to 16 CPUs
> (which I think is specified already but not implemented anywhere so
> far) or beyond, we need to grow fixmap, but then there would be
> free space again.
>
> I think we could actually move everything between VMALLOC_END and
> ffff0000 /into/ a larger fixmap. This would be a substantial change,
> but it would give us more vmalloc space for most configurations
> and more flexibility with what else we could put into fixmap.
>
> Something like:
>
> enum fixed_addresses {
> #ifdef CONFIG_HIGHMEM
>         FIX_KMAP_BEGIN,
>         FIX_KMAP_END = FIX_KMAP_BEGIN + (KM_TYPE_NR*NR_CPUS)-1,
> #endif
> #ifdef CONFIG_CPU_XSCALE
>         XSCALE_CLEAN_BEGIN,
>         XSCALE_CLEAN_END = XSCALE_CLEAN_START + 15,
> #endif
> #ifdef CONFIG_HAVE_TCM
>         ITCM_BEGIN,
>         ITCM_END = ITCM_SIZE / PAGE_SIZE - 1,
>         DTCM_BEGIN,
>         DTCM_END = DTCM_SIZE / PAGE_SIZE - 1,
> #endif
> #ifdef CONFIG_PCI
>         PCI_IO_BEGIN,
>         PCI_IO_END = PCI_IO_BEGIN + SZ_1M / PAGE_SIZE -1,
> #endif
>         __end_of_fixed_addresses,
> };
>
> Obviously we'd have to be careful to change all references to the
> current well-known addresses to use the fixmap infrastructure
> intead.
>
> Any opinion on this? I haven't found any downsides yet, other than
> having to do the work. I suppose we could even move the DEBUG_LL
> mapping in there, but that would require some more rework.
>
I think it is a good idea. Just need define the END_OF_FIXMAP address
to be 0xffff0000, and
all the fixmap can extend from high address to lower address.
Now, the 2nd level table is mapped, and the table address is stored in
top_pmd. If want to extend
the fixmap range, top_pmd need to be changed.

>         Arnd

SCU registers mapping for CA9/CA5 cores

[Russell King - ARM Linux]

On Fri, Jun 21, 2013 at 11:29:22AM +0200, Arnd Bergmann wrote:
> I think we could actually move everything between VMALLOC_END and
> ffff0000 /into/ a larger fixmap. This would be a substantial change,
> but it would give us more vmalloc space for most configurations
> and more flexibility with what else we could put into fixmap.

... at the expense of:
(a) having to look up the PTE table pointer every time we want to add or
remove a mapping,
(b) wasting the top 15 pages from being used.

Also, the placement of some of these pages _matters_ especially for the
VIPT aliasing caches, where cache colouring becomes important.  So an
enum in fixmap isn't going to work for those.

Really, the value to such a change is very low and is really just churn
for churn's sake.  "Oh we can do something different!"  No.

SCU registers mapping for CA9/CA5 cores

[Arnd Bergmann]

On Monday 24 June 2013, Russell King - ARM Linux wrote:
> On Fri, Jun 21, 2013 at 11:29:22AM +0200, Arnd Bergmann wrote:
> > I think we could actually move everything between VMALLOC_END and
> > ffff0000 into a larger fixmap. This would be a substantial change,
> > but it would give us more vmalloc space for most configurations
> > and more flexibility with what else we could put into fixmap.
> 
> ... at the expense of:
> (a) having to look up the PTE table pointer every time we want to add or
> remove a mapping,

Is that a frequent operation? I guess it's not important given the
argument below.

> (b) wasting the top 15 pages from being used.
> 
> Also, the placement of some of these pages matters especially for the
> VIPT aliasing caches, where cache colouring becomes important.  So an
> enum in fixmap isn't going to work for those.

x86 deals with alignment through some extra tricks, it's certainly doable
but I agree it's not improving readability if you do that.

> Really, the value to such a change is very low and is really just churn
> for churn's sake.  "Oh we can do something different!"  No.

Ok, fair enough.

I also saw the patch for early_ioremap, which would basically require
doing something like this. I'll comment on that email separately.

	Arnd