ioremap() on SH

ioremap() on SH

[Franck Bui-Huu]

Hello,

I'm puzzled by the implementation of ioremap() on SH architecture.

Here it is:

        if (likely(PXSEG(offset) < P3SEG && PXSEG(last_addr) < P3SEG)) {
                if (unlikely(flags & _PAGE_CACHABLE))
                        return (void __iomem *)P1SEGADDR(offset);

                return (void __iomem *)P2SEGADDR(offset);
        }

        return __ioremap(offset, size, flags);

where 'offset' is a physical address...

If I understand correctly PXSEG(x) is used to get the segment of a
_virtual_ address, so doing PXSEG(offset) looks weird.

Furthermore, it looks like the virtual to physical address translation is
similar to MIPS 32 bits architecture. Therefore only the low 512Mo of the
physical address space can be accessed through P1/P2 segments.

So this test:

        if (likely(PXSEG(offset) < P3SEG && PXSEG(last_addr) < P3SEG)) {

                /* simply map the address into P2 seg */

        }

where P3SEG = 0xc0000000 seems wrong too.

Why not simply doing something like this ?

        if (unlikely(offset >= offset + size))
                return NULL;

        if (offset + size < 0x20000000)
                return (void __iomem *)P2SEGADDR(offset);

        return __ioremap(offset, size, flags);

Thanks

                Franck

Re: ioremap() on SH

[Paul Mundt]

On Thu, Feb 14, 2008 at 03:36:31PM +0100, Franck Bui-Huu wrote:
> I'm puzzled by the implementation of ioremap() on SH architecture.
> 
> Here it is:
> 
>         if (likely(PXSEG(offset) < P3SEG && PXSEG(last_addr) < P3SEG)) {
>                 if (unlikely(flags & _PAGE_CACHABLE))
>                         return (void __iomem *)P1SEGADDR(offset);
> 
>                 return (void __iomem *)P2SEGADDR(offset);
>         }
> 
>         return __ioremap(offset, size, flags);
> 
> where 'offset' is a physical address...
> 
> If I understand correctly PXSEG(x) is used to get the segment of a
> _virtual_ address, so doing PXSEG(offset) looks weird.
> 
In 29-bit physical mode, there is no real differentiation. P1 and P2 are
both non-translatable sections that refer to the low 512M (P1 is cached,
and P2 is uncached). If you're using the MIPS terminology, these are
KSEG0 and KSEG1 respectively. In the nommu context the logic is
simplified since we don't really have that sort of segmentation, so
there's still a direct 1:1 correlation between physical and virtual there
also. Thus, we can take a P2 address with a cached remap and hand back
the P1 equivalent, and vice versa for cached->uncached transitions.

Page table mappings are only used for certain parts of P3 and other parts
of the address space. There are also non-translatable holes in the
address space further along that need special care, so there is further
special casing for those.

The comment above the __ioremap() prototype attempts to clarify some of
this. There are additional notes in arch/sh/mm/ioremap_32.c as well,
mostly relating to the special casing in the P3 address space.

There are also cases where address space behaviour is determined by
special PTE bits relative to a given chip-select. PCMCIA and CF via area
5 and area 6 are examples of this, and are traditionally why P3 remapping
requires special handling. Other P3 addresses don't generally require
special casing, and can use fairly standard page table mapping.

sh64 on the other hand accomplishes similar behaviour through the mapping
of 512M superpages in the DTLB (using wired entries), which can be seen
in arch/sh/kernel/head_64.S.

Hope that helps.

Re: ioremap() on SH

[Franck Bui-Huu]

On Thu 14/02 23:57, Paul Mundt wrote:
> On Thu, Feb 14, 2008 at 03:36:31PM +0100, Franck Bui-Huu wrote:
> > I'm puzzled by the implementation of ioremap() on SH architecture.
> >
> > Here it is:
> >
> >         if (likely(PXSEG(offset) < P3SEG && PXSEG(last_addr) < P3SEG)) {
> >                 if (unlikely(flags & _PAGE_CACHABLE))
> >                         return (void __iomem *)P1SEGADDR(offset);
> >
> >                 return (void __iomem *)P2SEGADDR(offset);
> >         }
> >
> >         return __ioremap(offset, size, flags);
> >
> > where 'offset' is a physical address...
> >
> > If I understand correctly PXSEG(x) is used to get the segment of a
> > _virtual_ address, so doing PXSEG(offset) looks weird.
> >
> In 29-bit physical mode, there is no real differentiation. P1 and P2 are
> both non-translatable sections that refer to the low 512M (P1 is cached,
> and P2 is uncached). If you're using the MIPS terminology, these are
> KSEG0 and KSEG1 respectively. In the nommu context the logic is
> simplified since we don't really have that sort of segmentation, so
> there's still a direct 1:1 correlation between physical and virtual there
> also. Thus, we can take a P2 address with a cached remap and hand back
> the P1 equivalent, and vice versa for cached->uncached transitions.
>
> Page table mappings are only used for certain parts of P3 and other parts
> of the address space. There are also non-translatable holes in the
> address space further along that need special care, so there is further
> special casing for those.
>
> The comment above the __ioremap() prototype attempts to clarify some of
> this. There are additional notes in arch/sh/mm/ioremap_32.c as well,
> mostly relating to the special casing in the P3 address space.
>
> There are also cases where address space behaviour is determined by
> special PTE bits relative to a given chip-select. PCMCIA and CF via area
> 5 and area 6 are examples of this, and are traditionally why P3 remapping
> requires special handling. Other P3 addresses don't generally require
> special casing, and can use fairly standard page table mapping.
>
> sh64 on the other hand accomplishes similar behaviour through the mapping
> of 512M superpages in the DTLB (using wired entries), which can be seen
> in arch/sh/kernel/head_64.S.
>
> Hope that helps.

Hm, not really, I'm afraid.

Unfortunately I don't see how it addresses the points I raised:

What does PXSEG(offset) mean if 'offset' is a physical address ?

From my understanding this macro used to retreive a *virtual* address
segment from a *virtual* address. However PXSEG(offset) retrieves the
*virtual* address segment from a *physical* address which seems a no
sense.

Why doing the following test "PXSEG(offset) < P3SEG" where P3SEG yields
to 0xc0000000. If offset is 0xa0000000 for example, assuming that 32 bits
physical mode exists, what's ioremap returning ?

                Franck

Re: ioremap() on SH

[Paul Mundt]

On Thu, Feb 14, 2008 at 05:42:32PM +0100, Franck Bui-Huu wrote:
> On Thu 14/02 23:57, Paul Mundt wrote:
> > On Thu, Feb 14, 2008 at 03:36:31PM +0100, Franck Bui-Huu wrote:
> > In 29-bit physical mode, there is no real differentiation. P1 and P2 are
> > both non-translatable sections that refer to the low 512M (P1 is cached,
> > and P2 is uncached). If you're using the MIPS terminology, these are
> > KSEG0 and KSEG1 respectively. In the nommu context the logic is
> > simplified since we don't really have that sort of segmentation, so
> > there's still a direct 1:1 correlation between physical and virtual there
> > also. Thus, we can take a P2 address with a cached remap and hand back
> > the P1 equivalent, and vice versa for cached->uncached transitions.
> 
> What does PXSEG(offset) mean if 'offset' is a physical address ?
> 
> From my understanding this macro used to retreive a *virtual* address
> segment from a *virtual* address. However PXSEG(offset) retrieves the
> *virtual* address segment from a *physical* address which seems a no
> sense.
> 
You've apparently neglected the importance of the term "non-translatable"
as was mentioned above. There is a 1:1 correlation here in the low-512MB.
P1 and P2 are _fixed_, you only have virtual<->physical translations in
P3/P0(U0), and occasionally P4. Otherwise it is only the cacheability of
the mapping that matters, which is determined by whether you access it as
a P1 or P2 address. As should be apparent, we do not take page faults for
non-translatable areas.

> Why doing the following test "PXSEG(offset) < P3SEG" where P3SEG yields
> to 0xc0000000. If offset is 0xa0000000 for example, assuming that 32 bits
> physical mode exists, what's ioremap returning ?
> 
0xa000000 is the P2 base, which is the uncached start. In this case
PXSEG(0xa0000000) = 0xa0000000, so it depends on what you specify as the
ioremap flags. If you want a cacheable mapping, you will get back
0x80000000 which is the cached equivalent of that address. If you want it
uncached, you will get back 0xa0000000. 0x80000000 and 0xa0000000 point
at exactly the same thing, just with different caching attributes.

If you wanted to do a dumb ioremap that assumed you always wanted
uncached and didn't try to deal with the special cases where we need page
table mappings, your ioremap would just be a wrapper to P2SEGADDR, and
this is in fact what we used to have.

Re: ioremap() on SH

[Franck Bui-Huu]

Paul Mundt wrote:
> 0xa000000 is the P2 base, which is the uncached start. In this case
> PXSEG(0xa0000000) = 0xa0000000, so it depends on what you specify as the
> ioremap flags. If you want a cacheable mapping, you will get back
> 0x80000000 which is the cached equivalent of that address. If you want it
> uncached, you will get back 0xa0000000. 0x80000000 and 0xa0000000 point
> at exactly the same thing, just with different caching attributes.
> 

Do you think that such mapping is correct ?

That is to access a device at 0xa000 0000 (physical address), I can use
the virtual address 0xa000 0000 or 0x8000 0000 ?

In this case shouldn't we use the TLB instead to access the device ?
IOW shouldn't ioremap() return a virtual address in P3 segment ?

Thanks
		Franck

Re: ioremap() on SH

[Stuart MENEFY]

Franck Bui-Huu wrote:
> Paul Mundt wrote:
>> 0xa000000 is the P2 base, which is the uncached start. In this case
>> PXSEG(0xa0000000) = 0xa0000000, so it depends on what you specify as the
>> ioremap flags. If you want a cacheable mapping, you will get back
>> 0x80000000 which is the cached equivalent of that address. If you want it
>> uncached, you will get back 0xa0000000. 0x80000000 and 0xa0000000 point
>> at exactly the same thing, just with different caching attributes.
>>
> 
> Do you think that such mapping is correct ?
> 
> That is to access a device at 0xa000 0000 (physical address), I can use
> the virtual address 0xa000 0000 or 0x8000 0000 ?

0xa000 0000 is not a 'correct' physical address in 29 bit mode. In this
example you should really be using physical 0. However traditionally
the SH kernel allowed this because in 29 bit mode it is not ambiguous.

In 32 bit mode this all changes. 0xa000 0000 is now a legitimate physical
address, and needs to be mapped using the TLB or PMB. The current
git kernel doesn't support ioremap for these addresses.

Stuart

Re: ioremap() on SH

[Franck Bui-Huu]

Stuart MENEFY wrote:
> 0xa000 0000 is not a 'correct' physical address in 29 bit mode. In this
> example you should really be using physical 0. However traditionally
> the SH kernel allowed this because in 29 bit mode it is not ambiguous.
> 

Weird: why would a driver use 0xa000 0000 when it can simply use 0 ?
Isn't this error prone ?

Current implementation assumes that if the passed physical address is
greater than 0xc000 0000 then it uses page tables.

Why the 0xc000 0000 limit has been chosen ? Why not simply using the
512M limit ?

> In 32 bit mode this all changes. 0xa000 0000 is now a legitimate physical
> address, and needs to be mapped using the TLB or PMB. The current
> git kernel doesn't support ioremap for these addresses.
> 

Well if you just chose the 512M boundary it works for both world, this
is what I tried to propose in my previous email. And it removes the weid
test "PXSEG(phys) < P3SEG" too.

But I'm probably missing something since I'm not familiar with the SH
world.

		Franck

Re: ioremap() on SH

[Stuart MENEFY]

Franck Bui-Huu wrote:
> Stuart MENEFY wrote:
>> 0xa000 0000 is not a 'correct' physical address in 29 bit mode. In this
>> example you should really be using physical 0. However traditionally
>> the SH kernel allowed this because in 29 bit mode it is not ambiguous.
> 
> Weird: why would a driver use 0xa000 0000 when it can simply use 0 ?
> Isn't this error prone ?

In 29 bit mode the mappings at P1 and P2 are hard wired, so I suspect it
fools people into thinking that the peripherals really are at those
address. In fact in older systems which are only 29 bit, it does work,
because the top three bits don't even exist, and so it doesn't matter what
you set them to.

It also means that people 'forget' to use ioremap at all, and abuse
ctrl_in/out, which is why the switch to 32 bit physical addressing isn't
as painless as it should be.

> Current implementation assumes that if the passed physical address is
> greater than 0xc000 0000 then it uses page tables.
> 
> Why the 0xc000 0000 limit has been chosen ? Why not simply using the
> 512M limit ?
 >
>> In 32 bit mode this all changes. 0xa000 0000 is now a legitimate physical
>> address, and needs to be mapped using the TLB or PMB. The current
>> git kernel doesn't support ioremap for these addresses.
> 
> Well if you just chose the 512M boundary it works for both world, this
> is what I tried to propose in my previous email. And it removes the weid
> test "PXSEG(phys) < P3SEG" too.

Is simply an efficiency issue. In 29 bit mode we don't want to use the page
tables as the P1/P2 mappings do the job just fine, without having to set up
the page tables and take faults.

> But I'm probably missing something since I'm not familiar with the SH
> world.

Unfortunately there is a lot more missing for full 32 bit mode support than
just a working ioremap(). Paul has been kindly picking up some of the changes
from the ST tree, but there is still a sizeable chunk to come, and we
have to work out what needs changing for the Renesas parts. In the ST tree,
in 29 bit mode, remapping anything between 0x2000 0000 (end of 29 bit physical)
and 0xdfff ffff (top of P3) is an error, in an attempt to remove this type of
abuse by driver writers!

Stuart