ARM page tables

ARM page tables

[Mason]

Hello everyone,

On my SoC, we have two MMIO regions mapped at init:

  {
   .virtual = (0xf0000000 + 0x00800000),
   .pfn =((unsigned long)((0x20000000) >> 12)),
   .length = 0x00200000,
   .type = 0,
  },
  {
   .virtual = (0xf0000000 +(0)),
   .pfn =((unsigned long)((0) >> 12)),
   .length = 0x00800000,
   .type = 0,
  },

I dumped the L1 page table descriptors, expecting to find
entries for the MMIO regions:
(AFAICT, the page table starts at 0xc0007000)

col_1 = entry number (hex)
col_2 = entry value
col_3 = string for (desc & 3)

300 00011452 SECTION
301 00111452 SECTION
302 00211452 SECTION
303 00311452 SECTION
304 00411452 SECTION
305 00511452 SECTION
306 00611452 SECTION
307 00711452 SECTION
308 20011452 SECTION
309 20111452 SECTION

If I understand correctly, entry 308 means:
VIRTUAL ADDRESS 0x3080_0000 is mapped to PHYSICAL ADDRESS 0x2000_0000
(Is my reading correct?)

But we asked to map PA 0x2000_0000 to VA 0xf080_0000 in iotable_init,
so I'm confused...

Also, looking at entry 300:
VA 0x3000_0000 to PA 0
but we asked for VA 0xf000_0000 didn't we?

Can someone clear my confusion?

Regards.

ARM page tables

[Russell King - ARM Linux]

On Tue, Dec 09, 2014 at 06:21:32PM +0100, Mason wrote:
> On my SoC, we have two MMIO regions mapped at init:
> 
>  {
>   .virtual = (0xf0000000 + 0x00800000),
>   .pfn =((unsigned long)((0x20000000) >> 12)),
>   .length = 0x00200000,
>   .type = 0,
>  },
>  {
>   .virtual = (0xf0000000 +(0)),
>   .pfn =((unsigned long)((0) >> 12)),
>   .length = 0x00800000,
>   .type = 0,
>  },
> 
> I dumped the L1 page table descriptors, expecting to find
> entries for the MMIO regions:
> (AFAICT, the page table starts at 0xc0007000)

No, it starts at 0xc0004000.

> col_1 = entry number (hex)
> col_2 = entry value
> col_3 = string for (desc & 3)
> 
> 300 00011452 SECTION
> 301 00111452 SECTION
> 302 00211452 SECTION
> 303 00311452 SECTION
> 304 00411452 SECTION
> 305 00511452 SECTION
> 306 00611452 SECTION
> 307 00711452 SECTION
> 308 20011452 SECTION
> 309 20111452 SECTION
> 
> If I understand correctly, entry 308 means:
> VIRTUAL ADDRESS 0x3080_0000 is mapped to PHYSICAL ADDRESS 0x2000_0000
> (Is my reading correct?)

No.  If you're talking about 0x308 from an offset of 0x3000 into the
page table, that's 0x308 * 1MB + 0xc0000000 = 0xf0800000.

> But we asked to map PA 0x2000_0000 to VA 0xf080_0000 in iotable_init,
> so I'm confused...

So, with the right starting point, it works out correctly.

-- 
FTTC broadband for 0.8mile line: currently at 9.5Mbps down 400kbps up
according to speedtest.net.

ARM page tables

[Mason]

Hello Russel,

On 09/12/2014 18:27, Russell King - ARM Linux wrote:
> On Tue, Dec 09, 2014 at 06:21:32PM +0100, Mason wrote:
>> On my SoC, we have two MMIO regions mapped at init:
>>
>>  {
>>   .virtual = (0xf0000000 + 0x00800000),
>>   .pfn =((unsigned long)((0x20000000) >> 12)),
>>   .length = 0x00200000,
>>   .type = 0,
>>  },
>>  {
>>   .virtual = (0xf0000000 +(0)),
>>   .pfn =((unsigned long)((0) >> 12)),
>>   .length = 0x00800000,
>>   .type = 0,
>>  },
>>
>> I dumped the L1 page table descriptors, expecting to find
>> entries for the MMIO regions:
>> (AFAICT, the page table starts at 0xc0007000)
> 
> No, it starts at 0xc0004000.

I should have suspected that I had the wrong start address. Thanks!
What's the name of the symbolic constant for that address?

The page table is allocated in pgd_alloc, right?
  new_pgd = __pgd_alloc();
#define __pgd_alloc()   (pgd_t *)__get_free_pages(GFP_KERNEL | __GFP_REPEAT, 2)

If that's the correct init code, how does it guarantee to always
return 0xc0004000?

>> col_1 = entry number (hex)
>> col_2 = entry value
>> col_3 = string for (desc & 3)
>>
>> 300 00011452 SECTION
>> 301 00111452 SECTION
>> 302 00211452 SECTION
>> 303 00311452 SECTION
>> 304 00411452 SECTION
>> 305 00511452 SECTION
>> 306 00611452 SECTION
>> 307 00711452 SECTION
>> 308 20011452 SECTION
>> 309 20111452 SECTION
>>
>> If I understand correctly, entry 308 means:
>> VIRTUAL ADDRESS 0x3080_0000 is mapped to PHYSICAL ADDRESS 0x2000_0000
>> (Is my reading correct?)
> 
> No.  If you're talking about 0x308 from an offset of 0x3000 into the
> page table, that's 0x308 * 1MB + 0xc0000000 = 0xf0800000.
> 
>> But we asked to map PA 0x2000_0000 to VA 0xf080_0000 in iotable_init,
>> so I'm confused...
> 
> So, with the right starting point, it works out correctly.

Indeed!

I am investigating this issue:

As I described, we map PA 0-8MB. But some code actually accesses
address 0xf0920000 (from kernel space) and I was expecting very
nasty things, like the equivalent of SIGSEGV in the kernel.
But nothing happens, no page fault, no BUG_ON, no panic, no
implosion... (Writes seem ignored, and reads always return 0)

I don't understand how it is possible to access unmapped virtual
addresses without the MMU throwing a fit... That's why I'm looking
at the page table.

Regards.

ARM page tables

[Russell King - ARM Linux]

On Tue, Dec 09, 2014 at 11:59:22PM +0100, Mason wrote:
> On 09/12/2014 18:27, Russell King - ARM Linux wrote:
> > On Tue, Dec 09, 2014 at 06:21:32PM +0100, Mason wrote:
> >>  {
> >>   .virtual = (0xf0000000 +(0)),
> >>   .pfn =((unsigned long)((0) >> 12)),
> >>   .length = 0x00800000,
> >>   .type = 0,
> >>  },
> What's the name of the symbolic constant for that address?

swapper_pg_dir

> >> 300 00011452 SECTION
> >> 301 00111452 SECTION
> >> 302 00211452 SECTION
> >> 303 00311452 SECTION
> >> 304 00411452 SECTION
> >> 305 00511452 SECTION
> >> 306 00611452 SECTION
> >> 307 00711452 SECTION
> >> 308 20011452 SECTION
> >> 309 20111452 SECTION
...
> I am investigating this issue:
> 
> As I described, we map PA 0-8MB. But some code actually accesses
> address 0xf0920000 (from kernel space) and I was expecting very
> nasty things, like the equivalent of SIGSEGV in the kernel.
> But nothing happens, no page fault, no BUG_ON, no panic, no
> implosion... (Writes seem ignored, and reads always return 0)

As we can see above, you asked for 0x00200000 bytes to be mapped from
0xf0800000.  So, 0xf0800000 to 0xf09fffff will be mapped.  The address
which you are accessing falls within this range.

So, there's a valid mapping.  Assuming nothing on the bus raises an
external abort, accesses to this address will succeed.

> I don't understand how it is possible to access unmapped virtual
> addresses without the MMU throwing a fit... That's why I'm looking
> at the page table.

You _have_ asked for that address to be mapped.

-- 
FTTC broadband for 0.8mile line: currently at 9.5Mbps down 400kbps up
according to speedtest.net.

ARM page tables

[Mason]

Hello Russel,

Russell King wrote:

> Mason wrote:
>  
>> What's the name of the address of the page table?
>
> swapper_pg_dir

Got it.
arch/arm/kernel/head.S

/*
  * swapper_pg_dir is the virtual address of the initial page table.
  */
.equ  swapper_pg_dir, KERNEL_RAM_VADDR - PG_DIR_SIZE

#define KERNEL_RAM_VADDR (PAGE_OFFSET + TEXT_OFFSET)

PAGE_OFFSET = 0xC0000000
TEXT_OFFSET = $(textofs-y) = 0x00008000
PG_DIR_SIZE = 0x4000

Thus, in my case, swapper_pg_dir = 0xC0004000

> As we can see above, you asked for 0x00200000 bytes to be mapped from
> 0xf0800000.  So, 0xf0800000 to 0xf09fffff will be mapped.  The address
> which you are accessing falls within this range.
>
> So, there's a valid mapping.  Assuming nothing on the bus raises an
> external abort, accesses to this address will succeed.

I was looking so hard for the tree, that I completely missed the forest.
Thanks for clearing my confusion.

By the way, wrt to the thread "Creating 16 MB super-sections for MMIO".
I'm thinking of writing a quick-and-dirty patch to test the impact on
our SoC of using a super-section for the 0-16MB region. Would there be
any interest in mainlining that?

Regards.