Re: fix CONFIG_HIGHPTE

[Andrew Morton <akpm@zip.com.au>]

William Lee Irwin III wrote:
> 
> Minimalistic fix. Perhaps rough at the edges but I can clean the
> ugliness ppl care about when they complain. 2.5.30 successfully booted
> & ran userspace on a 16-way NUMA-Q with 16GB of RAM with this patch
> and CONFIG_HIGHPTE enabled.

Thanks, Bill.  It doesn't seem any uglier than anything else highmem-related.

> ...
> +#define rmap_ptep_map(pte_paddr)                                       \
> +({                                                                     \
> +       unsigned long pfn = (unsigned long)(pte_paddr >> PAGE_SHIFT);   \
> +       unsigned long idx = __pte_offset(((unsigned long)pte_paddr));   \
> +       (pte_t *)kmap_atomic(pfn_to_page(pfn), KM_PTE2) + idx;          \
> +})

Could be an inline?

> +static inline rmap_ptep_map(pte_addr_t pte_paddr)
> +{
> +       return (pte_t *)pte_paddr;
> +}

Better try compiling that ;)

> ...
> --- 1.66/include/linux/mm.h     Thu Aug  1 12:30:06 2002
> +++ edited/include/linux/mm.h   Fri Aug  2 22:24:40 2002
> @@ -161,7 +161,7 @@
>         union {
>                 struct pte_chain * chain;       /* Reverse pte mapping pointer.
>                                          * protected by PG_chainlock */
> -               pte_t            * direct;
> +               pte_addr_t               direct;
>         } pte;

Four more bytes into struct page.  I bet that hurt.

> ...
>  struct pte_chain {
>         struct pte_chain * next;
> -       pte_t * ptep;
> +       pte_addr_t ptep;
>  };

We'll get fifteen pte_addr_t's per pte_chain on a P4 with the
array-of-pteps-per-pte_chain patch.

And we'll need that, to reduce load on KM_PTECHAIN.  Because
there's no point in pte_highmem without also having pte_chain_highmem,
yes?

Which means either going back to a custom allocator or teaching
slab about highmem and kmap_atomic.  (Probably a custom allocator;
internal fragmentation on 32/64/128 byte pte_chains won't be tooooo
bad, presumably).

We're piling more and more crap in there to support these pte_chains.
How much is too much?

Is it likely that large pages and/or shared pagetables would allow us to
place pagetables and pte_chains in the direct-mapped region, avoid all
this?