follow_page() performance regressions

follow_page() performance regressions

[Paul Mundt]

(adding Linus and l-k to Cc..)

On Tue, Jul 01, 2008 at 01:24:39PM +0900, Hideo Saito wrote:
> I have a question as to performance of kernel on linux-2.6.25.9, but
> this may be off-topic, because the problem is not related to the SH
> architecture directly.
> 
> When I tested Hackbench benchmark test on our platform(SH7780, 400MHz),
> the performance decreases as follows.
> 
> The result on linux-2.6.25.9:
> #tcsh> ./hackbench 40 ; time ./hackbench 40 ; time ./hackbench 40
> Time: 69.858
> Time: 69.810
> Time: 70.356
> 
> The result on linux-2.6.25.8:
> #tcsh> ./hackbench 40 ; time ./hackbench 40 ; time ./hackbench 40
> Time: 65.353
> Time: 65.622
> Time: 65.413
> 
> I confirmed that the cause is in the following changes. Isn't this
> decrease of performance avoided, though it seems that this code is
> changed in order to handle ZERO_PAGE?
> 
> --- a/mm/memory.c
> +++ b/mm/memory.c
>     pte = *ptep;
>     if (!pte_present(pte))
> -       goto unlock;
> +       goto no_page;
>     if ((flags & FOLL_WRITE) && !pte_write(pte))
>         goto unlock;
>     page = vm_normal_page(vma, address, pte);
>     if (unlikely(!page))
> -       goto unlock;
> +       goto bad_page;
> 
>     if (flags & FOLL_GET)
>         get_page(page);

git blame points to:

commit 89f5b7da2a6bad2e84670422ab8192382a5aeb9f
Author: Linus Torvalds <torvalds@linux-foundation.org>
Date:   Fri Jun 20 11:18:25 2008 -0700

    Reinstate ZERO_PAGE optimization in 'get_user_pages()' and fix XIP

...

So yes, the ZERO_PAGE handling in follow_page() is causing the slow-down here.

Re: follow_page() performance regressions

[Linus Torvalds]

On Tue, 1 Jul 2008, Paul Mundt wrote:
> 
> So yes, the ZERO_PAGE handling in follow_page() is causing the slow-down here.

I suspect it could be any of:

 - actual code generation differences in follow_page() (some SH person 
   needs to check that)

 - some cache issue on SH. ZERO_PAGE is a single page at a fixed virtual 
   address, while it used to populate the page tables with individual 
   pages. Normally, this should be *better* for caching, but maybe there 
   is some conflict? What kind of caches does SH have (virtually indexed?)

 - hackbench relying on follow_page() to populate the page tables, which 
   it no longer does for anonymous areas (using ZERO_PAGE directly 
   instead).

   Again, normally this would speed things up (fewer TLB misses etc), but 
   if it then causes a new page fault that used to have been covered by 
   follow_page(), who knows?

What does hackbench actually do? Anybody?

			Linus

Re: follow_page() performance regressions

[Linus Torvalds]

On Tue, 1 Jul 2008, Linus Torvalds wrote:
> 
>  - some cache issue on SH. ZERO_PAGE is a single page at a fixed virtual 
>    address, while it used to populate the page tables with individual 
>    pages. Normally, this should be *better* for caching, but maybe there 
>    is some conflict? What kind of caches does SH have (virtually indexed?)

Actually, this one is unlikely. ZERO_PAGE _used_ to be problematic on some 
virtually indexed caches, because the same physical page would get mapped 
into multiple places virtually, which in turn then caused some cache or 
TLB conflict or something (I forget - MIPS was the main offender).

But because we now use just the single kernel virtual address for 
accessing the ZERO_PAGE, that really shouldn't matter. Unless the 
particular user of follow_pages() that triggers in this case is doing 
something odd, of course.

		Linus

Re: follow_page() performance regressions

[Hideo Saito]

On Tue, 1 Jul 2008, Linus Torvalds wrote:
> 
> I suspect it could be any of:
> 
>  - actual code generation differences in follow_page() (some SH person 
>    needs to check that)
> 
>  - some cache issue on SH. ZERO_PAGE is a single page at a fixed virtual 
>    address, while it used to populate the page tables with individual 
>    pages. Normally, this should be *better* for caching, but maybe there 
>    is some conflict? What kind of caches does SH have (virtually indexed?)
> 
>  - hackbench relying on follow_page() to populate the page tables, which 
>    it no longer does for anonymous areas (using ZERO_PAGE directly 
>    instead).
> 
>    Again, normally this would speed things up (fewer TLB misses etc), but 
>    if it then causes a new page fault that used to have been covered by 
>    follow_page(), who knows?
> 
> What does hackbench actually do? Anybody?

I am reporter of this problem.
Sorry, my testing was not enough. This problem depends our platform or maybe SH architecture.

When I changed that the code of |goto no_page| and |goto bad_page| to both |goto unlock| at follow_page(), I did not notice that the code of |bad_page:| and |no_page:| was lost by optimizer. In the result, only the size of the text code of kernel became small 0x40 bytes. I think that the cause of this problem is in any of our platform or in the replacement of SH cache.