kernel compiling performance challenge

kernel compiling performance challenge

[Nick Piggin]

[-- Attachment #1: Type: text/plain, Size: 3381 bytes --]

I was recently disappointed to find that 2.4 kernels very slightly
edge out the latest 2.6 kernels in kernel compiling performance on
my dual Xeon, even when 2.6 has HZ set at 100.

So I spent a few days trying some different mm/ optimisations, and
managed to reduce total kernel residency (excluding idle time) by 7%
on that workload, and now manage to beat 2.4 by a good third of a
second.

Here's a diffprofile versus a plain 2.6.14-rc3 kernel:
        123   384.4% __get_zone_counts
         54     0.0% __page_set_anon_rmap
         37    17.0% find_lock_page
         28    31.8% lru_cache_add_active
         26    42.6% path_lookup
         25    22.3% kmem_cache_alloc
         20     6.2% find_vma
         19    17.3% _atomic_dec_and_lock
         18    26.5% __copy_from_user_ll
         17   188.9% shmem_nopage
         17    58.6% unmap_vmas
         15     0.0% __page_state
         14    31.1% copy_pte_range
         14    13.6% __wake_up_bit
         14     0.0% remove_vma
         14    46.7% exit_notify
         13    61.9% sys_close
         13    27.1% anon_vma_prepare
         13     0.0% unlink_file_vma
         13    92.9% do_generic_mapping_read
         12   109.1% free_pgd_range
         12     0.0% vm_stat_account
         12   100.0% sys_mmap2
         10    17.5% get_empty_filp
          .
          .
        -10   -21.7% _spin_unlock_irq
        -10   -90.9% flush_old_exec
        -10   -25.6% vfs_read
        -10    -4.6% __handle_mm_fault
        -12   -17.1% do_shmem_file_read
        -12   -60.0% cond_resched
        -12   -60.0% number
        -12   -46.2% sys_open
        -12  -100.0% __vm_stat_account
        -13   -46.4% inotify_inode_queue_event
        -13   -33.3% dput
        -13    -2.8% release_pages
        -14   -26.9% kmem_cache_free
        -14   -17.7% pte_alloc_map
        -14    -3.2% __link_path_walk
        -16  -100.0% __rmqueue
        -19   -10.9% sysenter_past_esp
        -19   -55.9% vfs_getattr
        -20   -25.0% zone_watermark_ok
        -21   -17.9% may_open
        -21   -15.8% strnlen_user
        -21    -8.8% __pagevec_lru_add_active
        -23  -100.0% remove_vm_struct
        -23    -5.1% zap_pte_range
        -28    -0.6% do_page_fault
        -33   -15.8% do_anonymous_page
        -40    -7.1% __d_lookup
        -44    -5.9% _spin_lock
        -75   -43.1% page_remove_rmap
        -79   -98.8% set_page_dirty
        -81   -23.5% free_hot_cold_page
        -81   -10.0% __copy_to_user_ll
        -94   -98.9% page_add_anon_rmap
       -108   -26.2% do_no_page
       -111   -72.1% prep_new_page
       -143  -100.0% bad_range
       -444   -87.2% __mod_page_state
       -548   -10.1% buffered_rmqueue
      -1634    -7.1% total

Depending on how much interest there is, I might keep a tree around
to collect performance improvements. If you have any more[*] I could
look at, please send them over. I'll eventually try to get things
merged.

* Not just for kbuild, or only mm related, but preferably something
   that I can easily measure on my little system.

Attached is a rollup against 2.6.14-rc3. I don't currently have any
webspace handy, so I can't host a broken-out tarball anywhere yet.

Sorry for the big attachment (actually most of it is Hugh's pagefault
scalability prep and my lockless pagecache prep that I'm working on
top of).

Nick

-- 
SUSE Labs, Novell Inc.


[-- Attachment #2: 2.6.14-rc3-kc1.patch.gz --]
[-- Type: application/x-tar, Size: 52581 bytes --]

Re: kernel compiling performance challenge

[Nick Piggin]

Nick Piggin wrote:

> Attached is a rollup against 2.6.14-rc3. I don't currently have any
> webspace handy, so I can't host a broken-out tarball anywhere yet.
> 

Hmm, a wayward hunk to net/core/skbuff.c slipped in there - it
is wrong. Just delete the net/core/skbuff.c part of the patch
before applying it.

-- 
SUSE Labs, Novell Inc.

Send instant messages to your online friends http://au.messenger.yahoo.com