Re: regression caused by cgroups optimization in 3.17-rc2

[Dave Hansen <dave@sr71.net>]

On 09/09/2014 07:50 AM, Johannes Weiner wrote:
> The mctz->lock is only taken when there is, or has been, soft limit
> excess.  However, the soft limit defaults to infinity, so unless you
> set it explicitly on the root level, I can't see how this could be
> mctz->lock contention.
> 
> It's more plausible that this is the res_counter lock for testing soft
> limit excess - for me, both these locks get inlined into check_events,
> could you please double check you got the right lock?

I got the wrong lock.  Here's how it looks after mainline, plus your free_pages_and_swap_cache() patch:

Samples: 2M of event 'cycles', Event count (approx.): 51647128377                            
+   60.60%     1.33%  page_fault2_processes              [.] testcase                       ▒
+   59.14%     0.41%  [kernel]                           [k] page_fault                     ◆
+   58.72%     0.01%  [kernel]                           [k] do_page_fault                  ▒
+   58.70%     0.08%  [kernel]                           [k] __do_page_fault                ▒
+   58.50%     0.29%  [kernel]                           [k] handle_mm_fault                ▒
+   40.14%     0.28%  [kernel]                           [k] do_cow_fault                   ▒
-   34.56%    34.56%  [kernel]                           [k] _raw_spin_lock                 ▒
   - _raw_spin_lock                                                                         ▒
      - 78.11% __res_counter_charge                                                         ▒
           res_counter_charge                                                               ▒
           try_charge                                                                       ▒
         - mem_cgroup_try_charge                                                            ▒
            + 99.99% do_cow_fault                                                           ▒
      - 10.30% res_counter_uncharge_until                                                   ▒
           res_counter_uncharge                                                             ▒
           uncharge_batch                                                                   ▒
           uncharge_list                                                                    ▒
           mem_cgroup_uncharge_list                                                         ▒
           release_pages                                                                    ▒
      + 4.75% free_pcppages_bulk                                                            ▒
      + 3.65% do_cow_fault                                                                  ▒
      + 2.24% get_page_from_freelist                                                        ▒

> You also said that this cost hasn't been there before, but I do see
> that trace in both v3.16 and v3.17-rc3 with roughly the same impact
> (although my machines show less contention than yours).  Could you
> please double check that this is in fact a regression independent of
> 05b843012335 ("mm: memcontrol: use root_mem_cgroup res_counter")?

Here's the same workload on the same machine with only Johannes' revert applied:

-   35.92%    35.92%  [kernel]                           [k] _raw_spin_lock                 ▒
   - _raw_spin_lock                                                                         ▒
      - 49.09% get_page_from_freelist                                                       ▒
         - __alloc_pages_nodemask                                                           ▒
            + 99.90% alloc_pages_vma                                                        ▒
      - 43.67% free_pcppages_bulk                                                           ▒
         - 100.00% free_hot_cold_page                                                       ▒
            + 99.93% free_hot_cold_page_list                                                ▒
      - 7.08% do_cow_fault                                                                  ▒
           handle_mm_fault                                                                  ▒
           __do_page_fault                                                                  ▒
           do_page_fault                                                                    ▒
           page_fault                                                                       ▒
           testcase                                                                         ▒

So I think it's probably part of the same regression.