Re: [RT] lockdep munching nr_list_entries like popcorn

Thomas Gleixner
2017-02-16T08:37:06Z

On Thu, 16 Feb 2017, Mike Galbraith wrote:

> 4.9.10-rt6-virgin on 72 core +SMT box.
>
> Below is 1 line per minute, box idling along daintily nibbling, I fire
> up a parallel kbuild loop at 40465, and box gobbles greedily.
>
> I have entries bumped to 128k, and chain bits to 18 so box will get
> booted and run for a while before lockdep says "I quit". With stock
> settings, this box will barely get booted. Seems the bigger the box,
> the sooner you're gonna run out. A NOPREEMPT kernel seems to nibble
> entries too, but nowhere remotely near as greedily as RT.

Right. RT adds a bunch of locks through the local lock mechanism.

> <...>-100309 [064] d....13 2885.873312: add_lock_to_list.isra.24.constprop.42+0x20/0x100: nr_list_entries: 40129
> <...>-92785 [047] d....12 3905.137578: add_lock_to_list.isra.24.constprop.42+0x20/0x100: nr_list_entries: 51287

That's odd.

> With stacktrace on, buffer contains 1010 __lru_cache_add+0x4f...
>
> (gdb) list *__lru_cache_add+0x4f
> 0xffffffff811dca9f is in __lru_cache_add (./include/linux/locallock.h:59).
> 54
> 55 static inline void __local_lock(struct local_irq_lock *lv)
> 56 {
> 57 if (lv->owner != current) {
> 58 spin_lock_local(&lv->lock);
> 59 LL_WARN(lv->owner);
> 60 LL_WARN(lv->nestcnt);
> 61 lv->owner = current;
> 62 }
> 63 lv->nestcnt++;
>
> ...which seems to be this.
>
> 0xffffffff811dca80 is in __lru_cache_add (mm/swap.c:397).
> 392 }
> 393 EXPORT_SYMBOL(mark_page_accessed);
> 394
> 395 static void __lru_cache_add(struct page *page)
> 396 {
> 397 struct pagevec *pvec = &get_locked_var(swapvec_lock, lru_add_pvec);
> 398
> 399 get_page(page);
> 400 if (!pagevec_add(pvec, page) || PageCompound(page))
> 401 __pagevec_lru_add(pvec);
>
> swapvec_lock? Oodles of 'em? Nope.

Well, it's a per cpu lock and the lru_cache_add() variants might be called
from a gazillion of different call chains, but yes, it does not make a lot
of sense. We'll have a look.

Thanks,

tglx