From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>,
akpm@linux-foundation.org, linux-mm@kvack.org,
linux-kernel@vger.kernel.org,
Cong Wang <xiyou.wangcong@gmail.com>,
Dave Hansen <dave.hansen@intel.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Mel Gorman <mgorman@suse.de>, Michal Hocko <mhocko@kernel.org>,
Petr Mladek <pmladek@suse.com>,
Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
Vlastimil Babka <vbabka@suse.cz>,
"yuwang.yuwang" <yuwang.yuwang@alibaba-inc.com>
Subject: Re: [PATCH] mm: don't warn about allocations which stall for too long
Date: Thu, 2 Nov 2017 17:53:13 +0900 [thread overview]
Message-ID: <20171102085313.GD655@jagdpanzerIV> (raw)
In-Reply-To: <20171031153225.218234b4@gandalf.local.home>
On (10/31/17 15:32), Steven Rostedt wrote:
[..]
> (new globals)
> static DEFINE_SPIN_LOCK(console_owner_lock);
> static struct task_struct console_owner;
> static bool waiter;
>
> console_unlock() {
>
> [ Assumes this part can not preempt ]
>
> spin_lock(console_owner_lock);
> console_owner = current;
> spin_unlock(console_owner_lock);
+ disables IRQs?
> for each message
> write message out to console
>
> if (READ_ONCE(waiter))
> break;
>
> spin_lock(console_owner_lock);
> console_owner = NULL;
> spin_unlock(console_owner_lock);
>
> [ preemption possible ]
otherwise
printk()
if (console_trylock())
console_unlock()
preempt_disable()
spin_lock(console_owner_lock);
console_owner = current;
spin_unlock(console_owner_lock);
.......
spin_lock(console_owner_lock);
IRQ
printk()
console_trylock() // fails so we go to busy-loop part
spin_lock(console_owner_lock); << deadlock
even if we would replace spin_lock(console_owner_lock) with IRQ
spin_lock, we still would need to protect against IRQs on the very
same CPU. right? IOW, we need to store smp_processor_id() of a CPU
currently doing console_unlock() and check it in vprintk_emit()?
and we need to protect the entire console_unlock() function. not
just the printing loop, otherwise the IRQ CPU will spin forever
waiting for itself to up() the console_sem.
this somehow reminds me of "static unsigned int logbuf_cpu", which
we used to have in vprintk_emit() and were happy to remove it...
the whole "console_unlock() is non-preemptible" can bite, I'm
afraid. it's not always printk()->console_unlock(), sometimes
it's console_lock()->console_unlock() that has to flush the
logbuf.
CPU0 CPU1 ~ CPU99
console_lock();
printk(); ... printk();
console_unlock()
preempt_disable();
for (;;)
call_console_drivers();
<<lockup>>
this pattern is not so unusual. _especially_ in the existing scheme
of things.
not to mention the problem of "the last printk()", which will take
over and do the flush.
CPU0 CPU1 ~ CPU99
console_lock();
printk(); ... printk();
console_unlock();
IRQ on CPU2
printk()
// take over console_sem
console_unlock()
and so on.
seems that there will be lots of if-s.
-ss
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>,
akpm@linux-foundation.org, linux-mm@kvack.org,
linux-kernel@vger.kernel.org,
Cong Wang <xiyou.wangcong@gmail.com>,
Dave Hansen <dave.hansen@intel.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Mel Gorman <mgorman@suse.de>, Michal Hocko <mhocko@kernel.org>,
Petr Mladek <pmladek@suse.com>,
Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
Vlastimil Babka <vbabka@suse.cz>,
"yuwang.yuwang" <yuwang.yuwang@alibaba-inc.com>
Subject: Re: [PATCH] mm: don't warn about allocations which stall for too long
Date: Thu, 2 Nov 2017 17:53:13 +0900 [thread overview]
Message-ID: <20171102085313.GD655@jagdpanzerIV> (raw)
In-Reply-To: <20171031153225.218234b4@gandalf.local.home>
On (10/31/17 15:32), Steven Rostedt wrote:
[..]
> (new globals)
> static DEFINE_SPIN_LOCK(console_owner_lock);
> static struct task_struct console_owner;
> static bool waiter;
>
> console_unlock() {
>
> [ Assumes this part can not preempt ]
>
> spin_lock(console_owner_lock);
> console_owner = current;
> spin_unlock(console_owner_lock);
+ disables IRQs?
> for each message
> write message out to console
>
> if (READ_ONCE(waiter))
> break;
>
> spin_lock(console_owner_lock);
> console_owner = NULL;
> spin_unlock(console_owner_lock);
>
> [ preemption possible ]
otherwise
printk()
if (console_trylock())
console_unlock()
preempt_disable()
spin_lock(console_owner_lock);
console_owner = current;
spin_unlock(console_owner_lock);
.......
spin_lock(console_owner_lock);
IRQ
printk()
console_trylock() // fails so we go to busy-loop part
spin_lock(console_owner_lock); << deadlock
even if we would replace spin_lock(console_owner_lock) with IRQ
spin_lock, we still would need to protect against IRQs on the very
same CPU. right? IOW, we need to store smp_processor_id() of a CPU
currently doing console_unlock() and check it in vprintk_emit()?
and we need to protect the entire console_unlock() function. not
just the printing loop, otherwise the IRQ CPU will spin forever
waiting for itself to up() the console_sem.
this somehow reminds me of "static unsigned int logbuf_cpu", which
we used to have in vprintk_emit() and were happy to remove it...
the whole "console_unlock() is non-preemptible" can bite, I'm
afraid. it's not always printk()->console_unlock(), sometimes
it's console_lock()->console_unlock() that has to flush the
logbuf.
CPU0 CPU1 ~ CPU99
console_lock();
printk(); ... printk();
console_unlock()
preempt_disable();
for (;;)
call_console_drivers();
<<lockup>>
this pattern is not so unusual. _especially_ in the existing scheme
of things.
not to mention the problem of "the last printk()", which will take
over and do the flush.
CPU0 CPU1 ~ CPU99
console_lock();
printk(); ... printk();
console_unlock();
IRQ on CPU2
printk()
// take over console_sem
console_unlock()
and so on.
seems that there will be lots of if-s.
-ss
next prev parent reply other threads:[~2017-11-02 8:53 UTC|newest]
Thread overview: 51+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-10-26 11:28 [PATCH] mm: don't warn about allocations which stall for too long Tetsuo Handa
2017-10-26 11:28 ` Tetsuo Handa
2017-10-26 11:41 ` Michal Hocko
2017-10-26 11:41 ` Michal Hocko
2017-11-08 10:30 ` peter enderborg
2017-11-09 8:52 ` Michal Hocko
2017-11-09 8:52 ` Michal Hocko
2017-11-09 9:34 ` peter enderborg
2017-11-09 10:09 ` Michal Hocko
2017-11-09 10:09 ` Michal Hocko
2017-11-09 10:19 ` Tetsuo Handa
2017-11-09 10:19 ` Tetsuo Handa
2017-10-26 14:37 ` Johannes Weiner
2017-10-26 14:37 ` Johannes Weiner
2017-10-31 19:32 ` Steven Rostedt
2017-11-01 8:30 ` Vlastimil Babka
2017-11-01 8:30 ` Vlastimil Babka
2017-11-01 13:38 ` Petr Mladek
2017-11-01 13:38 ` Petr Mladek
2017-11-01 15:36 ` Steven Rostedt
2017-11-01 15:36 ` Steven Rostedt
2017-11-02 11:46 ` Petr Mladek
2017-11-02 11:46 ` Petr Mladek
2017-11-02 14:49 ` Steven Rostedt
2017-11-02 14:49 ` Steven Rostedt
2017-11-01 15:33 ` Steven Rostedt
2017-11-01 15:33 ` Steven Rostedt
2017-11-01 17:42 ` Vlastimil Babka
2017-11-01 17:42 ` Vlastimil Babka
2017-11-01 17:54 ` Steven Rostedt
2017-11-01 17:54 ` Steven Rostedt
2017-11-02 8:53 ` Sergey Senozhatsky [this message]
2017-11-02 8:53 ` Sergey Senozhatsky
2017-11-02 9:14 ` Sergey Senozhatsky
2017-11-02 9:14 ` Sergey Senozhatsky
2017-11-02 14:55 ` Steven Rostedt
2017-11-02 14:55 ` Steven Rostedt
2017-11-02 12:55 ` Michal Hocko
2017-11-02 12:55 ` Michal Hocko
2017-11-02 15:56 ` Steven Rostedt
2017-11-02 15:56 ` Steven Rostedt
2017-11-02 17:06 ` [PATCH v2] printk: Add console owner and waiter logic to load balance console writes Steven Rostedt
2017-11-02 17:06 ` Steven Rostedt
2017-11-02 17:10 ` Steven Rostedt
2017-11-02 17:10 ` Steven Rostedt
2017-11-02 17:38 ` Steven Rostedt
2017-11-02 17:38 ` Steven Rostedt
2017-11-03 10:19 ` Jan Kara
2017-11-03 10:19 ` Jan Kara
2017-11-03 11:18 ` Steven Rostedt
2017-11-03 11:18 ` Steven Rostedt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171102085313.GD655@jagdpanzerIV \
--to=sergey.senozhatsky.work@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=dave.hansen@intel.com \
--cc=hannes@cmpxchg.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mhocko@kernel.org \
--cc=penguin-kernel@I-love.SAKURA.ne.jp \
--cc=pmladek@suse.com \
--cc=rostedt@goodmis.org \
--cc=sergey.senozhatsky@gmail.com \
--cc=vbabka@suse.cz \
--cc=xiyou.wangcong@gmail.com \
--cc=yuwang.yuwang@alibaba-inc.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.