From mboxrd@z Thu Jan 1 00:00:00 1970 From: Steven Rostedt Subject: Re: [PATCH] writeback: call writeback tracepoints withoud holding list_lock in wb_writeback() Date: Thu, 25 Feb 2016 14:54:32 -0500 Message-ID: <20160225145432.3749e5ec@gandalf.local.home> References: <1456354043-31420-1-git-send-email-yang.shi@linaro.org> <20160224214042.71c3493b@grimm.local.home> <56CF5848.7050806@linaro.org> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: tj@kernel.org, jack@suse.cz, axboe@fb.com, fengguang.wu@intel.com, tglx@linutronix.de, bigeasy@linutronix.de, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-rt-users@vger.kernel.org, linaro-kernel@lists.linaro.org To: "Shi, Yang" Return-path: In-Reply-To: <56CF5848.7050806@linaro.org> Sender: owner-linux-mm@kvack.org List-Id: linux-rt-users.vger.kernel.org On Thu, 25 Feb 2016 11:38:48 -0800 "Shi, Yang" wrote: > On 2/24/2016 6:40 PM, Steven Rostedt wrote: > > On Wed, 24 Feb 2016 14:47:23 -0800 > > Yang Shi wrote: > > > >> commit 5634cc2aa9aebc77bc862992e7805469dcf83dac ("writeback: update writeback > >> tracepoints to report cgroup") made writeback tracepoints report cgroup > >> writeback, but it may trigger the below bug on -rt kernel due to the list_lock > >> held for the for loop in wb_writeback(). > > > > list_lock is a sleeping mutex, it's not disabling preemption. Moving it > > doesn't make a difference. > > > >> > >> BUG: sleeping function called from invalid context at kernel/locking/rtmutex.c:930 > >> in_atomic(): 1, irqs_disabled(): 0, pid: 625, name: kworker/u16:3 > > > > Something else disabled preemption. And note, nothing in the tracepoint > > should have called a sleeping function. > > Yes, it makes me confused too. It sounds like the preempt_ip address is > not that accurate. Yep, but the change you made doesn't look to be the fix. > > > > > > >> INFO: lockdep is turned off. > >> Preemption disabled at:[] wb_writeback+0xec/0x830 Can you disassemble the vmlinux file to see exactly where that call is. I use gdb to find the right locations. gdb> li *0xffffffc000374a5c gdb> disass 0xffffffc000374a5c > >> > >> CPU: 7 PID: 625 Comm: kworker/u16:3 Not tainted 4.4.1-rt5 #20 > >> Hardware name: Freescale Layerscape 2085a RDB Board (DT) > >> Workqueue: writeback wb_workfn (flush-7:0) > >> Call trace: > >> [] dump_backtrace+0x0/0x200 > >> [] show_stack+0x24/0x30 > >> [] dump_stack+0x88/0xa8 > >> [] ___might_sleep+0x2ec/0x300 > >> [] rt_spin_lock+0x38/0xb8 > >> [] kernfs_path_len+0x30/0x90 > >> [] trace_event_raw_event_writeback_work_class+0xe8/0x2e8 > > > > How accurate is this trace back? Here's the code that is executed in > > this tracepoint: > > > > TP_fast_assign( > > struct device *dev = bdi->dev; > > if (!dev) > > dev = default_backing_dev_info.dev; > > strncpy(__entry->name, dev_name(dev), 32); > > __entry->nr_pages = work->nr_pages; > > __entry->sb_dev = work->sb ? work->sb->s_dev : 0; > > __entry->sync_mode = work->sync_mode; > > __entry->for_kupdate = work->for_kupdate; > > __entry->range_cyclic = work->range_cyclic; > > __entry->for_background = work->for_background; > > __entry->reason = work->reason; > > ), > > > > See anything that would sleep? > > According to the stack backtrace, kernfs_path_len calls slepping lock, > which is called by __trace_wb_cgroup_size(wb) in __dynamic_array(char, > cgroup, __trace_wb_cgroup_size(wb)). > > The below is the definition: > > DECLARE_EVENT_CLASS(writeback_work_class, > TP_PROTO(struct bdi_writeback *wb, struct wb_writeback_work *work), > TP_ARGS(wb, work), > TP_STRUCT__entry( > __array(char, name, 32) > __field(long, nr_pages) > __field(dev_t, sb_dev) > __field(int, sync_mode) > __field(int, for_kupdate) > __field(int, range_cyclic) > __field(int, for_background) > __field(int, reason) > __dynamic_array(char, cgroup, __trace_wb_cgroup_size(wb)) > Ah, thanks for pointing that out. I missed that. -- Steve -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org