From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754326AbdEIPlI (ORCPT ); Tue, 9 May 2017 11:41:08 -0400 Received: from mout.gmx.net ([212.227.15.15]:51043 "EHLO mout.gmx.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753824AbdEIPlG (ORCPT ); Tue, 9 May 2017 11:41:06 -0400 Message-ID: <1494344459.4440.17.camel@gmx.de> Subject: Re: get/put_cpu() usage in block/blk-mq.c From: Mike Galbraith To: Jens Axboe Cc: LKML Date: Tue, 09 May 2017 17:40:59 +0200 In-Reply-To: References: <1494310029.20252.44.camel@gmx.de> <1494342256.4440.3.camel@gmx.de> <842b9fc5-5a8e-d147-0b14-3c9f9540336d@fb.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.16.5 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:6oqKPUMsx/zjqIbI8+axtJ/R+MCuxuYZbCjVzgdG3iinHr8l1BL dEvKI/PrnOlUaKJtutBmNWHfxrvbXF9ql2NTxDUmgWVstoudZZxeAiYoREKUdmNGH+/WLSN dk5ogBeeG+KifSyQScr3hg3dRx0AU99oEpGI6IMOHpcmXD039iIA/gcrJmEYDN8k8fPGxtw pLb2lx8LQpqM/+sqZ2bhg== X-UI-Out-Filterresults: notjunk:1;V01:K0:LPhn82aEwaE=:tcfj9K1xK8J1Ho+wnOP9A5 yB9j1yz1WZdsK5IHP9nGjgijIxmUJaRNILWe0MzMtQlvAAnYxFweRAr6et9Qi6jvnT8aJskC8 zEx0HKtemMre09HNHy7AZWvJmeY1WzihKw4BI/nSOzUUcwb6y5JMv8TAD9HIAA0Gso+Ma8BEF g3s9ijKhA8nIZqzSwecAXZoJPIOr5xjXHNfXahJtm11B/kitomDTzIn+2ZcahnppduO00aZQX n+bElcQDLNe7zPRc+PEA/1Gfkcc7g5SsNKLru/qjJ0es4xlUl6fVBNtcsI/wMkGJ0YiMUrU/+ ZGPOAsqIQqwbN8PY10+2Il+VN5uZc3S+atFmBezk4wXIFN0QR7BMpYd3KRWeLLZOex4V0A3ek Ixb0nPxVuze0xWJUl8yTt+sYgTXUynyRTuW1/koW3OxA1YflY7z//F/xOaNW5KsjZU0ayCRtp YMd/m6KKihjExy2e2ghGlnyI0m+RVa1WU2V/IojyHqc1ZNqOVO7Xo6h5WNXnFfXVPJcmUqUGo di9THlO/uMseHrQC2k5R3LtcfnL7gbBkux1BkfBj0tEEXIvBiqRJQiDhUgrBNnzJMUytB8WS2 ejLih4n5zMTlsjzj1DlnXz51w85RUYCronTO7dhy6qbJBlk6wWncOZEu5L/vX0VMJaP/arEWR tokJdLeIEpSmM3zLqC6mUCZE1O8YSC9utWccdJKDogpDJyS4TTcqVQj6casMigZNjN5YH51OS ST4si0yVvz87z1Glf5Rddc2rMxf8+epSuTkUqbrgdezFOdIiUIU21+2i59YXTDOrT4RO67OnK u4FADM9 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 2017-05-09 at 09:15 -0600, Jens Axboe wrote: > On 05/09/2017 09:13 AM, Jens Axboe wrote: > > On 05/09/2017 09:04 AM, Mike Galbraith wrote: > > > On Tue, 2017-05-09 at 08:53 -0600, Jens Axboe wrote: > > > > On 05/09/2017 12:07 AM, Mike Galbraith wrote: > > > > > Hi Jens, > > > > > > > > > > I was about to fix up this splat.. > > > > > > > > > > [ 445.022141] loop: module loaded > > > > > [ 445.078116] BUG: using smp_processor_id() in preemptible > > > > > [00000000] code: loop0/3801 > > > > > [ 445.085873] caller is debug_smp_processor_id+0x17/0x20 > > > > > [ 445.091016] CPU: 7 PID: 3801 Comm: loop0 Tainted: G > > > > > E 4.12.0-default #40 > > > > > [ 445.098838] Hardware name: IBM System x3550 M3 -[7944K3G] > > > > > -/69Y5698 , BIOS -[D6E150AUS-1.10]- 12/15/2010 > > > > > [ 445.108564] Call Trace: > > > > > [ 445.111016] dump_stack+0x65/0x89 > > > > > [ 445.114330] check_preemption_disabled+0xde/0xf0 > > > > > [ 445.118945] debug_smp_processor_id+0x17/0x20 > > > > > [ 445.123300] blk_stat_add+0xb0/0x130 > > > > > [ 445.126876] __blk_mq_complete_request+0xb5/0x150 > > > > > [ 445.131575] blk_mq_complete_request+0x16/0x20 > > > > > [ 445.136020] loop_queue_work+0x5f/0xaa0 [loop] > > > > > [ 445.140461] ? _raw_spin_unlock_irq+0x21/0x40 > > > > > [ 445.144816] ? finish_task_switch+0x85/0x270 > > > > > [ 445.149085] ? __schedule+0x291/0x8c0 > > > > > [ 445.152747] kthread_worker_fn+0xc2/0x1d0 > > > > > [ 445.156754] kthread+0x114/0x150 > > > > > [ 445.159983] ? __kthread_init_worker+0xb0/0xb0 > > > > > [ 445.164424] ? kthread_park+0x60/0x60 > > > > > [ 445.168085] ret_from_fork+0x2c/0x40 > > > > > > > > Is it from this_cpu_ptr() in blk_stat_add()? > > > > > > Yeah. > > > > So why is this complaining, doesn't rcu_read_lock() disable > > preemption? > > Ah, I guess it doesn't if PREEMPT_RCU is set. How about the below? Should do it. I was about to run LTP (where it turned up) again anyway, I'll add this. No news is good news, what you should hear. > diff --git a/block/blk-stat.c b/block/blk-stat.c > index 6c2f40940439..022f5925a427 100644 > --- a/block/blk-stat.c > +++ b/block/blk-stat.c > @@ -96,13 +96,16 @@ void blk_stat_add(struct request *rq) > > rcu_read_lock(); > list_for_each_entry_rcu(cb, &q->stats->callbacks, list) { > - if (blk_stat_is_active(cb)) { > - bucket = cb->bucket_fn(rq); > - if (bucket < 0) > - continue; > - stat = &this_cpu_ptr(cb->cpu_stat)[bucket]; > - __blk_stat_add(stat, value); > - } > + if (!blk_stat_is_active(cb)) > + continue; > + > + bucket = cb->bucket_fn(rq); > + if (bucket < 0) > + continue; > + > + stat = &get_cpu_var(cb->cpu_stat)[bucket]; > + __blk_stat_add(stat, value); > + put_cpu_var(cb->cpu_stat); > } > rcu_read_unlock(); > } >