From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1751443AbdJEQxL (ORCPT <rfc822;w@1wt.eu>);
        Thu, 5 Oct 2017 12:53:11 -0400
Received: from mail.kernel.org ([198.145.29.99]:53122 "EHLO mail.kernel.org"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S1751331AbdJEQxJ (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 5 Oct 2017 12:53:09 -0400
DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5B77321908
Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.org
Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=acme@kernel.org
Date: Thu, 5 Oct 2017 13:53:05 -0300
From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Julia Cartwright <julia@ni.com>
Cc: bigeasy@linutronix.de, linux-rt-users@vger.kernel.org,
        linux-kernel@vger.kernel.org,
        Arnaldo Carvalho de Melo <acme@redhat.com>,
        Clark Williams <williams@redhat.com>,
        Dean Luick <dean.luick@intel.com>,
        Dennis Dalessandro <dennis.dalessandro@intel.com>,
        Doug Ledford <dledford@redhat.com>, Kaike Wan <kaike.wan@intel.com>,
        Leon Romanovsky <leonro@mellanox.com>, linux-rdma@vger.kernel.org,
        Peter Zijlstra <peterz@infradead.org>,
        Sebastian Andrzej Siewior <sebastian.siewior@linutronix.de>,
        Sebastian Sanchez <sebastian.sanchez@intel.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Thomas Gleixner <tglx@linutronix.de>
Subject: Re: [PATCH 1/2] IB/hfi1: Use preempt_{dis,en}able_nort()
Message-ID: <20171005165305.GN25388@kernel.org>
References: <20171003154920.31566-1-acme@kernel.org>
 <20171003154920.31566-2-acme@kernel.org>
 <20171005141744.GC21185@jcartwri.amer.corp.natinst.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20171005141744.GC21185@jcartwri.amer.corp.natinst.com>
X-Url: http://acmel.wordpress.com
User-Agent: Mutt/1.9.0 (2017-09-02)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Em Thu, Oct 05, 2017 at 09:17:44AM -0500, Julia Cartwright escreveu:
> On Tue, Oct 03, 2017 at 12:49:19PM -0300, Arnaldo Carvalho de Melo wrote:
> > +++ b/drivers/infiniband/hw/hfi1/pio.c
> > @@ -1421,7 +1421,7 @@ struct pio_buf *sc_buffer_alloc(struct send_context *sc, u32 dw_len,

> >  	/* there is enough room */

> > -	preempt_disable();
> > +	preempt_disable_nort();
> >  	this_cpu_inc(*sc->buffers_allocated);

> Have you tried this on RT w/ CONFIG_DEBUG_PREEMPT?

No

> I believe that the this_cpu_* operations perform a preemption check, which we'd trip.

Humm, looking at include/linux/percpu-defs.h on v4.11.12-rt14 I see
(trimmed to what we're discussing here):

#ifdef CONFIG_DEBUG_PREEMPT
extern void __this_cpu_preempt_check(const char *op);
#else
static inline void __this_cpu_preempt_check(const char *op) { }
#endif

#define __this_cpu_add(pcp, val)					\
({									\
	__this_cpu_preempt_check("add");				\
	raw_cpu_add(pcp, val);						\
})
#define __this_cpu_inc(pcp)		__this_cpu_add(pcp, 1)

/*
 * Operations with implied preemption/interrupt protection.  These
 * operations can be used without worrying about preemption or interrupt.
 */
#define this_cpu_add(pcp, val)          __pcpu_size_call(this_cpu_add_, pcp, val)
#define this_cpu_inc(pcp)               this_cpu_add(pcp, 1)
 
> You may also have to change these to the non-preempt checked variants.

So __this_cpu_inc() checks preemption but this_cpu_inc() doesn't and
thus we're ok here? Or am I getting lost in this maze of defines? :-)

- Arnaldo