From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51B52C49EA7 for ; Fri, 25 Jun 2021 12:42:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3983660241 for ; Fri, 25 Jun 2021 12:42:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231133AbhFYMpL (ORCPT ); Fri, 25 Jun 2021 08:45:11 -0400 Received: from Galois.linutronix.de ([193.142.43.55]:33180 "EHLO galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229934AbhFYMpK (ORCPT ); Fri, 25 Jun 2021 08:45:10 -0400 From: Thomas Gleixner DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1624624968; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=kUvHNoaJ8hCHdVSooVihvRmjh1gPNArcPRW12xQl1mg=; b=wE8ezas2TzVU1H2UhFE2Qqluhh4R0PTGEDvLHCOLfcRKKkYieNL6kUqFABsgS5f4EHK1zn uC6Dc5WymQYokWHBTcQ7vf1nHgivxUMx3MUK8qNxxXYiKhX4fN8cZcZ22L+ci4srGoE++t Cfol40KPMcXP9wqkZEedvJaVZS72WRmr1OjJRWD0k5yLXogloXgfks2oOZLF/kYlSwpkjg 398u/kH2ZY5BlU7n2MuHxv2DzPSczvIGQb4mn4qTJyDTSMw5H7sbfci6fXpvzvWN8U78yS 5xSOzMhEoEMYxpBxjgI38RARtNkGfSGE+3hUrlsDAYNe+gwwR6zNiEuyzHG7UA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1624624968; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=kUvHNoaJ8hCHdVSooVihvRmjh1gPNArcPRW12xQl1mg=; b=fxzpnqDLxdWf/HUlsC3sYIU73rs4RLRvicSCFfizzTYMn/oKBBJAb5U77/sulc9bMlCoHc NgfQdpgeg1qmR/BA== To: "Tian\, Kevin" , Alex Williamson Cc: Jason Gunthorpe , "Dey\, Megha" , "Raj\, Ashok" , "Pan\, Jacob jun" , "Jiang\, Dave" , "Liu\, Yi L" , "Lu\, Baolu" , "Williams\, Dan J" , "Luck\, Tony" , "Kumar\, Sanjay K" , LKML , KVM , Kirti Wankhede , Peter Zijlstra , Marc Zyngier , Bjorn Helgaas Subject: RE: Virtualizing MSI-X on IMS via VFIO In-Reply-To: <87o8buuyfy.ffs@nanos.tec.linutronix.de> References: <8735t7wazk.ffs@nanos.tec.linutronix.de> <20210624154434.11809b8f.alex.williamson@redhat.com> <87o8buuyfy.ffs@nanos.tec.linutronix.de> Date: Fri, 25 Jun 2021 14:42:48 +0200 Message-ID: <87im22uncn.ffs@nanos.tec.linutronix.de> MIME-Version: 1.0 Content-Type: text/plain Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Jun 25 2021 at 10:43, Thomas Gleixner wrote: > On Fri, Jun 25 2021 at 05:21, Kevin Tian wrote: >> p.s. one question to Thomas. As Alex cited above, software must >> not modify the Address, Data, or Steering Tag fields of an MSI-X >> entry while it is unmasked. However this rule might be violated >> today in below flow: >> >> request_irq() >> __setup_irq() >> irq_startup() >> __irq_startup() >> irq_enable() >> unmask_irq() <<<<<<<<<<<<< >> irq_setup_affinity() >> irq_do_set_affinity() >> msi_set_affinity() // when IR is disabled >> irq_msi_update_msg() >> pci_msi_domain_write_msg() <<<<<<<<<<<<<< >> >> Isn't above have msi-x entry updated after it's unmasked? > > Dammit, I could swear that we had masking at the core or PCI level at > some point. Let me dig into this. Indeed, that code path does not check irq_can_move_pcntxt(). It doesn't blow up in our face by chance because of this: __setup_irq() irq_activate() unmask() irq_setup_affinity() irq_activate() assigns a vector based on the affinity mask so irq_setup_affinity() ends up writing the same data again pointlessly. For some stupid reason the ordering of startup/setup_affinity is the way it is for historical reasons. I tried to reorder it at some point but that caused failure on !x86 so I went back to the status quo. All other affinity settings happen with the interrupt masked because we do that from actual interrupt context via irq_move_masked_irq() which does the right thing. Let me fix that proper for the startup case. Thanks, tglx