From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E703C433E0 for ; Thu, 11 Mar 2021 19:52:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5339264F86 for ; Thu, 11 Mar 2021 19:52:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229871AbhCKTwY (ORCPT ); Thu, 11 Mar 2021 14:52:24 -0500 Received: from mail.kernel.org ([198.145.29.99]:49532 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229490AbhCKTvz (ORCPT ); Thu, 11 Mar 2021 14:51:55 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 16F5C64F80; Thu, 11 Mar 2021 19:51:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1615492314; bh=2KlA7EfVuhHt9Lg7khxrZsPO9Aw6CY4WB7/VJn7+giM=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=G6o2P1PKtqem7HP4l0h6g27R6D8sLxNRlMz+vpy2lkFpuutiOOpcp9j9ZoGTLyjP0 eNxsvN85ObsVGUDXX+JOCpZ0W9XRopKRLCu/fY/C9GSiyijxEzH7DgJTGUtfRhKnY5 f8Y0WJ5qbwqzzH0Tlz6iYfweSZ8/U6R6dQoBqcQW22aqVG15RNSUtj6+7ft8YvQkZ9 zfMAEq0E1T8Pttquy/dtDzPW9CqAqtmXEP2UX4Oukg1VUBYncQ2IvfouLHRSPKtBn5 i/IufPLOqP0zteJcueUS1NVBT0bx6Q3a/yMoWlUHOoq/y4KgzgBuNSd89u7g6DDtDp IswmIV+KbYXmQ== Date: Thu, 11 Mar 2021 21:51:50 +0200 From: Leon Romanovsky To: Alexander Duyck Cc: Bjorn Helgaas , Bjorn Helgaas , Saeed Mahameed , Jason Gunthorpe , Jakub Kicinski , linux-pci , linux-rdma@vger.kernel.org, Netdev , Don Dutile , Alex Williamson , "David S . Miller" , Greg Kroah-Hartman Subject: Re: [PATCH mlx5-next v7 0/4] Dynamically assign MSI-X vectors count Message-ID: References: <20210311181729.GA2148230@bjorn-Precision-5520> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Thu, Mar 11, 2021 at 11:37:28AM -0800, Alexander Duyck wrote: > On Thu, Mar 11, 2021 at 10:17 AM Bjorn Helgaas wrote: > > > > On Wed, Mar 10, 2021 at 03:34:01PM -0800, Alexander Duyck wrote: > > > On Wed, Mar 10, 2021 at 11:09 AM Bjorn Helgaas wrote: > > > > On Sun, Mar 07, 2021 at 10:55:24AM -0800, Alexander Duyck wrote: > > > > > On Sun, Feb 28, 2021 at 11:55 PM Leon Romanovsky wrote: > > > > > > From: Leon Romanovsky > > > > > > > > > > > > @Alexander Duyck, please update me if I can add your ROB tag again > > > > > > to the series, because you liked v6 more. > > > > > > > > > > > > Thanks > > > > > > > > > > > > --------------------------------------------------------------------------------- > > > > > > Changelog > > > > > > v7: > > > > > > * Rebase on top v5.12-rc1 > > > > > > * More english fixes > > > > > > * Returned to static sysfs creation model as was implemented in v0/v1. <...> > > > representors rather than being actual PCIe devices. Having > > > functionality that only works when the VF driver is not loaded just > > > feels off. The VF sysfs directory feels like it is being used as a > > > subdirectory of the PF rather than being a device on its own. > > > > Moving "virtfnX_msix_count" to the PF seems like it would mitigate > > this somewhat. I don't know how to make this work while a VF driver > > is bound without making the VF feel even less like a PCIe device, > > i.e., we won't be able to use the standard MSI-X model. > > Yeah, I actually do kind of like that idea. In addition it would > address one of the things I pointed out as an issue before as you > could place the virtfn values that the total value in the same folder > so that it is all in one central spot rather than having to walk all > over the sysfs hierarchy to check the setting for each VF when trying > to figure out how the vectors are currently distributed. User binds specific VF with specific PCI ID to VM, so instead of changing MSI-X table for that specific VF, he will need to translate from virtfn25_msix_count to PCI ID. I also gave an example of my system where I have many PFs and VFs function numbers are not distributed nicely. On that system virtfn25_msix_count won't translate to AA:BB:CC.25 but to something else. Thanks