From mboxrd@z Thu Jan  1 00:00:00 1970
From: Shijith Thotton <shijith.thotton@caviumnetworks.com>
Subject: Re: [RFC PATCH] igb_uio: issue FLR during open and
 release of device file
Date: Thu, 1 Jun 2017 17:05:01 +0530
Message-ID: <20170601113501.GA2165@localhost.localdomain>
References: <1748341.rbpcFmWp0q@polaris>
 <1496228966-18573-1-git-send-email-shijith.thotton@caviumnetworks.com>
 <f55be7d7-24af-4ea8-2808-c2e50731dce1@intel.com>
 <20170531083010.228859c3@xeon-e3>
 <ec1a0f14-33e2-a7dc-85bc-3daa5a469c33@intel.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: Stephen Hemminger <stephen@networkplumber.org>,
 Gregory Etelson <gregory@weka.io>, dev@dpdk.org,
 Qi Zhang <qi.z.zhang@intel.com>, Wenzhuo Lu <wenzhuo.lu@intel.com>,
 Thomas Monjalon <thomas@monjalon.net>,
 Jianfeng Tan <jianfeng.tan@intel.com>
To: Ferruh Yigit <ferruh.yigit@intel.com>
Return-path: <dev-bounces@dpdk.org>
Received: from NAM02-CY1-obe.outbound.protection.outlook.com
 (mail-cys01nam02on0065.outbound.protection.outlook.com [104.47.37.65])
 by dpdk.org (Postfix) with ESMTP id B72217CB4
 for <dev@dpdk.org>; Thu,  1 Jun 2017 13:35:23 +0200 (CEST)
Content-Disposition: inline
In-Reply-To: <ec1a0f14-33e2-a7dc-85bc-3daa5a469c33@intel.com>
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

On Wed, May 31, 2017 at 06:11:40PM +0100, Ferruh Yigit wrote:
> On 5/31/2017 4:30 PM, Stephen Hemminger wrote:
> > On Wed, 31 May 2017 13:20:08 +0100
> > Ferruh Yigit <ferruh.yigit@intel.com> wrote:
> > 
> >> On 5/31/2017 12:09 PM, Shijith Thotton wrote:
> >>> Set UIO info device file operations open and release. Call pci reset
> >>> function inside open and release to clear device state at start and
> >>> end. Copied this behaviour from vfio_pci kernel module code. With this
> >>> change, it is not mandatory to issue FLR by PMD's during init and close.  
> >>
> >> Cc: Jianfeng Tan <jianfeng.tan@intel.com>
> >>
> >> Jianfeng also implemented following patch:
> >> http://dpdk.org/dev/patchwork/patch/17495/
> >>
> >> Which also implements release and open ops, for slightly different
> >> reason (prevent DMA access after app exit), but mainly both are to
> >> gracefully handle application exit status.
> >>
> >> btw, for Jianfeng's case, can adding pci_clear_master() in release and
> >> moving pci_set_master() to open help preventing unwanted DMA?
> >>
> >>
> >> Gregory,
> >>
> >> Can you please check if this patch fixes your issue?
> >>
> >> Thanks,
> >> ferruh
> > 
> > pci_reset should stop all DMA. It also clears master status.

Per Alex Williamson[1], bus master will be disabled after FLR. But a disable is
preferred after reset, since all device won't behave as per spec.

1. http://www.spinics.net/lists/kvm/msg115715.html

> 
> If so, should open() call pci_set_master(), currently it has been only
> called by igbuio_pci_probe() once?

DPDK has pci_uio_set_bus_master/pci_vfio_set_bus_master to set bus master.
Can we leave it to the application ? vfio leaves the device with bus master
disabled after open.