Re: [PATCH v3 1/2] eal/pci: introduce a PCI driver flag

[Jerin Jacob <jerin.jacob@caviumnetworks.com>]

-----Original Message-----
> Date: Tue, 13 Jun 2017 09:03:56 +0200
> From: Thomas Monjalon <thomas@monjalon.net>
> To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Cc: dev@dpdk.org, ferruh.yigit@intel.com
> Subject: Re: [dpdk-dev]  [PATCH v3 1/2] eal/pci: introduce a PCI driver flag
> 
> 13/06/2017 06:43, Jerin Jacob:
> > -----Original Message-----
> > > Date: Mon, 12 Jun 2017 18:21:33 +0200
> > > From: Thomas Monjalon <thomas@monjalon.net>
> > > To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > > Cc: dev@dpdk.org, ferruh.yigit@intel.com
> > > Subject: Re: [dpdk-dev]  [PATCH v3 1/2] eal/pci: introduce a PCI driver flag
> > > 
> > > 09/06/2017 12:27, Jerin Jacob:
> > > > Some ethdev devices like nicvf thunderx PMD need special treatment for
> > > > Secondary queue set(SQS) PCIe VF devices, where, it expects to not unmap
> > > > or free the memory without registering the ethdev subsystem.
> > > > 
> > > > Introducing a new RTE_PCI_DRV_KEEP_MAPPED_RES
> > > > PCI driver flag to request PCI subsystem to not unmap the mapped PCI
> > > > resources(PCI BAR address) if unsupported device detected.
> > > > 
> > > > Suggested-by: Ferruh Yigit <ferruh.yigit@intel.com>
> > > > Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > > [...]
> > > > --- a/lib/librte_eal/common/eal_common_pci.c
> > > > +++ b/lib/librte_eal/common/eal_common_pci.c
> > > > @@ -221,7 +221,12 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr,
> > > >  	ret = dr->probe(dr, dev);
> > > >  	if (ret) {
> > > >  		dev->driver = NULL;
> > > > -		if (dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING)
> > > > +		if ((dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) &&
> > > > +			/* Don't unmap if device is unsupported and
> > > > +			 * driver needs mapped resources.
> > > > +			 */
> > > > +			!(ret > 0 &&
> > > > +				(dr->drv_flags & RTE_PCI_DRV_KEEP_MAPPED_RES)))
> > > >  			rte_pci_unmap_device(dev);
> > > >  	}
> > > >  
> > > > --- a/lib/librte_eal/common/include/rte_pci.h
> > > > +++ b/lib/librte_eal/common/include/rte_pci.h
> > > > +/** Device driver needs to keep mapped resources if unsupported dev detected */
> > > > +#define RTE_PCI_DRV_KEEP_MAPPED_RES 0x0020
> > > 
> > > If I understand well, you want to map resources but not probe it?
> > 
> > Yes.
> > 
> > > Shouldn't it be less hacky to probe it as a (new) null class?
> > 
> > The Vendor and Class ID is same for those device too so we need to map
> > the PCI bar and have access to know the class of device. If you are concerned about
> > if it an common code change, My first version was without common code change.
> > http://dpdk.org/dev/patchwork/patch/24983/
> > 
> > Ferruh would like to have flag scheme, I think it make sense for
> > PMD maintenance perspective.
> 
> Yes
> 
> My idea was to have a new class of device interface to reserve those
> resources, so the probe function would succeed.
> Do you think it would be a good idea?

Currently Kernel PF code creates 12 SRIOV VF devices per port(one VF device has
8 queues === 96 queues(12VFs) for 96 cores(thunderx max cores)), out of that 1 VF
device is _primary_ which mapped to dpdk ethdev port. If probe succeeds for
another 11 VFs then too may ethdev NULL ports show up. We can support
up to 12 ports(12*12 VF = 144 ports). I think, it is not good from
end user perceptive. We already have unsupported device concept in eal device
framework(when probe returns > 0). IMHO, it OK to keep as it for
simplicity.