Re: [PATCH v8 03/14] eal/pci, ethdev: Remove assumption that port will not be detached

[Tetsuya Mukawa <mukawa-AlSX/UN32fvPDbFq/vQRIQ@public.gmane.org>]

Hi Thomas,

I appreciate for your all comments.

On 2015/02/17 9:36, Thomas Monjalon wrote:
> 2015-02-16 13:14, Tetsuya Mukawa:
>> To remove assumption, do like followings.
>>
>> This patch adds "RTE_PCI_DRV_DETACHABLE" to drv_flags of rte_pci_driver
>> structure. The flags indicate the driver can detach devices at runtime.
>> Also, remove assumption that port will not be detached.
>>
>> To remove the assumption.
>> - Add 'attached' member to rte_eth_dev structure.
>>   This member is used for indicating the port is attached, or not.
>> - Add rte_eth_dev_allocate_new_port().
>>   This function is used for allocating new port.
>>
>> v8:
>> - NONE_TRACE is changed to NO_TRACE.
>>   (Thanks to Iremonger, Bernard)
>> v5:
>> - Change parameters of rte_eth_dev_validate_port() to cleanup code.
>> v4:
>> - Use braces with 'for' loop.
>> - Fix indent of 'if' statement.
>>
>> Signed-off-by: Tetsuya Mukawa <mukawa-AlSX/UN32fvPDbFq/vQRIQ@public.gmane.org>
>> ---
>>  lib/librte_eal/common/include/rte_pci.h |   2 +
>>  lib/librte_ether/rte_ethdev.c           | 454 +++++++++++++-------------------
>>  lib/librte_ether/rte_ethdev.h           |   5 +
>>  3 files changed, 186 insertions(+), 275 deletions(-)
>>
>> diff --git a/lib/librte_eal/common/include/rte_pci.h b/lib/librte_eal/common/include/rte_pci.h
>> index 7b48b55..7f2d699 100644
>> --- a/lib/librte_eal/common/include/rte_pci.h
>> +++ b/lib/librte_eal/common/include/rte_pci.h
>> @@ -207,6 +207,8 @@ struct rte_pci_driver {
>>  #define RTE_PCI_DRV_FORCE_UNBIND 0x0004
>>  /** Device driver supports link state interrupt */
>>  #define RTE_PCI_DRV_INTR_LSC	0x0008
>> +/** Device driver supports detaching capability */
>> +#define RTE_PCI_DRV_DETACHABLE	0x0010
>>  
>>  /**< Internal use only - Macro used by pci addr parsing functions **/
>>  #define GET_PCIADDR_FIELD(in, fd, lim, dlm)                   \
>> diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c
>> index ea3a1fb..a79fa5b 100644
>> --- a/lib/librte_ether/rte_ethdev.c
>> +++ b/lib/librte_ether/rte_ethdev.c
>> @@ -175,6 +175,16 @@ enum {
>>  	STAT_QMAP_RX
>>  };
>>  
>> +enum {
>> +	DEV_INVALID = 0,
>> +	DEV_VALID,
>> +};
>> +
>> +enum {
>> +	DEV_DISCONNECTED = 0,
>> +	DEV_CONNECTED
>> +};
> The commit log explains what is an attached port but not what means
> valid or connected.
> These enums should have a comment to explain their usage.

Sure, I will add comment, and fix commit log.

>> +
>>  static inline void
>>  rte_eth_dev_data_alloc(void)
>>  {
>> @@ -201,19 +211,34 @@ rte_eth_dev_allocated(const char *name)
>>  {
>>  	unsigned i;
>>  
>> -	for (i = 0; i < nb_ports; i++) {
>> -		if (strcmp(rte_eth_devices[i].data->name, name) == 0)
>> +	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
>> +		if ((rte_eth_devices[i].attached == DEV_CONNECTED) &&
>> +		    strcmp(rte_eth_devices[i].data->name, name) == 0)
>>  			return &rte_eth_devices[i];
>>  	}
>>  	return NULL;
>>  }
>>  
>> +static uint8_t
>> +rte_eth_dev_allocate_new_port(void)
>> +{
>> +	unsigned i;
>> +
>> +	for (i = 0; i < RTE_MAX_ETHPORTS; i++) {
>> +		if (rte_eth_devices[i].attached == DEV_DISCONNECTED)
>> +			return i;
>> +	}
>> +	return RTE_MAX_ETHPORTS;
>> +}
> This function does not allocate a new port.
> It get the first free port id.

I will change the function name like below.
rte_eth_dev_find_free_port()

> Is uint8_t sill a good size for hotpluggable virtual device ids?

I am not sure it's enough, but uint8_t is widely used in "rte_ethdev.c"
as port id.
If someone reports it doesn't enough, I guess it will be the time to
write a patch to change all uint_8 in one patch.

>> +
>>  struct rte_eth_dev *
>>  rte_eth_dev_allocate(const char *name)
>>  {
>> +	uint8_t port_id;
>>  	struct rte_eth_dev *eth_dev;
>>  
>> -	if (nb_ports == RTE_MAX_ETHPORTS) {
>> +	port_id = rte_eth_dev_allocate_new_port();
>> +	if (port_id == RTE_MAX_ETHPORTS) {
>>  		PMD_DEBUG_TRACE("Reached maximum number of Ethernet ports\n");
>>  		return NULL;
>>  	}
>> @@ -226,10 +251,12 @@ rte_eth_dev_allocate(const char *name)
>>  		return NULL;
>>  	}
>>  
>> -	eth_dev = &rte_eth_devices[nb_ports];
>> -	eth_dev->data = &rte_eth_dev_data[nb_ports];
>> +	eth_dev = &rte_eth_devices[port_id];
>> +	eth_dev->data = &rte_eth_dev_data[port_id];
>>  	snprintf(eth_dev->data->name, sizeof(eth_dev->data->name), "%s", name);
>> -	eth_dev->data->port_id = nb_ports++;
>> +	eth_dev->data->port_id = port_id;
>> +	eth_dev->attached = DEV_CONNECTED;
>> +	nb_ports++;
>>  	return eth_dev;
>>  }
>>  
>> @@ -283,6 +310,7 @@ rte_eth_dev_init(struct rte_pci_driver *pci_drv,
>>  			(unsigned) pci_dev->id.device_id);
>>  	if (rte_eal_process_type() == RTE_PROC_PRIMARY)
>>  		rte_free(eth_dev->data->dev_private);
>> +	eth_dev->attached = DEV_DISCONNECTED;
>>  	nb_ports--;
>>  	return diag;
>>  }
>> @@ -308,10 +336,28 @@ rte_eth_driver_register(struct eth_driver *eth_drv)
>>  	rte_eal_pci_register(&eth_drv->pci_drv);
>>  }
>>  
>> +enum {
>> +	NO_TRACE = 0,
>> +	TRACE
>> +};
> What means this enum?
>
>> +
>> +static int
>> +rte_eth_dev_validate_port(uint8_t port_id, int trace)
>> +{
>> +	if (port_id >= RTE_MAX_ETHPORTS ||
>> +	    rte_eth_devices[port_id].attached != DEV_CONNECTED) {
>> +		if (trace) {
>> +			PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
>> +		}
> It's not common in DPDK to have a parameter to toggle log.
> It should be enabled globally via the log level.

Sure, I will change it.

>> +		return DEV_INVALID;
> It would be simpler to return 0 in a function called "is_valid_port".

I will change the function name.

>> +	} else
>> +		return DEV_VALID;
> So it could 1 (true) here.

Sure, I will change like above, also delete DEV_INVALID and DEV_VALID
definitions.

> [...]
>> --- a/lib/librte_ether/rte_ethdev.h
>> +++ b/lib/librte_ether/rte_ethdev.h
>> @@ -1539,6 +1539,7 @@ struct rte_eth_dev {
>>  	struct eth_dev_ops *dev_ops;    /**< Functions exported by PMD */
>>  	struct rte_pci_device *pci_dev; /**< PCI info. supplied by probing */
>>  	struct rte_eth_dev_cb_list callbacks; /**< User application callbacks */
>> +	uint8_t attached; /**< Flag indicating the port is attached */
>>  };
>>  
>>  struct rte_eth_dev_sriov {
>> @@ -1604,6 +1605,10 @@ extern struct rte_eth_dev rte_eth_devices[];
>>   * initialized by the [matching] Ethernet driver during the PCI probing phase.
>>   * All devices whose port identifier is in the range
>>   * [0,  rte_eth_dev_count() - 1] can be operated on by network applications.
>> + * immediately after invoking rte_eal_init().
>> + * If the application unplugs a port using hotplug function, The enabled port
>> + * numbers may be noncontiguous. In the case, the applications need to manage
>> + * enabled port by themselves.
>>   *
>>   * @return
>>   *   - The total number of usable Ethernet devices.
>>
>