[PATCH net 0/2] ibmvnic: Bug fixes for queue descriptor processing

[PATCH net 0/2] ibmvnic: Bug fixes for queue descriptor processing

[Thomas Falcon]

This series resolves a few issues in the ibmvnic driver's
RX buffer and TX completion processing. The first patch
includes memory barriers to synchronize queue descriptor
reads. The second patch fixes a memory leak that could
occur if the device returns a TX completion with an error
code in the descriptor, in which case the respective socket
buffer and other relevant data structures may not be freed
or updated properly.

Thomas Falcon (2):
  ibmvnic: Ensure that SCRQ entry reads are correctly ordered
  ibmvnic: Fix TX completion error handling

 drivers/net/ethernet/ibm/ibmvnic.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

-- 
1.8.3.1

[PATCH net 1/2] ibmvnic: Ensure that SCRQ entry reads are correctly ordered

[Thomas Falcon]

Ensure that received Subordinate Command-Response Queue (SCRQ)
entries are properly read in order by the driver. These queues
are used in the ibmvnic device to process RX buffer and TX completion
descriptors. dma_rmb barriers have been added after checking for a
pending descriptor to ensure the correct descriptor entry is checked
and after reading the SCRQ descriptor to ensure the entire
descriptor is read before processing.

Fixes: 032c5e828 ("Driver for IBM System i/p VNIC protocol")
Signed-off-by: Thomas Falcon <tlfalcon@linux.ibm.com>
---
 drivers/net/ethernet/ibm/ibmvnic.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c
index 2aa40b2..489ed5e 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -2403,6 +2403,8 @@ static int ibmvnic_poll(struct napi_struct *napi, int budget)
 
 		if (!pending_scrq(adapter, adapter->rx_scrq[scrq_num]))
 			break;
+		/* ensure that we do not prematurely exit the polling loop */
+		dma_rmb();
 		next = ibmvnic_next_scrq(adapter, adapter->rx_scrq[scrq_num]);
 		rx_buff =
 		    (struct ibmvnic_rx_buff *)be64_to_cpu(next->
@@ -3098,6 +3100,9 @@ static int ibmvnic_complete_tx(struct ibmvnic_adapter *adapter,
 		unsigned int pool = scrq->pool_index;
 		int num_entries = 0;
 
+		/* ensure that the correct descriptor entry is read */
+		dma_rmb();
+
 		next = ibmvnic_next_scrq(adapter, scrq);
 		for (i = 0; i < next->tx_comp.num_comps; i++) {
 			if (next->tx_comp.rcs[i]) {
@@ -3498,6 +3503,9 @@ static union sub_crq *ibmvnic_next_scrq(struct ibmvnic_adapter *adapter,
 	}
 	spin_unlock_irqrestore(&scrq->lock, flags);
 
+	/* ensure that the entire SCRQ descriptor is read */
+	dma_rmb();
+
 	return entry;
 }
 
-- 
1.8.3.1

Re: [PATCH net 1/2] ibmvnic: Ensure that SCRQ entry reads are correctly ordered

[Michael Ellerman]

Thomas Falcon <tlfalcon@linux.ibm.com> writes:
> Ensure that received Subordinate Command-Response Queue (SCRQ)
> entries are properly read in order by the driver. These queues
> are used in the ibmvnic device to process RX buffer and TX completion
> descriptors. dma_rmb barriers have been added after checking for a
> pending descriptor to ensure the correct descriptor entry is checked
> and after reading the SCRQ descriptor to ensure the entire
> descriptor is read before processing.
>
> Fixes: 032c5e828 ("Driver for IBM System i/p VNIC protocol")
> Signed-off-by: Thomas Falcon <tlfalcon@linux.ibm.com>
> ---
>  drivers/net/ethernet/ibm/ibmvnic.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c
> index 2aa40b2..489ed5e 100644
> --- a/drivers/net/ethernet/ibm/ibmvnic.c
> +++ b/drivers/net/ethernet/ibm/ibmvnic.c
> @@ -2403,6 +2403,8 @@ static int ibmvnic_poll(struct napi_struct *napi, int budget)
>  
>  		if (!pending_scrq(adapter, adapter->rx_scrq[scrq_num]))
>  			break;
> +		/* ensure that we do not prematurely exit the polling loop */
> +		dma_rmb();

I'd be happier if these comments were more specific about which read(s)
they are ordering vs which other read(s).

I'm sure it's obvious to you, but it may not be to a future author,
and/or after the code has been refactored over time.

>  		next = ibmvnic_next_scrq(adapter, adapter->rx_scrq[scrq_num]);
>  		rx_buff =
>  		    (struct ibmvnic_rx_buff *)be64_to_cpu(next->
> @@ -3098,6 +3100,9 @@ static int ibmvnic_complete_tx(struct ibmvnic_adapter *adapter,
>  		unsigned int pool = scrq->pool_index;
>  		int num_entries = 0;
>  
> +		/* ensure that the correct descriptor entry is read */
> +		dma_rmb();
> +
>  		next = ibmvnic_next_scrq(adapter, scrq);
>  		for (i = 0; i < next->tx_comp.num_comps; i++) {
>  			if (next->tx_comp.rcs[i]) {
> @@ -3498,6 +3503,9 @@ static union sub_crq *ibmvnic_next_scrq(struct ibmvnic_adapter *adapter,
>  	}
>  	spin_unlock_irqrestore(&scrq->lock, flags);
>  
> +	/* ensure that the entire SCRQ descriptor is read */
> +	dma_rmb();
> +
>  	return entry;
>  }

cheers

Re: [PATCH net 1/2] ibmvnic: Ensure that SCRQ entry reads are correctly ordered

[Thomas Falcon]

On 11/24/20 11:43 PM, Michael Ellerman wrote:
> Thomas Falcon <tlfalcon@linux.ibm.com> writes:
>> Ensure that received Subordinate Command-Response Queue (SCRQ)
>> entries are properly read in order by the driver. These queues
>> are used in the ibmvnic device to process RX buffer and TX completion
>> descriptors. dma_rmb barriers have been added after checking for a
>> pending descriptor to ensure the correct descriptor entry is checked
>> and after reading the SCRQ descriptor to ensure the entire
>> descriptor is read before processing.
>>
>> Fixes: 032c5e828 ("Driver for IBM System i/p VNIC protocol")
>> Signed-off-by: Thomas Falcon <tlfalcon@linux.ibm.com>
>> ---
>>   drivers/net/ethernet/ibm/ibmvnic.c | 8 ++++++++
>>   1 file changed, 8 insertions(+)
>>
>> diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c
>> index 2aa40b2..489ed5e 100644
>> --- a/drivers/net/ethernet/ibm/ibmvnic.c
>> +++ b/drivers/net/ethernet/ibm/ibmvnic.c
>> @@ -2403,6 +2403,8 @@ static int ibmvnic_poll(struct napi_struct *napi, int budget)
>>   
>>   		if (!pending_scrq(adapter, adapter->rx_scrq[scrq_num]))
>>   			break;
>> +		/* ensure that we do not prematurely exit the polling loop */
>> +		dma_rmb();
> I'd be happier if these comments were more specific about which read(s)
> they are ordering vs which other read(s).
>
> I'm sure it's obvious to you, but it may not be to a future author,
> and/or after the code has been refactored over time.

Thank you for reviewing! I will submit a v2 soon with clearer comments 
on the reads being ordered here.

Thanks,

Tom


>
>>   		next = ibmvnic_next_scrq(adapter, adapter->rx_scrq[scrq_num]);
>>   		rx_buff =
>>   		    (struct ibmvnic_rx_buff *)be64_to_cpu(next->
>> @@ -3098,6 +3100,9 @@ static int ibmvnic_complete_tx(struct ibmvnic_adapter *adapter,
>>   		unsigned int pool = scrq->pool_index;
>>   		int num_entries = 0;
>>   
>> +		/* ensure that the correct descriptor entry is read */
>> +		dma_rmb();
>> +
>>   		next = ibmvnic_next_scrq(adapter, scrq);
>>   		for (i = 0; i < next->tx_comp.num_comps; i++) {
>>   			if (next->tx_comp.rcs[i]) {
>> @@ -3498,6 +3503,9 @@ static union sub_crq *ibmvnic_next_scrq(struct ibmvnic_adapter *adapter,
>>   	}
>>   	spin_unlock_irqrestore(&scrq->lock, flags);
>>   
>> +	/* ensure that the entire SCRQ descriptor is read */
>> +	dma_rmb();
>> +
>>   	return entry;
>>   }
> cheers

Re: [PATCH net 1/2] ibmvnic: Ensure that SCRQ entry reads are correctly ordered

[Michael Ellerman]

Thomas Falcon <tlfalcon@linux.ibm.com> writes:
> On 11/24/20 11:43 PM, Michael Ellerman wrote:
>> Thomas Falcon <tlfalcon@linux.ibm.com> writes:
>>> Ensure that received Subordinate Command-Response Queue (SCRQ)
>>> entries are properly read in order by the driver. These queues
>>> are used in the ibmvnic device to process RX buffer and TX completion
>>> descriptors. dma_rmb barriers have been added after checking for a
>>> pending descriptor to ensure the correct descriptor entry is checked
>>> and after reading the SCRQ descriptor to ensure the entire
>>> descriptor is read before processing.
>>>
>>> Fixes: 032c5e828 ("Driver for IBM System i/p VNIC protocol")
>>> Signed-off-by: Thomas Falcon <tlfalcon@linux.ibm.com>
>>> ---
>>>   drivers/net/ethernet/ibm/ibmvnic.c | 8 ++++++++
>>>   1 file changed, 8 insertions(+)
>>>
>>> diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c
>>> index 2aa40b2..489ed5e 100644
>>> --- a/drivers/net/ethernet/ibm/ibmvnic.c
>>> +++ b/drivers/net/ethernet/ibm/ibmvnic.c
>>> @@ -2403,6 +2403,8 @@ static int ibmvnic_poll(struct napi_struct *napi, int budget)
>>>   
>>>   		if (!pending_scrq(adapter, adapter->rx_scrq[scrq_num]))
>>>   			break;
>>> +		/* ensure that we do not prematurely exit the polling loop */
>>> +		dma_rmb();
>> I'd be happier if these comments were more specific about which read(s)
>> they are ordering vs which other read(s).
>>
>> I'm sure it's obvious to you, but it may not be to a future author,
>> and/or after the code has been refactored over time.
>
> Thank you for reviewing! I will submit a v2 soon with clearer comments 
> on the reads being ordered here.

Thanks.

cheers

[PATCH net 2/2] ibmvnic: Fix TX completion error handling

[Thomas Falcon]

TX completions received with an error return code are not
being processed properly. When an error code is seen, do not
proceed to the next completion before cleaning up the existing
entry's data structures.

Fixes: 032c5e828 ("Driver for IBM System i/p VNIC protocol")
Signed-off-by: Thomas Falcon <tlfalcon@linux.ibm.com>
---
 drivers/net/ethernet/ibm/ibmvnic.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c
index 489ed5e..7097bcb 100644
--- a/drivers/net/ethernet/ibm/ibmvnic.c
+++ b/drivers/net/ethernet/ibm/ibmvnic.c
@@ -3105,11 +3105,9 @@ static int ibmvnic_complete_tx(struct ibmvnic_adapter *adapter,
 
 		next = ibmvnic_next_scrq(adapter, scrq);
 		for (i = 0; i < next->tx_comp.num_comps; i++) {
-			if (next->tx_comp.rcs[i]) {
+			if (next->tx_comp.rcs[i])
 				dev_err(dev, "tx error %x\n",
 					next->tx_comp.rcs[i]);
-				continue;
-			}
 			index = be32_to_cpu(next->tx_comp.correlators[i]);
 			if (index & IBMVNIC_TSO_POOL_MASK) {
 				tx_pool = &adapter->tso_pool[pool];
-- 
1.8.3.1