[PATCH 1/2 net] net/mlx4_core: Test interrupts fail if not all comp vectors called request_irq

[PATCH 1/2 net] net/mlx4_core: Test interrupts fail if not all comp vectors called request_irq

[clsoto]

From: Carol L Soto <clsoto@linux.vnet.ibm.com>

Test interrupts fails if not all completion vectors called
request_irq. This case can happen if only mlx4_en loads and
we have more completion vectors than rx rings. 

Fixes: c66fa19c405a ('net/mlx4: Add EQ pool')
Acked-by: Matan Barak <matanb@mellanox.com>
Signed-off-by: Carol L Soto <clsoto@linux.vnet.ibm.com>

---
 drivers/net/ethernet/mellanox/mlx4/eq.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/net/ethernet/mellanox/mlx4/eq.c b/drivers/net/ethernet/mellanox/mlx4/eq.c
index 8e81e53..c344884 100644
--- a/drivers/net/ethernet/mellanox/mlx4/eq.c
+++ b/drivers/net/ethernet/mellanox/mlx4/eq.c
@@ -1364,6 +1364,10 @@ int mlx4_test_interrupts(struct mlx4_dev *dev)
 	 * and performing a NOP command
 	 */
 	for(i = 0; !err && (i < dev->caps.num_comp_vectors); ++i) {
+		/* Make sure request_irq was called */
+		if (!priv->eq_table.eq[i].have_irq)
+			continue;
+
 		/* Temporary use polling for command completions */
 		mlx4_cmd_use_polling(dev);
 
-- 
Can we push this patch to 4.2-stable as the bug was introduced in 4.2-rc1 
1.8.3.1

[PATCH 2/2] Do not set shared_ports when nreq > MAX_MSIX

[clsoto]

From: Carol L Soto <clsoto@linux.vnet.ibm.com>

If we get MAX_MSIX interrupts would like to have each receive ring
with his own msix interrupt line.

Fixes: 9293267a3e2a ('net/mlx4_core: Capping number of requested MSIXs to MAX_MSIX')
Signed-off-by: Carol L Soto <clsoto@linux.vnet.ibm.com>

---
 drivers/net/ethernet/mellanox/mlx4/main.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
index 006757f..f03f513 100644
--- a/drivers/net/ethernet/mellanox/mlx4/main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/main.c
@@ -2673,10 +2673,8 @@ static void mlx4_enable_msi_x(struct mlx4_dev *dev)
 
 		nreq = min_t(int, dev->caps.num_eqs - dev->caps.reserved_eqs,
 			     nreq);
-		if (nreq > MAX_MSIX) {
+		if (nreq > MAX_MSIX)
 			nreq = MAX_MSIX;
-			shared_ports = true;
-		}
 
 		entries = kcalloc(nreq, sizeof *entries, GFP_KERNEL);
 		if (!entries)
-- 
1.8.3.1

Re: [PATCH 2/2] Do not set shared_ports when nreq > MAX_MSIX

[Or Gerlitz]

On Wed, Oct 7, 2015 at 12:27 AM,  <clsoto@linux.vnet.ibm.com> wrote:
> From: Carol L Soto <clsoto@linux.vnet.ibm.com>
>
> If we get MAX_MSIX interrupts would like to have each receive ring
> with his own msix interrupt line.

so 9293267a3e2a  was only partially correct? and/or not fully optimal?
please elaborate more on that in your change log.

>
> Fixes: 9293267a3e2a ('net/mlx4_core: Capping number of requested MSIXs to MAX_MSIX')
> Signed-off-by: Carol L Soto <clsoto@linux.vnet.ibm.com>

Carol, you didn't use net/mlx4: prefix as ask for mlx4 driver patch
titles, so please repost, but before that I'd like to see an ack from
Matan for this patch as well.

Or.

>
> ---
>  drivers/net/ethernet/mellanox/mlx4/main.c | 4 +---
>  1 file changed, 1 insertion(+), 3 deletions(-)
>
> diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
> index 006757f..f03f513 100644
> --- a/drivers/net/ethernet/mellanox/mlx4/main.c
> +++ b/drivers/net/ethernet/mellanox/mlx4/main.c
> @@ -2673,10 +2673,8 @@ static void mlx4_enable_msi_x(struct mlx4_dev *dev)
>
>                 nreq = min_t(int, dev->caps.num_eqs - dev->caps.reserved_eqs,
>                              nreq);
> -               if (nreq > MAX_MSIX) {
> +               if (nreq > MAX_MSIX)
>                         nreq = MAX_MSIX;
> -                       shared_ports = true;
> -               }
>
>                 entries = kcalloc(nreq, sizeof *entries, GFP_KERNEL);
>                 if (!entries)
> --
> 1.8.3.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] Do not set shared_ports when nreq > MAX_MSIX

[Carol Soto]

On 10/6/2015 4:39 PM, Or Gerlitz wrote:
> On Wed, Oct 7, 2015 at 12:27 AM,  <clsoto@linux.vnet.ibm.com> wrote:
>> From: Carol L Soto <clsoto@linux.vnet.ibm.com>
>>
>> If we get MAX_MSIX interrupts would like to have each receive ring
>> with his own msix interrupt line.
> so 9293267a3e2a  was only partially correct? and/or not fully optimal?
> please elaborate more on that in your change log.
just not fully optimal, with commit 9293267a3e2a if I have 64 MSIXs and 
2 ports I can get 8 rings for each port but then the rings will share 
the interrupt lines. For 64 MSIXs we can have each ring with his own 
interrupt line.

>> Fixes: 9293267a3e2a ('net/mlx4_core: Capping number of requested MSIXs to MAX_MSIX')
>> Signed-off-by: Carol L Soto <clsoto@linux.vnet.ibm.com>
> Carol, you didn't use net/mlx4: prefix as ask for mlx4 driver patch
> titles, so please repost, but before that I'd like to see an ack from
> Matan for this patch as well.
Sorry completely missed it. When Matan acks will resend it.
>
> Or.
>
>> ---
>>   drivers/net/ethernet/mellanox/mlx4/main.c | 4 +---
>>   1 file changed, 1 insertion(+), 3 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c b/drivers/net/ethernet/mellanox/mlx4/main.c
>> index 006757f..f03f513 100644
>> --- a/drivers/net/ethernet/mellanox/mlx4/main.c
>> +++ b/drivers/net/ethernet/mellanox/mlx4/main.c
>> @@ -2673,10 +2673,8 @@ static void mlx4_enable_msi_x(struct mlx4_dev *dev)
>>
>>                  nreq = min_t(int, dev->caps.num_eqs - dev->caps.reserved_eqs,
>>                               nreq);
>> -               if (nreq > MAX_MSIX) {
>> +               if (nreq > MAX_MSIX)
>>                          nreq = MAX_MSIX;
>> -                       shared_ports = true;
>> -               }
>>
>>                  entries = kcalloc(nreq, sizeof *entries, GFP_KERNEL);
>>                  if (!entries)
>> --
>> 1.8.3.1
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] Do not set shared_ports when nreq > MAX_MSIX

[Matan Barak]

On 10/7/2015 12:46 AM, Carol Soto wrote:
>
>
> On 10/6/2015 4:39 PM, Or Gerlitz wrote:
>> On Wed, Oct 7, 2015 at 12:27 AM,  <clsoto@linux.vnet.ibm.com> wrote:
>>> From: Carol L Soto <clsoto@linux.vnet.ibm.com>
>>>
>>> If we get MAX_MSIX interrupts would like to have each receive ring
>>> with his own msix interrupt line.
>> so 9293267a3e2a  was only partially correct? and/or not fully optimal?
>> please elaborate more on that in your change log.
> just not fully optimal, with commit 9293267a3e2a if I have 64 MSIXs and
> 2 ports I can get 8 rings for each port but then the rings will share
> the interrupt lines. For 64 MSIXs we can have each ring with his own
> interrupt line.
>
>>> Fixes: 9293267a3e2a ('net/mlx4_core: Capping number of requested
>>> MSIXs to MAX_MSIX')
>>> Signed-off-by: Carol L Soto <clsoto@linux.vnet.ibm.com>
>> Carol, you didn't use net/mlx4: prefix as ask for mlx4 driver patch
>> titles, so please repost, but before that I'd like to see an ack from
>> Matan for this patch as well.
> Sorry completely missed it. When Matan acks will resend it.

The logic seems correct to me. When there are more nreqs than we could 
possibly support (or want to support), there’s no reason to share the 
EQs between the different ports.
Thanks for your fix.

Regards,
Matan

>>
>> Or.
>>
>>> ---
>>>   drivers/net/ethernet/mellanox/mlx4/main.c | 4 +---
>>>   1 file changed, 1 insertion(+), 3 deletions(-)
>>>
>>> diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c
>>> b/drivers/net/ethernet/mellanox/mlx4/main.c
>>> index 006757f..f03f513 100644
>>> --- a/drivers/net/ethernet/mellanox/mlx4/main.c
>>> +++ b/drivers/net/ethernet/mellanox/mlx4/main.c
>>> @@ -2673,10 +2673,8 @@ static void mlx4_enable_msi_x(struct mlx4_dev
>>> *dev)
>>>
>>>                  nreq = min_t(int, dev->caps.num_eqs -
>>> dev->caps.reserved_eqs,
>>>                               nreq);
>>> -               if (nreq > MAX_MSIX) {
>>> +               if (nreq > MAX_MSIX)
>>>                          nreq = MAX_MSIX;
>>> -                       shared_ports = true;
>>> -               }
>>>
>>>                  entries = kcalloc(nreq, sizeof *entries, GFP_KERNEL);
>>>                  if (!entries)
>>> --
>>> 1.8.3.1
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [PATCH 2/2] Do not set shared_ports when nreq > MAX_MSIX

[Matan Barak]

On 10/7/2015 10:25 AM, Matan Barak wrote:
>
>
> On 10/7/2015 12:46 AM, Carol Soto wrote:
>>
>>
>> On 10/6/2015 4:39 PM, Or Gerlitz wrote:
>>> On Wed, Oct 7, 2015 at 12:27 AM,  <clsoto@linux.vnet.ibm.com> wrote:
>>>> From: Carol L Soto <clsoto@linux.vnet.ibm.com>
>>>>
>>>> If we get MAX_MSIX interrupts would like to have each receive ring
>>>> with his own msix interrupt line.
>>> so 9293267a3e2a  was only partially correct? and/or not fully optimal?
>>> please elaborate more on that in your change log.
>> just not fully optimal, with commit 9293267a3e2a if I have 64 MSIXs and
>> 2 ports I can get 8 rings for each port but then the rings will share
>> the interrupt lines. For 64 MSIXs we can have each ring with his own
>> interrupt line.
>>
>>>> Fixes: 9293267a3e2a ('net/mlx4_core: Capping number of requested
>>>> MSIXs to MAX_MSIX')
>>>> Signed-off-by: Carol L Soto <clsoto@linux.vnet.ibm.com>
>>> Carol, you didn't use net/mlx4: prefix as ask for mlx4 driver patch
>>> titles, so please repost, but before that I'd like to see an ack from
>>> Matan for this patch as well.
>> Sorry completely missed it. When Matan acks will resend it.
>
> The logic seems correct to me. When there are more nreqs than we could
> possibly support (or want to support), there’s no reason to share the
> EQs between the different ports.
> Thanks for your fix.
>

Please also clean the shared_ports variable - we don't need this anymore.

> Regards,
> Matan
>
>>>
>>> Or.
>>>
>>>> ---
>>>>   drivers/net/ethernet/mellanox/mlx4/main.c | 4 +---
>>>>   1 file changed, 1 insertion(+), 3 deletions(-)
>>>>
>>>> diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c
>>>> b/drivers/net/ethernet/mellanox/mlx4/main.c
>>>> index 006757f..f03f513 100644
>>>> --- a/drivers/net/ethernet/mellanox/mlx4/main.c
>>>> +++ b/drivers/net/ethernet/mellanox/mlx4/main.c
>>>> @@ -2673,10 +2673,8 @@ static void mlx4_enable_msi_x(struct mlx4_dev
>>>> *dev)
>>>>
>>>>                  nreq = min_t(int, dev->caps.num_eqs -
>>>> dev->caps.reserved_eqs,
>>>>                               nreq);
>>>> -               if (nreq > MAX_MSIX) {
>>>> +               if (nreq > MAX_MSIX)
>>>>                          nreq = MAX_MSIX;
>>>> -                       shared_ports = true;
>>>> -               }
>>>>
>>>>                  entries = kcalloc(nreq, sizeof *entries, GFP_KERNEL);
>>>>                  if (!entries)
>>>> --
>>>> 1.8.3.1
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>

Re: [PATCH 2/2] Do not set shared_ports when nreq > MAX_MSIX

[Carol Soto]

On 10/7/2015 3:08 AM, Matan Barak wrote:
>
>
> On 10/7/2015 10:25 AM, Matan Barak wrote:
>>
>>
>> On 10/7/2015 12:46 AM, Carol Soto wrote:
>>>
>>>
>>> On 10/6/2015 4:39 PM, Or Gerlitz wrote:
>>>> On Wed, Oct 7, 2015 at 12:27 AM, <clsoto@linux.vnet.ibm.com> wrote:
>>>>> From: Carol L Soto <clsoto@linux.vnet.ibm.com>
>>>>>
>>>>> If we get MAX_MSIX interrupts would like to have each receive ring
>>>>> with his own msix interrupt line.
>>>> so 9293267a3e2a  was only partially correct? and/or not fully optimal?
>>>> please elaborate more on that in your change log.
>>> just not fully optimal, with commit 9293267a3e2a if I have 64 MSIXs and
>>> 2 ports I can get 8 rings for each port but then the rings will share
>>> the interrupt lines. For 64 MSIXs we can have each ring with his own
>>> interrupt line.
>>>
>>>>> Fixes: 9293267a3e2a ('net/mlx4_core: Capping number of requested
>>>>> MSIXs to MAX_MSIX')
>>>>> Signed-off-by: Carol L Soto <clsoto@linux.vnet.ibm.com>
>>>> Carol, you didn't use net/mlx4: prefix as ask for mlx4 driver patch
>>>> titles, so please repost, but before that I'd like to see an ack from
>>>> Matan for this patch as well.
>>> Sorry completely missed it. When Matan acks will resend it.
>>
>> The logic seems correct to me. When there are more nreqs than we could
>> possibly support (or want to support), there’s no reason to share the
>> EQs between the different ports.
>> Thanks for your fix.
>>
>
> Please also clean the shared_ports variable - we don't need this anymore.
>
Thanks for the feedback, will resend.
Carol
>> Regards,
>> Matan
>>
>>>>
>>>> Or.
>>>>
>>>>> ---
>>>>>   drivers/net/ethernet/mellanox/mlx4/main.c | 4 +---
>>>>>   1 file changed, 1 insertion(+), 3 deletions(-)
>>>>>
>>>>> diff --git a/drivers/net/ethernet/mellanox/mlx4/main.c
>>>>> b/drivers/net/ethernet/mellanox/mlx4/main.c
>>>>> index 006757f..f03f513 100644
>>>>> --- a/drivers/net/ethernet/mellanox/mlx4/main.c
>>>>> +++ b/drivers/net/ethernet/mellanox/mlx4/main.c
>>>>> @@ -2673,10 +2673,8 @@ static void mlx4_enable_msi_x(struct mlx4_dev
>>>>> *dev)
>>>>>
>>>>>                  nreq = min_t(int, dev->caps.num_eqs -
>>>>> dev->caps.reserved_eqs,
>>>>>                               nreq);
>>>>> -               if (nreq > MAX_MSIX) {
>>>>> +               if (nreq > MAX_MSIX)
>>>>>                          nreq = MAX_MSIX;
>>>>> -                       shared_ports = true;
>>>>> -               }
>>>>>
>>>>>                  entries = kcalloc(nreq, sizeof *entries, 
>>>>> GFP_KERNEL);
>>>>>                  if (!entries)
>>>>> -- 
>>>>> 1.8.3.1
>>>>>
>>>>> -- 
>>>>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>

Re: [PATCH 2/2] Do not set shared_ports when nreq > MAX_MSIX

[David Miller]

From: Carol Soto <clsoto@linux.vnet.ibm.com>
Date: Tue, 6 Oct 2015 16:46:06 -0500

> 
> 
> On 10/6/2015 4:39 PM, Or Gerlitz wrote:
>> On Wed, Oct 7, 2015 at 12:27 AM,  <clsoto@linux.vnet.ibm.com> wrote:
>>> From: Carol L Soto <clsoto@linux.vnet.ibm.com>
>>>
>>> If we get MAX_MSIX interrupts would like to have each receive ring
>>> with his own msix interrupt line.
>> so 9293267a3e2a  was only partially correct? and/or not fully optimal?
>> please elaborate more on that in your change log.
> just not fully optimal, with commit 9293267a3e2a if I have 64 MSIXs
> and 2 ports I can get 8 rings for each port but then the rings will
> share the interrupt lines. For 64 MSIXs we can have each ring with his
> own interrupt line.
> 
>>> Fixes: 9293267a3e2a ('net/mlx4_core: Capping number of requested MSIXs
>>> to MAX_MSIX')
>>> Signed-off-by: Carol L Soto <clsoto@linux.vnet.ibm.com>
>> Carol, you didn't use net/mlx4: prefix as ask for mlx4 driver patch
>> titles, so please repost, but before that I'd like to see an ack from
>> Matan for this patch as well.
> Sorry completely missed it. When Matan acks will resend it.

Also please fix you subject to have a proper "mlx4: " subsystem prefix.

Re: [PATCH 2/2] Do not set shared_ports when nreq > MAX_MSIX

[Or Gerlitz]

On 10/8/2015 2:54 PM, David Miller wrote:
> Also please fix you subject to have a proper "mlx4: " subsystem prefix. 

Carol, I have the patch and will submit it with other fixes that we have.

Or.

Re: [PATCH 1/2 net] net/mlx4_core: Test interrupts fail if not all comp vectors called request_irq

[Or Gerlitz]

On Wed, Oct 7, 2015 at 12:27 AM,  <clsoto@linux.vnet.ibm.com> wrote:
> From: Carol L Soto <clsoto@linux.vnet.ibm.com>
>
> Test interrupts fails if not all completion vectors called
> request_irq. This case can happen if only mlx4_en loads and
> we have more completion vectors than rx rings.
>
> Fixes: c66fa19c405a ('net/mlx4: Add EQ pool')
> Acked-by: Matan Barak <matanb@mellanox.com>
> Signed-off-by: Carol L Soto <clsoto@linux.vnet.ibm.com>

Here too, I will submit this with other fixes we have, thanks a lot.