[PATCH] ntb_transport: fix qp count bug

[PATCH] ntb_transport: fix qp count bug

[Logan Gunthorpe]

In cases where there are more mw's than spads/2-2, the mw count gets
reduced to match the limitation. ntb_transport also tries to ensure that
there are fewer qps than mws but uses the full mw count instead of
the reduced one. When this happens, the math in
'ntb_transport_setup_qp_mw' will get confused and result in a kernel
paging request bug.

This patch fixes the bug by reducing qp_count to the reduced mw count
instead of the full mw count.

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Cc: Jon Mason <jdmason@kudzu.us>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Allen Hubbe <Allen.Hubbe@emc.com>
---
 drivers/ntb/ntb_transport.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/ntb/ntb_transport.c b/drivers/ntb/ntb_transport.c
index 02ca45fdd892..0a778d2cab94 100644
--- a/drivers/ntb/ntb_transport.c
+++ b/drivers/ntb/ntb_transport.c
@@ -1128,8 +1128,8 @@ static int ntb_transport_probe(struct ntb_client *self, struct ntb_dev *ndev)
 	qp_count = ilog2(qp_bitmap);
 	if (max_num_clients && max_num_clients < qp_count)
 		qp_count = max_num_clients;
-	else if (mw_count < qp_count)
-		qp_count = mw_count;
+	else if (nt->mw_count < qp_count)
+		qp_count = nt->mw_count;
 
 	qp_bitmap &= BIT_ULL(qp_count) - 1;
 
-- 
2.11.0

RE: [PATCH] ntb_transport: fix qp count bug

[Allen Hubbe]

From: Logan Gunthorpe
> In cases where there are more mw's than spads/2-2, the mw count gets
> reduced to match the limitation. ntb_transport also tries to ensure that
> there are fewer qps than mws but uses the full mw count instead of
> the reduced one. When this happens, the math in
> 'ntb_transport_setup_qp_mw' will get confused and result in a kernel
> paging request bug.
> 
> This patch fixes the bug by reducing qp_count to the reduced mw count
> instead of the full mw count.
> 
> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> Cc: Jon Mason <jdmason@kudzu.us>
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Allen Hubbe <Allen.Hubbe@emc.com>

Acked-by: Allen Hubbe <Allen.Hubbe@dell.com>

> ---
>  drivers/ntb/ntb_transport.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/ntb/ntb_transport.c b/drivers/ntb/ntb_transport.c
> index 02ca45fdd892..0a778d2cab94 100644
> --- a/drivers/ntb/ntb_transport.c
> +++ b/drivers/ntb/ntb_transport.c
> @@ -1128,8 +1128,8 @@ static int ntb_transport_probe(struct ntb_client *self, struct
> ntb_dev *ndev)
>  	qp_count = ilog2(qp_bitmap);
>  	if (max_num_clients && max_num_clients < qp_count)
>  		qp_count = max_num_clients;
> -	else if (mw_count < qp_count)
> -		qp_count = mw_count;
> +	else if (nt->mw_count < qp_count)
> +		qp_count = nt->mw_count;
> 
>  	qp_bitmap &= BIT_ULL(qp_count) - 1;
> 
> --
> 2.11.0

Re: [PATCH] ntb_transport: fix qp count bug

[Jon Mason]

On Tue, Jun 06, 2017 at 09:08:47AM -0400, Allen Hubbe wrote:
> From: Logan Gunthorpe
> > In cases where there are more mw's than spads/2-2, the mw count gets
> > reduced to match the limitation. ntb_transport also tries to ensure that
> > there are fewer qps than mws but uses the full mw count instead of
> > the reduced one. When this happens, the math in
> > 'ntb_transport_setup_qp_mw' will get confused and result in a kernel
> > paging request bug.
> > 
> > This patch fixes the bug by reducing qp_count to the reduced mw count
> > instead of the full mw count.
> > 
> > Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> > Cc: Jon Mason <jdmason@kudzu.us>
> > Cc: Dave Jiang <dave.jiang@intel.com>
> > Cc: Allen Hubbe <Allen.Hubbe@emc.com>
> 
> Acked-by: Allen Hubbe <Allen.Hubbe@dell.com>

Adding to my ntb branch with the following line:
Fixes: e26a5843f7f5 ("NTB: Split ntb_hw_intel and ntb_transport drivers")

This enables the stable kernels it is a bug fix and to pick it up.

Thanks,
Jon
> 
> > ---
> >  drivers/ntb/ntb_transport.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/ntb/ntb_transport.c b/drivers/ntb/ntb_transport.c
> > index 02ca45fdd892..0a778d2cab94 100644
> > --- a/drivers/ntb/ntb_transport.c
> > +++ b/drivers/ntb/ntb_transport.c
> > @@ -1128,8 +1128,8 @@ static int ntb_transport_probe(struct ntb_client *self, struct
> > ntb_dev *ndev)
> >  	qp_count = ilog2(qp_bitmap);
> >  	if (max_num_clients && max_num_clients < qp_count)
> >  		qp_count = max_num_clients;
> > -	else if (mw_count < qp_count)
> > -		qp_count = mw_count;
> > +	else if (nt->mw_count < qp_count)
> > +		qp_count = nt->mw_count;
> > 
> >  	qp_bitmap &= BIT_ULL(qp_count) - 1;
> > 
> > --
> > 2.11.0
> 
> 

[PATCH] ntb_netdev: set the net_device's parent

[Logan Gunthorpe]

At present, ntb_netdev devices end up under /sys/devices/virtual/net
completely unconnected to the ntb trees below them. This patch sets the
parent of the net_device (using SET_NETDEV_DEV) to the client_dev
device. This results in a better connected sysfs path for the network
device:

/sys/devices/pci0000:00/0000:00:03.0/0000:03:00.1/0000:03:00.1/ntb_netdev0/net/eth2

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Cc: Jon Mason <jdmason@kudzu.us>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Allen Hubbe <Allen.Hubbe@emc.com>
---
 drivers/net/ntb_netdev.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/ntb_netdev.c b/drivers/net/ntb_netdev.c
index 4daf3d0926a8..0250aa9ae2cb 100644
--- a/drivers/net/ntb_netdev.c
+++ b/drivers/net/ntb_netdev.c
@@ -418,6 +418,8 @@ static int ntb_netdev_probe(struct device *client_dev)
 	if (!ndev)
 		return -ENOMEM;
 
+	SET_NETDEV_DEV(ndev, client_dev);
+
 	dev = netdev_priv(ndev);
 	dev->ndev = ndev;
 	dev->pdev = pdev;
-- 
2.11.0

[PATCH] ntb_transport: fix qp count bug

[Logan Gunthorpe]

In cases where there are more mw's than spads/2-2, the mw count gets
reduced to match the limitation. ntb_transport also tries to ensure that
there are fewer qps than mws but uses the full mw count instead of
the reduced one. When this happens, the math in
'ntb_transport_setup_qp_mw' will get confused and result in a kernel
paging request bug.

This patch fixes the bug by reducing qp_count to the reduced mw count
instead of the full mw count.

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Cc: Jon Mason <jdmason@kudzu.us>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Allen Hubbe <Allen.Hubbe@emc.com>
---
 drivers/ntb/ntb_transport.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/ntb/ntb_transport.c b/drivers/ntb/ntb_transport.c
index 02ca45fdd892..0a778d2cab94 100644
--- a/drivers/ntb/ntb_transport.c
+++ b/drivers/ntb/ntb_transport.c
@@ -1128,8 +1128,8 @@ static int ntb_transport_probe(struct ntb_client *self, struct ntb_dev *ndev)
 	qp_count = ilog2(qp_bitmap);
 	if (max_num_clients && max_num_clients < qp_count)
 		qp_count = max_num_clients;
-	else if (mw_count < qp_count)
-		qp_count = mw_count;
+	else if (nt->mw_count < qp_count)
+		qp_count = nt->mw_count;
 
 	qp_bitmap &= BIT_ULL(qp_count) - 1;
 
-- 
2.11.0

Re: [PATCH] ntb_netdev: set the net_device's parent

[Logan Gunthorpe]

Sorry ignore this. I sent an old patch :(

Logan

On 08/12/17 05:01 PM, Logan Gunthorpe wrote:
> At present, ntb_netdev devices end up under /sys/devices/virtual/net
> completely unconnected to the ntb trees below them. This patch sets the
> parent of the net_device (using SET_NETDEV_DEV) to the client_dev
> device. This results in a better connected sysfs path for the network
> device:
> 
> /sys/devices/pci0000:00/0000:00:03.0/0000:03:00.1/0000:03:00.1/ntb_netdev0/net/eth2
> 
> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> Cc: Jon Mason <jdmason@kudzu.us>
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Allen Hubbe <Allen.Hubbe@emc.com>
> ---
>   drivers/net/ntb_netdev.c | 2 ++
>   1 file changed, 2 insertions(+)
> 
> diff --git a/drivers/net/ntb_netdev.c b/drivers/net/ntb_netdev.c
> index 4daf3d0926a8..0250aa9ae2cb 100644
> --- a/drivers/net/ntb_netdev.c
> +++ b/drivers/net/ntb_netdev.c
> @@ -418,6 +418,8 @@ static int ntb_netdev_probe(struct device *client_dev)
>   	if (!ndev)
>   		return -ENOMEM;
>   
> +	SET_NETDEV_DEV(ndev, client_dev);
> +
>   	dev = netdev_priv(ndev);
>   	dev->ndev = ndev;
>   	dev->pdev = pdev;
> 

Re: [PATCH] ntb_transport: fix qp count bug

[Logan Gunthorpe]

Sorry, ignore this. I sent an old patch.

Logan

On 08/12/17 05:01 PM, Logan Gunthorpe wrote:
> In cases where there are more mw's than spads/2-2, the mw count gets
> reduced to match the limitation. ntb_transport also tries to ensure that
> there are fewer qps than mws but uses the full mw count instead of
> the reduced one. When this happens, the math in
> 'ntb_transport_setup_qp_mw' will get confused and result in a kernel
> paging request bug.
> 
> This patch fixes the bug by reducing qp_count to the reduced mw count
> instead of the full mw count.
> 
> Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
> Cc: Jon Mason <jdmason@kudzu.us>
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Allen Hubbe <Allen.Hubbe@emc.com>
> ---
>   drivers/ntb/ntb_transport.c | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/ntb/ntb_transport.c b/drivers/ntb/ntb_transport.c
> index 02ca45fdd892..0a778d2cab94 100644
> --- a/drivers/ntb/ntb_transport.c
> +++ b/drivers/ntb/ntb_transport.c
> @@ -1128,8 +1128,8 @@ static int ntb_transport_probe(struct ntb_client *self, struct ntb_dev *ndev)
>   	qp_count = ilog2(qp_bitmap);
>   	if (max_num_clients && max_num_clients < qp_count)
>   		qp_count = max_num_clients;
> -	else if (mw_count < qp_count)
> -		qp_count = mw_count;
> +	else if (nt->mw_count < qp_count)
> +		qp_count = nt->mw_count;
>   
>   	qp_bitmap &= BIT_ULL(qp_count) - 1;
>   
>