* [PATCH 2/5] mlx5: Remove a set-but-not-used variable
From: Bart Van Assche @ 2016-12-06 1:18 UTC (permalink / raw)
To: Doug Ledford
Cc: Eli Cohen, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <9ff07804-492c-c67a-e729-b31e0f863027-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
This has been detected by building the mlx5 driver with W=1.
Signed-off-by: Bart Van Assche <bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
Cc: Eli Cohen <eli-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
drivers/infiniband/hw/mlx5/qp.c | 7 -------
1 file changed, 7 deletions(-)
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index d1e921816bfe..b2c6c13d44ac 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -2577,7 +2577,6 @@ static int __mlx5_ib_modify_qp(struct ib_qp *ibqp,
struct mlx5_ib_port *mibport = NULL;
enum mlx5_qp_state mlx5_cur, mlx5_new;
enum mlx5_qp_optpar optpar;
- int sqd_event;
int mlx5_st;
int err;
u16 op;
@@ -2724,12 +2723,6 @@ static int __mlx5_ib_modify_qp(struct ib_qp *ibqp,
if (qp->rq.wqe_cnt && cur_state == IB_QPS_RESET && new_state == IB_QPS_INIT)
context->db_rec_addr = cpu_to_be64(qp->db.dma);
- if (cur_state == IB_QPS_RTS && new_state == IB_QPS_SQD &&
- attr_mask & IB_QP_EN_SQD_ASYNC_NOTIFY && attr->en_sqd_async_notify)
- sqd_event = 1;
- else
- sqd_event = 0;
-
if (cur_state == IB_QPS_RESET && new_state == IB_QPS_INIT) {
u8 port_num = (attr_mask & IB_QP_PORT ? attr->port_num :
qp->port) - 1;
--
2.11.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 1/5] mlx5: Use NULL instead of 0 to represent a pointer
From: Bart Van Assche @ 2016-12-06 1:18 UTC (permalink / raw)
To: Doug Ledford
Cc: Eli Cohen, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <9ff07804-492c-c67a-e729-b31e0f863027-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
Detected by sparse.
Signed-off-by: Bart Van Assche <bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
Cc: Eli Cohen <eli-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
drivers/infiniband/hw/mlx5/main.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 32b09f059c84..abd200e3e299 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -127,7 +127,7 @@ static int mlx5_netdev_event(struct notifier_block *this,
if ((upper == ndev || (!upper && ndev == ibdev->roce.netdev))
&& ibdev->ib_active) {
- struct ib_event ibev = {0};
+ struct ib_event ibev = { NULL };
ibev.device = &ibdev->ib_dev;
ibev.event = (event == NETDEV_UP) ?
--
2.11.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 0/5] mlx5 patches for kernel v4.10
From: Bart Van Assche @ 2016-12-06 1:17 UTC (permalink / raw)
To: Doug Ledford
Cc: Eli Cohen, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Hello Doug,
The patches in this series is what I came up with while analyzing the
build output for the mlx5 driver and by analyzing an error message
reported by the mlx5 driver. It would be appreciated if these patches
would be considered for inclusion in kernel v4.10.
Thanks,
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH] IB/hfi1: Define platform_config_table_limits once
From: Bart Van Assche @ 2016-12-06 0:48 UTC (permalink / raw)
To: Doug Ledford
Cc: Dennis Dalessandro, Dean Luick,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Defining static data structures in a header file is wrong because
this causes the data structure to be instantiated once in every .c
file it is included in. Hence move the definition of a static
array from a header file into the only .c file in which it is used.
Signed-off-by: Bart Van Assche <bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
Cc: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Cc: Dean Luick <dean.luick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/hw/hfi1/firmware.c | 10 ++++++++++
drivers/infiniband/hw/hfi1/platform.h | 10 ----------
2 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/firmware.c b/drivers/infiniband/hw/hfi1/firmware.c
index 13db8eb4f4ec..acbc261b1858 100644
--- a/drivers/infiniband/hw/hfi1/firmware.c
+++ b/drivers/infiniband/hw/hfi1/firmware.c
@@ -239,6 +239,16 @@ static const u8 all_fabric_serdes_broadcast = 0xe1;
const u8 pcie_serdes_broadcast[2] = { 0xe2, 0xe3 };
static const u8 all_pcie_serdes_broadcast = 0xe0;
+static const u32 platform_config_table_limits[PLATFORM_CONFIG_TABLE_MAX] = {
+ 0,
+ SYSTEM_TABLE_MAX,
+ PORT_TABLE_MAX,
+ RX_PRESET_TABLE_MAX,
+ TX_PRESET_TABLE_MAX,
+ QSFP_ATTEN_TABLE_MAX,
+ VARIABLE_SETTINGS_TABLE_MAX
+};
+
/* forwards */
static void dispose_one_firmware(struct firmware_details *fdet);
static int load_fabric_serdes_firmware(struct hfi1_devdata *dd,
diff --git a/drivers/infiniband/hw/hfi1/platform.h b/drivers/infiniband/hw/hfi1/platform.h
index e2c21613c326..ad46dd6b4b3d 100644
--- a/drivers/infiniband/hw/hfi1/platform.h
+++ b/drivers/infiniband/hw/hfi1/platform.h
@@ -168,16 +168,6 @@ struct platform_config_cache {
struct platform_config_data config_tables[PLATFORM_CONFIG_TABLE_MAX];
};
-static const u32 platform_config_table_limits[PLATFORM_CONFIG_TABLE_MAX] = {
- 0,
- SYSTEM_TABLE_MAX,
- PORT_TABLE_MAX,
- RX_PRESET_TABLE_MAX,
- TX_PRESET_TABLE_MAX,
- QSFP_ATTEN_TABLE_MAX,
- VARIABLE_SETTINGS_TABLE_MAX
-};
-
/* This section defines default values and encodings for the
* fields defined for each table above
*/
--
2.11.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: [PATCH V1 rdma-core 0/2] Optimize RoCE address handle creation for userspace
From: Jason Gunthorpe @ 2016-12-05 23:44 UTC (permalink / raw)
To: Yishai Hadas
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, Yishai Hadas,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, monis-VPRAkNaXOzVWk0Htik3J/w,
majd-VPRAkNaXOzVWk0Htik3J/w
In-Reply-To: <7b8e90ac-c653-6a54-846d-75adeeb8059f-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
On Tue, Dec 06, 2016 at 12:01:13AM +0200, Yishai Hadas wrote:
> >I'm not sure where we are in the release cycle right now.. Doug - if
> >we didn't make a release with the 1.3 symbol tag (this was created
> >just before rdma-core) then this should go into 1.3 when it gets
> >merged..
>
> Jason,
> You pointed to use 1.4, the series followed that.
Yes, but I realized we may not have made a ibverbs release before
starting rdma-core.
> Doug,
> Can you please comment here ? we should have a clear definition what defined
> to be a "release" which requires a new number.
> In case we want to use 1.3 please let me know if I need to re-spin or you
> will change as part of the merge.
Doug/Leon/You have to check this stuff before merging any series that
touches a map file, beacuse it depends on when in the cycle the patch
gets merged.
After the first rdma-core release is done we should denote the map
stanzas with a comment indicating which release number they are for so
it is easier to keep track.
eg we are nearing the release for 4.9, so presumably Doug will tag a
rdma-core relase soon. If this series does not make that cutoff 1.4
would be correct.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH V1 rdma-core 0/2] Optimize RoCE address handle creation for userspace
From: Yishai Hadas @ 2016-12-05 22:01 UTC (permalink / raw)
To: Jason Gunthorpe, dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: Yishai Hadas, linux-rdma-u79uwXL29TY76Z2rM5mHXA,
monis-VPRAkNaXOzVWk0Htik3J/w, majd-VPRAkNaXOzVWk0Htik3J/w
In-Reply-To: <20161205170702.GA27306-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
On 12/5/2016 7:07 PM, Jason Gunthorpe wrote:
> On Sun, Dec 04, 2016 at 05:53:24PM +0200, Yishai Hadas wrote:
>
>> Test results:
>> Upon testing the above use case the provider couldn't be loaded as expected,
>> "libibverbs: Warning: couldn't load driver 'libmlx5-rdmav2.so':
>> libmlx5-rdmav2.so: cannot open shared object file: No such file or directory"
>
> The dynmic linker didn't print something too? That is surprising..
No more output, I was surprised as well about the error message, that's
why I used strace to follow the system calls. Looking at the man page of
dlopen can see that upon an error the most recent error is returned
which was in that case ENOENT as the linker continued searching in other
locations.
Just run with LD_LIBRARY_PATH=/usr/lib64/libibverbs which based on man
page will cause the linker to look only in that location, I get below
error as we expect it.
libibverbs: Warning: couldn't load driver 'libmlx5-rdmav2.so':
/usr/lib64/libibverbs/libmlx5-rdmav2.so: symbol ibv_cmd_create_ah,
version IBVERBS_1.0 not defined in file libibverbs.so.1 with link time
reference
> I'm not sure where we are in the release cycle right now.. Doug - if
> we didn't make a release with the 1.3 symbol tag (this was created
> just before rdma-core) then this should go into 1.3 when it gets
> merged..
Jason,
You pointed to use 1.4, the series followed that.
Doug,
Can you please comment here ? we should have a clear definition what
defined to be a "release" which requires a new number.
In case we want to use 1.3 please let me know if I need to re-spin or
you will change as part of the merge.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH v6 12/16] IB/pvrdma: Add Queue Pair support
From: Adit Ranadive @ 2016-12-05 21:21 UTC (permalink / raw)
To: Yuval Shaia
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
pv-drivers-pghWNbHTmq7QT0dZR+AlfA, jhansen-pghWNbHTmq7QT0dZR+AlfA,
asarwade-pghWNbHTmq7QT0dZR+AlfA,
georgezhang-pghWNbHTmq7QT0dZR+AlfA,
bryantan-pghWNbHTmq7QT0dZR+AlfA
In-Reply-To: <20161205172533.GA14581@yuval-lap>
On Mon, Dec 05, 2016 at 7:25:34PM +0200, Yuval Shaia wrote:
> > > +
> > > + /* Skip header page. */
> > > + qp->sq.offset = PAGE_SIZE;
> > > +
> > > + /* Recv queue pages are after send pages. */
> > > + qp->rq.offset = qp->npages_send * PAGE_SIZE;
> >
> > Unless i'm missing something here, per comment it should be:
> > qp->rq.offset = qp->sq.offset + qp->npages_send * PAGE_SIZE;
>
> Hi,
> Any comments about this question?
My comment [1] for the npages_send applies here as well. Since
we account for the extra page within the npages_send attribute, the
rq.offset calculated here is correct.
[1] http://marc.info/?l=linux-rdma&m=148069497625433&w=2
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* RE: [PATCH rdma-next 01/10] IB/core: Add raw packet protocol
From: Liran Liss @ 2016-12-05 20:14 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Hefty, Sean, Tom Talpey, Steve Wise, 'Doug Ledford',
'Leon Romanovsky',
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
'Steve Wise', Marciniszyn, Mike, Dalessandro, Dennis,
'Lijun Ou', 'Wei Hu(Xavier)', Latif, Faisal,
Yishai Hadas, 'Selvin Xavier', 'Devesh Sharma',
'Mitesh Ahuja', 'Christian Benvenuti',
'Dave Goodell'
In-Reply-To: <20161205171013.GA27784-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
> From: Jason Gunthorpe [mailto:jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org]
>
> > Anyway, returning to the initial matter at hand: I would like to start
> > with each port reporting a bit mask of the supported protocols on that
> > link (RoCE v1/v2, Raw Ethernet, iWARP, etc.) It will be used for
> > reporting device capabilities in general for tools, as well as by
> > applications that don't use rdmacm.
>
> Why don't we start by defining how it is supposed to even work and how to fix
> the RDMA CM before adding even more random capability bits?
>
> Jason
Extensions to RDMA CM is an important topic, which we can continue to discuss as devices that require it are introduced.
Protocol capabilities are needed for information purposes as well as for applications that do not use RDMA CM.
They are not "random", but depict what a device can do.
--Liran
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Enabling peer to peer device transactions for PCIe devices
From: Christoph Hellwig @ 2016-12-05 20:06 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Haggai Eran, John.Bridgman-5C7GfCeVMHo@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org,
Felix.Kuehling-5C7GfCeVMHo@public.gmane.org,
serguei.sagalovitch-5C7GfCeVMHo@public.gmane.org,
Paul.Blinzer-5C7GfCeVMHo@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
Stephen Bates, ben.sander-5C7GfCeVMHo@public.gmane.org,
Suravee.Suthikulpanit-5C7GfCeVMHo@public.gmane.org,
linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Alexander.Deucher-5C7GfCeVMHo@public.gmane.org, Max Gurtovoy,
christian.koenig-5C7GfCeVMHo@public.gmane.org,
"Linux-media-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-media@
In-Reply-To: <20161205194614.GA21132-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
On Mon, Dec 05, 2016 at 12:46:14PM -0700, Jason Gunthorpe wrote:
> In any event the allocator still needs to track which regions are in
> use and be able to hook 'free' from userspace. That does suggest it
> should be integrated into the nvme driver and not a bolt on driver..
Two totally different use cases:
- a card that exposes directly byte addressable storage as a PCI-e
bar. Thin of it as a nvdimm on a PCI-e card. That's the iopmem
case.
- the NVMe CMB which exposes a byte addressable indirection buffer for
I/O, but does not actually provide byte addressable persistent
storage. This is something that needs to be added to the NVMe driver
(and the block layer for the abstraction probably).
^ permalink raw reply
* Re: Enabling peer to peer device transactions for PCIe devices
From: Logan Gunthorpe @ 2016-12-05 19:59 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Haggai Eran, John.Bridgman-5C7GfCeVMHo@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org,
Felix.Kuehling-5C7GfCeVMHo@public.gmane.org,
serguei.sagalovitch-5C7GfCeVMHo@public.gmane.org,
Paul.Blinzer-5C7GfCeVMHo@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
Stephen Bates, ben.sander-5C7GfCeVMHo@public.gmane.org,
Suravee.Suthikulpanit-5C7GfCeVMHo@public.gmane.org,
linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Alexander.Deucher-5C7GfCeVMHo@public.gmane.org, Max Gurtovoy,
christian.koenig-5C7GfCeVMHo@public.gmane.org,
"Linux-media-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-media@
In-Reply-To: <20161205194614.GA21132-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
On 05/12/16 12:46 PM, Jason Gunthorpe wrote:
> NVMe might have to deal with pci-e hot-unplug, which is a similar
> problem-class to the GPU case..
Sure, but if the NVMe device gets hot-unplugged it means that all the
CMB mappings are useless and need to be torn down. This probably means
killing any process that has mappings open.
> In any event the allocator still needs to track which regions are in
> use and be able to hook 'free' from userspace. That does suggest it
> should be integrated into the nvme driver and not a bolt on driver..
Yup, that's correct. And yes, I've never suggested this to be a bolt on
driver -- I always expected for it to get integrated into the nvme
driver. (iopmem was not meant for this.)
Logan
^ permalink raw reply
* Re: Enabling peer to peer device transactions for PCIe devices
From: Jason Gunthorpe @ 2016-12-05 19:46 UTC (permalink / raw)
To: Logan Gunthorpe
Cc: Haggai Eran, John.Bridgman-5C7GfCeVMHo@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org,
Felix.Kuehling-5C7GfCeVMHo@public.gmane.org,
serguei.sagalovitch-5C7GfCeVMHo@public.gmane.org,
Paul.Blinzer-5C7GfCeVMHo@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
Stephen Bates, ben.sander-5C7GfCeVMHo@public.gmane.org,
Suravee.Suthikulpanit-5C7GfCeVMHo@public.gmane.org,
linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Alexander.Deucher-5C7GfCeVMHo@public.gmane.org, Max Gurtovoy,
christian.koenig-5C7GfCeVMHo@public.gmane.org,
"Linux-media-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-media@
In-Reply-To: <10356964-c454-47fb-7fb3-8bf2a418b11b-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
On Mon, Dec 05, 2016 at 12:27:20PM -0700, Logan Gunthorpe wrote:
>
>
> On 05/12/16 12:14 PM, Jason Gunthorpe wrote:
> >But CMB sounds much more like the GPU case where there is a
> >specialized allocator handing out the BAR to consumers, so I'm not
> >sure a general purpose chardev makes a lot of sense?
>
> I don't think it will ever need to be as complicated as the GPU case. There
> will probably only ever be a relatively small amount of memory behind the
> CMB and really the only users are those doing P2P work. Thus the specialized
> allocator could be pretty simple and I expect it would be fine to just
> return -ENOMEM if there is not enough memory.
NVMe might have to deal with pci-e hot-unplug, which is a similar
problem-class to the GPU case..
In any event the allocator still needs to track which regions are in
use and be able to hook 'free' from userspace. That does suggest it
should be integrated into the nvme driver and not a bolt on driver..
Jason
^ permalink raw reply
* Re: [PATCHv13 0/3] rdmacg: IB/core: rdma controller support
From: Tejun Heo @ 2016-12-05 19:35 UTC (permalink / raw)
To: Parav Pandit
Cc: cgroups, linux-doc, linux-kernel, linux-rdma, lizefan, hannes,
dledford, hch, liranl, sean.hefty, jgunthorpe, haggaie, corbet,
james.l.morris, serge, ogerlitz, matanb, akpm,
linux-security-module
In-Reply-To: <1480705637-2986-1-git-send-email-pandit.parav@gmail.com>
Parav, it's a bit too late for this cycle. Let's target v4.11. I'll
review the patches after the merge window. Please ping me if I don't.
Thanks.
--
tejun
^ permalink raw reply
* Re: Enabling peer to peer device transactions for PCIe devices
From: Logan Gunthorpe @ 2016-12-05 19:27 UTC (permalink / raw)
To: Jason Gunthorpe, Dan Williams
Cc: Haggai Eran, John.Bridgman-5C7GfCeVMHo@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org,
Felix.Kuehling-5C7GfCeVMHo@public.gmane.org,
serguei.sagalovitch-5C7GfCeVMHo@public.gmane.org,
Paul.Blinzer-5C7GfCeVMHo@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
Stephen Bates, ben.sander-5C7GfCeVMHo@public.gmane.org,
Suravee.Suthikulpanit-5C7GfCeVMHo@public.gmane.org,
linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Alexander.Deucher-5C7GfCeVMHo@public.gmane.org, Max Gurtovoy,
christian.koenig-5C7GfCeVMHo@public.gmane.org,
"Linux-media-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-media@
In-Reply-To: <20161205191438.GA20464-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
On 05/12/16 12:14 PM, Jason Gunthorpe wrote:
> But CMB sounds much more like the GPU case where there is a
> specialized allocator handing out the BAR to consumers, so I'm not
> sure a general purpose chardev makes a lot of sense?
I don't think it will ever need to be as complicated as the GPU case.
There will probably only ever be a relatively small amount of memory
behind the CMB and really the only users are those doing P2P work. Thus
the specialized allocator could be pretty simple and I expect it would
be fine to just return -ENOMEM if there is not enough memory.
Also, if it was implemented this way, if there was a need to make the
allocator more complicated it could easily be added later as the
userspace interface is just mmap to obtain a buffer.
Logan
^ permalink raw reply
* Re: Enabling peer to peer device transactions for PCIe devices
From: Jason Gunthorpe @ 2016-12-05 19:14 UTC (permalink / raw)
To: Dan Williams
Cc: Haggai Eran, John.Bridgman-5C7GfCeVMHo@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org,
Felix.Kuehling-5C7GfCeVMHo@public.gmane.org,
serguei.sagalovitch-5C7GfCeVMHo@public.gmane.org,
Paul.Blinzer-5C7GfCeVMHo@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
Stephen Bates, ben.sander-5C7GfCeVMHo@public.gmane.org,
Suravee.Suthikulpanit-5C7GfCeVMHo@public.gmane.org,
linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Alexander.Deucher-5C7GfCeVMHo@public.gmane.org, Max Gurtovoy,
christian.koenig-5C7GfCeVMHo@public.gmane.org,
"Linux-media-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-media@
In-Reply-To: <CAPcyv4iVHhOSxPrLMZ53Xw3CK+9cOWn9zEG8smMtqF_LAcKKpg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
On Mon, Dec 05, 2016 at 10:48:58AM -0800, Dan Williams wrote:
> On Mon, Dec 5, 2016 at 10:39 AM, Logan Gunthorpe <logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org> wrote:
> > On 05/12/16 11:08 AM, Dan Williams wrote:
> >>
> >> I've already recommended that iopmem not be a block device and instead
> >> be a device-dax instance. I also don't think it should claim the PCI
> >> ID, rather the driver that wants to map one of its bars this way can
> >> register the memory region with the device-dax core.
> >>
> >> I'm not sure there are enough device drivers that want to do this to
> >> have it be a generic /sys/.../resource_dmableX capability. It still
> >> seems to be an exotic one-off type of configuration.
> >
> >
> > Yes, this is essentially my thinking. Except I think the userspace interface
> > should really depend on the device itself. Device dax is a good choice for
> > many and I agree the block device approach wouldn't be ideal.
> >
> > Specifically for NVME CMB: I think it would make a lot of sense to just hand
> > out these mappings with an mmap call on /dev/nvmeX. I expect CMB buffers
> > would be volatile and thus you wouldn't need to keep track of where in the
> > BAR the region came from. Thus, the mmap call would just be an allocator
> > from BAR memory. If device-dax were used, userspace would need to lookup
> > which device-dax instance corresponds to which nvme drive.
>
> I'm not opposed to mapping /dev/nvmeX. However, the lookup is trivial
> to accomplish in sysfs through /sys/dev/char to find the sysfs path
> of
But CMB sounds much more like the GPU case where there is a
specialized allocator handing out the BAR to consumers, so I'm not
sure a general purpose chardev makes a lot of sense?
Jason
^ permalink raw reply
* Re: Enabling peer to peer device transactions for PCIe devices
From: Dan Williams @ 2016-12-05 18:48 UTC (permalink / raw)
To: Logan Gunthorpe
Cc: Haggai Eran, John.Bridgman-5C7GfCeVMHo@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org,
Felix.Kuehling-5C7GfCeVMHo@public.gmane.org,
serguei.sagalovitch-5C7GfCeVMHo@public.gmane.org,
Paul.Blinzer-5C7GfCeVMHo@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
Stephen Bates, Jason Gunthorpe,
ben.sander-5C7GfCeVMHo@public.gmane.org,
Suravee.Suthikulpanit-5C7GfCeVMHo@public.gmane.org,
linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Alexander.Deucher-5C7GfCeVMHo@public.gmane.org, Max Gurtovoy,
christian.koenig-5C7GfCeVMHo@public.gmane.org
In-Reply-To: <a3a1c239-297d-c091-7758-54acdf00f74e-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org>
On Mon, Dec 5, 2016 at 10:39 AM, Logan Gunthorpe <logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org> wrote:
> On 05/12/16 11:08 AM, Dan Williams wrote:
>>
>> I've already recommended that iopmem not be a block device and instead
>> be a device-dax instance. I also don't think it should claim the PCI
>> ID, rather the driver that wants to map one of its bars this way can
>> register the memory region with the device-dax core.
>>
>> I'm not sure there are enough device drivers that want to do this to
>> have it be a generic /sys/.../resource_dmableX capability. It still
>> seems to be an exotic one-off type of configuration.
>
>
> Yes, this is essentially my thinking. Except I think the userspace interface
> should really depend on the device itself. Device dax is a good choice for
> many and I agree the block device approach wouldn't be ideal.
>
> Specifically for NVME CMB: I think it would make a lot of sense to just hand
> out these mappings with an mmap call on /dev/nvmeX. I expect CMB buffers
> would be volatile and thus you wouldn't need to keep track of where in the
> BAR the region came from. Thus, the mmap call would just be an allocator
> from BAR memory. If device-dax were used, userspace would need to lookup
> which device-dax instance corresponds to which nvme drive.
>
I'm not opposed to mapping /dev/nvmeX. However, the lookup is trivial
to accomplish in sysfs through /sys/dev/char to find the sysfs path of
the device-dax instance under the nvme device, or if you already have
the nvme sysfs path the dax instance(s) will appear under the "dax"
sub-directory.
^ permalink raw reply
* Re: Enabling peer to peer device transactions for PCIe devices
From: Logan Gunthorpe @ 2016-12-05 18:39 UTC (permalink / raw)
To: Dan Williams, Jason Gunthorpe
Cc: Haggai Eran, John.Bridgman-5C7GfCeVMHo@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org,
Felix.Kuehling-5C7GfCeVMHo@public.gmane.org,
serguei.sagalovitch-5C7GfCeVMHo@public.gmane.org,
Paul.Blinzer-5C7GfCeVMHo@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
Stephen Bates, ben.sander-5C7GfCeVMHo@public.gmane.org,
Suravee.Suthikulpanit-5C7GfCeVMHo@public.gmane.org,
linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Alexander.Deucher-5C7GfCeVMHo@public.gmane.org, Max Gurtovoy,
christian.koenig-5C7GfCeVMHo@public.gmane.org,
"Linux-media-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-media@
In-Reply-To: <CAPcyv4iEXwvtDbZgnWzdKU6uN_sOGmXH1KtW_Nws6kUftJUigQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
On 05/12/16 11:08 AM, Dan Williams wrote:
> I've already recommended that iopmem not be a block device and instead
> be a device-dax instance. I also don't think it should claim the PCI
> ID, rather the driver that wants to map one of its bars this way can
> register the memory region with the device-dax core.
>
> I'm not sure there are enough device drivers that want to do this to
> have it be a generic /sys/.../resource_dmableX capability. It still
> seems to be an exotic one-off type of configuration.
Yes, this is essentially my thinking. Except I think the userspace
interface should really depend on the device itself. Device dax is a
good choice for many and I agree the block device approach wouldn't be
ideal.
Specifically for NVME CMB: I think it would make a lot of sense to just
hand out these mappings with an mmap call on /dev/nvmeX. I expect CMB
buffers would be volatile and thus you wouldn't need to keep track of
where in the BAR the region came from. Thus, the mmap call would just be
an allocator from BAR memory. If device-dax were used, userspace would
need to lookup which device-dax instance corresponds to which nvme drive.
Logan
^ permalink raw reply
* Re: Enabling peer to peer device transactions for PCIe devices
From: Dan Williams @ 2016-12-05 18:08 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Haggai Eran, John.Bridgman-5C7GfCeVMHo@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org,
Felix.Kuehling-5C7GfCeVMHo@public.gmane.org,
serguei.sagalovitch-5C7GfCeVMHo@public.gmane.org,
Paul.Blinzer-5C7GfCeVMHo@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
Stephen Bates, ben.sander-5C7GfCeVMHo@public.gmane.org,
Suravee.Suthikulpanit-5C7GfCeVMHo@public.gmane.org,
linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Alexander.Deucher-5C7GfCeVMHo@public.gmane.org, Max Gurtovoy,
christian.koenig-5C7GfCeVMHo@public.gmane.org,
"Linux-media-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-media@
In-Reply-To: <20161205180231.GA28133-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
On Mon, Dec 5, 2016 at 10:02 AM, Jason Gunthorpe
<jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> wrote:
> On Mon, Dec 05, 2016 at 09:40:38AM -0800, Dan Williams wrote:
>
>> > If it is kernel only with physical addresess we don't need a uAPI for
>> > it, so I'm not sure #1 is at all related to iopmem.
>> >
>> > Most people who want #1 probably can just mmap
>> > /sys/../pci/../resourceX to get a user handle to it, or pass around
>> > __iomem pointers in the kernel. This has been asked for before with
>> > RDMA.
>> >
>> > I'm still not really clear what iopmem is for, or why DAX should ever
>> > be involved in this..
>>
>> Right, by default remap_pfn_range() does not establish DMA capable
>> mappings. You can think of iopmem as remap_pfn_range() converted to
>> use devm_memremap_pages(). Given the extra constraints of
>> devm_memremap_pages() it seems reasonable to have those DMA capable
>> mappings be optionally established via a separate driver.
>
> Except the iopmem driver claims the PCI ID, and presents a block
> interface which is really *NOT* what people who have asked for this in
> the past have wanted. IIRC it was embedded stuff eg RDMA video
> directly out of a capture card or a similar kind of thinking.
>
> It is a good point about devm_memremap_pages limitations, but maybe
> that just says to create a /sys/.../resource_dmableX ?
>
> Or is there some reason why people want a filesystem on top of BAR
> memory? That does not seem to have been covered yet..
>
I've already recommended that iopmem not be a block device and instead
be a device-dax instance. I also don't think it should claim the PCI
ID, rather the driver that wants to map one of its bars this way can
register the memory region with the device-dax core.
I'm not sure there are enough device drivers that want to do this to
have it be a generic /sys/.../resource_dmableX capability. It still
seems to be an exotic one-off type of configuration.
^ permalink raw reply
* Re: Enabling peer to peer device transactions for PCIe devices
From: Jason Gunthorpe @ 2016-12-05 18:02 UTC (permalink / raw)
To: Dan Williams
Cc: Haggai Eran, John.Bridgman-5C7GfCeVMHo@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org,
Felix.Kuehling-5C7GfCeVMHo@public.gmane.org,
serguei.sagalovitch-5C7GfCeVMHo@public.gmane.org,
Paul.Blinzer-5C7GfCeVMHo@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
Stephen Bates, ben.sander-5C7GfCeVMHo@public.gmane.org,
Suravee.Suthikulpanit-5C7GfCeVMHo@public.gmane.org,
linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Alexander.Deucher-5C7GfCeVMHo@public.gmane.org, Max Gurtovoy,
christian.koenig-5C7GfCeVMHo@public.gmane.org,
"Linux-media-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-media@
In-Reply-To: <CAPcyv4hdMkXOxj9hUDpnftA7UTGDa498eBugdePp8EWr6S80gA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
On Mon, Dec 05, 2016 at 09:40:38AM -0800, Dan Williams wrote:
> > If it is kernel only with physical addresess we don't need a uAPI for
> > it, so I'm not sure #1 is at all related to iopmem.
> >
> > Most people who want #1 probably can just mmap
> > /sys/../pci/../resourceX to get a user handle to it, or pass around
> > __iomem pointers in the kernel. This has been asked for before with
> > RDMA.
> >
> > I'm still not really clear what iopmem is for, or why DAX should ever
> > be involved in this..
>
> Right, by default remap_pfn_range() does not establish DMA capable
> mappings. You can think of iopmem as remap_pfn_range() converted to
> use devm_memremap_pages(). Given the extra constraints of
> devm_memremap_pages() it seems reasonable to have those DMA capable
> mappings be optionally established via a separate driver.
Except the iopmem driver claims the PCI ID, and presents a block
interface which is really *NOT* what people who have asked for this in
the past have wanted. IIRC it was embedded stuff eg RDMA video
directly out of a capture card or a similar kind of thinking.
It is a good point about devm_memremap_pages limitations, but maybe
that just says to create a /sys/.../resource_dmableX ?
Or is there some reason why people want a filesystem on top of BAR
memory? That does not seem to have been covered yet..
Jason
^ permalink raw reply
* Re: Enabling peer to peer device transactions for PCIe devices
From: Dan Williams @ 2016-12-05 17:40 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Haggai Eran, John.Bridgman-5C7GfCeVMHo@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org,
Felix.Kuehling-5C7GfCeVMHo@public.gmane.org,
serguei.sagalovitch-5C7GfCeVMHo@public.gmane.org,
Paul.Blinzer-5C7GfCeVMHo@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
Stephen Bates, ben.sander-5C7GfCeVMHo@public.gmane.org,
Suravee.Suthikulpanit-5C7GfCeVMHo@public.gmane.org,
linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Alexander.Deucher-5C7GfCeVMHo@public.gmane.org, Max Gurtovoy,
christian.koenig-5C7GfCeVMHo@public.gmane.org,
"Linux-media-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-media@
In-Reply-To: <20161205171830.GB27784-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
On Mon, Dec 5, 2016 at 9:18 AM, Jason Gunthorpe
<jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org> wrote:
> On Sun, Dec 04, 2016 at 07:23:00AM -0600, Stephen Bates wrote:
>> Hi All
>>
>> This has been a great thread (thanks to Alex for kicking it off) and I
>> wanted to jump in and maybe try and put some summary around the
>> discussion. I also wanted to propose we include this as a topic for LFS/MM
>> because I think we need more discussion on the best way to add this
>> functionality to the kernel.
>>
>> As far as I can tell the people looking for P2P support in the kernel fall
>> into two main camps:
>>
>> 1. Those who simply want to expose static BARs on PCIe devices that can be
>> used as the source/destination for DMAs from another PCIe device. This
>> group has no need for memory invalidation and are happy to use
>> physical/bus addresses and not virtual addresses.
>
> I didn't think there was much on this topic except for the CMB
> thing.. Even that is really a mapped kernel address..
>
>> I think something like the iopmem patches Logan and I submitted recently
>> come close to addressing use case 1. There are some issues around
>> routability but based on feedback to date that does not seem to be a
>> show-stopper for an initial inclusion.
>
> If it is kernel only with physical addresess we don't need a uAPI for
> it, so I'm not sure #1 is at all related to iopmem.
>
> Most people who want #1 probably can just mmap
> /sys/../pci/../resourceX to get a user handle to it, or pass around
> __iomem pointers in the kernel. This has been asked for before with
> RDMA.
>
> I'm still not really clear what iopmem is for, or why DAX should ever
> be involved in this..
Right, by default remap_pfn_range() does not establish DMA capable
mappings. You can think of iopmem as remap_pfn_range() converted to
use devm_memremap_pages(). Given the extra constraints of
devm_memremap_pages() it seems reasonable to have those DMA capable
mappings be optionally established via a separate driver.
^ permalink raw reply
* [PATCH] MAINTAINERS: Update Intel RDMA RNIC driver maintainers
From: Shiraz Saleem @ 2016-12-05 17:28 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
e1000-rdma-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f, Shiraz Saleem
Signed-off-by: Shiraz Saleem <shiraz.saleem-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Faisal Latif <faisal.latif-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
MAINTAINERS | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index e1b090f..df9ef6c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6009,10 +6009,7 @@ F: drivers/net/ethernet/intel/*/
INTEL RDMA RNIC DRIVER
M: Faisal Latif <faisal.latif-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
-R: Chien Tin Tung <chien.tin.tung-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
-R: Mustafa Ismail <mustafa.ismail-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
-R: Shiraz Saleem <shiraz.saleem-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
-R: Tatyana Nikolova <tatyana.e.nikolova-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
+M: Shiraz Saleem <shiraz.saleem-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
L: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
S: Supported
F: drivers/infiniband/hw/i40iw/
--
2.8.0
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: [PATCH v6 12/16] IB/pvrdma: Add Queue Pair support
From: Yuval Shaia @ 2016-12-05 17:25 UTC (permalink / raw)
To: Adit Ranadive
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
pv-drivers-pghWNbHTmq7QT0dZR+AlfA, jhansen-pghWNbHTmq7QT0dZR+AlfA,
asarwade-pghWNbHTmq7QT0dZR+AlfA,
georgezhang-pghWNbHTmq7QT0dZR+AlfA,
bryantan-pghWNbHTmq7QT0dZR+AlfA
In-Reply-To: <20161202130726.GB3663@yuval-lap>
On Fri, Dec 02, 2016 at 03:07:27PM +0200, Yuval Shaia wrote:
> On Sun, Oct 02, 2016 at 07:10:32PM -0700, Adit Ranadive wrote:
> > + } else {
> > + qp->is_kernel = true;
> > +
> > + ret = pvrdma_set_sq_size(to_vdev(pd->device),
> > + &init_attr->cap,
> > + init_attr->qp_type, qp);
> > + if (ret)
> > + goto err_qp;
> > +
> > + ret = pvrdma_set_rq_size(to_vdev(pd->device),
> > + &init_attr->cap, qp);
> > + if (ret)
> > + goto err_qp;
> > +
> > + qp->npages = qp->npages_send + qp->npages_recv;
>
> Per next comment shouldn't it be:
> qp->npages = qp->npages_send + qp->npages_recv + PAGE_SIZE;
>
> Otherwise there will be only npages_send - 1 pages for send ring.
>
> > +
> > + /* Skip header page. */
> > + qp->sq.offset = PAGE_SIZE;
> > +
> > + /* Recv queue pages are after send pages. */
> > + qp->rq.offset = qp->npages_send * PAGE_SIZE;
>
> Unless i'm missing something here, per comment it should be:
> qp->rq.offset = qp->sq.offset + qp->npages_send * PAGE_SIZE;
Hi,
Any comments about this question?
>
> > + }
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Enabling peer to peer device transactions for PCIe devices
From: Jason Gunthorpe @ 2016-12-05 17:18 UTC (permalink / raw)
To: Stephen Bates
Cc: Haggai Eran, John.Bridgman-5C7GfCeVMHo@public.gmane.org,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-nvdimm-y27Ovi1pjclAfugRpC6u6w@public.gmane.org,
Felix.Kuehling-5C7GfCeVMHo@public.gmane.org,
serguei.sagalovitch-5C7GfCeVMHo@public.gmane.org,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org,
Paul.Blinzer-5C7GfCeVMHo@public.gmane.org,
ben.sander-5C7GfCeVMHo@public.gmane.org,
Suravee.Suthikulpanit-5C7GfCeVMHo@public.gmane.org,
linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Alexander.Deucher-5C7GfCeVMHo@public.gmane.org, Max Gurtovoy,
christian.koenig-5C7GfCeVMHo@public.gmane.org,
Linux-media-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <61a2fb07344aacd81111449d222de66e.squirrel-2RFepEojUI2m/tTznewQxVaTQe2KTcn/@public.gmane.org>
On Sun, Dec 04, 2016 at 07:23:00AM -0600, Stephen Bates wrote:
> Hi All
>
> This has been a great thread (thanks to Alex for kicking it off) and I
> wanted to jump in and maybe try and put some summary around the
> discussion. I also wanted to propose we include this as a topic for LFS/MM
> because I think we need more discussion on the best way to add this
> functionality to the kernel.
>
> As far as I can tell the people looking for P2P support in the kernel fall
> into two main camps:
>
> 1. Those who simply want to expose static BARs on PCIe devices that can be
> used as the source/destination for DMAs from another PCIe device. This
> group has no need for memory invalidation and are happy to use
> physical/bus addresses and not virtual addresses.
I didn't think there was much on this topic except for the CMB
thing.. Even that is really a mapped kernel address..
> I think something like the iopmem patches Logan and I submitted recently
> come close to addressing use case 1. There are some issues around
> routability but based on feedback to date that does not seem to be a
> show-stopper for an initial inclusion.
If it is kernel only with physical addresess we don't need a uAPI for
it, so I'm not sure #1 is at all related to iopmem.
Most people who want #1 probably can just mmap
/sys/../pci/../resourceX to get a user handle to it, or pass around
__iomem pointers in the kernel. This has been asked for before with
RDMA.
I'm still not really clear what iopmem is for, or why DAX should ever
be involved in this..
> For use-case 2 it looks like there are several options and some of them
> (like HMM) have been around for quite some time without gaining
> acceptance. I think there needs to be more discussion on this usecase and
> it could be some time before we get something upstreamable.
AFAIK, hmm makes parts easier, but isn't directly addressing this
need..
I think you need to get ZONE_DEVICE accepted for non-cachable PCI BARs
as the first step.
>From there is pretty clear we the DMA API needs to be updated to
support that use and work can be done to solve the various problems
there on the basis of using ZONE_DEVICE pages to figure out to the
PCI-E end points
Jason
^ permalink raw reply
* Re: Use ib_drain_qp instead of ib_drain_rq in ib_srp
From: Sagi Grimberg @ 2016-12-05 17:11 UTC (permalink / raw)
To: Max Gurtovoy, Bart Van Assche, Christoph Hellwig,
swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW,
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <97bc319e-ffa8-17f3-cebb-11d0a7d110d3-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Just noticed that send_cq has IB_POLL_DIRECT restriction in srp.
>
> what about iSER then ? we use there ib_drain_sq...
For iSER we need to make sure we don't post recvs or sends after we
drained.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 01/28] bnxt_en: Add bnxt_set_max_func_irqs().
From: Michael Chan @ 2016-12-05 17:10 UTC (permalink / raw)
To: David Miller; +Cc: Selvin Xavier, dledford, linux-rdma, Netdev
In-Reply-To: <20161205.114704.1700305660817315475.davem@davemloft.net>
On Mon, Dec 5, 2016 at 8:47 AM, David Miller <davem@davemloft.net> wrote:
>
> It really doesn't make any sense to only send 7 out of 28 of these
> patches to the networking list.
>
> In fact I would say that you need to split this series into two
> components.
OK. I will resend those bnxt_en patches later today as a separate series.
>
> One that goes into the networking tree, and once that's accepted you
> can submit the IB parts to that subsystem's maintainer.
^ permalink raw reply
* Re: [PATCH rdma-next 01/10] IB/core: Add raw packet protocol
From: Jason Gunthorpe @ 2016-12-05 17:10 UTC (permalink / raw)
To: Liran Liss
Cc: Hefty, Sean, Tom Talpey, Steve Wise, 'Doug Ledford',
'Leon Romanovsky',
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
'Steve Wise', Marciniszyn, Mike, Dalessandro, Dennis,
'Lijun Ou', 'Wei Hu(Xavier)', Latif, Faisal,
Yishai Hadas, 'Selvin Xavier', 'Devesh Sharma',
'Mitesh Ahuja', 'Christian Benvenuti',
'Dave Goodell', Moni
In-Reply-To: <HE1PR0501MB2812393A0A690DEC1E03DE3FB1800-692Kmc8YnlIVrnpjwTCbp8DSnupUy6xnnBOFsp37pqbUKgpGm//BTAC/G2K4zDHf@public.gmane.org>
On Sun, Dec 04, 2016 at 08:38:14PM +0000, Liran Liss wrote:
> Anyway, returning to the initial matter at hand: I would like to
> start with each port reporting a bit mask of the supported protocols
> on that link (RoCE v1/v2, Raw Ethernet, iWARP, etc.) It will be
> used for reporting device capabilities in general for tools, as well
> as by applications that don't use rdmacm.
Why don't we start by defining how it is supposed to even work and how
to fix the RDMA CM before adding even more random capability bits?
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox