public inbox for linux-rdma@vger.kernel.org
 help / color / mirror / Atom feed
From: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
To: Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>,
	Jason Gunthorpe <jgg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Cc: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>,
	RDMA mailing list
	<linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
	Daniel Jurgens <danielj-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>,
	Parav Pandit <parav-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Subject: [PATCH rdma-next v2 00/14] index e382a3ca759e..d4a471a76d82 100644
Date: Thu,  4 Jan 2018 17:25:30 +0200	[thread overview]
Message-ID: <20180104152544.28919-1-leon@kernel.org> (raw)

Changelog:
 v1 -> v2:
  * Dropped "IB/mlx5: Use correct mdev for vport queries in ib_virt"
 v0 -> v1:
  * Rebased to latest rdma/for-next
  * Enriched commit messages.

Thanks

-------------------------------------
>From Daniel:

This feature allows RDMA resources (pd, mr, cq, qp, etc) to be used with
both physical ports of capable mlx5 devices. When enabled a single IB
device with two ports will be registered instead of two single port
devices.

There are still two PCI devices underlying the two port device, the
capabilities indicate which device is the "master" device and which is
the slave.

When the add callback function is called for a slave device a list of IB
devices is searched for matching master device, indicated by the
capabilities and the system_image_guid. If a match is found the slave is
bound to the master device, otherwise it's placed on a list, in case
it's master becomes available in the future. When a master device is added it
searches the list of available slaves for a matching slave device. If a
match is found it binds the slave as its 2nd port. If no match as found
the device still appears as a dual port device, with the 2nd port down.
RDMA resources can still created that use the yet unavailable 2nd port.

Commands related to IB resources are all routed through the master
mlx5_core device. Port specific commands, like those for hardware
counters are routed to their respective port mlx5_core device. Since
devices can appear and disappear asynchronously a reference count on the
underlying mlx5_core device is maintained. Getting and putting this
reference is only necessary for commands destined to a specific port,
the master core device can be used freely, as it will exist while the IB
device exists.

SR-IOV devices follow the same pattern as the physical ones. VFs of a
master port can bind VFs of slave ports, if available, and operate as
dual port devices.

Examples of devices passed to a VM:
(master)         - One net device, one IB device that has two ports. The slave
                   port will always be down.
(slave)          - One net device, no IB devices.
(slave, slave)   - Two net devices and no IB devices.
(master, master) - Two net devices, two IB devices, each with two ports.
                   The slave port of each device will always be down.
(master, slave)  - Two net devices, one IB device, with two ports. Both
		    ports can be used.

There are no changes to the existing design for net devices.

The feature is disabled by default and it is enabled in firmware
with mlxconfig.

	Thanks
---------------------------------------

Daniel Jurgens (13):
  net/mlx5: Fix race for multiple RoCE enable
  net/mlx5: Set software owner ID during init HCA
  IB/core: Change roce_rescan_device to return void
  IB/mlx5: Reduce the use of num_port capability
  IB/mlx5: Make netdev notifications multiport capable
  {net,IB}/mlx5: Manage port association for multiport RoCE
  IB/mlx5: Move IB event processing onto a workqueue
  IB/mlx5: Implement dual port functionality in query routines
  IB/mlx5: Update counter implementation for dual port RoCE
  {net,IB}/mlx5: Change set_roce_gid to take a port number
  IB/mlx5: Route MADs for dual port RoCE
  IB/mlx5: Don't advertise RAW QP support in dual port mode
  net/mlx5: Set num_vhca_ports capability

Parav Pandit (1):
  IB/mlx5: Change debugfs to have per port contents

 drivers/infiniband/core/cache.c                    |   7 +-
 drivers/infiniband/core/core_priv.h                |   1 -
 drivers/infiniband/core/roce_gid_mgmt.c            |  13 +-
 drivers/infiniband/hw/mlx5/cong.c                  |  83 ++-
 drivers/infiniband/hw/mlx5/mad.c                   |  23 +-
 drivers/infiniband/hw/mlx5/main.c                  | 785 +++++++++++++++++----
 drivers/infiniband/hw/mlx5/mlx5_ib.h               |  52 +-
 drivers/infiniband/hw/mlx5/qp.c                    |   8 +-
 .../net/ethernet/mellanox/mlx5/core/fpga/conn.c    |  11 +-
 drivers/net/ethernet/mellanox/mlx5/core/fw.c       |  10 +-
 .../net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c  |   2 +-
 drivers/net/ethernet/mellanox/mlx5/core/lib/gid.c  |   5 +-
 drivers/net/ethernet/mellanox/mlx5/core/main.c     |  12 +-
 .../net/ethernet/mellanox/mlx5/core/mlx5_core.h    |   2 +-
 drivers/net/ethernet/mellanox/mlx5/core/vport.c    |  91 ++-
 include/linux/mlx5/device.h                        |   5 +
 include/linux/mlx5/driver.h                        |  29 +-
 include/linux/mlx5/mlx5_ifc.h                      |  32 +-
 include/linux/mlx5/vport.h                         |   4 +
 include/rdma/ib_verbs.h                            |   8 +
 20 files changed, 976 insertions(+), 207 deletions(-)

--
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

             reply	other threads:[~2018-01-04 15:25 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-01-04 15:25 Leon Romanovsky [this message]
     [not found] ` <20180104152544.28919-1-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2018-01-04 15:25   ` [PATCH rdma-next v2 01/14] net/mlx5: Fix race for multiple RoCE enable Leon Romanovsky
2018-01-04 15:25   ` [PATCH rdma-next v2 02/14] net/mlx5: Set software owner ID during init HCA Leon Romanovsky
2018-01-04 15:25   ` [PATCH rdma-next v2 03/14] IB/core: Change roce_rescan_device to return void Leon Romanovsky
2018-01-04 15:25   ` [PATCH rdma-next v2 04/14] IB/mlx5: Reduce the use of num_port capability Leon Romanovsky
2018-01-04 15:25   ` [PATCH rdma-next v2 05/14] IB/mlx5: Make netdev notifications multiport capable Leon Romanovsky
2018-01-04 15:25   ` [PATCH rdma-next v2 06/14] {net,IB}/mlx5: Manage port association for multiport RoCE Leon Romanovsky
     [not found]     ` <20180104152544.28919-7-leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2018-01-04 23:33       ` Jason Gunthorpe
     [not found]         ` <20180104233334.GA18993-uk2M96/98Pc@public.gmane.org>
2018-01-05  6:36           ` Leon Romanovsky
2018-01-05 13:47           ` Daniel Jurgens
2018-01-04 15:25   ` [PATCH rdma-next v2 07/14] IB/mlx5: Move IB event processing onto a workqueue Leon Romanovsky
2018-01-04 15:25   ` [PATCH rdma-next v2 08/14] IB/mlx5: Implement dual port functionality in query routines Leon Romanovsky
2018-01-04 15:25   ` [PATCH rdma-next v2 09/14] IB/mlx5: Change debugfs to have per port contents Leon Romanovsky
2018-01-04 15:25   ` [PATCH rdma-next v2 10/14] IB/mlx5: Update counter implementation for dual port RoCE Leon Romanovsky
2018-01-04 15:25   ` [PATCH rdma-next v2 11/14] {net,IB}/mlx5: Change set_roce_gid to take a port number Leon Romanovsky
2018-01-04 15:25   ` [PATCH rdma-next v2 12/14] IB/mlx5: Route MADs for dual port RoCE Leon Romanovsky
2018-01-04 15:25   ` [PATCH rdma-next v2 13/14] IB/mlx5: Don't advertise RAW QP support in dual port mode Leon Romanovsky
2018-01-04 15:25   ` [PATCH rdma-next v2 14/14] net/mlx5: Set num_vhca_ports capability Leon Romanovsky
2018-01-04 19:53   ` [PATCH rdma-next v2 00/14] index e382a3ca759e..d4a471a76d82 100644 Or Gerlitz
     [not found]     ` <CAJ3xEMjptQf99ZdGxkB6kTh9LkQ7-pjiObMh+knDWXFStPOg7Q-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-01-04 21:50       ` Daniel Jurgens
     [not found]         ` <2618f285-6fa4-06ac-4b4a-51d0b2660b61-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2018-01-05 14:59           ` Or Gerlitz
     [not found]             ` <CAJ3xEMhVxBqJUwdRn3BXJPnf9J-_gXtrWrQgun3bCEW7W7Oq1w-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-01-05 15:03               ` Daniel Jurgens
     [not found]                 ` <792bac90-b1eb-110f-399c-08d5309d92dd-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2018-01-05 15:22                   ` Or Gerlitz
     [not found]                     ` <CAJ3xEMjOGEQQf+y4wEEPiS_F9fhYmDwBmd=ucUDQX_9416gzKQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2018-01-05 15:57                       ` Daniel Jurgens
     [not found]                         ` <d15d980b-1d67-277e-60f8-0b7e92035a06-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
2018-01-05 20:44                           ` Or Gerlitz
2018-01-09  2:51   ` Jason Gunthorpe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180104152544.28919-1-leon@kernel.org \
    --to=leon-dgejt+ai2ygdnm+yrofe0a@public.gmane.org \
    --cc=danielj-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
    --cc=dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \
    --cc=jgg-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
    --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=parav-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox