* Re: [net-next RFC V5 3/5] virtio: intorduce an API to set affinity for a virtqueue
From: Michael S. Tsirkin @ 2012-07-29 20:40 UTC (permalink / raw)
To: Paolo Bonzini
Cc: krkumar2, habanero, kvm, netdev, mashirle, linux-kernel,
virtualization, edumazet, tahm, jwhan, davem, sri
In-Reply-To: <5012A7D3.4040800@redhat.com>
On Fri, Jul 27, 2012 at 04:38:11PM +0200, Paolo Bonzini wrote:
> Il 05/07/2012 12:29, Jason Wang ha scritto:
> > Sometimes, virtio device need to configure irq affiniry hint to maximize the
> > performance. Instead of just exposing the irq of a virtqueue, this patch
> > introduce an API to set the affinity for a virtqueue.
> >
> > The api is best-effort, the affinity hint may not be set as expected due to
> > platform support, irq sharing or irq type. Currently, only pci method were
> > implemented and we set the affinity according to:
> >
> > - if device uses INTX, we just ignore the request
> > - if device has per vq vector, we force the affinity hint
> > - if the virtqueues share MSI, make the affinity OR over all affinities
> > requested
> >
> > Signed-off-by: Jason Wang <jasowang@redhat.com>
>
> Hmm, I don't see any benefit from this patch, I need to use
> irq_set_affinity (which however is not exported) to actually bind IRQs
> to CPUs. Example:
>
> with irq_set_affinity_hint:
> 43: 89 107 100 97 PCI-MSI-edge virtio0-request
> 44: 178 195 268 199 PCI-MSI-edge virtio0-request
> 45: 97 100 97 155 PCI-MSI-edge virtio0-request
> 46: 234 261 213 218 PCI-MSI-edge virtio0-request
>
> with irq_set_affinity:
> 43: 721 0 0 1 PCI-MSI-edge virtio0-request
> 44: 0 746 0 1 PCI-MSI-edge virtio0-request
> 45: 0 0 658 0 PCI-MSI-edge virtio0-request
> 46: 0 0 1 547 PCI-MSI-edge virtio0-request
>
> I gathered these quickly after boot, but real benchmarks show the same
> behavior, and performance gets actually worse with virtio-scsi
> multiqueue+irq_set_affinity_hint than with irq_set_affinity.
>
> I also tried adding IRQ_NO_BALANCING, but the only effect is that I
> cannot set the affinity
>
> The queue steering algorithm I use in virtio-scsi is extremely simple
> and based on your tx code. See how my nice pinning is destroyed:
>
> # taskset -c 0 dd if=/dev/sda bs=1M count=1000 of=/dev/null iflag=direct
> # cat /proc/interrupts
> 43: 2690 2709 2691 2696 PCI-MSI-edge virtio0-request
> 44: 109 122 199 124 PCI-MSI-edge virtio0-request
> 45: 170 183 170 237 PCI-MSI-edge virtio0-request
> 46: 143 166 125 125 PCI-MSI-edge virtio0-request
>
> All my requests come from CPU#0 and thus go to the first virtqueue, but
> the interrupts are serviced all over the place.
>
> Did you set the affinity manually in your experiments, or perhaps there
> is a difference between scsi and networking... (interrupt mitigation?)
>
> Paolo
You need to run irqbalancer in guest to make it actually work. Do you?
^ permalink raw reply
* [PATCH] net/tun: fix ioctl() based info leaks
From: Mathias Krause @ 2012-07-29 20:58 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, Mathias Krause
The tun module leaks up to 36 bytes of memory by not initializing a
structure located on the stack that gets copied to user memory by the
TUNGETIFF and SIOCGIFHWADDR ioctl()s.
Signed-off-by: Mathias Krause <minipli@googlemail.com>
---
drivers/net/tun.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/net/tun.c b/drivers/net/tun.c
index 987aeef..cadeb94 100644
--- a/drivers/net/tun.c
+++ b/drivers/net/tun.c
@@ -1252,9 +1252,12 @@ static long __tun_chr_ioctl(struct file *file, unsigned int cmd,
int vnet_hdr_sz;
int ret;
- if (cmd == TUNSETIFF || _IOC_TYPE(cmd) == 0x89)
+ if (cmd == TUNSETIFF || _IOC_TYPE(cmd) == 0x89) {
if (copy_from_user(&ifr, argp, ifreq_len))
return -EFAULT;
+ } else {
+ memset(&ifr, 0, sizeof(ifr));
+ }
if (cmd == TUNGETFEATURES) {
/* Currently this just means: "what IFF flags are valid?".
--
1.7.10.4
^ permalink raw reply related
* Re: [PATCH] seeq: use PTR_RET at init_module of driver
From: Jiri Pirko @ 2012-07-29 21:34 UTC (permalink / raw)
To: Devendra Naga
Cc: David S. Miller, Jeff Kirsher, David Howells, Jiri Pirko,
Pradeep A Dalvi, netdev
In-Reply-To: <1343568527-12729-1-git-send-email-develkernel412222@gmail.com>
Sun, Jul 29, 2012 at 03:28:47PM CEST, develkernel412222@gmail.com wrote:
>the driver sees wether the dev_seeq pointer is having a error that can be
>read by using the PTR_ERR, and returns it at error case, other wise 0 at
>success case.
>
>the PTR_RET does the same thing, and use PTR_RET instead of redoing the
>code of PTR_RET
>
>Signed-off-by: Devendra Naga <develkernel412222@gmail.com>
>---
>This patch is actually made by using coccinelle, and also by looking
>at the PTR_RET function.
>
>as its implementation is
>static int PTR_RET(void *ptr)
>{
> if (IS_ERR(ptr)
> return PTR_ERR(ptr);
> return 0;
>}
>
>by using this i removed the following code and replaced with PTR_RET....
>
>
> drivers/net/ethernet/seeq/seeq8005.c | 4 +---
> 1 file changed, 1 insertion(+), 3 deletions(-)
>
>diff --git a/drivers/net/ethernet/seeq/seeq8005.c b/drivers/net/ethernet/seeq/seeq8005.c
>index 698edbb..d6e50de 100644
>--- a/drivers/net/ethernet/seeq/seeq8005.c
>+++ b/drivers/net/ethernet/seeq/seeq8005.c
>@@ -736,9 +736,7 @@ MODULE_PARM_DESC(irq, "SEEQ 8005 IRQ number");
> int __init init_module(void)
> {
> dev_seeq = seeq8005_probe(-1);
>- if (IS_ERR(dev_seeq))
>- return PTR_ERR(dev_seeq);
>- return 0;
>+ return PTR_RET(dev_seeq);
> }
>
> void __exit cleanup_module(void)
>--
>1.7.9.5
>
>--
>To unsubscribe from this list: send the line "unsubscribe netdev" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at http://vger.kernel.org/majordomo-info.html
Reviewed-by: Jiri Pirko <jiri@resnulli.us>
^ permalink raw reply
* RE: [PATCH 00/17] drivers: hv: kvp
From: KY Srinivasan @ 2012-07-29 22:50 UTC (permalink / raw)
To: Greg KH
Cc: olaf@aepfle.de, netdev@vger.kernel.org,
linux-kernel@vger.kernel.org, virtualization@lists.osdl.org,
apw@canonical.com, devel@linuxdriverproject.org,
ben@decadent.org.uk
In-Reply-To: <20120724155412.GA23918@kroah.com>
> -----Original Message-----
> From: Greg KH [mailto:gregkh@linuxfoundation.org]
> Sent: Tuesday, July 24, 2012 11:54 AM
> To: KY Srinivasan
> Cc: linux-kernel@vger.kernel.org; devel@linuxdriverproject.org;
> virtualization@lists.osdl.org; olaf@aepfle.de; apw@canonical.com;
> netdev@vger.kernel.org; ben@decadent.org.uk
> Subject: Re: [PATCH 00/17] drivers: hv: kvp
>
> On Tue, Jul 24, 2012 at 09:01:12AM -0700, K. Y. Srinivasan wrote:
> > This patchset expands the KVP (Key Value Pair) functionality to
> > implement the mechanism to GET/SET IP addresses in the guest. This
> > functionality is used in Windows Server 2012 to implement VM
> > replication functionality. The way IP configuration information
> > is managed is distro specific. Based on the feedback I have gotten
> > from Olaf, Greg, Steve, Ben and Mairus, I have chosen to seperate
> > distro specific code from this patch-set. Most of the GET operation
> > can be implemented in a way that is completely distro independent and
> > I have implemented that as such and is included in this patch-set.
> > Some of the attributes that can only be fetched in a distro
> > dependent way as well the mechanism for configuring an interface
> > (the SET operation) that is clearly distro specific is to be
> > implemented via external scripts that will be invoked via the KVP
> > code. We define here the interface to these scripts.
> >
> > Adding support for IP injection resulted in some changes to the
> > protocol between the user level daemon and the kernel driver.
> > These changes have been implemented in way that would retain
> > compatibility with older daemons. I would like to thank Olaf and
> > Greg for pointing out the compatibility issue.
>
> Due to this being the middle of the merge window, I will not be able to
> look at this until after 3.6-rc1 is out.
Thanks Greg. In the meantime, I have addressed all the comments that both Olaf
and Ben have posted on this patch-set. Since addressing these comments changed
some data structures, I think it will be best if you dropped this patch-set. I will post the
updated patch-set shortly.
Regards,
K. Y
^ permalink raw reply
* Re: [PATCH] net/tun: fix ioctl() based info leaks
From: richard -rw- weinberger @ 2012-07-29 23:11 UTC (permalink / raw)
To: Mathias Krause; +Cc: David S. Miller, netdev
In-Reply-To: <1343595494-10414-1-git-send-email-minipli@googlemail.com>
On Sun, Jul 29, 2012 at 10:58 PM, Mathias Krause <minipli@googlemail.com> wrote:
> The tun module leaks up to 36 bytes of memory by not initializing a
> structure located on the stack that gets copied to user memory by the
> TUNGETIFF and SIOCGIFHWADDR ioctl()s.
>
> Signed-off-by: Mathias Krause <minipli@googlemail.com>
> ---
> drivers/net/tun.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/net/tun.c b/drivers/net/tun.c
> index 987aeef..cadeb94 100644
> --- a/drivers/net/tun.c
> +++ b/drivers/net/tun.c
> @@ -1252,9 +1252,12 @@ static long __tun_chr_ioctl(struct file *file, unsigned int cmd,
> int vnet_hdr_sz;
> int ret;
>
> - if (cmd == TUNSETIFF || _IOC_TYPE(cmd) == 0x89)
> + if (cmd == TUNSETIFF || _IOC_TYPE(cmd) == 0x89) {
> if (copy_from_user(&ifr, argp, ifreq_len))
> return -EFAULT;
> + } else {
> + memset(&ifr, 0, sizeof(ifr));
> + }
>
> if (cmd == TUNGETFEATURES) {
> /* Currently this just means: "what IFF flags are valid?".
The fix makes sense to me.
Beside of the fix, why are you adding braces to if and else?
We don't use braces on single statements.
--
Thanks,
//richard
^ permalink raw reply
* [RFC v2 0/2] PCI-Express Non-Transparent Bridge Support
From: Jon Mason @ 2012-07-30 0:26 UTC (permalink / raw)
To: linux-kernel; +Cc: netdev, linux-pci, Dave Jiang
Version 2 of the patch that adds support for PCI-Express Non-Transparent
Bridges in Linux. This version incorporates the changes suggested by
Stephen Hemminger <shemminger@vyatta.com>
Greg KH <gregkh@linuxfoundation.org>
chetan loke <loke.chetan@gmail.com>
Jiri Pirko <jiri@resnulli.us>
Thanks,
Jon
^ permalink raw reply
* [RFC v2 1/2] PCI-Express Non-Transparent Bridge Support
From: Jon Mason @ 2012-07-30 0:26 UTC (permalink / raw)
To: linux-kernel; +Cc: netdev, linux-pci, Dave Jiang
In-Reply-To: <1343607994-32415-1-git-send-email-jon.mason@intel.com>
A PCI-Express non-transparent bridge (NTB) is a point-to-point PCIe bus
connecting 2 systems, providing electrical isolation between the two subsystems.
A non-transparent bridge is functionally similar to a transparent bridge except
that both sides of the bridge have their own independent address domains. The
host on one side of the bridge will not have the visibility of the complete
memory or I/O space on the other side of the bridge. To communicate across the
non-transparent bridge, each NTB endpoint has one (or more) apertures exposed to
the local system. Writes to these apertures are mirrored to memory on the
remote system. Communications can also occur through the use of doorbell
registers that initiate interrupts to the alternate domain, and scratch-pad
registers accessible from both sides.
The NTB device driver is needed to configure these memory windows, doorbell, and
scratch-pad registers as well as use them in such a way as they can be turned
into a viable communication channel to the remote system. ntb_hw.[ch]
determines the usage model (NTB to NTB or NTB to Root Port) and abstracts away
the underlying hardware to provide access and a common interface to the doorbell
registers, scratch pads, and memory windows. These hardware interfaces are
exported so that other, non-mainlined kernel drivers can access these.
ntb_transport.[ch] also uses the exported interfaces in ntb_hw.[ch] to setup a
communication channel(s) and provide a reliable way of transferring data from
one side to the other, which it then exports so that "client" drivers can access
them. These client drivers are used to provide a standard kernel interface
(i.e., Ethernet device) to NTB, such that Linux can transfer data from one
system to the other in a standard way.
Signed-off-by: Jon Mason <jon.mason@intel.com>
---
MAINTAINERS | 6 +
drivers/Kconfig | 2 +
drivers/Makefile | 1 +
drivers/ntb/Kconfig | 13 +
drivers/ntb/Makefile | 3 +
drivers/ntb/ntb_hw.c | 1178 ++++++++++++++++++++++++++++++++++++
drivers/ntb/ntb_hw.h | 206 +++++++
drivers/ntb/ntb_regs.h | 150 +++++
drivers/ntb/ntb_transport.c | 1387 +++++++++++++++++++++++++++++++++++++++++++
include/linux/ntb.h | 92 +++
10 files changed, 3038 insertions(+), 0 deletions(-)
create mode 100644 drivers/ntb/Kconfig
create mode 100644 drivers/ntb/Makefile
create mode 100644 drivers/ntb/ntb_hw.c
create mode 100644 drivers/ntb/ntb_hw.h
create mode 100644 drivers/ntb/ntb_regs.h
create mode 100644 drivers/ntb/ntb_transport.c
create mode 100644 include/linux/ntb.h
diff --git a/MAINTAINERS b/MAINTAINERS
index bd45164..9d57552 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4844,6 +4844,12 @@ S: Maintained
F: Documentation/scsi/NinjaSCSI.txt
F: drivers/scsi/nsp32*
+NTB DRIVER
+M: Jon Mason <jon.mason@intel.com>
+S: Supported
+F: drivers/ntb/
+F: include/linux/ntb.h
+
NTFS FILESYSTEM
M: Anton Altaparmakov <anton@tuxera.com>
L: linux-ntfs-dev@lists.sourceforge.net
diff --git a/drivers/Kconfig b/drivers/Kconfig
index bfc9186..ebc16d3 100644
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -148,4 +148,6 @@ source "drivers/iio/Kconfig"
source "drivers/vme/Kconfig"
+source "drivers/ntb/Kconfig"
+
endmenu
diff --git a/drivers/Makefile b/drivers/Makefile
index 2ba29ff..39bba94 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -137,3 +137,4 @@ obj-$(CONFIG_EXTCON) += extcon/
obj-$(CONFIG_MEMORY) += memory/
obj-$(CONFIG_IIO) += iio/
obj-$(CONFIG_VME_BUS) += vme/
+obj-$(CONFIG_NTB) += ntb/
diff --git a/drivers/ntb/Kconfig b/drivers/ntb/Kconfig
new file mode 100644
index 0000000..f69df793
--- /dev/null
+++ b/drivers/ntb/Kconfig
@@ -0,0 +1,13 @@
+config NTB
+ tristate "Intel Non-Transparent Bridge support"
+ depends on PCI
+ depends on X86
+ help
+ The PCI-E Non-transparent bridge hardware is a point-to-point PCI-E bus
+ connecting 2 systems. When configured, writes to the device's PCI
+ mapped memory will be mirrored to a buffer on the remote system. The
+ ntb Linux driver uses this point-to-point communication as a method to
+ transfer data from one system to the other.
+
+ If unsure, say N.
+
diff --git a/drivers/ntb/Makefile b/drivers/ntb/Makefile
new file mode 100644
index 0000000..15cb59f
--- /dev/null
+++ b/drivers/ntb/Makefile
@@ -0,0 +1,3 @@
+obj-$(CONFIG_NTB) += ntb.o
+
+ntb-objs := ntb_hw.o ntb_transport.o
diff --git a/drivers/ntb/ntb_hw.c b/drivers/ntb/ntb_hw.c
new file mode 100644
index 0000000..11cb526
--- /dev/null
+++ b/drivers/ntb/ntb_hw.c
@@ -0,0 +1,1178 @@
+/*
+ * This file is provided under a dual BSD/GPLv2 license. When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright(c) 2012 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+ * The full GNU General Public License is included in this distribution
+ * in the file called LICENSE.GPL.
+ *
+ * BSD LICENSE
+ *
+ * Copyright(c) 2012 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copy
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * Intel PCIe NTB Linux driver
+ *
+ * Contact Information:
+ * Jon Mason <jon.mason@intel.com>
+ */
+#include <linux/debugfs.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/slab.h>
+#include "ntb_hw.h"
+#include "ntb_regs.h"
+
+#define NTB_NAME "Intel(R) PCI-E Non-Transparent Bridge Driver"
+#define NTB_VER "0.21"
+
+MODULE_DESCRIPTION(NTB_NAME);
+MODULE_VERSION(NTB_VER);
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_AUTHOR("Intel Corporation");
+
+enum {
+ NTB_CONN_CLASSIC = 0,
+ NTB_CONN_B2B,
+ NTB_CONN_RP,
+};
+
+enum {
+ NTB_DEV_USD = 0,
+ NTB_DEV_DSD,
+};
+
+enum {
+ SNB_HW = 0,
+ BWD_HW,
+};
+
+/* Translate memory window 0,1 to BAR 2,4 */
+#define MW_TO_BAR(mw) (mw * 2 + 2)
+
+static DEFINE_PCI_DEVICE_TABLE(ntb_pci_tbl) = {
+ {PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_NTB_B2B_BWD)},
+ {PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_NTB_B2B_JSF)},
+ {PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_NTB_CLASSIC_JSF)},
+ {PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_NTB_RP_JSF)},
+ {PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_NTB_RP_SNB)},
+ {PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_NTB_B2B_SNB)},
+ {PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_NTB_CLASSIC_SNB)},
+ {0}
+};
+MODULE_DEVICE_TABLE(pci, ntb_pci_tbl);
+
+/**
+ * ntb_register_event_callback() - register event callback
+ * @ndev: pointer to ntb_device instance
+ * @func: callback function to register
+ *
+ * This function registers a callback for any HW driver events such as link
+ * up/down, power management notices and etc.
+ *
+ * RETURNS: An appropriate -ERRNO error value on error, or zero for success.
+ */
+int ntb_register_event_callback(struct ntb_device *ndev,
+ void (*func)(void *handle, unsigned int event))
+{
+ if (ndev->event_cb)
+ return -EINVAL;
+
+ ndev->event_cb = func;
+
+ return 0;
+}
+
+/**
+ * ntb_unregister_event_callback() - unregisters the event callback
+ * @ndev: pointer to ntb_device instance
+ *
+ * This function unregisters the existing callback from transport
+ */
+void ntb_unregister_event_callback(struct ntb_device *ndev)
+{
+ ndev->event_cb = NULL;
+}
+
+/**
+ * ntb_register_db_callback() - register a callback for doorbell interrupt
+ * @ndev: pointer to ntb_device instance
+ * @idx: doorbell index to register callback, zero based
+ * @func: callback function to register
+ *
+ * This function registers a callback function for the doorbell interrupt
+ * on the primary side. The function will unmask the doorbell as well to
+ * allow interrupt.
+ *
+ * RETURNS: An appropriate -ERRNO error value on error, or zero for success.
+ */
+int ntb_register_db_callback(struct ntb_device *ndev, unsigned int idx,
+ void *data, void (*func)(void *data, int db_num))
+{
+ unsigned long mask;
+
+ if (idx >= ndev->max_cbs || ndev->db_cb[idx].callback) {
+ dev_warn(&ndev->pdev->dev, "Invalid Index.\n");
+ return -EINVAL;
+ }
+
+ ndev->db_cb[idx].callback = func;
+ ndev->db_cb[idx].data = data;
+
+ /* unmask interrupt */
+ mask = readw(ndev->reg_ofs.pdb_mask);
+ clear_bit(idx * ndev->bits_per_vector, &mask);
+ writew(mask, ndev->reg_ofs.pdb_mask);
+
+ return 0;
+}
+
+/**
+ * ntb_unregister_db_callback() - unregister a callback for doorbell interrupt
+ * @ndev: pointer to ntb_device instance
+ * @idx: doorbell index to register callback, zero based
+ *
+ * This function unregisters a callback function for the doorbell interrupt
+ * on the primary side. The function will also mask the said doorbell.
+ */
+void ntb_unregister_db_callback(struct ntb_device *ndev, unsigned int idx)
+{
+ unsigned long mask;
+
+ if (idx >= ndev->max_cbs || !ndev->db_cb[idx].callback)
+ return;
+
+ mask = readw(ndev->reg_ofs.pdb_mask);
+ set_bit(idx * ndev->bits_per_vector, &mask);
+ writew(mask, ndev->reg_ofs.pdb_mask);
+
+ ndev->db_cb[idx].callback = NULL;
+}
+
+/**
+ * ntb_find_transport() - find the transport pointer
+ * @transport: pointer to pci device
+ *
+ * Given the pci device pointer, return the transport pointer passed in when
+ * the transport attached when it was inited.
+ *
+ * RETURNS: pointer to transport.
+ */
+void *ntb_find_transport(struct pci_dev *pdev)
+{
+ struct ntb_device *ndev = pci_get_drvdata(pdev);
+ return ndev->ntb_transport;
+}
+
+/**
+ * ntb_register_transport() - Register NTB transport with NTB HW driver
+ * @transport: transport identifier
+ *
+ * This function allows a transport to reserve the hardware driver for
+ * NTB usage.
+ *
+ * RETURNS: pointer to ntb_device, NULL on error.
+ */
+struct ntb_device *ntb_register_transport(struct pci_dev *pdev, void *transport)
+{
+ struct ntb_device *ndev = pci_get_drvdata(pdev);
+
+ if (ndev->ntb_transport)
+ return NULL;
+
+ ndev->ntb_transport = transport;
+ return ndev;
+}
+
+/**
+ * ntb_unregister_transport() - Unregister the transport with the NTB HW driver
+ * @ndev - ntb_device of the transport to be freed
+ *
+ * This function unregisters the transport from the HW driver and performs any
+ * necessary cleanups.
+ */
+void ntb_unregister_transport(struct ntb_device *ndev)
+{
+ int i;
+
+ if (!ndev->ntb_transport)
+ return;
+
+ for (i = 0; i < ndev->max_cbs; i++)
+ ntb_unregister_db_callback(ndev, i);
+
+ ntb_unregister_event_callback(ndev);
+ ndev->ntb_transport = NULL;
+}
+
+/**
+ * ntb_get_max_spads() - get the total scratch regs usable
+ * @ndev: pointer to ntb_device instance
+ *
+ * This function returns the max 32bit scratchpad registers usable by the
+ * upper layer.
+ *
+ * RETURNS: total number of scratch pad registers available
+ */
+int ntb_get_max_spads(struct ntb_device *ndev)
+{
+ return ndev->limits.max_spads;
+}
+
+/**
+ * ntb_write_local_spad() - write to the secondary scratchpad register
+ * @ndev: pointer to ntb_device instance
+ * @idx: index to the scratchpad register, 0 based
+ * @val: the data value to put into the register
+ *
+ * This function allows writing of a 32bit value to the indexed scratchpad
+ * register. The register resides on the secondary (external) side.
+ *
+ * RETURNS: An appropriate -ERRNO error value on error, or zero for success.
+ */
+int ntb_write_local_spad(struct ntb_device *ndev, unsigned int idx, u32 val)
+{
+ if (idx >= ndev->limits.max_spads)
+ return -EINVAL;
+
+ dev_dbg(&ndev->pdev->dev, "Writing %x to local scratch pad index %d\n",
+ val, idx);
+ writel(val, ndev->reg_ofs.spad_read + idx * 4);
+
+ return 0;
+}
+
+/**
+ * ntb_read_local_spad() - read from the primary scratchpad register
+ * @ndev: pointer to ntb_device instance
+ * @idx: index to scratchpad register, 0 based
+ * @val: pointer to 32bit integer for storing the register value
+ *
+ * This function allows reading of the 32bit scratchpad register on
+ * the primary (internal) side.
+ *
+ * RETURNS: An appropriate -ERRNO error value on error, or zero for success.
+ */
+int ntb_read_local_spad(struct ntb_device *ndev, unsigned int idx, u32 *val)
+{
+ if (idx >= ndev->limits.max_spads)
+ return -EINVAL;
+
+ *val = readl(ndev->reg_ofs.spad_write + idx * 4);
+ dev_dbg(&ndev->pdev->dev,
+ "Reading %x from local scratch pad index %d\n", *val, idx);
+
+ return 0;
+}
+
+/**
+ * ntb_write_remote_spad() - write to the secondary scratchpad register
+ * @ndev: pointer to ntb_device instance
+ * @idx: index to the scratchpad register, 0 based
+ * @val: the data value to put into the register
+ *
+ * This function allows writing of a 32bit value to the indexed scratchpad
+ * register. The register resides on the secondary (external) side.
+ *
+ * RETURNS: An appropriate -ERRNO error value on error, or zero for success.
+ */
+int ntb_write_remote_spad(struct ntb_device *ndev, unsigned int idx, u32 val)
+{
+ if (idx >= ndev->limits.max_spads)
+ return -EINVAL;
+
+ dev_dbg(&ndev->pdev->dev, "Writing %x to remote scratch pad index %d\n",
+ val, idx);
+ writel(val, ndev->reg_ofs.spad_write + idx * 4);
+
+ return 0;
+}
+
+/**
+ * ntb_read_remote_spad() - read from the primary scratchpad register
+ * @ndev: pointer to ntb_device instance
+ * @idx: index to scratchpad register, 0 based
+ * @val: pointer to 32bit integer for storing the register value
+ *
+ * This function allows reading of the 32bit scratchpad register on
+ * the primary (internal) side.
+ *
+ * RETURNS: An appropriate -ERRNO error value on error, or zero for success.
+ */
+int ntb_read_remote_spad(struct ntb_device *ndev, unsigned int idx, u32 *val)
+{
+ if (idx >= ndev->limits.max_spads)
+ return -EINVAL;
+
+ *val = readl(ndev->reg_ofs.spad_read + idx * 4);
+ dev_dbg(&ndev->pdev->dev,
+ "Reading %x from remote scratch pad index %d\n", *val, idx);
+
+ return 0;
+}
+
+/**
+ * ntb_get_mw_vbase() - get virtual addr for the NTB memory window
+ * @ndev: pointer to ntb_device instance
+ * @mw: memory window number
+ *
+ * This function provides the base virtual address of the memory window
+ * specified.
+ *
+ * RETURNS: pointer to virtual address, or NULL on error.
+ */
+void *ntb_get_mw_vbase(struct ntb_device *ndev, unsigned int mw)
+{
+ if (mw > NTB_NUM_MW)
+ return NULL;
+
+ return ndev->mw[mw].vbase;
+}
+
+/**
+ * ntb_get_mw_size() - return size of NTB memory window
+ * @ndev: pointer to ntb_device instance
+ * @mw: memory window number
+ *
+ * This function provides the physical size of the memory window specified
+ *
+ * RETURNS: the size of the memory window or zero on error
+ */
+resource_size_t ntb_get_mw_size(struct ntb_device *ndev, unsigned int mw)
+{
+ if (mw > NTB_NUM_MW)
+ return 0;
+
+ return ndev->mw[mw].bar_sz;
+}
+
+/**
+ * ntb_set_mw_addr - set the memory window address
+ * @ndev: pointer to ntb_device instance
+ * @mw: memory window number
+ * @addr: base address for data
+ *
+ * This function sets the base physical address of the memory window. This
+ * memory address is where data from the remote system will be transfered into
+ * or out of depending on how the transport is configured.
+ */
+void ntb_set_mw_addr(struct ntb_device *ndev, unsigned int mw, u64 addr)
+{
+ if (mw > NTB_NUM_MW)
+ return;
+
+ dev_dbg(&ndev->pdev->dev, "Writing addr %Lx to BAR %d\n", addr,
+ MW_TO_BAR(mw));
+
+ ndev->mw[mw].phys_addr = addr;
+
+ switch (MW_TO_BAR(mw)) {
+ case NTB_BAR_23:
+ writeq(addr, ndev->reg_ofs.sbar2_xlat);
+ break;
+ case NTB_BAR_45:
+ writeq(addr, ndev->reg_ofs.sbar4_xlat);
+ break;
+ }
+}
+
+/**
+ * ntb_ring_sdb() - Set the doorbell on the secondary/external side
+ * @ndev: pointer to ntb_device instance
+ * @db: doorbell to ring
+ *
+ * This function allows triggering of a doorbell on the secondary/external
+ * side that will initiate an interrupt on the remote host
+ *
+ * RETURNS: An appropriate -ERRNO error value on error, or zero for success.
+ */
+void ntb_ring_sdb(struct ntb_device *ndev, unsigned int db)
+{
+ dev_dbg(&ndev->pdev->dev, "%s: ringing doorbell %d\n", __func__, db);
+
+ if (ndev->hw_type == BWD_HW)
+ writeq((u64) 1 << db, ndev->reg_ofs.sdb);
+ else
+ writew(((1 << ndev->bits_per_vector) - 1) <<
+ (db * ndev->bits_per_vector), ndev->reg_ofs.sdb);
+}
+
+static void ntb_link_event(struct ntb_device *ndev, int link_state)
+{
+ unsigned int event;
+
+ if (ndev->link_status == link_state)
+ return;
+
+ if (link_state == NTB_LINK_UP) {
+ u16 status;
+
+ dev_info(&ndev->pdev->dev, "Link Up\n");
+ ndev->link_status = NTB_LINK_UP;
+ event = NTB_EVENT_HW_LINK_UP;
+
+ if (ndev->hw_type == BWD_HW)
+ status = readw(ndev->reg_ofs.lnk_stat);
+ else {
+ int rc = pci_read_config_word(ndev->pdev,
+ SNB_LINK_STATUS_OFFSET,
+ &status);
+ if (rc)
+ return;
+ }
+ dev_info(&ndev->pdev->dev, "Link Width %d, Link Speed %d\n",
+ (status & NTB_LINK_WIDTH_MASK) >> 4,
+ (status & NTB_LINK_SPEED_MASK));
+ } else {
+ dev_info(&ndev->pdev->dev, "Link Down\n");
+ ndev->link_status = NTB_LINK_DOWN;
+ event = NTB_EVENT_HW_LINK_DOWN;
+ }
+
+ /* notify the upper layer if we have an event change */
+ if (ndev->event_cb)
+ ndev->event_cb(ndev->ntb_transport, event);
+}
+
+static int ntb_link_status(struct ntb_device *ndev)
+{
+ int link_state;
+
+ if (ndev->hw_type == BWD_HW) {
+ u32 ntb_cntl;
+
+ ntb_cntl = readl(ndev->reg_ofs.lnk_cntl);
+ if (ntb_cntl & BWD_CNTL_LINK_DOWN)
+ link_state = NTB_LINK_DOWN;
+ else
+ link_state = NTB_LINK_UP;
+ } else {
+ u16 status;
+ int rc;
+
+ rc = pci_read_config_word(ndev->pdev, SNB_LINK_STATUS_OFFSET,
+ &status);
+ if (rc)
+ return rc;
+
+ if (status & NTB_LINK_STATUS_ACTIVE)
+ link_state = NTB_LINK_UP;
+ else
+ link_state = NTB_LINK_DOWN;
+ }
+
+ ntb_link_event(ndev, link_state);
+
+ return 0;
+}
+
+/* BWD doesn't have link status interrupt, poll on that platform */
+static void ntb_handle_heartbeat(struct work_struct *work)
+{
+ struct ntb_device *ndev = container_of(work, struct ntb_device,
+ hb_timer.work);
+ unsigned long ts = jiffies;
+
+ /* If we haven't gotten an interrupt in a while, check the BWD link
+ * status bit
+ */
+ if (ts > ndev->last_ts + NTB_HB_TIMEOUT) {
+ int rc = ntb_link_status(ndev);
+ if (rc)
+ dev_err(&ndev->pdev->dev,
+ "Error determining link status\n");
+ }
+
+ schedule_delayed_work(&ndev->hb_timer, NTB_HB_TIMEOUT);
+}
+
+static int ntb_xeon_setup(struct ntb_device *ndev)
+{
+ int rc;
+ u8 val;
+
+ ndev->hw_type = SNB_HW;
+
+ rc = pci_read_config_byte(ndev->pdev, NTB_PPD_OFFSET, &val);
+ if (rc)
+ return rc;
+
+ switch (val & SNB_PPD_CONN_TYPE) {
+ case NTB_CONN_B2B:
+ ndev->conn_type = NTB_CONN_B2B;
+ break;
+ case NTB_CONN_CLASSIC:
+ case NTB_CONN_RP:
+ default:
+ dev_err(&ndev->pdev->dev, "Only B2B supported at this time\n");
+ return -EINVAL;
+ }
+
+ if (val & SNB_PPD_DEV_TYPE)
+ ndev->dev_type = NTB_DEV_DSD;
+ else
+ ndev->dev_type = NTB_DEV_USD;
+
+ ndev->reg_ofs.pdb = ndev->reg_base + SNB_PDOORBELL_OFFSET;
+ ndev->reg_ofs.pdb_mask = ndev->reg_base + SNB_PDBMSK_OFFSET;
+ ndev->reg_ofs.sbar2_xlat = ndev->reg_base + SNB_SBAR2XLAT_OFFSET;
+ ndev->reg_ofs.sbar4_xlat = ndev->reg_base + SNB_SBAR4XLAT_OFFSET;
+ ndev->reg_ofs.lnk_cntl = ndev->reg_base + SNB_NTBCNTL_OFFSET;
+ ndev->reg_ofs.lnk_stat = ndev->reg_base + SNB_LINK_STATUS_OFFSET;
+ ndev->reg_ofs.spad_read = ndev->reg_base + SNB_SPAD_OFFSET;
+ ndev->reg_ofs.spci_cmd = ndev->reg_base + SNB_PCICMD_OFFSET;
+
+ if (ndev->conn_type == NTB_CONN_B2B) {
+ ndev->reg_ofs.sdb = ndev->reg_base + SNB_B2B_DOORBELL_OFFSET;
+ ndev->reg_ofs.spad_write = ndev->reg_base + SNB_B2B_SPAD_OFFSET;
+ ndev->limits.max_spads = SNB_MAX_SPADS;
+ } else {
+ ndev->reg_ofs.sdb = ndev->reg_base + SNB_SDOORBELL_OFFSET;
+ ndev->reg_ofs.spad_write = ndev->reg_base + SNB_SPAD_OFFSET;
+ ndev->limits.max_spads = SNB_MAX_COMPAT_SPADS;
+ }
+
+ ndev->limits.max_db_bits = SNB_MAX_DB_BITS;
+ ndev->limits.msix_cnt = SNB_MSIX_CNT;
+ ndev->bits_per_vector = SNB_DB_BITS_PER_VEC;
+
+ return 0;
+}
+
+static int ntb_bwd_setup(struct ntb_device *ndev)
+{
+ int rc;
+ u32 val;
+
+ ndev->hw_type = BWD_HW;
+
+ rc = pci_read_config_dword(ndev->pdev, NTB_PPD_OFFSET, &val);
+ if (rc)
+ return rc;
+
+ switch ((val & BWD_PPD_CONN_TYPE) >> 8) {
+ case NTB_CONN_B2B:
+ ndev->conn_type = NTB_CONN_B2B;
+ break;
+ case NTB_CONN_RP:
+ default:
+ dev_err(&ndev->pdev->dev, "Only B2B supported at this time\n");
+ return -EINVAL;
+ }
+
+ if (val & BWD_PPD_DEV_TYPE)
+ ndev->dev_type = NTB_DEV_DSD;
+ else
+ ndev->dev_type = NTB_DEV_USD;
+
+ /* Initiate PCI-E link training */
+ rc = pci_write_config_dword(ndev->pdev, NTB_PPD_OFFSET,
+ val | BWD_PPD_INIT_LINK);
+ if (rc)
+ return rc;
+
+ ndev->reg_ofs.pdb = ndev->reg_base + BWD_PDOORBELL_OFFSET;
+ ndev->reg_ofs.pdb_mask = ndev->reg_base + BWD_PDBMSK_OFFSET;
+ ndev->reg_ofs.sbar2_xlat = ndev->reg_base + BWD_SBAR2XLAT_OFFSET;
+ ndev->reg_ofs.sbar4_xlat = ndev->reg_base + BWD_SBAR4XLAT_OFFSET;
+ ndev->reg_ofs.lnk_cntl = ndev->reg_base + BWD_NTBCNTL_OFFSET;
+ ndev->reg_ofs.lnk_stat = ndev->reg_base + BWD_LINK_STATUS_OFFSET;
+ ndev->reg_ofs.spad_read = ndev->reg_base + BWD_SPAD_OFFSET;
+ ndev->reg_ofs.spci_cmd = ndev->reg_base + BWD_PCICMD_OFFSET;
+
+ if (ndev->conn_type == NTB_CONN_B2B) {
+ ndev->reg_ofs.sdb = ndev->reg_base + BWD_B2B_DOORBELL_OFFSET;
+ ndev->reg_ofs.spad_write = ndev->reg_base + BWD_B2B_SPAD_OFFSET;
+ ndev->limits.max_spads = BWD_MAX_SPADS;
+ } else {
+ ndev->reg_ofs.sdb = ndev->reg_base + BWD_PDOORBELL_OFFSET;
+ ndev->reg_ofs.spad_write = ndev->reg_base + BWD_SPAD_OFFSET;
+ ndev->limits.max_spads = BWD_MAX_COMPAT_SPADS;
+ }
+
+ ndev->limits.max_db_bits = BWD_MAX_DB_BITS;
+ ndev->limits.msix_cnt = BWD_MSIX_CNT;
+ ndev->bits_per_vector = BWD_DB_BITS_PER_VEC;
+
+ /* Since bwd doesn't have a link interrupt, setup a heartbeat timer */
+ INIT_DELAYED_WORK(&ndev->hb_timer, ntb_handle_heartbeat);
+ schedule_delayed_work(&ndev->hb_timer, NTB_HB_TIMEOUT);
+
+ return 0;
+}
+
+static int __devinit ntb_device_setup(struct ntb_device *ndev)
+{
+ int rc;
+
+ switch (ndev->pdev->device) {
+ case PCI_DEVICE_ID_INTEL_NTB_2ND_SNB:
+ case PCI_DEVICE_ID_INTEL_NTB_RP_JSF:
+ case PCI_DEVICE_ID_INTEL_NTB_RP_SNB:
+ case PCI_DEVICE_ID_INTEL_NTB_CLASSIC_JSF:
+ case PCI_DEVICE_ID_INTEL_NTB_CLASSIC_SNB:
+ case PCI_DEVICE_ID_INTEL_NTB_B2B_JSF:
+ case PCI_DEVICE_ID_INTEL_NTB_B2B_SNB:
+ rc = ntb_xeon_setup(ndev);
+ break;
+ case PCI_DEVICE_ID_INTEL_NTB_B2B_BWD:
+ rc = ntb_bwd_setup(ndev);
+ break;
+ default:
+ rc = -ENODEV;
+ }
+
+ /* Enable Bus Master and Memory Space on the secondary side */
+ writew(PCI_COMMAND_MEMORY | PCI_COMMAND_MASTER, ndev->reg_ofs.spci_cmd);
+
+ return rc;
+}
+
+static void ntb_device_free(struct ntb_device *ndev)
+{
+ if (ndev->hw_type == BWD_HW)
+ cancel_delayed_work_sync(&ndev->hb_timer);
+}
+
+static irqreturn_t bwd_callback_msix_irq(int irq, void *data)
+{
+ struct ntb_db_cb *db_cb = data;
+ struct ntb_device *ndev = db_cb->ndev;
+
+ dev_dbg(&ndev->pdev->dev, "MSI-X irq %d received for DB %d\n", irq,
+ db_cb->db_num);
+
+ if (db_cb->callback)
+ db_cb->callback(db_cb->data, db_cb->db_num);
+
+ /* No need to check for the specific HB irq, any interrupt means
+ * we're connected.
+ */
+ ndev->last_ts = jiffies;
+
+ writeq((u64) 1 << db_cb->db_num, ndev->reg_ofs.pdb);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t xeon_callback_msix_irq(int irq, void *data)
+{
+ struct ntb_db_cb *db_cb = data;
+ struct ntb_device *ndev = db_cb->ndev;
+
+ dev_dbg(&ndev->pdev->dev, "MSI-X irq %d received for DB %d\n", irq,
+ db_cb->db_num);
+
+ if (db_cb->callback)
+ db_cb->callback(db_cb->data, db_cb->db_num);
+
+ /* On Sandybridge, there are 16 bits in the interrupt register
+ * but only 4 vectors. So, 5 bits are assigned to the first 3
+ * vectors, with the 4th having a single bit for link
+ * interrupts.
+ */
+ writew(((1 << ndev->bits_per_vector) - 1) <<
+ (db_cb->db_num * ndev->bits_per_vector), ndev->reg_ofs.pdb);
+
+ return IRQ_HANDLED;
+}
+
+/* Since we do not have a HW doorbell in BWD, this is only used in JF/JT */
+static irqreturn_t xeon_event_msix_irq(int irq, void *dev)
+{
+ struct ntb_device *ndev = dev;
+ int rc;
+
+ dev_dbg(&ndev->pdev->dev, "MSI-X irq %d received for Events\n", irq);
+
+ rc = ntb_link_status(ndev);
+ if (rc)
+ dev_err(&ndev->pdev->dev, "Error determining link status\n");
+
+ /* bit 15 is always the link bit */
+ writew(1 << ndev->limits.max_db_bits, ndev->reg_ofs.pdb);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t ntb_interrupt(int irq, void *dev)
+{
+ struct ntb_device *ndev = dev;
+ unsigned int i = 0;
+
+ if (ndev->hw_type == BWD_HW) {
+ u64 pdb = readq(ndev->reg_ofs.pdb);
+
+ dev_dbg(&ndev->pdev->dev, "irq %d - pdb = %Lx\n", irq, pdb);
+
+ while (pdb) {
+ i = __ffs(pdb);
+ pdb &= pdb - 1;
+ bwd_callback_msix_irq(irq, &ndev->db_cb[i]);
+ }
+ } else {
+ u16 pdb = readw(ndev->reg_ofs.pdb);
+
+ dev_dbg(&ndev->pdev->dev, "irq %d - pdb = %x sdb %x\n", irq,
+ pdb, readw(ndev->reg_ofs.sdb));
+
+ if (pdb & SNB_DB_HW_LINK) {
+ xeon_event_msix_irq(irq, dev);
+ pdb &= ~SNB_DB_HW_LINK;
+ }
+
+ while (pdb) {
+ i = __ffs(pdb);
+ pdb &= pdb - 1;
+ xeon_callback_msix_irq(irq, &ndev->db_cb[i]);
+ }
+ }
+
+ return IRQ_HANDLED;
+}
+
+static int ntb_setup_msix(struct ntb_device *ndev)
+{
+ struct pci_dev *pdev = ndev->pdev;
+ struct msix_entry *msix;
+ int msix_entries;
+ int rc, i, pos;
+ u16 val;
+
+ pos = pci_find_capability(pdev, PCI_CAP_ID_MSIX);
+ if (!pos) {
+ rc = -EIO;
+ goto err;
+ }
+
+ rc = pci_read_config_word(pdev, pos + PCI_MSIX_FLAGS, &val);
+ if (rc)
+ goto err;
+
+ msix_entries = msix_table_size(val);
+ if (msix_entries > ndev->limits.msix_cnt) {
+ rc = -EINVAL;
+ goto err;
+ }
+
+ ndev->msix_entries = kmalloc(sizeof(struct msix_entry) * msix_entries,
+ GFP_KERNEL);
+ if (!ndev->msix_entries) {
+ rc = -ENOMEM;
+ goto err;
+ }
+
+ for (i = 0; i < msix_entries; i++)
+ ndev->msix_entries[i].entry = i;
+
+ rc = pci_enable_msix(pdev, ndev->msix_entries, msix_entries);
+ if (rc < 0)
+ goto err1;
+ if (rc > 0) {
+ /* On SNB, the link interrupt is always tied to 4th vector. If
+ * we can't get all 4, then we can't use MSI-X.
+ */
+ if (ndev->hw_type != BWD_HW) {
+ rc = -EIO;
+ goto err1;
+ }
+
+ dev_warn(&pdev->dev,
+ "Only %d MSI-X vectors. Limiting the number of queues to that number.\n",
+ rc);
+ msix_entries = rc;
+ }
+
+ for (i = 0; i < msix_entries; i++) {
+ msix = &ndev->msix_entries[i];
+ WARN_ON(!msix->vector);
+
+ /* Use the last MSI-X vector for Link status */
+ if (ndev->hw_type == BWD_HW) {
+ rc = request_irq(msix->vector, bwd_callback_msix_irq, 0,
+ "ntb-callback-msix", &ndev->db_cb[i]);
+ if (rc)
+ goto err2;
+ } else {
+ if (i == msix_entries - 1) {
+ rc = request_irq(msix->vector,
+ xeon_event_msix_irq, 0,
+ "ntb-event-msix", ndev);
+ if (rc)
+ goto err2;
+ } else {
+ rc = request_irq(msix->vector,
+ xeon_callback_msix_irq, 0,
+ "ntb-callback-msix",
+ &ndev->db_cb[i]);
+ if (rc)
+ goto err2;
+ }
+ }
+ }
+
+ ndev->num_msix = msix_entries;
+ if (ndev->hw_type == BWD_HW)
+ ndev->max_cbs = msix_entries;
+ else
+ ndev->max_cbs = msix_entries - 1;
+
+ return 0;
+
+err2:
+ while (--i >= 0) {
+ msix = &ndev->msix_entries[i];
+ if (ndev->hw_type != BWD_HW && i == ndev->num_msix - 1)
+ free_irq(msix->vector, ndev);
+ else
+ free_irq(msix->vector, &ndev->db_cb[i]);
+ }
+ pci_disable_msix(pdev);
+err1:
+ kfree(ndev->msix_entries);
+ dev_err(&pdev->dev, "Error allocating MSI-X interrupt\n");
+err:
+ ndev->num_msix = 0;
+ return rc;
+}
+
+static int ntb_setup_msi(struct ntb_device *ndev)
+{
+ struct pci_dev *pdev = ndev->pdev;
+ int rc;
+
+ rc = pci_enable_msi(pdev);
+ if (rc)
+ return rc;
+
+ rc = request_irq(pdev->irq, ntb_interrupt, 0, "ntb-msi", ndev);
+ if (rc) {
+ pci_disable_msi(pdev);
+ dev_err(&pdev->dev, "Error allocating MSI interrupt\n");
+ return rc;
+ }
+
+ return 0;
+}
+
+static int ntb_setup_intx(struct ntb_device *ndev)
+{
+ struct pci_dev *pdev = ndev->pdev;
+ int rc;
+
+ pci_msi_off(pdev);
+
+ /* Verify intx is enabled */
+ pci_intx(pdev, 1);
+
+ rc = request_irq(pdev->irq, ntb_interrupt, IRQF_SHARED, "ntb-intx",
+ ndev);
+ if (rc)
+ return rc;
+
+ return 0;
+}
+
+static int __devinit ntb_setup_interrupts(struct ntb_device *ndev)
+{
+ int rc;
+
+ /* On BWD, disable all interrupts. On SNB, disable all but Link
+ * Interrupt. The rest will be unmasked as callbacks are registered.
+ */
+ if (ndev->hw_type == BWD_HW)
+ writeq(~0, ndev->reg_ofs.pdb_mask);
+ else
+ writew(~(1 << ndev->limits.max_db_bits),
+ ndev->reg_ofs.pdb_mask);
+
+ rc = ntb_setup_msix(ndev);
+ if (!rc)
+ goto done;
+
+ ndev->bits_per_vector = 1;
+ ndev->max_cbs = ndev->limits.max_db_bits;
+
+ rc = ntb_setup_msi(ndev);
+ if (!rc)
+ goto done;
+
+ rc = ntb_setup_intx(ndev);
+ if (rc) {
+ dev_err(&ndev->pdev->dev, "no usable interrupts\n");
+ return rc;
+ }
+
+done:
+ return 0;
+}
+
+static void __devexit ntb_free_interrupts(struct ntb_device *ndev)
+{
+ struct pci_dev *pdev = ndev->pdev;
+
+ /* mask interrupts */
+ if (ndev->hw_type == BWD_HW)
+ writeq(~0, ndev->reg_ofs.pdb_mask);
+ else
+ writew(~0, ndev->reg_ofs.pdb_mask);
+
+ if (ndev->num_msix) {
+ struct msix_entry *msix;
+ u32 i;
+
+ for (i = 0; i < ndev->num_msix; i++) {
+ msix = &ndev->msix_entries[i];
+ if (ndev->hw_type != BWD_HW && i == ndev->num_msix - 1)
+ free_irq(msix->vector, ndev);
+ else
+ free_irq(msix->vector, &ndev->db_cb[i]);
+ }
+ pci_disable_msix(pdev);
+ } else {
+ free_irq(pdev->irq, ndev);
+
+ if (pci_dev_msi_enabled(pdev))
+ pci_disable_msi(pdev);
+ }
+}
+
+static int __devinit ntb_create_callbacks(struct ntb_device *ndev)
+{
+ int i;
+
+ /* Checken-egg issue. We won't know how many callbacks are necessary
+ * until we see how many MSI-X vectors we get, but these pointers need
+ * to be passed into the MSI-X register fucntion. So, we allocate the
+ * max, knowing that they might not all be used, to work around this.
+ */
+ ndev->db_cb = kcalloc(ndev->limits.max_db_bits,
+ sizeof(struct ntb_db_cb),
+ GFP_KERNEL);
+ if (!ndev->db_cb)
+ return -ENOMEM;
+
+ for (i = 0; i < ndev->limits.max_db_bits; i++) {
+ ndev->db_cb[i].db_num = i;
+ ndev->db_cb[i].ndev = ndev;
+ }
+
+ return 0;
+}
+
+static void ntb_free_callbacks(struct ntb_device *ndev)
+{
+ int i;
+
+ for (i = 0; i < ndev->limits.max_db_bits; i++)
+ ntb_unregister_db_callback(ndev, i);
+
+ kfree(ndev->db_cb);
+}
+
+static int __devinit
+ntb_pci_probe(struct pci_dev *pdev,
+ __attribute__((unused)) const struct pci_device_id *id)
+{
+ struct ntb_device *ndev;
+ int rc, i;
+
+ ndev = kzalloc(sizeof(struct ntb_device), GFP_KERNEL);
+ if (!ndev)
+ return -ENOMEM;
+
+ ndev->pdev = pdev;
+ ndev->link_status = NTB_LINK_DOWN;
+ pci_set_drvdata(pdev, ndev);
+
+ rc = pci_enable_device(pdev);
+ if (rc)
+ goto err;
+
+ pci_set_master(ndev->pdev);
+
+ rc = pci_request_selected_regions(pdev, NTB_BAR_MASK, KBUILD_MODNAME);
+ if (rc)
+ goto err1;
+
+ ndev->reg_base = pci_ioremap_bar(pdev, NTB_BAR_MMIO);
+ if (!ndev->reg_base) {
+ dev_warn(&pdev->dev, "Cannot remap BAR 0\n");
+ rc = -EIO;
+ goto err2;
+ }
+
+ for (i = 0; i < NTB_NUM_MW; i++) {
+ ndev->mw[i].bar_sz = pci_resource_len(pdev, MW_TO_BAR(i));
+ ndev->mw[i].vbase =
+ ioremap_wc(pci_resource_start(pdev, MW_TO_BAR(i)),
+ ndev->mw[i].bar_sz);
+ dev_info(&pdev->dev, "MW %d size %d\n", i,
+ (u32) pci_resource_len(pdev, MW_TO_BAR(i)));
+ if (!ndev->mw[i].vbase) {
+ dev_warn(&pdev->dev, "Cannot remap BAR %d\n",
+ MW_TO_BAR(i));
+ rc = -EIO;
+ goto err3;
+ }
+ }
+
+ rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
+ if (rc) {
+ rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (rc)
+ goto err3;
+
+ dev_warn(&pdev->dev, "Cannot DMA highmem\n");
+ }
+
+ rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ if (rc) {
+ rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ if (rc)
+ goto err3;
+
+ dev_warn(&pdev->dev, "Cannot DMA consistent highmem\n");
+ }
+
+ rc = ntb_device_setup(ndev);
+ if (rc)
+ goto err3;
+
+ rc = ntb_create_callbacks(ndev);
+ if (rc)
+ goto err4;
+
+ rc = ntb_setup_interrupts(ndev);
+ if (rc)
+ goto err5;
+
+ /* The scratchpad registers keep the values between rmmod/insmod,
+ * blast them now
+ */
+ for (i = 0; i < ndev->limits.max_spads; i++) {
+ ntb_write_local_spad(ndev, i, 0);
+ ntb_write_remote_spad(ndev, i, 0);
+ }
+
+ rc = ntb_transport_init(pdev);
+ if (rc)
+ goto err6;
+
+ /* Let's bring the NTB link up */
+ writel(NTB_CNTL_BAR23_SNOOP | NTB_CNTL_BAR45_SNOOP,
+ ndev->reg_ofs.lnk_cntl);
+
+ return 0;
+
+err6:
+ ntb_free_interrupts(ndev);
+err5:
+ ntb_free_callbacks(ndev);
+err4:
+ ntb_device_free(ndev);
+err3:
+ for (i--; i >= 0; i--)
+ iounmap(ndev->mw[i].vbase);
+ iounmap(ndev->reg_base);
+err2:
+ pci_release_selected_regions(pdev, NTB_BAR_MASK);
+err1:
+ pci_disable_device(pdev);
+err:
+ kfree(ndev);
+
+ dev_err(&pdev->dev, "Error loading %s module\n", KBUILD_MODNAME);
+ return rc;
+}
+
+static void __devexit ntb_pci_remove(struct pci_dev *pdev)
+{
+ struct ntb_device *ndev = pci_get_drvdata(pdev);
+ int i;
+ u32 ntb_cntl;
+
+ /* Bring NTB link down */
+ ntb_cntl = readl(ndev->reg_ofs.lnk_cntl);
+ ntb_cntl |= NTB_LINK_DISABLE;
+ writel(ntb_cntl, ndev->reg_ofs.lnk_cntl);
+
+ ntb_transport_free(ndev->ntb_transport);
+
+ ntb_free_interrupts(ndev);
+ ntb_free_callbacks(ndev);
+ ntb_device_free(ndev);
+
+ for (i = 0; i < NTB_NUM_MW; i++)
+ iounmap(ndev->mw[i].vbase);
+
+ iounmap(ndev->reg_base);
+ pci_release_selected_regions(pdev, NTB_BAR_MASK);
+ pci_disable_device(pdev);
+ kfree(ndev);
+}
+
+static struct pci_driver ntb_pci_driver = {
+ .name = KBUILD_MODNAME,
+ .id_table = ntb_pci_tbl,
+ .probe = ntb_pci_probe,
+ .remove = __devexit_p(ntb_pci_remove),
+};
+
+static int __init ntb_init_module(void)
+{
+ pr_info("%s: %s, version %s\n", KBUILD_MODNAME, NTB_NAME, NTB_VER);
+
+ return pci_register_driver(&ntb_pci_driver);
+}
+module_init(ntb_init_module);
+
+static void __exit ntb_exit_module(void)
+{
+ pci_unregister_driver(&ntb_pci_driver);
+
+ pr_info("%s: Driver removed\n", KBUILD_MODNAME);
+}
+module_exit(ntb_exit_module);
diff --git a/drivers/ntb/ntb_hw.h b/drivers/ntb/ntb_hw.h
new file mode 100644
index 0000000..a1ddd77
--- /dev/null
+++ b/drivers/ntb/ntb_hw.h
@@ -0,0 +1,206 @@
+/*
+ * This file is provided under a dual BSD/GPLv2 license. When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright(c) 2012 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+ * The full GNU General Public License is included in this distribution
+ * in the file called LICENSE.GPL.
+ *
+ * BSD LICENSE
+ *
+ * Copyright(c) 2012 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copy
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * Intel PCIe NTB Linux driver
+ *
+ * Contact Information:
+ * Jon Mason <jon.mason@intel.com>
+ */
+
+#define PCI_DEVICE_ID_INTEL_NTB_B2B_JSF 0x3725
+#define PCI_DEVICE_ID_INTEL_NTB_CLASSIC_JSF 0x3726
+#define PCI_DEVICE_ID_INTEL_NTB_RP_JSF 0x3727
+#define PCI_DEVICE_ID_INTEL_NTB_RP_SNB 0x3C08
+#define PCI_DEVICE_ID_INTEL_NTB_B2B_SNB 0x3C0D
+#define PCI_DEVICE_ID_INTEL_NTB_CLASSIC_SNB 0x3C0E
+#define PCI_DEVICE_ID_INTEL_NTB_2ND_SNB 0x3C0F
+#define PCI_DEVICE_ID_INTEL_NTB_B2B_BWD 0x0C4E
+
+#define msix_table_size(control) ((control & PCI_MSIX_FLAGS_QSIZE)+1)
+
+#define NTB_BAR_MMIO 0
+#define NTB_BAR_23 2
+#define NTB_BAR_45 4
+#define NTB_BAR_MASK ((1 << NTB_BAR_MMIO) | (1 << NTB_BAR_23) |\
+ (1 << NTB_BAR_45))
+
+#define NTB_LINK_DOWN 0
+#define NTB_LINK_UP 1
+
+#define NTB_HB_TIMEOUT msecs_to_jiffies(1000)
+
+#define NTB_NUM_MW 2
+
+enum ntb_hw_event {
+ NTB_EVENT_SW_EVENT0 = (1 << 0),
+ NTB_EVENT_SW_EVENT1 = (1 << 1),
+ NTB_EVENT_SW_EVENT2 = (1 << 2),
+ NTB_EVENT_HW_ERROR = (1 << 3),
+ NTB_EVENT_HW_LINK_UP = (1 << 4),
+ NTB_EVENT_HW_LINK_DOWN = (1 << 5),
+};
+
+struct ntb_mw {
+ dma_addr_t phys_addr;
+ void __iomem *vbase;
+ resource_size_t bar_sz;
+};
+
+struct ntb_db_cb {
+ void (*callback) (void *data, int db_num);
+ unsigned int db_num;
+ void *data;
+ struct ntb_device *ndev;
+};
+
+struct ntb_device {
+ struct pci_dev *pdev;
+ struct msix_entry *msix_entries;
+ void __iomem *reg_base;
+ struct ntb_mw mw[NTB_NUM_MW];
+ struct {
+ unsigned int max_spads;
+ unsigned int max_db_bits;
+ unsigned int msix_cnt;
+ } limits;
+ struct {
+ void __iomem *pdb;
+ void __iomem *pdb_mask;
+ void __iomem *sdb;
+ void __iomem *sbar2_xlat;
+ void __iomem *sbar4_xlat;
+ void __iomem *spad_write;
+ void __iomem *spad_read;
+ void __iomem *lnk_cntl;
+ void __iomem *lnk_stat;
+ void __iomem *spci_cmd;
+ } reg_ofs;
+ void *ntb_transport;
+ void (*event_cb)(void *handle, enum ntb_hw_event event);
+
+ struct ntb_db_cb *db_cb;
+ unsigned char hw_type;
+ unsigned char conn_type;
+ unsigned char dev_type;
+ unsigned char num_msix;
+ unsigned char bits_per_vector;
+ unsigned char max_cbs;
+ unsigned char link_status;
+ struct delayed_work hb_timer;
+ unsigned long last_ts;
+};
+
+/**
+ * ntb_hw_link_status() - return the hardware link status
+ * @ndev: pointer to ntb_device instance
+ *
+ * Returns true if the hardware is connected to the remote system
+ *
+ * RETURNS: true or false based on the hardware link state
+ */
+static inline bool ntb_hw_link_status(struct ntb_device *ndev)
+{
+ return ndev->link_status == NTB_LINK_UP;
+}
+
+/**
+ * ntb_query_pdev() - return the pci_dev pointer
+ * @ndev: pointer to ntb_device instance
+ *
+ * Given the ntb pointer return the pci_dev pointerfor the NTB hardware device
+ *
+ * RETURNS: a pointer to the ntb pci_dev
+ */
+static inline struct pci_dev *ntb_query_pdev(struct ntb_device *ndev)
+{
+ return ndev->pdev;
+}
+
+/**
+ * ntb_query_max_cbs() - return the maximum number of callback tuples
+ * @ndev: pointer to ntb_device instance
+ *
+ * The number of callbacks can vary depending on the platform and MSI-X/MSI
+ * enablement
+ *
+ * RETURNS: the maximum number of callback tuples (3, 15, or 33)
+ */
+static inline unsigned int ntb_query_max_cbs(struct ntb_device *ndev)
+{
+ return ndev->max_cbs;
+}
+
+struct ntb_device *ntb_register_transport(struct pci_dev *pdev,
+ void *transport);
+void ntb_unregister_transport(struct ntb_device *ndev);
+void ntb_set_mw_addr(struct ntb_device *ndev, unsigned int mw, u64 addr);
+int ntb_register_db_callback(struct ntb_device *ndev, unsigned int idx,
+ void *data, void (*db_cb_func) (void *data,
+ int db_num));
+void ntb_unregister_db_callback(struct ntb_device *ndev, unsigned int idx);
+int ntb_register_event_callback(struct ntb_device *ndev,
+ void (*event_cb_func) (void *handle,
+ unsigned int event));
+void ntb_unregister_event_callback(struct ntb_device *ndev);
+int ntb_get_max_spads(struct ntb_device *ndev);
+int ntb_write_local_spad(struct ntb_device *ndev, unsigned int idx, u32 val);
+int ntb_read_local_spad(struct ntb_device *ndev, unsigned int idx, u32 *val);
+int ntb_write_remote_spad(struct ntb_device *ndev, unsigned int idx, u32 val);
+int ntb_read_remote_spad(struct ntb_device *ndev, unsigned int idx, u32 *val);
+void *ntb_get_mw_vbase(struct ntb_device *ndev, unsigned int mw);
+resource_size_t ntb_get_mw_size(struct ntb_device *ndev, unsigned int mw);
+void ntb_ring_sdb(struct ntb_device *ndev, unsigned int idx);
+void *ntb_find_transport(struct pci_dev *pdev);
+
+int ntb_transport_init(struct pci_dev *pdev);
+void ntb_transport_free(void *transport);
diff --git a/drivers/ntb/ntb_regs.h b/drivers/ntb/ntb_regs.h
new file mode 100644
index 0000000..c7b8a24
--- /dev/null
+++ b/drivers/ntb/ntb_regs.h
@@ -0,0 +1,150 @@
+/*
+ * This file is provided under a dual BSD/GPLv2 license. When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright(c) 2012 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+ * The full GNU General Public License is included in this distribution
+ * in the file called LICENSE.GPL.
+ *
+ * BSD LICENSE
+ *
+ * Copyright(c) 2012 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copy
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * Intel PCIe NTB Linux driver
+ *
+ * Contact Information:
+ * Jon Mason <jon.mason@intel.com>
+ */
+
+#define NTB_LINK_ENABLE 0x0000
+#define NTB_LINK_DISABLE 0x0002
+#define NTB_LINK_STATUS_ACTIVE 0x2000
+#define NTB_LINK_SPEED_MASK 0x000f
+#define NTB_LINK_WIDTH_MASK 0x03f0
+
+#define SNB_MSIX_CNT 4
+#define SNB_MAX_SPADS 16
+#define SNB_MAX_COMPAT_SPADS 8
+/* Reserve the uppermost bit for link interrupt */
+#define SNB_MAX_DB_BITS 15
+#define SNB_DB_BITS_PER_VEC 5
+
+#define SNB_DB_HW_LINK 0x8000
+
+#define SNB_PCICMD_OFFSET 0x0504
+#define SNB_DEVCTRL_OFFSET 0x0598
+#define SNB_LINK_STATUS_OFFSET 0x01A2
+
+#define SNB_PBAR2LMT_OFFSET 0x0000
+#define SNB_PBAR4LMT_OFFSET 0x0008
+#define SNB_PBAR2XLAT_OFFSET 0x0010
+#define SNB_PBAR4XLAT_OFFSET 0x0018
+#define SNB_SBAR2LMT_OFFSET 0x0020
+#define SNB_SBAR4LMT_OFFSET 0x0028
+#define SNB_SBAR2XLAT_OFFSET 0x0030
+#define SNB_SBAR4XLAT_OFFSET 0x0038
+#define SNB_SBAR0BASE_OFFSET 0x0040
+#define SNB_SBAR2BASE_OFFSET 0x0048
+#define SNB_SBAR4BASE_OFFSET 0x0050
+#define SNB_NTBCNTL_OFFSET 0x0058
+#define SNB_SBDF_OFFSET 0x005C
+#define SNB_PDOORBELL_OFFSET 0x0060
+#define SNB_PDBMSK_OFFSET 0x0062
+#define SNB_SDOORBELL_OFFSET 0x0064
+#define SNB_SDBMSK_OFFSET 0x0066
+#define SNB_USMEMMISS 0x0070
+#define SNB_SPAD_OFFSET 0x0080
+#define SNB_SPADSEMA4_OFFSET 0x00c0
+#define SNB_WCCNTRL_OFFSET 0x00e0
+#define SNB_B2B_SPAD_OFFSET 0x0100
+#define SNB_B2B_DOORBELL_OFFSET 0x0140
+#define SNB_B2B_XLAT_OFFSET 0x0144
+
+#define BWD_MSIX_CNT 34
+#define BWD_MAX_SPADS 16
+#define BWD_MAX_COMPAT_SPADS 16
+#define BWD_MAX_DB_BITS 34
+#define BWD_DB_BITS_PER_VEC 1
+
+#define BWD_PCICMD_OFFSET 0xb004
+#define BWD_MBAR23_OFFSET 0xb018
+#define BWD_MBAR45_OFFSET 0xb020
+#define BWD_DEVCTRL_OFFSET 0xb048
+#define BWD_LINK_STATUS_OFFSET 0xb052
+
+#define BWD_SBAR2XLAT_OFFSET 0x0008
+#define BWD_SBAR4XLAT_OFFSET 0x0010
+#define BWD_PDOORBELL_OFFSET 0x0020
+#define BWD_PDBMSK_OFFSET 0x0028
+#define BWD_NTBCNTL_OFFSET 0x0060
+#define BWD_EBDF_OFFSET 0x0064
+#define BWD_SPAD_OFFSET 0x0080
+#define BWD_SPADSEMA_OFFSET 0x00c0
+#define BWD_STKYSPAD_OFFSET 0x00c4
+#define BWD_PBAR2XLAT_OFFSET 0x8008
+#define BWD_PBAR4XLAT_OFFSET 0x8010
+#define BWD_B2B_DOORBELL_OFFSET 0x8020
+#define BWD_B2B_SPAD_OFFSET 0x8080
+#define BWD_B2B_SPADSEMA_OFFSET 0x80c0
+#define BWD_B2B_STKYSPAD_OFFSET 0x80c4
+
+#define NTB_CNTL_BAR23_SNOOP (1 << 2)
+#define NTB_CNTL_BAR45_SNOOP (1 << 6)
+#define BWD_CNTL_LINK_DOWN (1 << 16)
+
+#define NTB_PPD_OFFSET 0x00D4
+#define SNB_PPD_CONN_TYPE 0x0003
+#define SNB_PPD_DEV_TYPE 0x0010
+#define BWD_PPD_INIT_LINK 0x0004
+#define BWD_PPD_CONN_TYPE 0x0300
+#define BWD_PPD_DEV_TYPE 0x1000
+
+#define BWD_PBAR2XLAT_USD_ADDR 0x0000004000000000
+#define BWD_PBAR4XLAT_USD_ADDR 0x0000008000000000
+#define BWD_MBAR23_USD_ADDR 0x000000410000000C
+#define BWD_MBAR45_USD_ADDR 0x000000810000000C
+#define BWD_PBAR2XLAT_DSD_ADDR 0x0000004100000000
+#define BWD_PBAR4XLAT_DSD_ADDR 0x0000008100000000
+#define BWD_MBAR23_DSD_ADDR 0x000000400000000C
+#define BWD_MBAR45_DSD_ADDR 0x000000800000000C
diff --git a/drivers/ntb/ntb_transport.c b/drivers/ntb/ntb_transport.c
new file mode 100644
index 0000000..6d152d3
--- /dev/null
+++ b/drivers/ntb/ntb_transport.c
@@ -0,0 +1,1387 @@
+/*
+ * This file is provided under a dual BSD/GPLv2 license. When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright(c) 2012 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+ * The full GNU General Public License is included in this distribution
+ * in the file called LICENSE.GPL.
+ *
+ * BSD LICENSE
+ *
+ * Copyright(c) 2012 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copy
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * Intel PCIe NTB Linux driver
+ *
+ * Contact Information:
+ * Jon Mason <jon.mason@intel.com>
+ */
+#include <linux/debugfs.h>
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/errno.h>
+#include <linux/export.h>
+#include <linux/interrupt.h>
+#include <linux/kthread.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+#include <linux/ntb.h>
+#include "ntb_hw.h"
+
+static int transport_mtu = 0x4014;
+module_param(transport_mtu, uint, 0644);
+MODULE_PARM_DESC(transport_mtu, "Maximum size of NTB transport packets");
+
+static unsigned int max_num_conns = 2;
+module_param(max_num_conns, uint, 0644);
+MODULE_PARM_DESC(max_num_conns, "Maximum number of NTB transport connections");
+
+struct ntb_queue_entry {
+ /* ntb_queue list reference */
+ struct list_head entry;
+ /* pointers to data to be transfered */
+ void *callback_data;
+ void *buf;
+ unsigned int len;
+ unsigned int flags;
+};
+
+struct ntb_transport;
+
+struct ntb_transport_qp {
+ struct ntb_transport *transport;
+ struct ntb_device *ndev;
+ void *cb_data;
+
+ bool client_ready;
+ bool qp_link;
+ u8 qp_num; /* Only 64 QP's are allowed. 0-63 */
+
+ void (*tx_handler) (void *data, struct ntb_transport_qp *qp);
+ struct tasklet_struct tx_work;
+ struct list_head tx_pend_q;
+ struct list_head tx_comp_q;
+ struct list_head tx_free_q;
+ spinlock_t ntb_tx_pend_q_lock;
+ spinlock_t ntb_tx_comp_q_lock;
+ spinlock_t ntb_tx_free_q_lock;
+ void *tx_mw_begin;
+ void *tx_mw_end;
+ void *tx_offset;
+
+ void (*rx_handler) (void *data, struct ntb_transport_qp *qp);
+ struct tasklet_struct rx_work;
+ struct list_head rx_pend_q;
+ struct list_head rx_comp_q;
+ struct list_head rx_free_q;
+ spinlock_t ntb_rx_pend_q_lock;
+ spinlock_t ntb_rx_comp_q_lock;
+ spinlock_t ntb_rx_free_q_lock;
+ void *rx_buff_begin;
+ void *rx_buff_end;
+ void *rx_offset;
+
+ void (*event_handler) (void *data, int status);
+ struct delayed_work link_work;
+
+ struct dentry *debugfs_dir;
+ struct dentry *debugfs_stats;
+
+ /* Stats */
+ u64 rx_bytes;
+ u64 rx_pkts;
+ u64 rx_ring_empty;
+ u64 rx_err_no_buf;
+ u64 rx_err_oflow;
+ u64 rx_err_ver;
+ u64 tx_bytes;
+ u64 tx_pkts;
+ u64 tx_ring_full;
+};
+
+struct ntb_transport_mw {
+ size_t size;
+ void *virt_addr;
+ dma_addr_t dma_addr;
+};
+
+struct ntb_transport {
+ struct ntb_device *ndev;
+ struct ntb_transport_mw mw[NTB_NUM_MW];
+ struct ntb_transport_qp *qps;
+ unsigned int max_qps;
+ unsigned long qp_bitmap;
+ bool transport_link;
+ struct delayed_work link_work;
+ struct dentry *debugfs_dir;
+ struct list_head list;
+};
+
+enum {
+ DESC_DONE_FLAG = 1 << 0,
+ LINK_DOWN_FLAG = 1 << 1,
+ HW_ERROR_FLAG = 1 << 2,
+};
+
+struct ntb_payload_header {
+ u64 ver;
+ unsigned int len;
+ unsigned int flags;
+};
+
+enum {
+ MW0_SZ = 0,
+ MW1_SZ,
+ NUM_QPS,
+ QP_LINKS,
+ MAX_SPAD,
+};
+
+#define QP_TO_MW(qp) ((qp) % NTB_NUM_MW)
+#define NTB_QP_DEF_NUM_ENTRIES 100
+#define NTB_LINK_DOWN_TIMEOUT 10
+
+static LIST_HEAD(ntb_transport_list);
+
+static int debugfs_open(struct inode *inode, struct file *filp)
+{
+ filp->private_data = inode->i_private;
+ return 0;
+}
+
+static ssize_t debugfs_read(struct file *filp, char __user *ubuf, size_t count,
+ loff_t *offp)
+{
+ struct ntb_transport_qp *qp;
+ char buf[512];
+ ssize_t ret, out_offset, out_count;
+
+ out_count = 512;
+
+ qp = filp->private_data;
+ out_offset = 0;
+ out_offset += snprintf(buf + out_offset, out_count - out_offset,
+ "NTB Transport stats\n");
+ out_offset += snprintf(buf + out_offset, out_count - out_offset,
+ "rx_bytes - %llu\n", qp->rx_bytes);
+ out_offset += snprintf(buf + out_offset, out_count - out_offset,
+ "rx_pkts - %llu\n", qp->rx_pkts);
+ out_offset += snprintf(buf + out_offset, out_count - out_offset,
+ "rx_ring_empty - %llu\n", qp->rx_ring_empty);
+ out_offset += snprintf(buf + out_offset, out_count - out_offset,
+ "rx_err_no_buf - %llu\n", qp->rx_err_no_buf);
+ out_offset += snprintf(buf + out_offset, out_count - out_offset,
+ "rx_er_oflow - %llu\n", qp->rx_err_oflow);
+ out_offset += snprintf(buf + out_offset, out_count - out_offset,
+ "rx_err_ver - %llu\n", qp->rx_err_ver);
+ out_offset += snprintf(buf + out_offset, out_count - out_offset,
+ "rx_offset - %p\n", qp->rx_offset);
+ out_offset += snprintf(buf + out_offset, out_count - out_offset,
+ "tx_bytes - %llu\n", qp->tx_bytes);
+ out_offset += snprintf(buf + out_offset, out_count - out_offset,
+ "tx_pkts - %llu\n", qp->tx_pkts);
+ out_offset += snprintf(buf + out_offset, out_count - out_offset,
+ "tx_ring_full - %llu\n", qp->tx_ring_full);
+ out_offset += snprintf(buf + out_offset, out_count - out_offset,
+ "tx_offset - %p\n", qp->tx_offset);
+
+ ret = simple_read_from_buffer(ubuf, count, offp, buf, out_offset);
+ return ret;
+}
+
+static const struct file_operations ntb_qp_debugfs_stats = {
+ .owner = THIS_MODULE,
+ .open = debugfs_open,
+ .read = debugfs_read,
+};
+
+static void ntb_list_add_head(spinlock_t *lock, struct list_head *entry,
+ struct list_head *list)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(lock, flags);
+ list_add(entry, list);
+ spin_unlock_irqrestore(lock, flags);
+}
+
+static void ntb_list_add_tail(spinlock_t *lock, struct list_head *entry,
+ struct list_head *list)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(lock, flags);
+ list_add_tail(entry, list);
+ spin_unlock_irqrestore(lock, flags);
+}
+
+static struct ntb_queue_entry *ntb_list_rm_head(spinlock_t *lock,
+ struct list_head *list)
+{
+ struct ntb_queue_entry *entry;
+ unsigned long flags;
+
+ spin_lock_irqsave(lock, flags);
+ if (list_empty(list)) {
+ entry = NULL;
+ goto out;
+ }
+ entry = list_first_entry(list, struct ntb_queue_entry, entry);
+ list_del(&entry->entry);
+out:
+ spin_unlock_irqrestore(lock, flags);
+
+ return entry;
+}
+
+static int ntb_transport_setup_qp_mw(struct ntb_transport *nt,
+ unsigned int qp_num)
+{
+ struct ntb_transport_qp *qp = &nt->qps[qp_num];
+ u8 mw_num = QP_TO_MW(qp_num);
+ unsigned int size, num_qps_mw;
+
+ WARN_ON(nt->mw[mw_num].virt_addr == 0);
+
+ if (nt->max_qps % NTB_NUM_MW && !mw_num)
+ num_qps_mw = nt->max_qps / NTB_NUM_MW +
+ (nt->max_qps % NTB_NUM_MW - mw_num);
+ else
+ num_qps_mw = nt->max_qps / NTB_NUM_MW;
+
+ size = nt->mw[mw_num].size / num_qps_mw;
+ pr_debug("orig size = %d, num qps = %d, size = %d\n",
+ (int) nt->mw[mw_num].size, nt->max_qps, size);
+
+ qp->rx_buff_begin = nt->mw[mw_num].virt_addr +
+ (qp_num / NTB_NUM_MW * size);
+ qp->rx_buff_end = qp->rx_buff_begin + size;
+ pr_info("QP %d - RX Buff start %p end %p\n", qp->qp_num,
+ qp->rx_buff_begin, qp->rx_buff_end);
+ qp->rx_offset = qp->rx_buff_begin;
+
+ qp->tx_mw_begin = ntb_get_mw_vbase(nt->ndev, mw_num) +
+ (qp_num / NTB_NUM_MW * size);
+ qp->tx_mw_end = qp->tx_mw_begin + size;
+ pr_info("QP %d - TX MW start %p end %p\n", qp->qp_num, qp->tx_mw_begin,
+ qp->tx_mw_end);
+ qp->tx_offset = qp->tx_mw_begin;
+
+ qp->rx_pkts = 0;
+ qp->tx_pkts = 0;
+
+ return 0;
+}
+
+static int ntb_set_mw(struct ntb_transport *nt, int num_mw, unsigned int size)
+{
+ struct ntb_transport_mw *mw = &nt->mw[num_mw];
+ struct pci_dev *pdev = ntb_query_pdev(nt->ndev);
+ void *offset;
+
+ /* Alloc memory for receiving data. Must be 4k aligned */
+ mw->size = ALIGN(size, 4096);
+
+ mw->virt_addr = dma_alloc_coherent(&pdev->dev, mw->size, &mw->dma_addr,
+ GFP_KERNEL);
+ if (!mw->virt_addr) {
+ pr_err("Unable to allocate MW buffer of size %d\n",
+ (int) mw->size);
+ return -ENOMEM;
+ }
+
+ /* setup the hdr offsets with 0's */
+ for (offset = mw->virt_addr;
+ offset + sizeof(struct ntb_payload_header) < mw->virt_addr + size;
+ offset += transport_mtu + sizeof(struct ntb_payload_header))
+ memset(offset, 0, sizeof(struct ntb_payload_header));
+
+ /* Notify HW the memory location of the receive buffer */
+ ntb_set_mw_addr(nt->ndev, num_mw, mw->dma_addr);
+
+ return 0;
+}
+
+static void ntb_transport_event_callback(void *data, enum ntb_hw_event event)
+{
+ struct ntb_transport *nt = data;
+
+ if (event == NTB_EVENT_HW_ERROR)
+ BUG();
+
+ if (event == NTB_EVENT_HW_LINK_UP)
+ schedule_delayed_work(&nt->link_work, 0);
+
+ if (event == NTB_EVENT_HW_LINK_DOWN) {
+ int i;
+
+ nt->transport_link = NTB_LINK_DOWN;
+
+ /* Pass along the info to any clients */
+ for (i = 0; i < nt->max_qps; i++)
+ if (!test_bit(i, &nt->qp_bitmap)) {
+ struct ntb_transport_qp *qp = &nt->qps[i];
+
+ if (qp->event_handler &&
+ qp->qp_link != NTB_LINK_DOWN)
+ qp->event_handler(qp->cb_data,
+ NTB_LINK_DOWN);
+
+ qp->qp_link = NTB_LINK_DOWN;
+ }
+
+ /* The scratchpad registers keep the values if the remote side
+ * goes down, blast them now to give them a sane value the next
+ * time they are accessed
+ */
+ for (i = 0; i < MAX_SPAD; i++) {
+ ntb_write_local_spad(nt->ndev, i, 0);
+ ntb_write_remote_spad(nt->ndev, i, 0);
+ }
+ }
+}
+
+static void ntb_transport_link_work(struct work_struct *work)
+{
+ struct ntb_transport *nt = container_of(work, struct ntb_transport,
+ link_work.work);
+ struct ntb_device *ndev = nt->ndev;
+ u32 val;
+ int rc, i;
+
+ /* send the local info */
+ rc = ntb_write_remote_spad(ndev, MW0_SZ, ntb_get_mw_size(ndev, 0));
+ if (rc) {
+ pr_err("Error writing %x to remote spad %d\n",
+ (u32) ntb_get_mw_size(ndev, 0), MW0_SZ);
+ goto out;
+ }
+
+ rc = ntb_write_remote_spad(ndev, MW1_SZ, ntb_get_mw_size(ndev, 1));
+ if (rc) {
+ pr_err("Error writing %x to remote spad %d\n",
+ (u32) ntb_get_mw_size(ndev, 1), MW1_SZ);
+ goto out;
+ }
+
+ rc = ntb_write_remote_spad(ndev, NUM_QPS, nt->max_qps);
+ if (rc) {
+ pr_err("Error writing %x to remote spad %d\n",
+ nt->max_qps, NUM_QPS);
+ goto out;
+ }
+
+ rc = ntb_write_remote_spad(ndev, QP_LINKS, 0);
+ if (rc) {
+ pr_err("Error writing %x to remote spad %d\n", 0, QP_LINKS);
+ goto out;
+ }
+
+ /* Query the remote side for its info */
+ rc = ntb_read_remote_spad(ndev, NUM_QPS, &val);
+ if (rc) {
+ pr_err("Error reading remote spad %d\n", NUM_QPS);
+ goto out;
+ }
+
+ if (val != nt->max_qps)
+ goto out;
+ pr_info("Remote max number of qps = %d\n", val);
+
+ rc = ntb_read_remote_spad(ndev, MW0_SZ, &val);
+ if (rc) {
+ pr_err("Error reading remote spad %d\n", MW0_SZ);
+ goto out;
+ }
+
+ if (!val)
+ goto out;
+ pr_info("Remote MW0 size = %d\n", val);
+
+ rc = ntb_set_mw(nt, 0, val);
+ if (rc)
+ goto out;
+
+ rc = ntb_read_remote_spad(ndev, MW1_SZ, &val);
+ if (rc) {
+ pr_err("Error reading remote spad %d\n", MW1_SZ);
+ goto out;
+ }
+
+ if (!val)
+ goto out;
+ pr_info("Remote MW1 size = %d\n", val);
+
+ rc = ntb_set_mw(nt, 1, val);
+ if (rc)
+ goto out;
+
+ for (i = 0; i < nt->max_qps; i++) {
+ struct ntb_transport_qp *qp = &nt->qps[i];
+
+ rc = ntb_transport_setup_qp_mw(nt, i);
+ if (rc)
+ goto out;
+
+ if (qp->client_ready)
+ schedule_delayed_work(&qp->link_work, 0);
+ }
+
+ nt->transport_link = NTB_LINK_UP;
+
+ return;
+
+out:
+ if (ntb_hw_link_status(ndev))
+ schedule_delayed_work(&nt->link_work,
+ msecs_to_jiffies(NTB_LINK_DOWN_TIMEOUT));
+}
+
+static void ntb_qp_link_work(struct work_struct *work)
+{
+ struct ntb_transport_qp *qp;
+ struct ntb_transport *nt;
+ int rc, val;
+
+ qp = container_of(work, struct ntb_transport_qp, link_work.work);
+ nt = qp->transport;
+
+ WARN_ON(nt->transport_link != NTB_LINK_UP);
+
+ rc = ntb_read_local_spad(nt->ndev, QP_LINKS, &val);
+ if (rc) {
+ pr_err("Error reading spad %d\n", QP_LINKS);
+ return;
+ }
+
+ rc = ntb_write_remote_spad(nt->ndev, QP_LINKS, val | 1 << qp->qp_num);
+ if (rc)
+ pr_err("Error writing %x to remote spad %d\n",
+ val | 1 << qp->qp_num, QP_LINKS);
+
+ /* query remote spad for qp ready bits */
+ rc = ntb_read_remote_spad(nt->ndev, QP_LINKS, &val);
+ if (rc)
+ pr_err("Error reading remote spad %d\n", QP_LINKS);
+
+ pr_debug("Remote QP link status = %x\n", val);
+
+ /* See if the remote side is up */
+ if (1 << qp->qp_num & val) {
+ qp->qp_link = NTB_LINK_UP;
+
+ if (qp->event_handler)
+ qp->event_handler(qp->cb_data, NTB_LINK_UP);
+ } else if (ntb_hw_link_status(nt->ndev))
+ schedule_delayed_work(&qp->link_work,
+ msecs_to_jiffies(NTB_LINK_DOWN_TIMEOUT));
+}
+
+static void ntb_transport_init_queue(struct ntb_transport *nt,
+ unsigned int qp_num)
+{
+ struct ntb_transport_qp *qp;
+
+ qp = &nt->qps[qp_num];
+ qp->qp_num = qp_num;
+ qp->transport = nt;
+ qp->ndev = nt->ndev;
+ qp->qp_link = NTB_LINK_DOWN;
+
+ if (nt->debugfs_dir) {
+ char debugfs_name[4];
+
+ snprintf(debugfs_name, 4, "qp%d", qp_num);
+ qp->debugfs_dir = debugfs_create_dir(debugfs_name,
+ nt->debugfs_dir);
+
+ qp->debugfs_stats = debugfs_create_file("stats", S_IRUSR,
+ qp->debugfs_dir, qp,
+ &ntb_qp_debugfs_stats);
+ }
+
+ INIT_DELAYED_WORK(&qp->link_work, ntb_qp_link_work);
+
+ spin_lock_init(&qp->ntb_rx_comp_q_lock);
+ spin_lock_init(&qp->ntb_rx_pend_q_lock);
+ spin_lock_init(&qp->ntb_rx_free_q_lock);
+ spin_lock_init(&qp->ntb_tx_comp_q_lock);
+ spin_lock_init(&qp->ntb_tx_pend_q_lock);
+ spin_lock_init(&qp->ntb_tx_free_q_lock);
+
+ INIT_LIST_HEAD(&qp->rx_pend_q);
+ INIT_LIST_HEAD(&qp->rx_comp_q);
+ INIT_LIST_HEAD(&qp->rx_free_q);
+ INIT_LIST_HEAD(&qp->tx_pend_q);
+ INIT_LIST_HEAD(&qp->tx_comp_q);
+ INIT_LIST_HEAD(&qp->tx_free_q);
+}
+
+int ntb_transport_init(struct pci_dev *pdev)
+{
+ struct ntb_transport *nt;
+ int rc, i;
+
+ nt = kzalloc(sizeof(struct ntb_transport), GFP_KERNEL);
+ if (!nt)
+ return -ENOMEM;
+
+ if (debugfs_initialized())
+ nt->debugfs_dir = debugfs_create_dir(KBUILD_MODNAME, NULL);
+ else
+ nt->debugfs_dir = NULL;
+
+ nt->ndev = ntb_register_transport(pdev, nt);
+ if (!nt->ndev) {
+ rc = -EIO;
+ goto err;
+ }
+
+ nt->max_qps = ntb_query_max_cbs(nt->ndev);
+ if (!nt->max_qps) {
+ rc = -EIO;
+ goto err1;
+ }
+ nt->max_qps = min(nt->max_qps, max_num_conns);
+
+ nt->qps = kcalloc(nt->max_qps, sizeof(struct ntb_transport_qp),
+ GFP_KERNEL);
+ if (!nt->qps) {
+ rc = -ENOMEM;
+ goto err1;
+ }
+
+ nt->qp_bitmap = ((u64) 1 << nt->max_qps) - 1;
+
+ for (i = 0; i < nt->max_qps; i++)
+ ntb_transport_init_queue(nt, i);
+
+ rc = ntb_register_event_callback(nt->ndev,
+ ntb_transport_event_callback);
+ if (rc)
+ goto err2;
+
+ list_add(&nt->list, &ntb_transport_list);
+
+ INIT_DELAYED_WORK(&nt->link_work, ntb_transport_link_work);
+
+ if (ntb_hw_link_status(nt->ndev))
+ schedule_delayed_work(&nt->link_work, 0);
+
+ return 0;
+
+err2:
+ kfree(nt->qps);
+err1:
+ ntb_unregister_transport(nt->ndev);
+err:
+ debugfs_remove_recursive(nt->debugfs_dir);
+ kfree(nt);
+ return rc;
+}
+
+void ntb_transport_free(void *transport)
+{
+ struct ntb_transport *nt = transport;
+ struct pci_dev *pdev;
+ int i;
+
+ nt->transport_link = NTB_LINK_DOWN;
+
+ /* verify that all the qp's are freed */
+ while (!nt->qp_bitmap) {
+ i = ffz(nt->qp_bitmap);
+ if (i == ((u64) 1 << nt->max_qps))
+ break;
+ ntb_transport_free_queue(&nt->qps[i]);
+ }
+
+ list_del(&nt->list);
+
+ cancel_delayed_work_sync(&nt->link_work);
+
+ debugfs_remove_recursive(nt->debugfs_dir);
+
+ ntb_unregister_event_callback(nt->ndev);
+
+ pdev = ntb_query_pdev(nt->ndev);
+
+ for (i = 0; i < NTB_NUM_MW; i++)
+ if (nt->mw[i].virt_addr)
+ dma_free_coherent(&pdev->dev, nt->mw[i].size,
+ nt->mw[i].virt_addr,
+ nt->mw[i].dma_addr);
+
+ kfree(nt->qps);
+ ntb_unregister_transport(nt->ndev);
+ kfree(nt);
+}
+
+static void ntb_rx_copy_task(struct ntb_transport_qp *qp,
+ struct ntb_queue_entry *entry, void *offset)
+{
+ struct ntb_payload_header *hdr = offset;
+
+ entry->len = hdr->len;
+ offset += sizeof(struct ntb_payload_header);
+ memcpy(entry->buf, offset, entry->len);
+
+ /* Ensure that the data is fully copied out before clearing the flag */
+ wmb();
+ hdr->flags = 0;
+ ntb_list_add_tail(&qp->ntb_rx_comp_q_lock, &entry->entry,
+ &qp->rx_comp_q);
+
+ if (qp->rx_handler && qp->client_ready)
+ qp->rx_handler(qp->cb_data, qp);
+}
+
+static int ntb_process_rxc(struct ntb_transport_qp *qp)
+{
+ struct ntb_payload_header *hdr;
+ struct ntb_queue_entry *entry;
+ void *offset;
+
+ entry = ntb_list_rm_head(&qp->ntb_rx_pend_q_lock, &qp->rx_pend_q);
+ if (!entry) {
+ hdr = qp->rx_offset;
+ pr_debug("no buffer - HDR ver %llu, len %d, flags %x\n",
+ hdr->ver, hdr->len, hdr->flags);
+ qp->rx_err_no_buf++;
+ return -ENOMEM;
+ }
+
+ offset = qp->rx_offset;
+ hdr = offset;
+
+ if (!(hdr->flags & DESC_DONE_FLAG)) {
+ ntb_list_add_tail(&qp->ntb_rx_pend_q_lock, &entry->entry,
+ &qp->rx_pend_q);
+ qp->rx_ring_empty++;
+ return -EAGAIN;
+ }
+
+ if (hdr->ver != qp->rx_pkts) {
+ pr_debug("qp %d: version mismatch, expected %llu - got %llu\n",
+ qp->qp_num, qp->rx_pkts, hdr->ver);
+ ntb_list_add_tail(&qp->ntb_rx_pend_q_lock, &entry->entry,
+ &qp->rx_pend_q);
+ qp->rx_err_ver++;
+ return -EIO;
+ }
+
+ if (hdr->flags & NTB_LINK_DOWN) {
+ pr_info("qp %d: Link Down\n", qp->qp_num);
+ qp->qp_link = NTB_LINK_DOWN;
+ schedule_delayed_work(&qp->link_work,
+ msecs_to_jiffies(NTB_LINK_DOWN_TIMEOUT));
+
+ if (qp->event_handler)
+ qp->event_handler(qp->cb_data, NTB_LINK_DOWN);
+
+ ntb_list_add_tail(&qp->ntb_rx_pend_q_lock, &entry->entry,
+ &qp->rx_pend_q);
+
+ /* Ensure that the data is fully copied out before clearing the
+ * done flag
+ */
+ wmb();
+ hdr->flags = 0;
+ goto out;
+ }
+
+ pr_debug("rx offset %p, ver %llu - %d payload received, buf size %d\n",
+ qp->rx_offset, hdr->ver, hdr->len, entry->len);
+
+ if (hdr->len <= entry->len)
+ ntb_rx_copy_task(qp, entry, offset);
+ else {
+ ntb_list_add_tail(&qp->ntb_rx_pend_q_lock, &entry->entry,
+ &qp->rx_pend_q);
+
+ /* Ensure that the data is fully copied out before clearing the
+ * done flag
+ */
+ wmb();
+ hdr->flags = 0;
+ qp->rx_err_oflow++;
+ pr_err("RX overflow! Wanted %d got %d\n", hdr->len, entry->len);
+ }
+
+ qp->rx_bytes += hdr->len;
+ qp->rx_pkts++;
+
+out:
+ qp->rx_offset =
+ (qp->rx_offset +
+ ((transport_mtu + sizeof(struct ntb_payload_header)) * 2) >=
+ qp->rx_buff_end) ? qp->rx_buff_begin : qp->rx_offset +
+ transport_mtu + sizeof(struct ntb_payload_header);
+
+ return 0;
+}
+
+static void ntb_transport_rx(unsigned long data)
+{
+ struct ntb_transport_qp *qp = (struct ntb_transport_qp *)data;
+ int rc;
+
+ do {
+ rc = ntb_process_rxc(qp);
+ } while (!rc);
+}
+
+static void ntb_transport_rxc_db(void *data, int db_num)
+{
+ struct ntb_transport_qp *qp = data;
+
+ pr_debug("%s: doorbell %d received\n", __func__, db_num);
+
+ tasklet_schedule(&qp->rx_work);
+}
+
+static void ntb_tx_copy_task(struct ntb_transport_qp *qp,
+ struct ntb_queue_entry *entry,
+ void *offset)
+{
+ struct ntb_payload_header *hdr = offset;
+
+ offset += sizeof(struct ntb_payload_header);
+ memcpy_toio(offset, entry->buf, entry->len);
+
+ hdr->len = entry->len;
+ hdr->ver = qp->tx_pkts;
+
+ /* Ensure that the data is fully copied out before setting the flag */
+ wmb();
+ hdr->flags = entry->flags | DESC_DONE_FLAG;
+
+ ntb_ring_sdb(qp->ndev, qp->qp_num);
+
+ /* The entry length can only be zero if the packet is intended to be a
+ * "link down" or similar. Since no payload is being sent in these
+ * cases, there is nothing to add to the completion queue.
+ */
+ if (entry->len > 0) {
+ qp->tx_bytes += entry->len;
+
+ /* Add fully transmitted data to completion queue */
+ ntb_list_add_tail(&qp->ntb_tx_comp_q_lock, &entry->entry,
+ &qp->tx_comp_q);
+
+ if (qp->tx_handler)
+ qp->tx_handler(qp->cb_data, qp);
+ } else
+ ntb_list_add_tail(&qp->ntb_tx_free_q_lock, &entry->entry,
+ &qp->tx_free_q);
+}
+
+static int ntb_process_tx(struct ntb_transport_qp *qp,
+ struct ntb_queue_entry *entry)
+{
+ struct ntb_payload_header *hdr;
+ void *offset;
+
+ offset = qp->tx_offset;
+ hdr = offset;
+
+ pr_debug("%lld - offset %p, tx %p, entry len %d flags %x buff %p\n",
+ qp->tx_pkts, offset, qp->tx_offset, entry->len, entry->flags,
+ entry->buf);
+ if (hdr->flags) {
+ ntb_list_add_head(&qp->ntb_tx_pend_q_lock, &entry->entry,
+ &qp->tx_pend_q);
+ qp->tx_ring_full++;
+ return -EAGAIN;
+ }
+
+ if (entry->len > transport_mtu) {
+ pr_err("Trying to send pkt size of %d\n", entry->len);
+ entry->flags = HW_ERROR_FLAG;
+
+ ntb_list_add_tail(&qp->ntb_tx_comp_q_lock, &entry->entry,
+ &qp->tx_comp_q);
+
+ if (qp->tx_handler)
+ qp->tx_handler(qp->cb_data, qp);
+
+ return 0;
+ }
+
+ ntb_tx_copy_task(qp, entry, offset);
+
+ qp->tx_offset =
+ (qp->tx_offset +
+ ((transport_mtu + sizeof(struct ntb_payload_header)) * 2) >=
+ qp->tx_mw_end) ? qp->tx_mw_begin : qp->tx_offset + transport_mtu +
+ sizeof(struct ntb_payload_header);
+
+ qp->tx_pkts++;
+
+ return 0;
+}
+
+static void ntb_transport_tx(unsigned long data)
+{
+ struct ntb_transport_qp *qp = (struct ntb_transport_qp *)data;
+ struct ntb_queue_entry *entry;
+ int rc;
+
+ do {
+ entry = ntb_list_rm_head(&qp->ntb_tx_pend_q_lock,
+ &qp->tx_pend_q);
+ if (!entry)
+ break;
+
+ rc = ntb_process_tx(qp, entry);
+ } while (!rc);
+}
+
+static void ntb_send_link_down(struct ntb_transport_qp *qp)
+{
+ struct ntb_queue_entry *entry;
+ int i;
+
+ if (qp->qp_link == NTB_LINK_DOWN)
+ return;
+
+ qp->qp_link = NTB_LINK_DOWN;
+
+ for (i = 0; i < NTB_LINK_DOWN_TIMEOUT; i++) {
+ entry = ntb_list_rm_head(&qp->ntb_tx_free_q_lock,
+ &qp->tx_free_q);
+ if (entry)
+ break;
+ msleep(100);
+ }
+
+ entry->callback_data = NULL;
+ entry->buf = NULL;
+ entry->len = 0;
+ entry->flags = LINK_DOWN_FLAG;
+
+ ntb_list_add_tail(&qp->ntb_tx_pend_q_lock, &entry->entry,
+ &qp->tx_pend_q);
+ tasklet_schedule(&qp->tx_work);
+}
+
+/**
+ * ntb_transport_create_queue - Create a new NTB transport layer queue
+ * @rx_handler: receive callback function
+ * @tx_handler: transmit callback function
+ * @event_handler: event callback function
+ *
+ * Create a new NTB transport layer queue and provide the queue with a callback
+ * routine for both transmit and receive. The receive callback routine will be
+ * used to pass up data when the transport has received it on the queue. The
+ * transmit callback routine will be called when the transport has completed the
+ * transmission of the data on the queue and the data is ready to be freed.
+ *
+ * RETURNS: pointer to newly created ntb_queue, NULL on error.
+ */
+struct ntb_transport_qp *
+ntb_transport_create_queue(void *data, struct pci_dev *pdev,
+ const struct ntb_queue_handlers *handlers)
+{
+ struct ntb_queue_entry *entry;
+ struct ntb_transport_qp *qp;
+ struct ntb_transport *nt;
+ unsigned int free_queue;
+ int rc, i;
+
+ nt = ntb_find_transport(pdev);
+ if (!nt)
+ goto err;
+
+ free_queue = ffs(nt->qp_bitmap);
+ if (!free_queue)
+ goto err;
+
+ /* decrement free_queue to make it zero based */
+ free_queue--;
+
+ clear_bit(free_queue, &nt->qp_bitmap);
+
+ qp = &nt->qps[free_queue];
+ qp->cb_data = data;
+ qp->rx_handler = handlers->rx_handler;
+ qp->tx_handler = handlers->tx_handler;
+ qp->event_handler = handlers->event_handler;
+
+ for (i = 0; i < NTB_QP_DEF_NUM_ENTRIES; i++) {
+ entry = kzalloc(sizeof(struct ntb_queue_entry), GFP_ATOMIC);
+ if (!entry)
+ goto err1;
+
+ ntb_list_add_tail(&qp->ntb_rx_free_q_lock, &entry->entry,
+ &qp->rx_free_q);
+ }
+
+ for (i = 0; i < NTB_QP_DEF_NUM_ENTRIES; i++) {
+ entry = kzalloc(sizeof(struct ntb_queue_entry), GFP_ATOMIC);
+ if (!entry)
+ goto err2;
+
+ ntb_list_add_tail(&qp->ntb_tx_free_q_lock, &entry->entry,
+ &qp->tx_free_q);
+ }
+
+ tasklet_init(&qp->rx_work, ntb_transport_rx, (unsigned long) qp);
+ tasklet_init(&qp->tx_work, ntb_transport_tx, (unsigned long) qp);
+
+ rc = ntb_register_db_callback(qp->ndev, free_queue, qp,
+ ntb_transport_rxc_db);
+ if (rc)
+ goto err3;
+
+ pr_info("NTB Transport QP %d created\n", qp->qp_num);
+
+ return qp;
+
+err3:
+ tasklet_disable(&qp->rx_work);
+ tasklet_disable(&qp->tx_work);
+err2:
+ while ((entry =
+ ntb_list_rm_head(&qp->ntb_tx_free_q_lock, &qp->tx_free_q)))
+ kfree(entry);
+err1:
+ while ((entry =
+ ntb_list_rm_head(&qp->ntb_rx_free_q_lock, &qp->rx_free_q)))
+ kfree(entry);
+ set_bit(free_queue, &nt->qp_bitmap);
+err:
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(ntb_transport_create_queue);
+
+/**
+ * ntb_transport_free_queue - Frees NTB transport queue
+ * @qp: NTB queue to be freed
+ *
+ * Frees NTB transport queue
+ */
+void ntb_transport_free_queue(struct ntb_transport_qp *qp)
+{
+ struct ntb_queue_entry *entry;
+
+ if (!qp)
+ return;
+
+ cancel_delayed_work_sync(&qp->link_work);
+
+ ntb_unregister_db_callback(qp->ndev, qp->qp_num);
+ tasklet_disable(&qp->rx_work);
+ tasklet_disable(&qp->tx_work);
+
+ while ((entry =
+ ntb_list_rm_head(&qp->ntb_rx_free_q_lock, &qp->rx_free_q)))
+ kfree(entry);
+
+ while ((entry =
+ ntb_list_rm_head(&qp->ntb_rx_pend_q_lock, &qp->rx_pend_q))) {
+ pr_warn("Freeing item from a non-empty queue\n");
+ kfree(entry);
+ }
+
+ while ((entry =
+ ntb_list_rm_head(&qp->ntb_rx_comp_q_lock, &qp->rx_comp_q))) {
+ pr_warn("Freeing item from a non-empty queue\n");
+ kfree(entry);
+ }
+
+ while ((entry =
+ ntb_list_rm_head(&qp->ntb_tx_free_q_lock, &qp->tx_free_q)))
+ kfree(entry);
+
+ while ((entry =
+ ntb_list_rm_head(&qp->ntb_tx_pend_q_lock, &qp->tx_pend_q))) {
+ pr_warn("Freeing item from a non-empty queue\n");
+ kfree(entry);
+ }
+
+ while ((entry =
+ ntb_list_rm_head(&qp->ntb_tx_comp_q_lock, &qp->tx_comp_q))) {
+ pr_warn("Freeing item from a non-empty queue\n");
+ kfree(entry);
+ }
+
+ set_bit(qp->qp_num, &qp->transport->qp_bitmap);
+
+ pr_info("NTB Transport QP %d freed\n", qp->qp_num);
+}
+EXPORT_SYMBOL_GPL(ntb_transport_free_queue);
+
+/**
+ * ntb_transport_rx_remove - Dequeues enqueued rx packet
+ * @qp: NTB queue to be freed
+ * @len: pointer to variable to write enqueued buffers length
+ *
+ * Dequeues unused buffers from receive queue. Should only be used during
+ * shutdown of qp.
+ *
+ * RETURNS: NULL error value on error, or void* for success.
+ */
+void *ntb_transport_rx_remove(struct ntb_transport_qp *qp, unsigned int *len)
+{
+ struct ntb_queue_entry *entry;
+ void *buf;
+
+ if (!qp || qp->client_ready == NTB_LINK_UP)
+ return NULL;
+
+ entry = ntb_list_rm_head(&qp->ntb_rx_pend_q_lock, &qp->rx_pend_q);
+ if (!entry)
+ return NULL;
+
+ buf = entry->callback_data;
+ *len = entry->len;
+
+ ntb_list_add_tail(&qp->ntb_rx_free_q_lock, &entry->entry,
+ &qp->rx_free_q);
+
+ return buf;
+}
+EXPORT_SYMBOL_GPL(ntb_transport_rx_remove);
+
+/**
+ * ntb_transport_rx_enqueue - Enqueue a new NTB queue entry
+ * @qp: NTB transport layer queue the entry is to be enqueued on
+ * @cb: per buffer pointer for callback function to use
+ * @data: pointer to data buffer that incoming packets will be copied into
+ * @len: length of the data buffer
+ *
+ * Enqueue a new receive buffer onto the transport queue into which a NTB
+ * payload can be received into.
+ *
+ * RETURNS: An appropriate -ERRNO error value on error, or zero for success.
+ */
+int ntb_transport_rx_enqueue(struct ntb_transport_qp *qp, void *cb, void *data,
+ unsigned int len)
+{
+ struct ntb_queue_entry *entry;
+
+ if (!qp)
+ return -EINVAL;
+
+ entry = ntb_list_rm_head(&qp->ntb_rx_free_q_lock, &qp->rx_free_q);
+ if (!entry)
+ return -ENOMEM;
+
+ entry->callback_data = cb;
+ entry->buf = data;
+ entry->len = len;
+
+ ntb_list_add_tail(&qp->ntb_rx_pend_q_lock, &entry->entry,
+ &qp->rx_pend_q);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(ntb_transport_rx_enqueue);
+
+/**
+ * ntb_transport_tx_enqueue - Enqueue a new NTB queue entry
+ * @qp: NTB transport layer queue the entry is to be enqueued on
+ * @cb: per buffer pointer for callback function to use
+ * @data: pointer to data buffer that will be sent
+ * @len: length of the data buffer
+ *
+ * Enqueue a new transmit buffer onto the transport queue from which a NTB
+ * payload will be transmitted.
+ *
+ * RETURNS: An appropriate -ERRNO error value on error, or zero for success.
+ */
+int ntb_transport_tx_enqueue(struct ntb_transport_qp *qp, void *cb, void *data,
+ unsigned int len)
+{
+ struct ntb_queue_entry *entry;
+
+ if (!qp || qp->qp_link != NTB_LINK_UP || !len)
+ return -EINVAL;
+
+ entry = ntb_list_rm_head(&qp->ntb_tx_free_q_lock, &qp->tx_free_q);
+ if (!entry) {
+ /* ring full, kick it */
+ tasklet_schedule(&qp->tx_work);
+ return -ENOMEM;
+ }
+
+ entry->callback_data = cb;
+ entry->buf = data;
+ entry->len = len;
+ entry->flags = 0;
+
+ ntb_list_add_tail(&qp->ntb_tx_pend_q_lock, &entry->entry,
+ &qp->tx_pend_q);
+
+ tasklet_schedule(&qp->tx_work);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(ntb_transport_tx_enqueue);
+
+/**
+ * ntb_transport_tx_dequeue - Dequeue a NTB queue entry
+ * @qp: NTB transport layer queue to be dequeued from
+ * @len: length of the data buffer
+ *
+ * This function will dequeue a buffer from the transmit complete queue.
+ * Entries will only be enqueued on this queue after having been
+ * transfered to the remote side.
+ *
+ * RETURNS: callback pointer of the buffer from the transport queue, or NULL
+ * on empty
+ */
+void *ntb_transport_tx_dequeue(struct ntb_transport_qp *qp, unsigned int *len)
+{
+ struct ntb_queue_entry *entry;
+ void *buf;
+
+ if (!qp)
+ return NULL;
+
+ entry = ntb_list_rm_head(&qp->ntb_tx_comp_q_lock, &qp->tx_comp_q);
+ if (!entry)
+ return NULL;
+
+ buf = entry->callback_data;
+ if (entry->flags != HW_ERROR_FLAG)
+ *len = entry->len;
+ else
+ *len = -EIO;
+
+ ntb_list_add_tail(&qp->ntb_tx_free_q_lock, &entry->entry,
+ &qp->tx_free_q);
+
+ return buf;
+}
+EXPORT_SYMBOL_GPL(ntb_transport_tx_dequeue);
+
+/**
+ * ntb_transport_rx_dequeue - Dequeue a NTB queue entry
+ * @qp: NTB transport layer queue to be dequeued from
+ * @len: length of the data buffer
+ *
+ * This function will dequeue a buffer from the receive complete queue.
+ * Entries will only be enqueued on this queue after having been fully received.
+ *
+ * RETURNS: callback pointer of the buffer from the transport queue, or NULL
+ * on empty
+ */
+void *ntb_transport_rx_dequeue(struct ntb_transport_qp *qp, unsigned int *len)
+{
+ struct ntb_queue_entry *entry;
+ void *buf;
+
+ if (!qp)
+ return NULL;
+
+ entry = ntb_list_rm_head(&qp->ntb_rx_comp_q_lock, &qp->rx_comp_q);
+ if (!entry)
+ return NULL;
+
+ buf = entry->callback_data;
+ *len = entry->len;
+
+ ntb_list_add_tail(&qp->ntb_rx_free_q_lock, &entry->entry,
+ &qp->rx_free_q);
+
+ return buf;
+}
+EXPORT_SYMBOL_GPL(ntb_transport_rx_dequeue);
+
+/**
+ * ntb_transport_link_up - Notify NTB transport of client readiness to use queue
+ * @qp: NTB transport layer queue to be enabled
+ *
+ * Notify NTB transport layer of client readiness to use queue
+ */
+void ntb_transport_link_up(struct ntb_transport_qp *qp)
+{
+ if (!qp)
+ return;
+
+ qp->client_ready = NTB_LINK_UP;
+
+ if (qp->transport->transport_link == NTB_LINK_UP)
+ schedule_delayed_work(&qp->link_work, 0);
+}
+EXPORT_SYMBOL_GPL(ntb_transport_link_up);
+
+/**
+ * ntb_transport_link_down - Notify NTB transport to no longer enqueue data
+ * @qp: NTB transport layer queue to be disabled
+ *
+ * Notify NTB transport layer of client's desire to no longer receive data on
+ * transport queue specified. It is the client's responsibility to ensure all
+ * entries on queue are purged or otherwise handled appropraitely.
+ */
+void ntb_transport_link_down(struct ntb_transport_qp *qp)
+{
+ int rc, val;
+
+ if (!qp)
+ return;
+
+ qp->client_ready = NTB_LINK_DOWN;
+
+ cancel_delayed_work_sync(&qp->link_work);
+ qp->qp_link = NTB_LINK_DOWN;
+
+ rc = ntb_read_local_spad(qp->ndev, QP_LINKS, &val);
+ if (rc) {
+ pr_err("Error reading spad %d\n", QP_LINKS);
+ return;
+ }
+
+ rc = ntb_write_remote_spad(qp->ndev, QP_LINKS,
+ val & ~(1 << qp->qp_num));
+ if (rc)
+ pr_err("Error writing %x to remote spad %d\n",
+ val & ~(1 << qp->qp_num), QP_LINKS);
+
+ if (qp->transport->transport_link == NTB_LINK_UP)
+ ntb_send_link_down(qp);
+}
+EXPORT_SYMBOL_GPL(ntb_transport_link_down);
+
+/**
+ * ntb_transport_link_query - Query transport link state
+ * @qp: NTB transport layer queue to be queried
+ *
+ * Query connectivity to the remote system of the NTB transport queue
+ *
+ * RETURNS: true for link up or false for link down
+ */
+bool ntb_transport_link_query(struct ntb_transport_qp *qp)
+{
+ return qp->qp_link == NTB_LINK_UP;
+}
+EXPORT_SYMBOL_GPL(ntb_transport_link_query);
+
+/**
+ * ntb_transport_qp_num - Query the qp number
+ * @qp: NTB transport layer queue to be queried
+ *
+ * Query qp number of the NTB transport queue
+ *
+ * RETURNS: a zero based number specifying the qp number
+ */
+unsigned char ntb_transport_qp_num(struct ntb_transport_qp *qp)
+{
+ return qp->qp_num;
+}
+EXPORT_SYMBOL_GPL(ntb_transport_qp_num);
+
+/**
+ * ntb_transport_max_size - Query the max payload size of a qp
+ * @qp: NTB transport layer queue to be queried
+ *
+ * Query the maximum payload size permissible on the given qp
+ *
+ * RETURNS: the max payload size of a qp
+ */
+unsigned int
+ntb_transport_max_size(__attribute__((unused)) struct ntb_transport_qp *qp)
+{
+ return transport_mtu;
+}
+EXPORT_SYMBOL_GPL(ntb_transport_max_size);
+
+
+
+static int ntb_client_probe(const struct ntb_client *drvr)
+{
+ struct ntb_transport *nt;
+ int rc = -EINVAL;
+
+ list_for_each_entry(nt, &ntb_transport_list, list) {
+ rc = drvr->probe(ntb_query_pdev(nt->ndev));
+ if (rc)
+ break;
+ }
+
+ return rc;
+}
+
+/**
+ * register_ntb_client - Register NTB client driver
+ * @drvr: NTB client driver to be registered
+ *
+ * Register an NTB client driver with the NTB transport layer
+ *
+ * RETURNS: An appropriate -ERRNO error value on error, or zero for success.
+ */
+int register_ntb_client(const struct ntb_client *drvr)
+{
+ if (!list_empty(&ntb_transport_list))
+ return ntb_client_probe(drvr);
+ else
+ return 0;
+}
+EXPORT_SYMBOL_GPL(register_ntb_client);
+
+static void ntb_client_remove(const struct ntb_client *drvr)
+{
+ struct ntb_transport *nt;
+
+ list_for_each_entry(nt, &ntb_transport_list, list)
+ drvr->remove(ntb_query_pdev(nt->ndev));
+}
+
+/**
+ * unregister_ntb_client - Unregister NTB client driver
+ * @drvr: NTB client driver to be unregistered
+ *
+ * Unregister an NTB client driver with the NTB transport layer
+ *
+ * RETURNS: An appropriate -ERRNO error value on error, or zero for success.
+ */
+void unregister_ntb_client(const struct ntb_client *drvr)
+{
+ if (!list_empty(&ntb_transport_list))
+ ntb_client_remove(drvr);
+}
+EXPORT_SYMBOL_GPL(unregister_ntb_client);
diff --git a/include/linux/ntb.h b/include/linux/ntb.h
new file mode 100644
index 0000000..027bb0b
--- /dev/null
+++ b/include/linux/ntb.h
@@ -0,0 +1,92 @@
+/*
+ * This file is provided under a dual BSD/GPLv2 license. When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright(c) 2012 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+ * The full GNU General Public License is included in this distribution
+ * in the file called LICENSE.GPL.
+ *
+ * BSD LICENSE
+ *
+ * Copyright(c) 2012 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copy
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * Intel PCIe NTB Linux driver
+ *
+ * Contact Information:
+ * Jon Mason <jon.mason@intel.com>
+ */
+
+struct ntb_transport_qp;
+
+struct ntb_client {
+ char *name;
+ int (*probe) (struct pci_dev *pdev);
+ void (*remove) (struct pci_dev *pdev);
+};
+
+int register_ntb_client(const struct ntb_client *drvr);
+void unregister_ntb_client(const struct ntb_client *drvr);
+
+struct ntb_queue_handlers {
+ void (*tx_handler) (void *data, struct ntb_transport_qp *qp);
+ void (*rx_handler) (void *data, struct ntb_transport_qp *qp);
+ void (*event_handler) (void *data, int status);
+};
+
+unsigned char ntb_transport_qp_num(struct ntb_transport_qp *qp);
+unsigned int ntb_transport_max_size(struct ntb_transport_qp *qp);
+struct ntb_transport_qp *
+ntb_transport_create_queue(void *data, struct pci_dev *pdev,
+ const struct ntb_queue_handlers *handlers);
+void ntb_transport_free_queue(struct ntb_transport_qp *qp);
+int ntb_transport_rx_enqueue(struct ntb_transport_qp *qp, void *cb, void *data,
+ unsigned int len);
+int ntb_transport_tx_enqueue(struct ntb_transport_qp *qp, void *cb, void *data,
+ unsigned int len);
+void *ntb_transport_tx_dequeue(struct ntb_transport_qp *qp, unsigned int *len);
+void *ntb_transport_rx_dequeue(struct ntb_transport_qp *qp, unsigned int *len);
+void *ntb_transport_rx_remove(struct ntb_transport_qp *qp, unsigned int *len);
+void ntb_transport_link_up(struct ntb_transport_qp *qp);
+void ntb_transport_link_down(struct ntb_transport_qp *qp);
+bool ntb_transport_link_query(struct ntb_transport_qp *qp);
--
1.7.5.4
^ permalink raw reply related
* [RFC v2 2/2] net: Add support for NTB virtual ethernet device
From: Jon Mason @ 2012-07-30 0:26 UTC (permalink / raw)
To: linux-kernel; +Cc: netdev, linux-pci, Dave Jiang
In-Reply-To: <1343607994-32415-1-git-send-email-jon.mason@intel.com>
A virtual ethernet device that uses the NTB transport API to send/receive data.
Signed-off-by: Jon Mason <jon.mason@intel.com>
---
drivers/net/Kconfig | 4 +
drivers/net/Makefile | 1 +
drivers/net/ntb_netdev.c | 417 ++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 422 insertions(+), 0 deletions(-)
create mode 100644 drivers/net/ntb_netdev.c
diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig
index 0c2bd80..9bf8a71 100644
--- a/drivers/net/Kconfig
+++ b/drivers/net/Kconfig
@@ -178,6 +178,10 @@ config NETPOLL_TRAP
config NET_POLL_CONTROLLER
def_bool NETPOLL
+config NTB_NETDEV
+ tristate "Virtual Ethernet over NTB"
+ depends on NTB
+
config RIONET
tristate "RapidIO Ethernet over messaging driver support"
depends on RAPIDIO
diff --git a/drivers/net/Makefile b/drivers/net/Makefile
index 3d375ca..9890148 100644
--- a/drivers/net/Makefile
+++ b/drivers/net/Makefile
@@ -69,3 +69,4 @@ obj-$(CONFIG_USB_IPHETH) += usb/
obj-$(CONFIG_USB_CDC_PHONET) += usb/
obj-$(CONFIG_HYPERV_NET) += hyperv/
+obj-$(CONFIG_NTB_NETDEV) += ntb_netdev.o
diff --git a/drivers/net/ntb_netdev.c b/drivers/net/ntb_netdev.c
new file mode 100644
index 0000000..cb19a0b
--- /dev/null
+++ b/drivers/net/ntb_netdev.c
@@ -0,0 +1,417 @@
+/*
+ * This file is provided under a dual BSD/GPLv2 license. When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * Copyright(c) 2012 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA.
+ * The full GNU General Public License is included in this distribution
+ * in the file called LICENSE.GPL.
+ *
+ * BSD LICENSE
+ *
+ * Copyright(c) 2012 Intel Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copy
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ * Intel PCIe NTB Network Linux driver
+ *
+ * Contact Information:
+ * Jon Mason <jon.mason@intel.com>
+ */
+#include <linux/etherdevice.h>
+#include <linux/ethtool.h>
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/ntb.h>
+
+#define NTB_NETDEV_VER "0.5"
+
+MODULE_DESCRIPTION(KBUILD_MODNAME);
+MODULE_VERSION(NTB_NETDEV_VER);
+MODULE_LICENSE("Dual BSD/GPL");
+MODULE_AUTHOR("Intel Corporation");
+
+struct ntb_netdev {
+ struct list_head list;
+ struct pci_dev *pdev;
+ struct net_device *ndev;
+ struct ntb_transport_qp *qp;
+};
+
+#define NTB_TX_TIMEOUT_MS 1000
+#define NTB_RXQ_SIZE 100
+
+static LIST_HEAD(dev_list);
+
+static void ntb_netdev_event_handler(void *data, int status)
+{
+ struct net_device *ndev = data;
+ struct ntb_netdev *dev = netdev_priv(ndev);
+
+ netdev_dbg(ndev, "Event %x, Link %x\n", status,
+ ntb_transport_link_query(dev->qp));
+
+ /* Currently, only link status event is supported */
+ if (status)
+ netif_carrier_on(ndev);
+ else
+ netif_carrier_off(ndev);
+}
+
+static void ntb_netdev_rx_handler(void *data, struct ntb_transport_qp *qp)
+{
+ struct net_device *ndev = data;
+ struct sk_buff *skb;
+ int len, rc;
+
+ while ((skb = ntb_transport_rx_dequeue(qp, &len))) {
+ netdev_dbg(ndev, "%s: %d byte payload received\n", __func__,
+ len);
+
+ skb_put(skb, len);
+ skb->protocol = eth_type_trans(skb, ndev);
+ skb->ip_summed = CHECKSUM_NONE;
+
+ if (netif_rx(skb) == NET_RX_DROP) {
+ ndev->stats.rx_errors++;
+ ndev->stats.rx_dropped++;
+ } else {
+ ndev->stats.rx_packets++;
+ ndev->stats.rx_bytes += len;
+ }
+
+ skb = netdev_alloc_skb(ndev, ndev->mtu + ETH_HLEN);
+ if (!skb) {
+ ndev->stats.rx_errors++;
+ ndev->stats.rx_frame_errors++;
+ break;
+ }
+
+ rc = ntb_transport_rx_enqueue(qp, skb, skb->data,
+ ndev->mtu + ETH_HLEN);
+ if (rc) {
+ ndev->stats.rx_errors++;
+ ndev->stats.rx_fifo_errors++;
+ break;
+ }
+ }
+}
+
+static void ntb_netdev_tx_handler(void *data, struct ntb_transport_qp *qp)
+{
+ struct net_device *ndev = data;
+ struct sk_buff *skb;
+ int len;
+
+ while ((skb = ntb_transport_tx_dequeue(qp, &len))) {
+ ndev->stats.tx_packets++;
+ ndev->stats.tx_bytes += skb->len;
+ dev_kfree_skb(skb);
+ }
+
+ if (ndev && netif_queue_stopped(ndev))
+ netif_wake_queue(ndev);
+}
+
+static netdev_tx_t ntb_netdev_start_xmit(struct sk_buff *skb,
+ struct net_device *ndev)
+{
+ struct ntb_netdev *dev = netdev_priv(ndev);
+ int rc;
+
+ netdev_dbg(ndev, "ntb_transport_tx_enqueue\n");
+
+ rc = ntb_transport_tx_enqueue(dev->qp, skb, skb->data, skb->len);
+ if (rc)
+ goto err;
+
+ return NETDEV_TX_OK;
+
+err:
+ ndev->stats.tx_dropped++;
+ ndev->stats.tx_errors++;
+ netif_stop_queue(ndev);
+ return NETDEV_TX_BUSY;
+}
+
+static int ntb_netdev_open(struct net_device *ndev)
+{
+ struct ntb_netdev *dev = netdev_priv(ndev);
+ struct sk_buff *skb;
+ int rc, i, len;
+
+ /* Add some empty rx bufs */
+ for (i = 0; i < NTB_RXQ_SIZE; i++) {
+ skb = netdev_alloc_skb(ndev, ndev->mtu + ETH_HLEN);
+ if (!skb) {
+ rc = -ENOMEM;
+ goto err;
+ }
+
+ rc = ntb_transport_rx_enqueue(dev->qp, skb, skb->data,
+ ndev->mtu + ETH_HLEN);
+ if (rc == -EINVAL)
+ goto err;
+ }
+
+ netif_carrier_off(ndev);
+ ntb_transport_link_up(dev->qp);
+
+ return 0;
+
+err:
+ while ((skb = ntb_transport_rx_remove(dev->qp, &len)))
+ dev_kfree_skb(skb);
+ return rc;
+}
+
+static int ntb_netdev_close(struct net_device *ndev)
+{
+ struct ntb_netdev *dev = netdev_priv(ndev);
+ struct sk_buff *skb;
+ int len;
+
+ ntb_transport_link_down(dev->qp);
+
+ while ((skb = ntb_transport_rx_remove(dev->qp, &len)))
+ dev_kfree_skb(skb);
+
+ return 0;
+}
+
+static int ntb_netdev_change_mtu(struct net_device *ndev, int new_mtu)
+{
+ struct ntb_netdev *dev = netdev_priv(ndev);
+ struct sk_buff *skb;
+ int len, rc;
+
+ if (new_mtu > ntb_transport_max_size(dev->qp) - ETH_HLEN)
+ return -EINVAL;
+
+ if (!netif_running(ndev)) {
+ ndev->mtu = new_mtu;
+ return 0;
+ }
+
+ /* Bring down the link and dispose of posted rx entries */
+ ntb_transport_link_down(dev->qp);
+
+ if (ndev->mtu < new_mtu) {
+ int i;
+
+ for (i = 0; (skb = ntb_transport_rx_remove(dev->qp, &len)); i++)
+ dev_kfree_skb(skb);
+
+ for (; i; i--) {
+ skb = netdev_alloc_skb(ndev, new_mtu + ETH_HLEN);
+ if (!skb) {
+ rc = -ENOMEM;
+ goto err;
+ }
+
+ rc = ntb_transport_rx_enqueue(dev->qp, skb, skb->data,
+ new_mtu + ETH_HLEN);
+ if (rc) {
+ dev_kfree_skb(skb);
+ goto err;
+ }
+ }
+ }
+
+ ndev->mtu = new_mtu;
+
+ ntb_transport_link_up(dev->qp);
+
+ return 0;
+
+err:
+ ntb_transport_link_down(dev->qp);
+
+ while ((skb = ntb_transport_rx_remove(dev->qp, &len)))
+ dev_kfree_skb(skb);
+
+ netdev_err(ndev, "Error changing MTU, device inoperable\n");
+ return rc;
+}
+
+static void ntb_netdev_tx_timeout(struct net_device *ndev)
+{
+ if (netif_running(ndev))
+ netif_wake_queue(ndev);
+}
+
+static const struct net_device_ops ntb_netdev_ops = {
+ .ndo_open = ntb_netdev_open,
+ .ndo_stop = ntb_netdev_close,
+ .ndo_start_xmit = ntb_netdev_start_xmit,
+ .ndo_change_mtu = ntb_netdev_change_mtu,
+ .ndo_tx_timeout = ntb_netdev_tx_timeout,
+ .ndo_set_mac_address = eth_mac_addr,
+};
+
+static void ntb_get_drvinfo(struct net_device *ndev,
+ struct ethtool_drvinfo *info)
+{
+ struct ntb_netdev *dev = netdev_priv(ndev);
+
+ strlcpy(info->driver, KBUILD_MODNAME, sizeof(info->driver));
+ strlcpy(info->version, NTB_NETDEV_VER, sizeof(info->version));
+ strlcpy(info->bus_info, pci_name(dev->pdev), sizeof(info->bus_info));
+}
+
+static int ntb_get_settings(struct net_device *dev, struct ethtool_cmd *cmd)
+{
+ cmd->supported = SUPPORTED_Backplane;
+ cmd->advertising = ADVERTISED_Backplane;
+ cmd->speed = SPEED_UNKNOWN;
+ ethtool_cmd_speed_set(cmd, SPEED_UNKNOWN);
+ cmd->duplex = DUPLEX_FULL;
+ cmd->port = PORT_OTHER;
+ cmd->phy_address = 0;
+ cmd->transceiver = XCVR_DUMMY1;
+ cmd->autoneg = AUTONEG_ENABLE;
+ cmd->maxtxpkt = 0;
+ cmd->maxrxpkt = 0;
+
+ return 0;
+}
+
+static const struct ethtool_ops ntb_ethtool_ops = {
+ .get_drvinfo = ntb_get_drvinfo,
+ .get_link = ethtool_op_get_link,
+ .get_settings = ntb_get_settings,
+};
+
+static const struct ntb_queue_handlers ntb_netdev_handlers = {
+ .tx_handler = ntb_netdev_tx_handler,
+ .rx_handler = ntb_netdev_rx_handler,
+ .event_handler = ntb_netdev_event_handler,
+};
+
+static int __devinit ntb_netdev_probe(struct pci_dev *pdev)
+{
+ struct net_device *ndev;
+ struct ntb_netdev *dev;
+ int rc;
+
+ ndev = alloc_etherdev(sizeof(struct ntb_netdev));
+ if (!ndev)
+ return -ENOMEM;
+
+ dev = netdev_priv(ndev);
+ dev->ndev = ndev;
+ dev->pdev = pdev;
+ BUG_ON(!dev->pdev);
+ ndev->features = NETIF_F_HIGHDMA;
+
+ //ndev->priv_flags |= IFF_LIVE_ADDR_CHANGE;
+
+ ndev->hw_features = ndev->features;
+ ndev->watchdog_timeo = msecs_to_jiffies(NTB_TX_TIMEOUT_MS);
+
+ random_ether_addr(ndev->perm_addr);
+ memcpy(ndev->dev_addr, ndev->perm_addr, ndev->addr_len);
+
+ ndev->netdev_ops = &ntb_netdev_ops;
+ SET_ETHTOOL_OPS(ndev, &ntb_ethtool_ops);
+
+ dev->qp = ntb_transport_create_queue(ndev, pdev, &ntb_netdev_handlers);
+ if (!dev->qp) {
+ rc = -EIO;
+ goto err;
+ }
+
+ ndev->mtu = ntb_transport_max_size(dev->qp) - ETH_HLEN;
+
+ rc = register_netdev(ndev);
+ if (rc)
+ goto err1;
+
+ list_add(&dev->list, &dev_list);
+ pr_info("%s: %s created\n", KBUILD_MODNAME, ndev->name);
+ return 0;
+
+err1:
+ ntb_transport_free_queue(dev->qp);
+err:
+ free_netdev(ndev);
+ return rc;
+}
+
+static void __exit ntb_netdev_remove(struct pci_dev *pdev)
+{
+ struct net_device *ndev;
+ struct ntb_netdev *dev;
+
+ list_for_each_entry(dev, &dev_list, list) {
+ if (dev->pdev == pdev)
+ break;
+ }
+ if (dev == NULL)
+ return;
+
+ ndev = dev->ndev;
+
+ unregister_netdev(ndev);
+ ntb_transport_free_queue(dev->qp);
+ free_netdev(ndev);
+}
+
+static struct ntb_client ntb_netdev_client = {
+ .name = KBUILD_MODNAME,
+ .probe = ntb_netdev_probe,
+ .remove = ntb_netdev_remove,
+};
+
+static int __init ntb_netdev_init_module(void)
+{
+ return register_ntb_client(&ntb_netdev_client);
+}
+
+module_init(ntb_netdev_init_module);
+
+static void __exit ntb_netdev_exit_module(void)
+{
+ unregister_ntb_client(&ntb_netdev_client);
+ pr_info("%s: Driver removed\n", KBUILD_MODNAME);
+}
+
+module_exit(ntb_netdev_exit_module);
--
1.7.5.4
^ permalink raw reply related
* Re: [PATCH 2/7] netpoll: make __netpoll_cleanup non-block
From: Cong Wang @ 2012-07-30 1:42 UTC (permalink / raw)
To: Neil Horman
Cc: bridge, Jiri Pirko, netdev, Jay Vosburgh, linux-kernel,
Eric Dumazet, Joe Perches, Cong Wang, Stephen Hemminger,
David S. Miller
In-Reply-To: <20120727184031.GB20938@hmsreliant.think-freely.org>
On Fri, 2012-07-27 at 14:40 -0400, Neil Horman wrote:
> Here, and above I see you using an rcu_head to defer cleanup, until after all
> pointer uses are dropped, but I don't see any modification of code points that
> dereference any struct netpoll pointers to include
> rcu_read_lock()/rcu_read_unlock(). Without those using rcu to defer cleanup is
> pointless, as the rcu code won't know when its safe to run. You're no better
> off that you would be just calling __netpoll_cleanup directly.
Hi, Neil,
Actually they are protected by rcu_read_lock_bh() which is implied by
local_irq_disable(), see:
commit f0f9deae9e7c421fa0c1c627beb8e174325e1ba7
Author: Herbert Xu <herbert@gondor.apana.org.au>
Date: Fri Sep 17 16:55:03 2010 -0700
netpoll: Disable IRQ around RCU dereference in netpoll_rx
We cannot use rcu_dereference_bh safely in netpoll_rx as we may
be called with IRQs disabled. We could however simply disable
IRQs as that too causes BH to be disabled and is safe in either
case.
Thanks to John Linville for discovering this bug and providing
a patch.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: David S. Miller <davem@davemloft.net>
Thanks.
^ permalink raw reply
* Re: [PATCH 4/7] bridge: call NETDEV_RELEASE notifier in br_del_if()
From: Cong Wang @ 2012-07-30 1:59 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: netdev, bridge, David S. Miller, linux-kernel
In-Reply-To: <20120727085021.1d4ef810@nehalam.linuxnetplumber.net>
On Fri, 2012-07-27 at 08:50 -0700, Stephen Hemminger wrote:
> On Fri, 27 Jul 2012 23:38:01 +0800
> Cong Wang <amwang@redhat.com> wrote:
>
> > When a bridge interface deletes its underlying ports, it should
> > notify netconsole too, like what bonding interface does.
> >
> > Cc: "David S. Miller" <davem@davemloft.net>
> > Signed-off-by: Cong Wang <amwang@redhat.com>
> > ---
> > net/bridge/br_if.c | 1 +
> > 1 files changed, 1 insertions(+), 0 deletions(-)
> >
> > diff --git a/net/bridge/br_if.c b/net/bridge/br_if.c
> > index e1144e1..d243914 100644
> > --- a/net/bridge/br_if.c
> > +++ b/net/bridge/br_if.c
> > @@ -427,6 +427,7 @@ int br_del_if(struct net_bridge *br, struct net_device *dev)
> > if (!p || p->br != br)
> > return -EINVAL;
> >
> > + call_netdevice_notifiers(NETDEV_RELEASE, br->dev);
> > del_nbp(p);
> >
> > spin_lock_bh(&br->lock);
>
> Since you can have multiple ports attached to the bridge, this
> doesn't seem correct. Don't you want the netconsole to keep going
> on the other ports of the bridge?
>
> What exactly is the problem with having netconsole persist?
Hmm, I saw an incorrect log message when deleting the last port from the
bridge when netconsole is setup on it. After rethinking it today, you
are right we should not simply disable netconsole when one port is
detached, as we have no way to know if that port is used to reach the
netconsole server.
So, please ignore this patch.
Thanks.
^ permalink raw reply
* [PATCH v4] ipv6: fix incorrect route 'expires' value passed to userspace
From: Li Wei @ 2012-07-30 2:01 UTC (permalink / raw)
To: davem; +Cc: eric.dumazet, netdev, Li Wei
In-Reply-To: <1343199114.2626.11088.camel@edumazet-glaptop>
When userspace use RTM_GETROUTE to dump route table, with an already
expired route entry, we always got an 'expires' value(2147157)
calculated base on INT_MAX.
The reason of this problem is in the following satement:
rt->dst.expires - jiffies < INT_MAX
gcc promoted the type of both sides of '<' to unsigned long, thus
a small negative value would be considered greater than INT_MAX.
With the help of Eric Dumazet, do the out of bound checks in
rtnl_put_cacheinfo(), _after_ conversion to clock_t.
Signed-off-by: Li Wei <lw@cn.fujitsu.com>
---
In fact, all the code was reconstructed by Eric, I just put the
commit log and resent it to the maillist, thanks Eric!
net/core/rtnetlink.c | 8 ++++++--
net/ipv6/route.c | 8 ++------
2 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index bc9e380..5ff949d 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c
@@ -625,9 +625,13 @@ int rtnl_put_cacheinfo(struct sk_buff *skb, struct dst_entry *dst, u32 id,
.rta_id = id,
};
- if (expires)
- ci.rta_expires = jiffies_to_clock_t(expires);
+ if (expires) {
+ unsigned long clock;
+ clock = jiffies_to_clock_t(abs(expires));
+ clock = min_t(unsigned long, clock, INT_MAX);
+ ci.rta_expires = (expires > 0) ? clock : -clock;
+ }
return nla_put(skb, RTA_CACHEINFO, sizeof(ci), &ci);
}
EXPORT_SYMBOL_GPL(rtnl_put_cacheinfo);
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index cf02cb9..8e80fd2 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -2480,12 +2480,8 @@ static int rt6_fill_node(struct net *net,
goto nla_put_failure;
if (nla_put_u32(skb, RTA_PRIORITY, rt->rt6i_metric))
goto nla_put_failure;
- if (!(rt->rt6i_flags & RTF_EXPIRES))
- expires = 0;
- else if (rt->dst.expires - jiffies < INT_MAX)
- expires = rt->dst.expires - jiffies;
- else
- expires = INT_MAX;
+
+ expires = (rt->rt6i_flags & RTF_EXPIRES) ? rt->dst.expires - jiffies : 0;
if (rtnl_put_cacheinfo(skb, &rt->dst, 0, expires, rt->dst.error) < 0)
goto nla_put_failure;
--
1.7.10.1
^ permalink raw reply related
* Re: [net-next RFC V5 4/5] virtio_net: multiqueue support
From: Jason Wang @ 2012-07-30 3:26 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: krkumar2, habanero, mashirle, kvm, netdev, sri, linux-kernel,
virtualization, edumazet, Sasha Levin, jwhan, davem, tahm
In-Reply-To: <20120729094451.GB8977@redhat.com>
On 07/29/2012 05:44 PM, Michael S. Tsirkin wrote:
> On Sat, Jul 21, 2012 at 02:02:58PM +0200, Sasha Levin wrote:
>> On 07/20/2012 03:40 PM, Michael S. Tsirkin wrote:
>>>> - err = init_vqs(vi);
>>>>> + if (virtio_has_feature(vdev, VIRTIO_NET_F_CTRL_VQ))
>>>>> + vi->has_cvq = true;
>>>>> +
>>> How about we disable multiqueue if there's no cvq?
>>> Will make logic a bit simpler, won't it?
>> multiqueues don't really depend on cvq. Does this added complexity really justifies adding an artificial limit?
> Well !cvq support is a legacy feature: the reason we support it
> in driver is to avoid breaking on old hosts. Adding more code to that
> path just doesn't make much sense since old hosts won't have mq.
>
After some thought about this, maybe there's no need to the cvq for the
negotiation if we want support only two modes ( 1 tx/rx queue pair and N
tx/rx queue pairs). We can do this just through the feature bit negotiation.
^ permalink raw reply
* Re: [net-next RFC V5 4/5] virtio_net: multiqueue support
From: Jason Wang @ 2012-07-30 3:29 UTC (permalink / raw)
To: Sasha Levin
Cc: krkumar2, habanero, mashirle, kvm, Michael S. Tsirkin, netdev,
linux-kernel, virtualization, edumazet, tahm, jwhan, davem, sri
In-Reply-To: <500D1926.5020407@gmail.com>
On 07/23/2012 05:28 PM, Sasha Levin wrote:
> On 07/23/2012 07:54 AM, Jason Wang wrote:
>> On 07/21/2012 08:02 PM, Sasha Levin wrote:
>>> On 07/20/2012 03:40 PM, Michael S. Tsirkin wrote:
>>>>> - err = init_vqs(vi);
>>>>>> + if (virtio_has_feature(vdev, VIRTIO_NET_F_CTRL_VQ))
>>>>>> + vi->has_cvq = true;
>>>>>> +
>>>> How about we disable multiqueue if there's no cvq?
>>>> Will make logic a bit simpler, won't it?
>>> multiqueues don't really depend on cvq. Does this added complexity really justifies adding an artificial limit?
>>>
>> Yes, it does not depends on cvq. Cvq were just used to negotiate the number of queues a guest wishes to use which is really useful (at least for now). Since multiqueue can not out-perform for single queue in every kinds of workloads or benchmark, so we want to let guest driver use single queue by default even when multiqueue were enabled by management software and let use to enalbe it through ethtool. So user could not feel regression when it switch to use a multiqueue capable driver and backend.
> Why would you limit it to a single vq if the user has specified a different number of vqs (>1) in the virtio-net device config?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
The only reason is to prevent the user from seeing the regression. The
performance of small packet sending is wrose than single queue, it tends
to send more but small packets when multiqueue is enabled. If we make
multiqueue bahave as good as single queue, we can remove this limit.
^ permalink raw reply
* Re: [net-next RFC V5 2/5] virtio_ring: move queue_index to vring_virtqueue
From: Jason Wang @ 2012-07-30 3:30 UTC (permalink / raw)
To: Paolo Bonzini
Cc: krkumar2, habanero, kvm, mst, netdev, mashirle, linux-kernel,
virtualization, edumazet, Sasha Levin, jwhan, sri, davem, tahm
In-Reply-To: <5010FDB2.5040505@redhat.com>
On 07/26/2012 04:20 PM, Paolo Bonzini wrote:
> Il 05/07/2012 13:40, Sasha Levin ha scritto:
>> @@ -275,7 +274,7 @@ static void vm_del_vq(struct virtqueue *vq)
>> vring_del_virtqueue(vq);
>>
>> /* Select and deactivate the queue */
>> - writel(info->queue_index, vm_dev->base + VIRTIO_MMIO_QUEUE_SEL);
>> + writel(virtqueue_get_queue_index(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_SEL);
>> writel(0, vm_dev->base + VIRTIO_MMIO_QUEUE_PFN);
>>
> This accesses vq after vring_del_virtqueue has freed it.
>
> Paolo
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
Yes, so need a temporary variable before vring_del_virtqueue().
Thanks.
^ permalink raw reply
* Re: [PATCH RESEND net-next V2] IB/ipoib: break linkage to neighbouring system
From: Roland Dreier @ 2012-07-30 3:48 UTC (permalink / raw)
To: Or Gerlitz; +Cc: davem, netdev, eric.dumazet, cl, Shlomo Pongratz
In-Reply-To: <1343150651-12568-1-git-send-email-ogerlitz@mellanox.com>
thanks, applied.
^ permalink raw reply
* [PATCH 1/2] net: Allow to create links with given ifindex
From: Pavel Emelyanov @ 2012-07-30 4:34 UTC (permalink / raw)
To: Linux Netdev List, David Miller
Currently the RTM_NEWLINK results in -EOPNOTSUPP if the ifinfomsg->ifi_index
is not zero. I propose to allow requesting ifindices on link creation. This
is required by the checkpoint-restore to correctly restore a net namespace
(i.e. -- a container). The question what to do with pre-created devices such
as lo or sit fbdev is open, but for manually created devices this can be
solved by this patch.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
---
diff --git a/net/core/dev.c b/net/core/dev.c
index 0ebaea1..5966e2f 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -5533,7 +5533,12 @@ int register_netdevice(struct net_device *dev)
}
}
- dev->ifindex = dev_new_index(net);
+ ret = -EBUSY;
+ if (!dev->ifindex)
+ dev->ifindex = dev_new_index(net);
+ else if (__dev_get_by_index(net, dev->ifindex))
+ goto err_uninit;
+
if (dev->iflink == -1)
dev->iflink = dev->ifindex;
diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c
index 334b930..76e19aa 100644
--- a/net/core/rtnetlink.c
+++ b/net/core/rtnetlink.c
@@ -1801,8 +1801,6 @@ replay:
return -ENODEV;
}
- if (ifm->ifi_index)
- return -EOPNOTSUPP;
if (tb[IFLA_MAP] || tb[IFLA_MASTER] || tb[IFLA_PROTINFO])
return -EOPNOTSUPP;
@@ -1828,10 +1826,14 @@ replay:
return PTR_ERR(dest_net);
dev = rtnl_create_link(net, dest_net, ifname, ops, tb);
-
- if (IS_ERR(dev))
+ if (IS_ERR(dev)) {
err = PTR_ERR(dev);
- else if (ops->newlink)
+ goto out;
+ }
+
+ dev->ifindex = ifm->ifi_index;
+
+ if (ops->newlink)
err = ops->newlink(net, dev, tb, data);
else
err = register_netdevice(dev);
^ permalink raw reply related
* [PATCH 2/2] veth: Allow to create peer link with given ifindex
From: Pavel Emelyanov @ 2012-07-30 4:36 UTC (permalink / raw)
To: Linux Netdev List, David Miller
In-Reply-To: <50160EEF.6050406@parallels.com>
The ifinfomsg is in there (thanks kaber@ for foreseeing this long time ago),
so take the given ifidex and register netdev with it.
Signed-off-by: Pavel Emelyanov <xemul@parallels.com>
---
diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 5852361..496c026 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -348,6 +348,9 @@ static int veth_newlink(struct net *src_net, struct net_device *dev,
if (tbp[IFLA_ADDRESS] == NULL)
eth_hw_addr_random(peer);
+ if (ifmp)
+ peer->ifindex = ifmp->ifi_index;
+
err = register_netdevice(peer);
put_net(net);
net = NULL;
^ permalink raw reply related
* Re: [PATCH] bnx2x: remove cast around the kmalloc in bnx2x_prev_mark_path
From: Eilon Greenstein @ 2012-07-30 4:50 UTC (permalink / raw)
To: netdev
In-Reply-To: <1343567963-12389-1-git-send-email-develkernel412222@gmail.com>
On Sun, 2012-07-29 at 19:04 +0545, Devendra Naga wrote:
> casting the void pointer is redundant (Documentation/CodingStyle)
>
> Signed-off-by: Devendra Naga <develkernel412222@gmail.com>
Thanks Devendra, obviously, I’m acking this patch. I’m not so sure if it
should go into net though, and net-next is closed right now but this is
up to Dave.
Acked-by: Eilon Greenstein <eilong@broadcom.com>
^ permalink raw reply
* Re: [PATCH] sctp: Make "Invalid Stream Identifier" ERROR follows SACK when bundling
From: Xufeng Zhang @ 2012-07-30 4:58 UTC (permalink / raw)
To: Vlad Yasevich
Cc: Neil Horman, xufeng zhang, sri, davem, linux-sctp, netdev,
linux-kernel
In-Reply-To: <5013069F.3080306@gmail.com>
On 7/28/12, Vlad Yasevich <vyasevich@gmail.com> wrote:
> here is an untested prototype of what I was talking about. This should
> handle multiple data chunks.
Yes, it works if only the end of the DATA chunk in a packet has
invalid stream identifier
and I have verified this patch by my test case, but what happens if
there are multiple
DATA chunks which have invalid stream identifier in a packet?
Consider the below example:
A packet has several chunks bundling together: "COOKIE_ECHO DATA DATA", both
of the two DATA chunks have invalid stream identifier, then the
response will be
"COOKIE_ACK ERROR SACK ERROR", right?
Thanks,
Xufeng Zhang
>
> -vlad
>
> ---
> include/net/sctp/command.h | 1 +
> net/sctp/sm_sideeffect.c | 22 ++++++++++++++++++++++
> net/sctp/sm_statefuns.c | 18 ++++++++++--------
> 3 files changed, 33 insertions(+), 8 deletions(-)
>
> diff --git a/include/net/sctp/command.h b/include/net/sctp/command.h
> index 712b3be..4043445 100644
> --- a/include/net/sctp/command.h
> +++ b/include/net/sctp/command.h
> @@ -110,6 +110,7 @@ typedef enum {
> SCTP_CMD_SEND_NEXT_ASCONF, /* Send the next ASCONF after ACK */
> SCTP_CMD_PURGE_ASCONF_QUEUE, /* Purge all asconf queues.*/
> SCTP_CMD_SET_ASOC, /* Restore association context */
> + SCTP_CMD_GEN_BAD_STREAM, /* Issue an Invalid Stream error */
> SCTP_CMD_LAST
> } sctp_verb_t;
>
> diff --git a/net/sctp/sm_sideeffect.c b/net/sctp/sm_sideeffect.c
> index 1ff51c9..c5a1322 100644
> --- a/net/sctp/sm_sideeffect.c
> +++ b/net/sctp/sm_sideeffect.c
> @@ -1036,6 +1036,22 @@ static void sctp_cmd_send_asconf(struct
> sctp_association *asoc)
> }
> }
>
> +static void sctp_cmd_make_inv_stream_err(sctp_cmd_seq_t *commands,
> + struct sctp_association *asoc,
> + struct sctp_chunk *chunk,
> + struct sctp_datahdr *data_hdr)
> +{
> + struct sctp_chunk *err;
> +
> + err = sctp_make_op_error(asoc, chunk, SCTP_ERROR_INV_STRM,
> + &data_hdr->stream,
> + sizeof(data_hdr->stream),
> + sizeof(u16));
> + if (err)
> + sctp_add_cmd_sf(commands, SCTP_CMD_REPLY,
> + SCTP_CHUNK(err));
> +}
> +
>
> /* These three macros allow us to pull the debugging code out of the
> * main flow of sctp_do_sm() to keep attention focused on the real
> @@ -1700,6 +1716,12 @@ static int sctp_cmd_interpreter(sctp_event_t
> event_type,
> asoc = cmd->obj.asoc;
> break;
>
> + case SCTP_CMD_GEN_BAD_STREAM:
> + sctp_cmd_make_inv_stream_err(commands,
> + asoc, chunk,
> + (struct sctp_datahdr *)cmd->obj.ptr);
> + break;
> +
> default:
> pr_warn("Impossible command: %u, %p\n",
> cmd->verb, cmd->obj.ptr);
> diff --git a/net/sctp/sm_statefuns.c b/net/sctp/sm_statefuns.c
> index 891f5db..57532e3 100644
> --- a/net/sctp/sm_statefuns.c
> +++ b/net/sctp/sm_statefuns.c
> @@ -2972,6 +2972,12 @@ discard_noforce:
> if (chunk->end_of_packet)
> sctp_add_cmd_sf(commands, SCTP_CMD_GEN_SACK, force);
>
> + /* Queue the INVALID STREAM error after the SACK if one is needed. */
> + if (SCTP_IERROR_BAD_STREAM == error) {
> + sctp_add_cmd_sf(commands, SCTP_CMD_GEN_BAD_STREAM,
> + SCTP_PTR(chunk->subh.data_hdr));
> + }
> +
> return SCTP_DISPOSITION_DISCARD;
> consume:
> return SCTP_DISPOSITION_CONSUME;
> @@ -3044,6 +3050,10 @@ sctp_disposition_t
> sctp_sf_eat_data_fast_4_4(const struct sctp_endpoint *ep,
> */
> sctp_add_cmd_sf(commands, SCTP_CMD_GEN_SHUTDOWN, SCTP_NULL());
> sctp_add_cmd_sf(commands, SCTP_CMD_GEN_SACK, SCTP_FORCE());
> + if (SCTP_IERROR_BAD_STREAM == error) {
> + sctp_add_cmd_sf(commands, SCTP_CMD_GEN_BAD_STREAM,
> + SCTP_PTR(chunk->subh.data_hdr));
> + }
> sctp_add_cmd_sf(commands, SCTP_CMD_TIMER_RESTART,
> SCTP_TO(SCTP_EVENT_TIMEOUT_T2_SHUTDOWN));
> }
> @@ -6140,14 +6150,6 @@ static int sctp_eat_data(const struct
> sctp_association *asoc,
> if (sid >= asoc->c.sinit_max_instreams) {
> /* Mark tsn as received even though we drop it */
> sctp_add_cmd_sf(commands, SCTP_CMD_REPORT_TSN, SCTP_U32(tsn));
> -
> - err = sctp_make_op_error(asoc, chunk, SCTP_ERROR_INV_STRM,
> - &data_hdr->stream,
> - sizeof(data_hdr->stream),
> - sizeof(u16));
> - if (err)
> - sctp_add_cmd_sf(commands, SCTP_CMD_REPLY,
> - SCTP_CHUNK(err));
> return SCTP_IERROR_BAD_STREAM;
> }
>
> -- 1.7.7.6
>
>
>
^ permalink raw reply
* [PATCH net 1/5] tg3: Request APE_LOCK_PHY before PHY access
From: Michael Chan @ 2012-07-30 5:15 UTC (permalink / raw)
To: davem; +Cc: netdev
to prevent PHY access conflict with APE firmware.
Signed-off-by: Michael Chan <mchan@broadcom.com>
---
drivers/net/ethernet/broadcom/tg3.c | 37 +++++++++++++++++++++++++++++++++++
drivers/net/ethernet/broadcom/tg3.h | 1 +
2 files changed, 38 insertions(+), 0 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index 9a009fd..a528f9a 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -672,6 +672,12 @@ static int tg3_ape_lock(struct tg3 *tp, int locknum)
else
bit = 1 << tp->pci_fn;
break;
+ case TG3_APE_LOCK_PHY0:
+ case TG3_APE_LOCK_PHY1:
+ case TG3_APE_LOCK_PHY2:
+ case TG3_APE_LOCK_PHY3:
+ bit = APE_LOCK_REQ_DRIVER;
+ break;
default:
return -EINVAL;
}
@@ -723,6 +729,12 @@ static void tg3_ape_unlock(struct tg3 *tp, int locknum)
else
bit = 1 << tp->pci_fn;
break;
+ case TG3_APE_LOCK_PHY0:
+ case TG3_APE_LOCK_PHY1:
+ case TG3_APE_LOCK_PHY2:
+ case TG3_APE_LOCK_PHY3:
+ bit = APE_LOCK_GRANT_DRIVER;
+ break;
default:
return;
}
@@ -1052,6 +1064,8 @@ static int tg3_readphy(struct tg3 *tp, int reg, u32 *val)
udelay(80);
}
+ tg3_ape_lock(tp, tp->phy_ape_lock);
+
*val = 0x0;
frame_val = ((tp->phy_addr << MI_COM_PHY_ADDR_SHIFT) &
@@ -1086,6 +1100,8 @@ static int tg3_readphy(struct tg3 *tp, int reg, u32 *val)
udelay(80);
}
+ tg3_ape_unlock(tp, tp->phy_ape_lock);
+
return ret;
}
@@ -1105,6 +1121,8 @@ static int tg3_writephy(struct tg3 *tp, int reg, u32 val)
udelay(80);
}
+ tg3_ape_lock(tp, tp->phy_ape_lock);
+
frame_val = ((tp->phy_addr << MI_COM_PHY_ADDR_SHIFT) &
MI_COM_PHY_ADDR_MASK);
frame_val |= ((reg << MI_COM_REG_ADDR_SHIFT) &
@@ -1135,6 +1153,8 @@ static int tg3_writephy(struct tg3 *tp, int reg, u32 val)
udelay(80);
}
+ tg3_ape_unlock(tp, tp->phy_ape_lock);
+
return ret;
}
@@ -13648,6 +13668,23 @@ static int __devinit tg3_phy_probe(struct tg3 *tp)
tg3_flag_set(tp, PAUSE_AUTONEG);
tp->link_config.flowctrl = FLOW_CTRL_TX | FLOW_CTRL_RX;
+ if (tg3_flag(tp, ENABLE_APE)) {
+ switch (tp->pci_fn) {
+ case 0:
+ tp->phy_ape_lock = TG3_APE_LOCK_PHY0;
+ break;
+ case 1:
+ tp->phy_ape_lock = TG3_APE_LOCK_PHY1;
+ break;
+ case 2:
+ tp->phy_ape_lock = TG3_APE_LOCK_PHY2;
+ break;
+ case 3:
+ tp->phy_ape_lock = TG3_APE_LOCK_PHY3;
+ break;
+ }
+ }
+
if (tg3_flag(tp, USE_PHYLIB))
return tg3_phy_init(tp);
diff --git a/drivers/net/ethernet/broadcom/tg3.h b/drivers/net/ethernet/broadcom/tg3.h
index a1b75cd..6fb45a5 100644
--- a/drivers/net/ethernet/broadcom/tg3.h
+++ b/drivers/net/ethernet/broadcom/tg3.h
@@ -3107,6 +3107,7 @@ struct tg3 {
int old_link;
u8 phy_addr;
+ u8 phy_ape_lock;
/* PHY info */
u32 phy_id;
--
1.7.1
^ permalink raw reply related
* [PATCH net 2/5] tg3: Fix Read DMA workaround for 5719 A0.
From: Michael Chan @ 2012-07-30 5:15 UTC (permalink / raw)
To: davem; +Cc: netdev
In-Reply-To: <1343625345-5155-1-git-send-email-mchan@broadcom.com>
The workaround was mis-applied to all 5719 and 5720 chips.
Signed-off-by: Michael Chan <mchan@broadcom.com>
---
drivers/net/ethernet/broadcom/tg3.c | 3 +--
1 files changed, 1 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index a528f9a..f0434df 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -9086,8 +9086,7 @@ static int tg3_reset_hw(struct tg3 *tp, int reset_phy)
GET_ASIC_REV(tp->pci_chip_rev_id) == ASIC_REV_57780 ||
tg3_flag(tp, 57765_PLUS)) {
val = tr32(TG3_RDMA_RSRVCTRL_REG);
- if (GET_ASIC_REV(tp->pci_chip_rev_id) == ASIC_REV_5719 ||
- GET_ASIC_REV(tp->pci_chip_rev_id) == ASIC_REV_5720) {
+ if (tp->pci_chip_rev_id == CHIPREV_ID_5719_A0) {
val &= ~(TG3_RDMA_RSRVCTRL_TXMRGN_MASK |
TG3_RDMA_RSRVCTRL_FIFO_LWM_MASK |
TG3_RDMA_RSRVCTRL_FIFO_HWM_MASK);
--
1.7.1
^ permalink raw reply related
* [PATCH net 3/5] tg3: Add New 5719 Read DMA workaround
From: Michael Chan @ 2012-07-30 5:15 UTC (permalink / raw)
To: davem; +Cc: netdev
In-Reply-To: <1343625345-5155-2-git-send-email-mchan@broadcom.com>
After Power-on-reset, the 5719's TX DMA length registers may contain
uninitialized values and cause TX DMA to stall. Check for invalid
values and set a register bit to flush the TX channels. The bit
needs to be turned off after the DMA channels have been flushed.
Signed-off-by: Michael Chan <mchan@broadcom.com>
---
drivers/net/ethernet/broadcom/tg3.c | 23 +++++++++++++++++++++++
drivers/net/ethernet/broadcom/tg3.h | 7 ++++++-
2 files changed, 29 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index f0434df..50045ed 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -9276,6 +9276,19 @@ static int tg3_reset_hw(struct tg3 *tp, int reset_phy)
tw32_f(RDMAC_MODE, rdmac_mode);
udelay(40);
+ if (GET_ASIC_REV(tp->pci_chip_rev_id) == ASIC_REV_5719) {
+ for (i = 0; i < TG3_NUM_RDMA_CHANNELS; i++) {
+ if (tr32(TG3_RDMA_LENGTH + (i << 2)) > TG3_MAX_MTU(tp))
+ break;
+ }
+ if (i < TG3_NUM_RDMA_CHANNELS) {
+ val = tr32(TG3_LSO_RD_DMA_CRPTEN_CTRL);
+ val |= TG3_LSO_RD_DMA_TX_LENGTH_WA;
+ tw32(TG3_LSO_RD_DMA_CRPTEN_CTRL, val);
+ tg3_flag_set(tp, 5719_RDMA_BUG);
+ }
+ }
+
tw32(RCVDCC_MODE, RCVDCC_MODE_ENABLE | RCVDCC_MODE_ATTN_ENABLE);
if (!tg3_flag(tp, 5705_PLUS))
tw32(MBFREE_MODE, MBFREE_MODE_ENABLE);
@@ -9635,6 +9648,16 @@ static void tg3_periodic_fetch_stats(struct tg3 *tp)
TG3_STAT_ADD32(&sp->tx_ucast_packets, MAC_TX_STATS_UCAST);
TG3_STAT_ADD32(&sp->tx_mcast_packets, MAC_TX_STATS_MCAST);
TG3_STAT_ADD32(&sp->tx_bcast_packets, MAC_TX_STATS_BCAST);
+ if (unlikely(tg3_flag(tp, 5719_RDMA_BUG) &&
+ (sp->tx_ucast_packets.low + sp->tx_mcast_packets.low +
+ sp->tx_bcast_packets.low) > TG3_NUM_RDMA_CHANNELS)) {
+ u32 val;
+
+ val = tr32(TG3_LSO_RD_DMA_CRPTEN_CTRL);
+ val &= ~TG3_LSO_RD_DMA_TX_LENGTH_WA;
+ tw32(TG3_LSO_RD_DMA_CRPTEN_CTRL, val);
+ tg3_flag_clear(tp, 5719_RDMA_BUG);
+ }
TG3_STAT_ADD32(&sp->rx_octets, MAC_RX_STATS_OCTETS);
TG3_STAT_ADD32(&sp->rx_fragments, MAC_RX_STATS_FRAGMENTS);
diff --git a/drivers/net/ethernet/broadcom/tg3.h b/drivers/net/ethernet/broadcom/tg3.h
index 6fb45a5..6d52cb2 100644
--- a/drivers/net/ethernet/broadcom/tg3.h
+++ b/drivers/net/ethernet/broadcom/tg3.h
@@ -1376,7 +1376,11 @@
#define TG3_LSO_RD_DMA_CRPTEN_CTRL 0x00004910
#define TG3_LSO_RD_DMA_CRPTEN_CTRL_BLEN_BD_4K 0x00030000
#define TG3_LSO_RD_DMA_CRPTEN_CTRL_BLEN_LSO_4K 0x000c0000
-/* 0x4914 --> 0x4c00 unused */
+#define TG3_LSO_RD_DMA_TX_LENGTH_WA 0x02000000
+/* 0x4914 --> 0x4be0 unused */
+
+#define TG3_NUM_RDMA_CHANNELS 4
+#define TG3_RDMA_LENGTH 0x00004be0
/* Write DMA control registers */
#define WDMAC_MODE 0x00004c00
@@ -2959,6 +2963,7 @@ enum TG3_FLAGS {
TG3_FLAG_L1PLLPD_EN,
TG3_FLAG_APE_HAS_NCSI,
TG3_FLAG_4K_FIFO_LIMIT,
+ TG3_FLAG_5719_RDMA_BUG,
TG3_FLAG_RESET_TASK_PENDING,
TG3_FLAG_5705_PLUS,
TG3_FLAG_IS_5788,
--
1.7.1
^ permalink raw reply related
* [PATCH net 4/5] tg3: Fix race condition in tg3_get_stats64()
From: Michael Chan @ 2012-07-30 5:15 UTC (permalink / raw)
To: davem; +Cc: netdev
In-Reply-To: <1343625345-5155-3-git-send-email-mchan@broadcom.com>
Spinlock should be taken before checking for tp->hw_stats.
Signed-off-by: Michael Chan <mchan@broadcom.com>
---
drivers/net/ethernet/broadcom/tg3.c | 6 ++++--
1 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index 50045ed..f03614b 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -12524,10 +12524,12 @@ static struct rtnl_link_stats64 *tg3_get_stats64(struct net_device *dev,
{
struct tg3 *tp = netdev_priv(dev);
- if (!tp->hw_stats)
+ spin_lock_bh(&tp->lock);
+ if (!tp->hw_stats) {
+ spin_unlock_bh(&tp->lock);
return &tp->net_stats_prev;
+ }
- spin_lock_bh(&tp->lock);
tg3_get_nstats(tp, stats);
spin_unlock_bh(&tp->lock);
--
1.7.1
^ permalink raw reply related
* [PATCH net 5/5] tg3: Update version to 3.124
From: Michael Chan @ 2012-07-30 5:15 UTC (permalink / raw)
To: davem; +Cc: netdev
In-Reply-To: <1343625345-5155-4-git-send-email-mchan@broadcom.com>
Signed-off-by: Michael Chan <mchan@broadcom.com>
---
drivers/net/ethernet/broadcom/tg3.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index f03614b..bf906c5 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -92,7 +92,7 @@ static inline void _tg3_flag_clear(enum TG3_FLAGS flag, unsigned long *bits)
#define DRV_MODULE_NAME "tg3"
#define TG3_MAJ_NUM 3
-#define TG3_MIN_NUM 123
+#define TG3_MIN_NUM 124
#define DRV_MODULE_VERSION \
__stringify(TG3_MAJ_NUM) "." __stringify(TG3_MIN_NUM)
#define DRV_MODULE_RELDATE "March 21, 2012"
--
1.7.1
^ permalink raw reply related
* Re: [net-next RFC V5 4/5] virtio_net: multiqueue support
From: Jason Wang @ 2012-07-30 5:15 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: krkumar2, habanero, mashirle, kvm, netdev, linux-kernel,
virtualization, edumazet, tahm, jwhan, davem, sri
In-Reply-To: <20120729095021.GC8977@redhat.com>
On 07/29/2012 05:50 PM, Michael S. Tsirkin wrote:
> On Mon, Jul 23, 2012 at 01:48:35PM +0800, Jason Wang wrote:
>>>> + }
>>>>
>>>> - if (virtio_has_feature(vi->vdev, VIRTIO_NET_F_CTRL_VQ)) {
>>>> + ret = vi->vdev->config->find_vqs(vi->vdev, total_vqs, vqs, callbacks,
>>>> + (const char **)names);
>>>> + if (ret)
>>>> + goto err;
>>>> +
>>>> + if (vi->has_cvq)
>>>> vi->cvq = vqs[2];
>>>>
>>>> - if (virtio_has_feature(vi->vdev, VIRTIO_NET_F_CTRL_VLAN))
>>>> - vi->dev->features |= NETIF_F_HW_VLAN_FILTER;
>>>> + for (i = 0; i< vi->num_queue_pairs * 2; i += 2) {
>>>> + int j = i == 0 ? i : i + vi->has_cvq;
>>>> + vi->rq[i / 2]->vq = vqs[j];
>>>> + vi->sq[i / 2]->vq = vqs[j + 1];
>>> Same here.
>> Consider the code is really simple, seem no need to use helpers.
> Well it was not simple to at least one reader :)
> The problem is not this logic is complex,
> it is that it is spread all over the code.
>
> If we had e.g. vnet_tx_vqn_to_queuenum vnet_tx_queuenum_to_vqn
> and same for rx, then the logic would all be
> in one place, and have a tidy comment on top explaining
> the VQ numbering scheme.
>
Looks reasonable, thanks.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox